Codec2 3200 carries M17 voice, and its classic decoder is the weak spot. It’s limited to 8 kHz and has that recognizable buzzy, synthetic sound. Instead of designing a new codec, I wanted to know how far a better receiver could go with exactly the same bits on the air.
The answer is FARGAN, the low-complexity neural vocoder from Xiph’s Opus codec (Valin et al.). The idea is simple. The Codec2-mod decoder unpacks each 8-byte frame into its usual parameters: pitch, voicing, energy and LSPs. Instead of feeding those to the sinusoidal synthesizer, we map them to 20 features per 10 ms and let FARGAN generate 16 kHz speech.
Data: the Opus speech corpus, processed with Opus’s own augmentation recipe, but with features computed by the real Codec2-mod encoder and decoder. An important early lesson: never pair noisy features with a clean target. The network can’t denoise, and my first attempts sounded like speech through a pipe.
Training: Opus’ two stage recipe on rented GPUs1. Stage 1 was 400 epochs (about 14.5 hours), and stage 2 was 50 adversarial epochs (about 37 hours on an RTX 5090). The adversarial stage made a clearly audible difference from its first few epochs.
C library: the model uses about 818,000 parameters, runs on Opus’ own C inference code, and takes about 1.6 ms of CPU time per 20 ms frame on a single x86 core. The ARM64 NEON build compiles and produces matching output, real hardware timing test will follow.
The result is Codec2-mod-neural, a drop-in alternative to codec2_decode(): 8 bytes in, 320 samples at 16 kHz out. Nothing changes for transmitters, and every existing Codec2 decoder still works on the same stream.
Code, training scripts, and the model: https://github.com/M17-Project/Codec2-mod-neural
Next up: integration into DroidStar and mvoice, and later a look at running it on LinHT. ViSQOL benchmarks will also be ran along with synthetic BER tests to assess synthesizer’s noise immunity.
- M17 Foundation covered the expense (200 USD) with most of it left for future runs. ↩︎
Leave a Reply