Over 100 open speech recognition models for spontaneous speech in 19 African languages. Fine-tuning lowers macro WER from 52.3 (best zero-shot model covering all 19 languages) to 31.5, with controlled comparisons of decoders, training granularity, and transfer.
2026 · arXiv:2606.02375
We benchmark zero-shot and fine-tuned speech recognition on WAXAL, a corpus of spontaneous speech in 19 African languages, covering Whisper, MMS, Omnilingual ASR, and SeamlessM4T-v2. Fine-tuning lowers macro WER from 52.3 for the best zero-shot model covering all 19 languages (Omnilingual-7B) to 36.2 for MMS-300M and 31.5 for Omnilingual with an LLM decoder, although zero-shot models are already competitive on a few languages.
Word error rate (%) on the WAXAL test set (utterances of 1.5 to 40 s, the same for every model). Lower is better.
| Language | Family | Best zero-shot | Whisper T | Whisper S | MMS-300M | Omni CTC | Omni LLM |
|---|---|---|---|---|---|---|---|
| Acholi | Nilotic | 51.7 | 59.6 | 43.9 | 41.5 | 37 | 33.1 |
| Akan | Niger-Congo (Kwa) | 47.2 | 38.6 | 33.4 | 33.9 | 32.4 | 29.2 |
| Amharic | Afro-Asiatic (Semitic) | 34.1 | 39.2 | 29.6 | 32.7 | 34.8 | 25 |
| Dagaare | Niger-Congo (Gur) | 46.8 | 37.2 | 34.9 | 34.2 | 34.4 | 28.9 |
| Dagbani | Niger-Congo (Gur) | 59.3 | 42.5 | 34.1 | 34.3 | 33.1 | 29.9 |
| Ewe | Niger-Congo (Kwa) | 49.5 | 36 | 32.9 | 30.8 | 32.4 | 29.1 |
| Fula | Niger-Congo (Atlantic) | 52.7 | 47 | 42.7 | 39.8 | 37.4 | 38.7 |
| Ikposo | Niger-Congo (Kwa) | 84.3 | 76.3 | 72.9 | 70.7 | 69.2 | 69.2 |
| Lingala | Niger-Congo (Bantu) | 29.5 | 43.2 | 35.4 | 36.1 | 30.7 | 30.1 |
| Luganda | Niger-Congo (Bantu) | 31.9 | 39.8 | 24.9 | 16.7 | 14.8 | 11.2 |
| Malagasy | Austronesian | 21.8 | 24 | 18.8 | 17.6 | 16.4 | 11.7 |
| Masaaba | Niger-Congo (Bantu) | 66.3 | 60.6 | 58.2 | 47.9 | 47.1 | 45 |
| Nyankole | Niger-Congo (Bantu) | 42.9 | 50.6 | 46.4 | 38.1 | 35.4 | 31.2 |
| Oromo | Afro-Asiatic (Cushitic) | 31.9 | 29.3 | 24.6 | 25.5 | 25.5 | 20.7 |
| Shona | Niger-Congo (Bantu) | 25.4 | 31.1 | 26.4 | 23.4 | 23.3 | 21.3 |
| Sidama | Afro-Asiatic (Cushitic) | 33.8 | 36.4 | 31.1 | 35.5 | 32 | 28.2 |
| Soga | Niger-Congo (Bantu) | 61.4 | 64 | 54.3 | 46.9 | 44 | 42.8 |
| Tigrinya | Afro-Asiatic (Semitic) | 61.3 | 51.3 | 43.4 | 44.7 | 45.9 | 37 |
| Wolaytta | Afro-Asiatic (Omotic) | 85.2 | 44.7 | 40.1 | 38.2 | 40 | 36.6 |
| Macro Average | – | 44.8 | 38.3 | 36.2 | 35 | 31.5 |
Bold = best fine-tuned model per language · Underlined = zero-shot better than every fine-tuned model · Omni = Omnilingual ASR, 300M encoder
Per-language models for all 19 languages, multilingual models, and research ablations. All on Hugging Face under CC-BY-4.0 (MMS-based models CC-BY-NC-4.0).
Most accurate: lowest WER among the fine-tuned models on 18 of 19 languages (macro 31.5). Slower than real time on one CPU thread; best with a GPU.
The same pretrained encoder with a CTC head (macro 35.0). About four times faster than real time on a single CPU thread.
Strong edge model (macro 36.2) with no repetition loops; the better edge model on most Bantu languages.
Macro 38.3; the better edge model on Amharic, Tigrinya, Oromo and Sidama.
Smallest and fastest: about 17 times faster than real time on one CPU thread, for the tightest budgets.
One model per language family, and one for all 19 languages, within about one WER point of per-language models.
from transformers import pipeline
# Replace {language} with any of the 19 ISO codes
# e.g. lug, amh, sna, ewe, orm ...
asr = pipeline("automatic-speech-recognition",
model="waxal-benchmarking/mms-300m-waxal-{language}")
result = asr("audio.wav")
print(result["text"])31 researchers across 3 continents
Linguistic Acknowledgements: Ajara Oyinloye · Abubakari Sadic Mohammed · Hafiz Adjei · Aliga Norah Lele · Marie-Louise B. Ndamuso · Odong Diana
@article{waxalnet2026,
title = {WaxalNet: Benchmarking and Adapting Speech Recognition for Spontaneous Speech in 19 African Languages},
author = {Olufemi, Victor Tolulope and Babatunde, Oreoluwa and Njema, Ramsey and
Gbotemi, Bolarinwa and Yen, Wanchi Lucia and Uzodinma, John and
Ajayi, Sunday and Williams, Oluwademilade and Moshood, Kausar and
Anyaele, Innocent Elendu and Arefaine, Akebert Tesfahunegn and
Hunzwi, Candace and Daniel, Wongel Dawit and Namuganga, Emmilly Immaculate and
Kadima, Cleophas and Bahizire, Athanase Biluge and Ranaivoson, Onitsiky and
Aaron, Emmanuel and Ladislaus, Nicholaus Dismas and Muhammed, Idris and
Simenya, Jonathan Enoch and Koome, Martin and Endaylalu, Matewos Tegete and
Adeyemo, Peter Ifeoluwa and Birindwa, Hondi Prisca and Eze-Mbey, Ukachi Agnes and
Oduro-Yeboah, Yacoba and Aremu, Toluwani and Adjovi, Pericles and
Ngueajio, Mikel K and Mitra, Prasenjit},
year = {2026},
note = {arXiv preprint arXiv:2606.02375}
}