learn.ifs.name

PUMA: language tokens for low-resource multilingual ASR

Oukid et al., 2026, PUMA (Findings of ACL 2026)

Subscriber deck — free during early access

5 cards in “4.3 Supervised results” · whole deck

4.3 Supervised results

  1. Monolingual training: how does PUMA compare with MMS?
    Lower WER on 9 of 10 languages. The biggest gains are Kabyle (0.551 → 0.307) and Hausa (0.143 → 0.029). Shona is the exception, slightly worse (0.310 → 0.313).
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §4.3.1, pp. 375–376, table 1
  2. Progressive training: what happens to PUMA's WER compared with monolingual?
    PUMA beats MMS on all ten languages. Some languages improve over their monolingual scores (Twi 0.338 → 0.285, Yoruba 0.274 → 0.251, Shona 0.313 → 0.297, Xhosa 0.319 → 0.304), while others get somewhat worse (e.g. Hausa 0.029 → 0.055).
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §4.3.2, pp. 375–376, tables 1, 2
  3. Simultaneous training: average WER of PUMA vs MMS vs Whisper-large?
    PUMA 0.314, Whisper-large 0.339, MMS 0.419 (Whisper-medium 0.402, Whisper-small 0.594). PUMA has the lowest WER and best ROUGE, but Whisper-large has the higher BLEU (0.499 vs 0.474).
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §4.3.3, p. 376, table 3
  4. In simultaneous training, on which languages does Whisper-large still beat PUMA in WER?
    Shona (0.305 vs 0.321), Xhosa (0.262 vs 0.329) and Fon (0.200 vs 0.207).
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), Appendix A, pp. 380–381, table 6
  5. How was Whisper fine-tuned in PUMA's experiments, and why only there?
    Whisper small (244M), medium (769M) and large (1.5B) were fine-tuned only in the simultaneous setting, because of compute limits.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §4.3, p. 375

Sources