learn.ifs.name

PUMA: language tokens for low-resource multilingual ASR

Oukid et al., 2026, PUMA (Findings of ACL 2026)

Subscriber deck — free during early access

4 cards in “4.4 Zero-shot & efficiency” · whole deck

4.4 Zero-shot & efficiency

  1. What is PUMA's zero-shot evaluation?
    Testing on FLEURS for seven of the languages (Wolof, Hausa, Yoruba, Igbo, Shona, Xhosa, Arabic), i.e. on read speech from a different corpus than the training data.
  2. Zero-shot FLEURS average WER: PUMA vs MMS, Whisper-large, SeamlessM4T and Omnilingual ASR?
    PUMA 0.407; MMS 0.482; SeamlessM4T v2 0.652; Whisper-large 0.792. Omnilingual ASR is better: CTC-1B 0.372 and LLM-7B 0.274 (best overall).
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §4.4, p. 377, table 4
  3. On which FLEURS languages does MMS beat PUMA zero-shot?
    Hausa (0.264 vs 0.324), Shona (0.245 vs 0.255) and Xhosa (0.401 vs 0.420). PUMA wins on the average, not everywhere.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), Appendix B, pp. 381–382, tables 7, 9
  4. How many parameters does PUMA train, and how does that scale with languages?
    About 30M (ULP, language tokens, LM head) on a fixed 1B backbone. Adding a language adds just one token vector. MMS also trains about 30M in this ten-language setup, but its adapters grow linearly with every language; the other systems compared train 244M to 7B parameters.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §§4.3, 4.4, pp. 375, 377, fig. 2

Sources