learn.ifs.name

PUMA: language tokens for low-resource multilingual ASR

Oukid et al., 2026, PUMA (Findings of ACL 2026)

Subscriber deck — free during early access

42 cards

1 Problem & idea

  1. What is PUMA, in one sentence?
    Projected Universal Multilingual ASR: one shared multilingual ASR model that becomes language-aware through a small shared Universal Language Projection (ULP) module plus a learnable token per language, instead of language-specific adapters.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), Abstract, p. 371
  2. Why is ASR still poor for many African languages, according to PUMA's introduction?
    Limited annotated data, rich morphology, and high phonetic variability.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §1, p. 371
  3. What scaling problem of existing multilingual ASR approaches does PUMA target?
    Adapters, conditioning modules or per-language fine-tuning add parameters for every language, so the model grows as languages are added and scaling to new languages gets expensive.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §1, p. 371
  4. List PUMA's four claimed contributions.
    (1) A lightweight architecture with no language-specific adapters. (2) The ULP module with learnable language tokens and full parameter sharing. (3) Evaluation on ten low-resource languages under monolingual, progressive and simultaneous training. (4) Competitive or better results than larger models, supervised and zero-shot, with far fewer trainable parameters.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §1, pp. 371–372

2 Related work

  1. What weakness of fully shared multilingual end-to-end ASR models does the paper point out?
    With all parameters shared and no explicit language signal, they struggle to capture language-specific phonology and morphology.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §2, p. 372
  2. What do self-supervised speech models (wav2vec 2.0, XLSR, XLS-R, USM) do well and badly, per PUMA's related work?
    They transfer well to new languages after fine-tuning, but they don't explicitly model language identity.
  3. How does MMS (Massively Multilingual Speech) handle many languages, and what is the cost?
    A shared 1B-parameter wav2vec 2.0 backbone plus a small adapter per language. Each adapter adds over 2M parameters, so the total grows linearly with the number of languages.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §2–3.1, pp. 372–373
  4. Where does PUMA position itself relative to Whisper and Omnilingual ASR?
    Those reach strong (including zero-shot) coverage through very large capacity. PUMA aims instead at a compact, scalable model for low-resource settings.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §2, p. 372

3.1 Architecture

  1. Name PUMA's five components, in order.
    Feature Extractor (FE) → Feature Projection (FP) → Universal Language Projection (ULP) → Encoder → LM head.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §3, pp. 372–373, fig. 1
  2. Which PUMA components are frozen and which are trained?
    Frozen: the feature extractor, feature projection and encoder, all taken from a pretrained wav2vec 2.0 model. Trained: the ULP module, the language tokens and the LM head (about 30M parameters).
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §§3.1, 4.3, pp. 373, 375, fig. 1
  3. Equation for the features entering the ULP (Eq. 1)?
    \(h = FP(FE(x)) \in \mathbb{R}^{N \times L \times D_{fp}}\), where N is the batch size, L the number of frames, and \(D_{fp}\) the projection dimension.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §3.1, p. 373
  4. What is a PUMA language token?
    A learnable vector \(t_l \in \mathbb{R}^{1 \times D_{fp}}\), one per language l, that tells the shared ULP which language the input is in.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §3.1, p. 373
  5. How is the language token combined with the audio features (Eq. 3)?
    It is prepended along the time axis: \(h' = \mathrm{concat}(t_l, h) \in \mathbb{R}^{N \times (L+1) \times D_{fp}}\).
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §3.1, p. 373
  6. What is inside the ULP module?
    n = 4 shared Transformer blocks (RMSNorm, multi-head self-attention, feed-forward) applied to the token-plus-features sequence: \(u = \mathrm{ULP}(h'; \theta)\). Through self-attention every frame can attend to the language token.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §§3.1, 4.2, pp. 373, 375, fig. 1
  7. Why is the language token removed after the ULP (Eq. 5), and why does it matter for CTC?
    The token doesn't correspond to any span of audio. Keeping it would add an extra position and break the alignment between output frames and the transcript that the CTC loss relies on.
  8. How does the ULP output reach the encoder (Eq. 6)?
    As a residual: \(e = \mathrm{Encoder}(h + u)\). The ULP learns a language-specific correction added to the original features, and the frozen wav2vec 2.0 encoder processes the sum.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §3.1, p. 373
  9. What does the LM head compute (Eq. 7)?
    A per-frame softmax over the character vocabulary: \(z = \mathrm{Softmax}(We + b)\), with \(W \in \mathbb{R}^{D_{enc} \times M}\), where M is the vocabulary size.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §3.1, p. 373
  10. Contrast how MMS and PUMA make a shared model language-specific.
    MMS swaps in a separate set of adapter weights \(\phi_l\) per language. PUMA keeps all weights shared and changes only a single input vector, the language token \(t_l\).
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §3.1, p. 373

3.2 Training regimes

  1. What loss is PUMA trained with (Eq. 12)?
    CTC loss over the per-frame output distributions, with a blank token for frames that don't emit a character.
  2. What are PUMA's three training regimes?
    Monolingual: a separate model per language. Progressive: languages added one after another. Simultaneous: all languages trained together.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §3.2, p. 374
  3. How does progressive training fight catastrophic forgetting?
    Each new stage starts from the previous weights and replays a small subset of data from the languages already learned. The paper doesn't say how much data is replayed or in what order languages are added.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §§3.2, 4.3.2, pp. 374–375
  4. How is simultaneous training set up, and what risk does the paper note?
    Training on the union of all language datasets, \(D = \bigcup_l D_l\), with mini-batches sampled across languages. The risk is gradient interference between languages.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §3.2, p. 374

4.1–4.2 Data & setup

  1. Which ten languages does PUMA evaluate on?
    Kabyle, Wolof, Hausa, Yoruba, Igbo, Shona, Xhosa, Fon, Twi, and Arabic.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §4.1, p. 375
  2. How much training data per language, at what sampling rate, and with what output vocabulary?
    About 15 hours of transcribed speech per language on average, resampled to 16 kHz. The character sets are merged into one shared vocabulary of 204 characters.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §4.1, p. 375
  3. Key PUMA hyperparameters?
    AdamW, learning rate \(10^{-3}\), weight decay \(10^{-3}\), effective batch size 32 (via gradient accumulation). ULP: 4 Transformer blocks, hidden dimension 768, 8 heads, projection dimension 1280. One NVIDIA A100 80 GB.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §4.2, p. 375
  4. Which evaluation metrics does PUMA report?
    WER (primary), plus BLEU and ROUGE-1/2/L, which measure n-gram overlap between hypothesis and reference.

4.3 Supervised results

  1. Monolingual training: how does PUMA compare with MMS?
    Lower WER on 9 of 10 languages. The biggest gains are Kabyle (0.551 → 0.307) and Hausa (0.143 → 0.029). Shona is the exception, slightly worse (0.310 → 0.313).
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §4.3.1, pp. 375–376, table 1
  2. Progressive training: what happens to PUMA's WER compared with monolingual?
    PUMA beats MMS on all ten languages. Some languages improve over their monolingual scores (Twi 0.338 → 0.285, Yoruba 0.274 → 0.251, Shona 0.313 → 0.297, Xhosa 0.319 → 0.304), while others get somewhat worse (e.g. Hausa 0.029 → 0.055).
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §4.3.2, pp. 375–376, tables 1, 2
  3. Simultaneous training: average WER of PUMA vs MMS vs Whisper-large?
    PUMA 0.314, Whisper-large 0.339, MMS 0.419 (Whisper-medium 0.402, Whisper-small 0.594). PUMA has the lowest WER and best ROUGE, but Whisper-large has the higher BLEU (0.499 vs 0.474).
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §4.3.3, p. 376, table 3
  4. In simultaneous training, on which languages does Whisper-large still beat PUMA in WER?
    Shona (0.305 vs 0.321), Xhosa (0.262 vs 0.329) and Fon (0.200 vs 0.207).
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), Appendix A, pp. 380–381, table 6
  5. How was Whisper fine-tuned in PUMA's experiments, and why only there?
    Whisper small (244M), medium (769M) and large (1.5B) were fine-tuned only in the simultaneous setting, because of compute limits.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §4.3, p. 375

4.4 Zero-shot & efficiency

  1. What is PUMA's zero-shot evaluation?
    Testing on FLEURS for seven of the languages (Wolof, Hausa, Yoruba, Igbo, Shona, Xhosa, Arabic), i.e. on read speech from a different corpus than the training data.
  2. Zero-shot FLEURS average WER: PUMA vs MMS, Whisper-large, SeamlessM4T and Omnilingual ASR?
    PUMA 0.407; MMS 0.482; SeamlessM4T v2 0.652; Whisper-large 0.792. Omnilingual ASR is better: CTC-1B 0.372 and LLM-7B 0.274 (best overall).
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §4.4, p. 377, table 4
  3. On which FLEURS languages does MMS beat PUMA zero-shot?
    Hausa (0.264 vs 0.324), Shona (0.245 vs 0.255) and Xhosa (0.401 vs 0.420). PUMA wins on the average, not everywhere.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), Appendix B, pp. 381–382, tables 7, 9
  4. How many parameters does PUMA train, and how does that scale with languages?
    About 30M (ULP, language tokens, LM head) on a fixed 1B backbone. Adding a language adds just one token vector. MMS also trains about 30M in this ten-language setup, but its adapters grow linearly with every language; the other systems compared train 244M to 7B parameters.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §§4.3, 4.4, pp. 375, 377, fig. 2

4.5 Ablation & limits

  1. What does the PUMA ablation remove, and what is the overall effect?
    It removes the language tokens from the ULP (simultaneous setting). WER rises for every language, slightly for Wolof, Igbo, Shona and Xhosa, strongly for Fon (0.207 → 0.296) and Arabic (0.384 → 0.505).
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §4.5, pp. 377–378, table 5
  2. What limitation do the authors themselves state?
    Zero-shot performance is limited by the deliberately lightweight design, compared with large systems like Omnilingual ASR that use massive capacity and LLM-based representations.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), Limitations, p. 378
  3. What future work does the paper propose?
    Multi-token language representations (capturing phonetic, orthographic and prosodic traits), Mixture-of-Experts strategies, and frugal LLM-inspired architectures within a constrained parameter budget.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §5, Limitations, p. 378

Reading it critically

  1. PUMA's training data: what should make you cautious about the very low Hausa WER (0.029)?
    The Hausa training set cited is a synthetic (XTTS-generated) ASR dataset, and the Yoruba, Igbo and Fon sets are cited as TTS datasets. Synthetic or studio TTS speech is much easier than natural speech. On FLEURS, Hausa WER is 0.324 and MMS does better.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §4.1, Appendix B, pp. 375, 381, dataset citations in the reference list
  2. The ablation text reports Arabic changes as a “relative WER increase of 12.1%” and BLEU drop of 11.3%. What's wrong?
    Those are absolute differences (0.384 → 0.505 and 0.345 → 0.232). The relative changes are about +31.5% WER and −32.8% BLEU. Always recompute relative vs absolute from the table.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §4.5, pp. 377–378, table 5
  3. Compare the MMS rows in Tables 1 (monolingual) and 2 (progressive). What do you notice?
    They are identical, so MMS appears not to have been retrained in the progressive setting; the comparison there is against monolingual MMS.
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §4.3.1–4.3.2, p. 376, tables 1, 2
  4. What details would you need to reproduce PUMA that the paper does not give?
    The exact wav2vec 2.0 / MMS checkpoint used, train/validation/test split sizes, the language order and replay amount in progressive training, and the decoding method (greedy vs beam search, any LM).
    • Oukid et al., 2026, PUMA (Findings of ACL 2026), §§3.1, 3.2, 4.1, pp. 373–375

Sources