Language Models are Unsupervised Multitask Learners
Radford et al., 2019, Language Models are Unsupervised Multitask Learners (OpenAI technical report)
Subscriber deck — free during early access
1 cards in “3.7. Translation” · whole deck
3.7. Translation
- How well did GPT-2 translate, and why was it surprising?Prompted with example pairs “english sentence = french sentence”: 5 BLEU English→French and 11.5 French→English, beating some unsupervised MT baselines but far below the best (33.5). Non-English pages had been filtered out, leaving only about 10 MB of French in WebText.
- Radford et al., 2019, Language Models are Unsupervised Multitask Learners (OpenAI technical report), 3.7. Translation
Source
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. In OpenAI technical report. https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf