Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

1. Introduction

1. What is the main idea and purpose of the T5 paper?
   Treat every text problem as text-to-text (text in, text out), so one model, objective, training procedure and decoding process apply to every task; use this unified setup for a systematic empirical survey of transfer learning choices, then scale up the best ones (up to 11B parameters) for state-of-the-art results.
   • Raffel et al., 2020, Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (Journal of Machine Learning Research), 1. Introduction, pp. 1–3
2. Is T5 mainly a new method?
   No: its authors stress that the goal is not to propose new methods but to give a comprehensive perspective on where transfer learning for NLP stands, through a survey and controlled comparison of existing techniques, plus new resources (C4, code, models).
   • Raffel et al., 2020, Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (Journal of Machine Learning Research), 1. Introduction, pp. 1–3