The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
Penedo et al., 2024, The FineWeb Datasets (Advances in Neural Information Processing Systems 37 (NeurIPS 2024), Datasets and Benchmarks Track)
Subscriber deck — free during early access
1 cards in “5 Bias analyses” · whole deck
5 Bias analyses
- What biases did the FineWeb bias analysis find?Overrepresentation of words reflecting hegemonic norms (e.g. “man”, “christian”), with the most skewed associations between religion words and dating (e.g. “christian dating”); FineWeb-Edu’s associations looked more like educational history and health content.
- Penedo et al., 2024, The FineWeb Datasets (Advances in Neural Information Processing Systems 37 (NeurIPS 2024), Datasets and Benchmarks Track), §5
Source
- Guilherme Penedo, Hynek Kydlíček, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra and Thomas Wolf. 2024. The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale. In Advances in Neural Information Processing Systems 37 (NeurIPS 2024), Datasets and Benchmarks Track. https://papers.nips.cc/paper_files/paper/2024/hash/370df50ccfdf8bde18f8f9c2d9151bda-Abstract-Datasets_and_Benchmarks_Track.html