Pile

Diverse, open-source language modeling dataset consisting of 825 GiB of data compiled from 22 smaller, high-quality datasets. The Pile is designed to provide a comprehensive and diverse range of text data to support the training and evaluation of language models, offering a rich resource for natural language processing research and development. It is particularly valuable for researchers and developers working on advanced language models and AI applications.

Pile was discovered on August 16th 2019 and it currently has a search volume of 673K with a growth of +14%.