A grassroots collective of researchers working to open source AI research.
News & Announcements
GPT-NeoX
Announcing GPT-NeoX-20B, a 20 billion parameter model trained in collaboration with CoreWeave.
Read the announcement >GPT-J
GPT-J-6B, a 6 billion parameter model trained on the Pile, is now available for use with our new codebase, Mesh Transformer JAX.
Mesh Transformer JAX on GitHub >Blog
We believe the creation and open source release of a large language model is a net good to AI safety. We explain why.
Why Release a Large Language Model? >Blog
Rotary Positional Embedding (RoPE) is a new type of position encoding that unifies absolute and relative approaches. We put it to the test.
Rotary Embeddings: A Relative Revolution >GPT-Neo
GPT-Neo 1.3B and 2.7B are now available on Hugging Face Model Hub! Run the models with Transformers or call for them through their on-demand Inference API.
EleutherAI on Model Hub >GPT-Neo
GPT-Neo 1.3B and 2.7B, trained on the Pile, are now available to run with the GPT-Neo framework.
GPT-Neo on GitHub >The Pile
We are proud to announce the release of the Pile, a free and publicly available 825GB dataset of diverse English text for language modeling!
Visit the Pile >