dalle-mini

Model Details

• Developed by: Boris Dayma, Suraj Patil, Pedro Cuenca, Khalid Saifullah, Tanishq Abraham, Phúc Lê, Luke, Luke Melas, Ritobrata Ghosh
• Model type: Transformer-based text-to-image generation model
• Language(s): English
• License: Apache 2.0
• Model Description: This is a model that can be used to generate images based on text prompts. As the model developers wrote in the project report about DALL·E mini, “OpenAI had the first impressive model for generating images with DALL·E. DALL·E mini is an attempt at reproducing those results with an open-source model.”
• Resources for more information: See OpenAI’s website for more information about DALL·E, including the DALL·E model card. See the project report for more information from the model’s developers. To learn more about DALL·E Mega, see the DALL·E Mega training journal.
• Cite as:

@misc{Dayma_DALL·E_Mini_2021,
      author = {Dayma, Boris and Patil, Suraj and Cuenca, Pedro and Saifullah, Khalid and Abraham, Tanishq and Lê Khắc, Phúc and Melas, Luke and Ghosh, Ritobrata},
      doi = {10.5281/zenodo.5146400},
      month = {7},
      title = {DALL·E Mini},
      url = {https://github.com/borisdayma/dalle-mini},
      year = {2021}
}

Uses

Direct Use

The model is intended to be used to generate images based on text prompts for research and personal consumption. Intended uses include supporting creativity, creating humorous content, and providing generations for people curious about the model’s behavior. Intended uses exclude those described in the Misuse and Out-of-Scope Use section.

Downstream Use

The model could also be used for downstream use cases, including:

• Research efforts, such as probing and better understanding the limitations and biases of generative models to further improve the state of science
• Development of educational or creative tools
• Generation of artwork and use in design and artistic processes.
• Other uses that are newly discovered by users. This currently includes poetry illustration (give a poem as prompt), fan art (putting a character in various other visual universes), visual puns, fairy tale illustrations (give a fantasy situation as prompt), concept mashups (applying a texture to something completely different), style transfers (portraits in the style of), … We hope you will find your own application!

Downstream uses exclude the uses described in Misuse and Out-of-Scope Use.

Misuse, Malicious Use, and Out-of-Scope Use

The model should not be used to intentionally create or disseminate images that create hostile or alienating environments for people. This includes generating images that people would foreseeably find disturbing, distressing, or offensive; or content that propagates historical or current stereotypes.

Out-of-Scope Use

The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.

Misuse and Malicious Use

Using the model to generate content that is cruel to individuals is a misuse of this model. This includes:

• Generating demeaning, dehumanizing, or otherwise harmful representations of people or their environments, cultures, religions, etc.
• Intentionally promoting or propagating discriminatory content or harmful stereotypes.
• Impersonating individuals without their consent.
• Sexual content without consent of the people who might see it.
• Mis- and disinformation
• Representations of egregious violence and gore
• Sharing of copyrighted or licensed material in violation of its terms of use.
• Sharing content that is an alteration of copyrighted or licensed material in violation of its terms of use.

Limitations and Bias

Limitations

The model developers discuss the limitations of the model further in the DALL·E Mini technical report:

• Faces and people in general are not generated properly.
• Animals are usually unrealistic.
• It is hard to predict where the model excels or falls short…Good prompt engineering will lead to the best results.
• The model has only been trained with English descriptions and will not perform as well in other languages

Bias

CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes.

The model was trained on unfiltered data from the Internet, limited to pictures with English descriptions. Text and images from communities and cultures using other languages were not utilized. This affects