Atext-to-image model is amachine learning model which takes an inputnatural language prompt and produces an image matching that description.

Text-to-image models began to be developed in the mid-2010s during the beginnings of theAI boom, as a result of advances indeep neural networks. In 2022, the output of state-of-the-art text-to-image models—such as OpenAI'sDALL-E 2,Google Brain'sImagen, Stability AI'sStable Diffusion, andMidjourney—began to be considered to approach the quality ofreal photographs and human-drawnart.

Text-to-image models are generallylatent diffusion models, which combine alanguage model, which transforms the input text into alatent representation, and agenerative image model, which produces an image conditioned on that representation. The most effective models have generally been trained on massive amounts of image and text datascraped from the web.^[1]

Before the rise ofdeep learning,^[when?] attempts to build text-to-image models were limited tocollages by arranging existing component images, such as from a database ofclip art.^[2][3]

The inverse task,image captioning, was more tractable, and a number of image captioning deep learning models came prior to the first text-to-image models.^[4]

A stop sign is flying in blue skies. by AlignDRAW (2015)^[5]

A stop sign is flying in blue skies by OpenAI's DALL-E 2 (2022), DALL-E 3 (2023), and GPT Image 1 (2025)

The first modern text-to-image model, alignDRAW, was introduced in 2015 by researchers from theUniversity of Toronto. alignDRAW extended the previously-introduced DRAW architecture (which used arecurrentvariational autoencoder with anattention mechanism) to be conditioned on text sequences.^[4] Images generated by alignDRAW were in smallresolution (32×32 pixels, attained fromresizing) and were considered to be 'low in diversity'. The model was able to generalize to objects not represented in the training data (such as a red school bus) and appropriately handled novel prompts such as "a stop sign is flying in blue skies", exhibiting output that it was not merely "memorizing" data from thetraining set.^[4][6]

In 2016, Reed, Akata, Yan et al. became the first to usegenerative adversarial networks for the text-to-image task.^[6][7] With models trained on narrow, domain-specific datasets, they were able to generate "visually plausible" images of birds and flowers from text captions like"an all black bird with a distinct thick, rounded bill". A model trained on the more diverseCOCO (Common Objects in Context) dataset produced images which were "from a distance... encouraging", but which lacked coherence in their details.^[6] Later systems include VQGAN-CLIP,^[8] XMC-GAN, and GauGAN2.^[9]

One of the first text-to-image models to capture widespread public attention wasOpenAI'sDALL-E, atransformer system announced in January 2021.^[10] A successor capable of generating more complex and realistic images, DALL-E 2, was unveiled in April 2022,^[11] followed byStable Diffusion that was publicly released in August 2022.^[12] In August 2022,text-to-image personalization allows to teach the model a new concept using a small set of images of a new object that was not included in the training set of the text-to-image foundation model. This is achieved bytextual inversion, namely, finding a new text term that correspond to these images.

Following other text-to-image models,language model-poweredtext-to-video platforms such as Runway, Make-A-Video,^[13] Imagen Video,^[14] Midjourney,^[15] and Phenaki^[16] can generate video from text and/or text/image prompts.^[17]

This sectionrelies largely or entirely upon asingle source. Relevant discussion may be found on thetalk page. Please helpimprove this article by introducingcitations to additional sources at this section.(December 2024) (Learn how and when to remove this message)

Text-to-image models have been built using a variety of architectures. The text encoding step may be performed with arecurrent neural network such as along short-term memory (LSTM) network, thoughtransformer models have since become a more popular option. For the image generation step, conditionalgenerative adversarial networks (GANs) have been commonly used, withdiffusion models also becoming a popular option in recent years. Rather than directly training a model to output a high-resolution image conditioned on a text embedding, a popular technique is to train a model to generate low-resolution images, and use one or more auxiliary deep learning models to upscale it, filling in finer details.

Text-to-image models are trained on large datasets of (text, image) pairs, often scraped from the web. With their 2022 Imagen model, Google Brain reported positive results from using alarge language model trained separately on a text-only corpus (with its weights subsequently frozen), a departure from the theretofore standard approach.^[18]

Training a text-to-image model requires a dataset of images paired with text captions. One dataset commonly used for this purpose is the COCO dataset. Released by Microsoft in 2014, COCO consists of around 123,000 images depicting a diversity of objects with five captions per image, generated by human annotators. Originally, the main focus of COCO was on the recognition of objects and scenes in images. Oxford-120 Flowers and CUB-200 Birds are smaller datasets of around 10,000 images each, restricted to flowers and birds, respectively. It is considered less difficult to train a high-quality text-to-image model with these datasets because of their narrow range of subject matter.^[7]

One of the largest open datasets for training text-to-image models is LAION-5B, containing more than 5 billion image-text pairs. This dataset was created using web scraping and automatic filtering based on similarity to high-quality artwork and professional photographs. Because of this, however, it also contains controversial content, which has led to discussions about the ethics of its use.

Some modern AI platforms not only generate images from text but also create synthetic datasets to improve model training and fine-tuning. These datasets help avoid copyright issues and expand the diversity of training data.^[19]

Evaluating and comparing the quality of text-to-image models is a problem involving assessing multiple desirable properties. A desideratum specific to text-to-image models is that generated images semantically align with the text captions used to generate them. A number of schemes have been devised for assessing these qualities, some automated and others based on human judgement.^[7]

A common algorithmic metric for assessing image quality and diversity is theInception Score (IS), which is based on the distribution of labels predicted by a pretrainedInceptionv3image classification model when applied to a sample of images generated by the text-to-image model. The score is increased when the image classification model predicts a single label with high probability, a scheme intended to favour "distinct" generated images. Another popular metric is the relatedFréchet inception distance, which compares the distribution of generated images and real training images according to features extracted by one of the final layers of a pretrained image classification model.^[7]

• Artificial intelligence art
• Text-to-video model
• AI slop

• Text-to-image generation

• Articles with short description
• Short description matches Wikidata
• Use mdy dates from November 2024
• All articles with vague or ambiguous time
• Vague or ambiguous time from October 2024
• Pages using multiple image with auto scaled images
• Articles needing additional references from December 2024
• All articles needing additional references
• Articles with excerpts