🖼️

Image prompt engineering

Generating images is a task AI solved very solidly. You can generate what you want with great specificity, there are vibrant communities with copious amounts of resources to help you piece together your prompt, and there are multiple different models out there, each offering a set of strengths it is particularly good at. Here are some strategies and practices to get you started on this journey.

Leading image generators

🖼️
Dall-E - has a simple interface that makes it easy to get started generating images. Just go to the website and enter your text into the Generate box to create an image. DALL-E 2 will then produce four versions of the prompt, which you can edit or download. You will need to pay for credits to start using it.

image
🖼️
MidJourney - runs through Discord, a communication and interest group platform, so you will need a Discord account. There is a slight learning curve in figuring out how to use it. (you need to type in "/imagine" in the input chat box to generate your images. Full guide can be found here). But in return you get beautiful art that you can create with a few words. And you can start experimenting with it for free.

image

🖼️
Stable Diffusion - a competitor with MidJourney to be the best of the best. The layout of Stable Diffusion is more cluttered than DALL-E 2 and Midjourney, but still something you can intuitively pick up on by poking around. This is a paid platform. Trial users get 200 free credits which should suffice to get your feet wet.

image

Methods for image prompt engineering:

#1 Snag a prompt for a style you like and edit it:

1. Go to one of these community storehouses of different image prompts.

Important: an image prompt for one model (like Midjourney) may not perform well in another model (like Stable Diffusion). Edit and reuse the prompts with the same model in which the original user ran with.

icon
Midjourney Discord - a bottomless storehouse of all the images ever generated on Midjourney together with the prompts that created them.
image
icon
Stable Foundation Discord - Community channel for sharing prompts and advice on generating with Stable Diffusion.

image

What is Discord? Discord is a website which hosts community channels for developers, tech enthusiasts and just about any interest group you can think of. You will need to create an account and click the join link below to get into the referred Discord channels.

2. Look up image styles that you are interested in.

Example: you can look up cartoon style, or photo-realistic style, or in style of an artist like Van Gogh.

image

3. Copy the prompt of an image whose style you like.

image

4. Modify the key words in the prompt to generate the content you are looking     for while sticking to the same style.

image

#2 Use a prompting assistant

There are tools which either suggest prompts to an image you already have .

For Stable Diffusion:

icon
Img2prompt - Get a suggested text prompt from an uploaded image

For MidJourney:

icon
/discover command - go to a Discord channel on Midjourney where you generate content and type in /discover into the input. You will be offered an opportunity to upload your own image and Midjourney will generate you a prompt

Model agnostic:

icon
Prompt parrot - a slightly different approach. This tool beefs up your prompt with a bunch of additional descriptors that will help generate a more precise and quality result. Results may vary.
image

#3 Ask ChatGPT for advice

Asking ChatGPT to generate an image prompt is a fool's errand, because much of the data on which ChatGPT was trained has little information contained about image generators. But it is still helpful for brainstorming different descriptors you can use for image prompting. To do so:

  1. Go to GPT3.5 or GPT4
  2. Give GPT the following instructions "Please create a table that breaks down an image scene composition into the following elements, where each of these elements is a column.”
  3. Type in the elements of an image which you are interested in modifying. For example: "Composition, camera angle, style, focal point, textures, details, color palette, lighting, location, time of day, mood, medium, etc."
  4. Ask it to make a table of all the options: "Fill the table with 10 rows of data, where: composition="<whatever you want the theme of the composition to be>"
image

In response, you will receive a table like this, from which you can choose the specifiers you are most interested in.

image