AI Image Editing With Words: How Instruction Edits Work

After reading this, you'll be able to change a specific part of an image by describing the change in plain words, keep the rest of the picture intact, and know why an edit sometimes drifts. That's the core of modern AI image editing: you show the model a picture, tell it what to do, and it returns a new version.

What instruction-based editing actually is

Older editing meant masks, layers and selection tools. Instruction-based editing flips that. You give the model an existing image plus a sentence such as "make the mug olive green", and it produces an edited image.

The idea has a clear research origin. The 2022 paper InstructPix2Pix: Learning to Follow Image Editing Instructions trained a model that takes an input image and a written instruction and returns the edit. Its training data was generated by pairing a language model (GPT-3) with a text-to-image model (Stable Diffusion). The authors report that it works in a single forward pass with no per-example fine-tuning, so edits take seconds, and that it generalizes to real images and instructions written by people.

Today's image models build on that idea. Google's Gemini API image generation docs describe providing an image and using text prompts to "add, remove, or modify elements, change the style, or adjust the color grading." OpenAI's image models support the same pattern through an edits endpoint.

Why edits drift

A generative model doesn't move pixels around like a photo editor. It produces a new image that is conditioned on your original. That's why a "small" edit can quietly change a face, a logo or the background.

OpenAI's cookbook on generating images with high input fidelity is useful here. It says a high-fidelity setting lets you "make subtle edits to an image without altering unrelated areas," and that it matters for faces, logos and other details that must stay accurate. Two caveats from that page: higher fidelity uses more input tokens, and the recipe is marked as archived, so it may reference outdated models or APIs. The lesson holds either way: if preservation is a setting in your tool, use it. If it isn't, you have to ask for preservation in words.

The edit prompt formula

Good edit prompts are short and specific. A reliable structure:

1. The action: change, add, remove, replace, restyle.
2. The target: the one object or area you mean.
3. The new state: colour, material, style, position.
4. The guard: what must stay the same.

Put together:

• Make the mug on the desk olive green. Keep everything else exactly the same.
• Remove the cardboard box from the left of the frame. Fill the space with the wooden floor.
• Replace the grey sky with a soft sunset. Keep the lighting on the person unchanged.
• Change the jacket from denim to black leather. Do not change the face, hair or pose.

The guard sentence isn't decoration. Google's docs use exactly this move in their own example: "Update this infographic to be in Spanish. Do not change any other elements of the image." Both OpenAI's examples ("Make the mug olive green", "Remove the mug from the desk") and Google's lean on the same habit: say the change, then say what to leave alone.

Edit one thing at a time

The most common mistake is stacking five changes in one instruction. The model has to guess priorities, and something usually gets dropped or altered.

Google's docs call multi-turn conversation "the recommended way to iterate on images." In practice that means a chain of small steps:

1. Fix the composition first (remove or add objects).
2. Then adjust colour and materials.
3. Then change the lighting or mood.
4. Finish with style changes.

If a step goes wrong, go back to the previous version and rephrase, rather than piling a correction onto a damaged image. Each pass is another regeneration, so tiny errors can compound. Save the good intermediate versions.

Name things the way the model sees them

"Fix it" and "make it better" give the model nothing to work with. Compare:

• Vague: Make it look more professional.
• Specific: Replace the cluttered shelf behind the speaker with a plain light grey wall. Keep the speaker's face and clothing unchanged.

When an image has several similar objects, point with position or a visible trait: "the second mug from the left", "the person in the red scarf". If the model keeps editing the wrong thing, describe the target by what's around it.

Text, faces and logos need extra care

Three areas break most often.

• Faces: The OpenAI cookbook says faces are preserved far more accurately with high input fidelity than in standard mode. If you're combining faces from several photos, it recommends merging them into a single composite image before the request, and putting the image with the face you care about first, because only the first input is preserved with extra richness in texture.
• Logos and product details: Say explicitly that the logo, label or packaging must not change, and check the result at full size. Even then, small lettering can warp. For product work, see AI Product Photography: Packshots, Lifestyle Shots, Props.
• Text in the image: Editing words is possible, as Google's translated-infographic example shows, but results vary run to run. Quote the exact new text and keep it short. More on that in How to Get Readable Text Inside AI Images.

Where the limits are

Be honest with yourself about what this technique can't do.

• It isn't pixel-exact. Even a well-guarded edit may shift fine texture, grain or small details elsewhere in the frame.
• Results vary between runs. The same instruction can succeed once and fail the next time, so generate a couple of options before judging a model.
• Some model variants are weaker at iteration. Google's docs say Nano Banana 2 Lite is "not optimized for multiple reference inputs or multi-turn sequential editing," so check the notes for the exact model you're using.
• Generated images from Google's models carry a SynthID watermark, according to the same docs.
• You need the rights to any image you upload. Google's best-practice notes say so directly.

Putting it to work in Mediafy Studio

Image models such as Nano Banana 2, GPT Image 2 and Seedream 5 Pro are available in Mediafy Studio, and the credit cost shows on the Generate button before you run anything (about 2 to 3 credits for an image today, though costs change). That makes small, repeated tries cheap enough to compare. If your instruction is short and rough, the Enhance button can rewrite it into a fuller prompt, but read the result: make sure the "keep everything else the same" guard survived. For background on each model's habits, see Nano Banana 2 Explained: What Google's Model Does Well and GPT Image for Design Work: Layouts, Text and Edits. If you want to rewrite a prompt from scratch, AI Image Prompt Structure: Subject, Setting, Light, Lens helps.

A quick checklist before you hit generate

• Does the prompt name one change?
• Is the target described clearly enough to find in the image?
• Is there a guard sentence for faces, logos or layout?
• Have I kept the previous good version?
• Will I compare two or three runs before deciding?

If you'd like to go deeper on workflow, the AI Creator Lab course covers it in more depth.

Ready to try your own edits? Sign in at Mediafy Studio and run your first prompt.

Sources

• Brooks, Holynski, Efros (UC Berkeley, arXiv), "InstructPix2Pix: Learning to Follow Image Editing Instructions": https://arxiv.org/abs/2211.09800
• Google AI for Developers, "Image generation with Gemini" (Gemini API docs): https://ai.google.dev/gemini-api/docs/image-generation
• OpenAI Cookbook, "Generate images with high input fidelity": https://developers.openai.com/cookbook/examples/generate_images_with_high_input_fidelity