Instruction-Based Image Editing and Inpainting
Changing, removing or filling parts of an image from a written instruction or a mask.
Tier Grades
Tier A: Frontier AI labs
feasible
Big labs ship editing models such as FLUX Kontext.
Tier B: Indonesian AI labs
feasible
Building synthetic edit data from open models, as InstructPix2Pix did, fits an Indonesian lab.
Tier C: Institution units such as ITB
feasible
A 3-author team produced InstructPix2Pix; a university unit can work on it.
Tier D: Organisations such as SEMAI (PoT, PoC, PoV)
feasible
Product photo editing and virtual try-on give fast retail proofs of value.
Where this comes from
Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.
Anchors
- ComputeTraining compute is not in the sources read.
- DataTraining data is generated with GPT-3 and Stable Diffusion; size not in the abstract.
- Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): 3 authors on the InstructPix2Pix paper (arXiv 2211.09800).
- Time horizonMonths.
- Shelf-life signalsGeneral image models edit as well as specialist editors, Editing provenance labels become mandatory
- Pipeline roleTier A releases editors; Tier C builds local edit datasets; Tier D uses editors in retail and media pilots.
Where this comes from
Team size is a proxy (author counts of exemplar papers), not a headcount.
Who and What
Exemplars
Observed actors
Linked models
FLUX.1 Kontextimage generationexemplar
Step1X-Editimage generationexemplar
InstructPix2Piximage generationexemplar
LaMaimage generationexemplar
GPT Image 2.5 Sunburstimage generationproduced by
Midjourney V8.2image generationproduced by
JoyAI-Image-Editimage generationproduced by
Riverflow 2.0image generationproduced by
Seedream 4.5image generationproduced byFLUX.2 [pro]image generationproduced byFLUX.2 [flex]image generationproduced by
ChronoEdit-14Bimage generationproduced byFLUX.1 Tools (Fill, Depth, Canny, Redux)image generationproduced byIC-Lightimage generationproduced by
PowerPaintimage generationproduced by
DragGANimage generationproduced byGLIDEimage generationproduced by
AOT-GANimage generationproduced by
Boundlessimage generationproduced by
GMCNNimage generationproduced by
DeepFill v2image generationproduced byPartial Convolution Inpaintingimage generationproduced byDeepFill v1image generationproduced byDeepFillimage generationproduced byLinks
R&D TopicInstruction-Based Image Editing and Inpainting
ModelsFLUX.1 KontextBlack Forest Labs, image generationInstructPix2PixUC Berkeley, image generationStep1X-EditStepFun, image generationLaMaSamsung, image generation+23 more
FLUX.1 KontextBlack Forest Labs, image generationInstructPix2PixUC Berkeley, image generationStep1X-EditStepFun, image generationLaMaSamsung, image generation+23 moreUse Casesnone recorded
Who Runs Itnone recorded
Models 27
DeepFillUniversity of Illinois Urbana-Champaign, Adobe Inc., image generationproduced byGPT Image 2.5 SunburstOpenAI, image generationproduced byMidjourney V8.2Midjourney, image generationproduced byDeepFill v1University of Illinois Urbana-Champaign, Adobe Inc., image generationproduced byDeepFill v2University of Illinois Urbana-Champaign, Adobe Inc., image generationproduced byFLUX.1 KontextBlack Forest Labs, image generationexemplarFLUX.1 Tools (Fill, Depth, Canny, Redux)Black Forest Labs, image generationproduced byFLUX.2 [flex]Black Forest Labs, image generationproduced byFLUX.2 [pro]Black Forest Labs, image generationproduced byGPT Image 1OpenAI, image generationproduced byGPT Image 1 miniOpenAI, image generationproduced by
Ideogram 4.5Ideogram, image generationproduced by+15
Seedream 4.5ByteDance, image generationproduced byAOT-GANSun Yat-sen University, Microsoft Research, image generationproduced byGLIDEOpenAI, image generationproduced byGMCNNThe Chinese University of Hong Kong, Tencent, image generationproduced byLaMaSamsung, Skolkovo Institute of Science and Technology, image generationexemplarPowerPaintTsinghua University, Shanghai AI Lab, image generationproduced byRiverflow 2.0Sourceful, image generationproduced byBoundlessGoogle, image generationproduced byChronoEdit-14BNVIDIA, image generationproduced byDragGANMax Planck Institute for Informatics, image generationproduced byInstructPix2PixUC Berkeley, image generationexemplarJoyAI-Image-EditJD.com, image generationproduced byPartial Convolution InpaintingNVIDIA, image generationproduced byStep1X-EditStepFun, image generationexemplarIC-Lightimage generationproduced byPapers
- SDEdit: Guided Image Synthesis and Editing with Stochastic Differential EquationsChenlin Meng et al., arXiv preprint, 2021 founding
- FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent SpaceBlack Forest Labs et al., arXiv preprint, 2025 state of the art
- MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image EditingKai Zhang et al., NeurIPS 2023 (per arXiv comment), 2023 state of the art
- RePaint: Inpainting using Denoising Diffusion Probabilistic ModelsAndreas Lugmayr et al., arXiv preprint, 2022 state of the art
- Prompt-to-Prompt Image Editing with Cross Attention ControlAmir Hertz et al., arXiv preprint, 2022 state of the art
Where this comes from
The papers to read first: where the direction began, its current best, and a survey when one exists.
Sources
- arxiv.org/abs/2211.09800 https://arxiv.org/abs/2211.09800
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Instruction-Based Image Editing and Inpainting", https://aurion.semai.id/topics/instruction-based-image-editing-and-inpainting/, records of October 2026.