Skip to content

AlonzoLeeeooo/awesome-image-inpainting-studies

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

58 Commits
LICENSE
LICENSE
 
 
README.md
README.md
 
 
reference.bib
reference.bib
 
 

Repository files navigation

A Collection of Image Inpainting Studies

This GitHub repository summarizes papers and resources related to the image inpainting task.

If you have any suggestions about this repository, please feel free to start a new issue or pull requests.

🔥 [Nov. 19th] We have released our latest paper titled "StableV2V: Stablizing Shape Consistency in Video-to-Video Editing", with the correponding code, model weights, and a testing benchmark DAVIS-Edit open-sourced. Feel free to check them out from the links!

🔥 News

Contents

To-Do Lists

  • Recent Papers
    • Update ECCV 2024 Papers
    • Update CVPR 2024 Papers
    • Update AAAI 2024 Papers
      • Update PDFs and References
    • Update ICLR 2024 Papers
    • Update NeurIPS 2024 Papers
    • Regular Maintenance of Preprint arXiv Papers and Missed Papers
  • Previous Papers
    • Published Papers on Conferences
      • Update CVPR papers
      • Update CVPRW papers
      • Update ICCV papers
      • Update ICCVW papers
      • Update ECCV papers
      • Update AAAI papers
      • Update IJCAI papers
      • Update ACM MM papers
      • Update WACV papers
      • Update NeurIPS papers
      • Update ICLR papers
    • Published Papers on Journals
      • Update TMM papers
      • Update TIP papers
      • Update TPAMI papers
      • Update TCSVT papers

<🎯Back to Top>

Papers

🔥 Text-Guided Image Inpainting

  • Year 2024
  • Year 2023
    • CVPR
      • NÜWA-LIP: Language-guided Image Inpainting with Defect-free VQGAN [Paper] [Code]
      • Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting [Paper]
      • SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model [Paper]
      • Paint by Example: Exemplar-based Image Editing with Diffusion Models [Paper] [Code] [Demo]
    • CVPRW
      • Scene Graph Driven Text-Prompt Generation for Image Inpainting [Paper]
    • ACM MM
      • Uni-paint: A Unified Framework for Multimodal Image Inpainting with Pretrained Diffusion Model [Paper] [Code]
    • SIGGRAPH
    • WACV
      • An Unified Framework for Language Guided Image Completion [Paper]
      • Interactive Image Manipulation with Complex Text Instructions [Paper]
  • Year 2022
    • CVPR
      • Blended Diffusion for Text-driven Editing of Natural Images [Paper] [Code]
  • Year 2021
    • CVPRW
      • Grounded, Controllable and Debiased Image Completion With Lexical Semantics [Paper]
    • ACM MM
      • Adversarial Learning with Mask Reconstruction for Text-Guided Image Inpainting [Paper]
  • Year 2020
    • ACM MM
      • MMFL: Multimodal Fusion Learning for Text-Guided Image Inpainting [Paper] [Code]
      • Text-Guided Neural Image Inpainting [Paper] [Code]
      • Text-Guided Image Inpainting [Paper]

<🎯Back to Top>

Vanilla Image Inpainting

  • Year 2024
    • CVPR
      • Structure Matters: Tackling the Semantic Discrepancy in Diffusion Models for Image Inpainting [Paper] [Code]
      • Don't Look into the Dark: Latent Codes for Pluralistic Image Inpainting [Paper]
    • WACV
      • Efficient Layout-Guided Image Inpainting for Mobile Use [Paper]
      • LatentPaint: Image Inpainting in Latent Space with Diffusion Models [Paper]
    • AAAI
      • Text Image Inpainting via Global Structure-Guided Diffusion Models [Paper] [Code]
      • SyFormer: Structure-Guided Synergism Transformer for Large-Portion Image Inpainting [Paper] [Code]
      • Large Occluded Human Image Completion via Image-Prior Cooperating [Paper] [Code]
    • ICLR
      • Image Inpainting via Tractable Steering of Diffusion Models [Paper]
      • Image Inpainting via Iteratively Decoupled Probabilistic Modeling [Paper] [Code]
    • TMM
      • Mutual Dual-Task Generator With Adaptive Attention Fusion for Image Inpainting [Paper]
    • TCSVT
      • Transformer-Based Image Inpainting Detection via Label Decoupling and Constrained Adversarial Training [Paper]
    • arXiv
      • HINT: High-quality INPainting Transformer with Mask-Aware Encoding and Enhanced Attention [Paper]
      • Rethinking Referring Object Removal [Paper]

<🎯Back to Top>

  • Year 2023
    • CVPRW
      • Image Inpainting with Hypergraphs for Resolution Improvement in Scanning Acoustic Microscopy [Paper]
      • Internal Diverse Image Completion [Paper] [Code]
    • ICCVW
      • TransInpaint: Transformer-based Image Inpainting with Context Adaptation [Paper]
    • AAAI
      • CoordFill: Efficient High-Resolution Image Inpainting via Parameterized Coordinate Querying [Paper] [Code]
      • Generative Image Inpainting with Segmentation Confusion Adversarial Training and Contrastive Learning [Paper] [Code]
    • WACV
      • Keys to Better Image Inpainting: Structure and Texture Go Hand in Hand [Paper] [Code]
      • GeoFill: Reference-Based Image Inpainting with Better Geometric Understanding [Paper]
      • Image Completion with Heterogeneously Filtered Spectral Hints [Paper] [Code]
    • TMM
      • Learning Adaptive Patch Generators for Mask-Robust Image Inpainting [Paper]
      • W-Net: Structure and Texture Interaction for Image Inpainting [Paper] [Code]
    • TIP
      • Progressive Contextual Aggregation Empowered by Pixel-Wise Confidence Scoring for Image Inpainting [Paper]
      • MagConv: Mask-Guided Convolution for Image Inpainting [Paper]
      • Context Adaptive Network for Image Inpainting [Paper]
    • TPAMI
      • Partial Convolution for Padding, Inpainting, and Image Synthesis [Paper] [Code]
    • TCSVT
      • Deep Image Inpainting With Enhanced Normalization and Contextual Attention [Paper]

<🎯Back to Top>

  • Year 2022
    • CVPR
      • MISF: Multi-level Interactive Siamese Filtering for High-Fidelity Image Inpainting [Paper] [Code]
      • MAT: Mask-Aware Transformer for Large Hole Image Inpainting [Paper] [Code]
      • Reduce Information Loss in Transformers for Pluralistic Image Inpainting [Paper] [Code]
      • Dual-path Image Inpainting with Auxiliary GAN Inversion [Paper]
      • RePaint: Inpainting using Denoising Diffusion Probabilistic Models [Paper] [Code]
      • TFill: Bridging Global Context Interactions for High-Fidelity Image Completion [Paper] [Code] [Project] [Video]
    • CVPRW
      • Zoom-to-Inpaint: Image Inpainting with High-Frequency Details [Paper] [Code]
      • Boundary-aware Image Inpainting with Multiple Auxiliary Cues [Paper] [Code]
      • Comparison of CoModGANs, LaMa and GLIDE for Art Inpainting Completing M.C Escher's Print Gallery [Paper]
      • Image Multi-Inpainting via Progressive Generative Adversarial Networks [Paper]
      • NTIRE 2022 Image Inpainting Challenge: Report [Paper] [Code]
      • GLaMa: Joint Spatial and Frequency Loss for General Image Inpainting [Paper]
      • PersonGONE: Image Inpainting for Automated Checkout Solution [Paper] [Code]
    • ECCV
      • MAE-FAR: Learning Prior Feature and Attention Enhanced Image Inpainting [Paper] [Code]
      • InvertFill: High-Fidelity Image Inpainting with GAN Inversion [Paper] [Code]
      • CM-GAN: Image Inpainting with Cascaded Modulation GAN and Object-Aware Training [Paper] [Code]
      • Unbiased Multi-Modality Guidance for Image Inpainting [Paper] [Code]
      • Inpainting at Modern Camera Resolution by Guided PatchMatch with Auto-Curation [Paper] [Code]
      • Hourglass Attention Network for Image Inpainting [Paper] [Code]
      • Diverse Image Inpainting with Normalizing Flow [Paper]
      • Perceptual Artifacts Localization for Inpainting [Paper] [Code]
      • Iterative Geometry-Aware Cross Guidance Network for Stereo Image Inpainting [Paper]
    • ACM MM
      • Delving Globally into Texture and Structure for Image Inpainting [Paper] [Code]
      • Atrous Pyramid Transformer with Spectral Convolution for Image Inpainting [Paper]
      • Deep Multi-Resolution Mutual Learning for Image Inpainting [Paper]
      • T-former: An Efficient Transformer for Image Inpainting [Paper] [Code]
    • WACV
      • LaMa: Resolution-robust Large Mask Inpainting with Fourier Convolutions [Paper] [Code]
    • NeurIPS
      • Cross-Image Context for Single Image Inpainting [Paper]
    • TMM
      • Deep Generative Model for Image Inpainting With Local Binary Pattern Learning and Spatial Attention [Paper] [Code]
    • TIP
      • Image Inpainting With Local and Global Refinement [Paper]
      • Pseudo Decoder Guided Light-Weight Architecture for Image Inpainting [Paper]
    • TPAMI
      • Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation [Paper] [Code] [Video]
    • TCSVT
      • Dual-Pyramidal Image Inpainting With Dynamic Normalization [Paper]
    • arXiv
      • GRIG: Few-Shot Generative Residual Image Inpainting [Paper] [Code] [Project]
      • Feature Refinement to Improve High Resolution Image Inpainting [Paper]

<🎯Back to Top>

  • Year 2021
    • CVPR
      • TransFill: Reference-Guided Image Inpainting by Merging Multiple Color and Spatial Transformations [Paper] [Code]
      • Image Inpainting With External-Internal Learning and Monochromic Bottleneck [Paper] [Code]
      • Image Inpainting Guided by Coherence Priors of Semantics and Textures [Paper]
      • PD-GAN: Probabilistic Diverse GAN for Image Inpainting [Paper] [Code]
      • Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAE [Paper] [Code]
    • ICCV
      • Distillation-guided Image Inpainting [Paper]
      • WaveFill: A Wavelet-based Generation Network for Image Inpainting [Paper] [Code]
      • CR-Fill: Generative Image Inpainting with Auxiliary Contextual Reconstruction [Paper] [Code]
      • Parallel Multi-Resolution Fusion Network for Image Inpainting [Paper]
      • ICT: High-Fidelity Pluralistic Image Completion with Transformers [Paper] [Code] [Project]
    • IJCAI
      • Context-Aware Image Inpainting with Learned Semantic Priors [Paper] [Code]
    • ACM MM
      • BAT-Fill: Diverse Image Inpainting with Bidirectional and Autoregressive Transformers [Paper] [Code]
      • JPGNet: Joint Predictive Filtering and Generative Network for Image Inpainting [Paper] [Code]
      • Learning Contextual Transformer Network for Image Inpainting [Paper]
      • GCM-Net: Towards Effective Global Context Modeling for Image Inpainting [Paper] [Code]
      • An Adaptive Iterative Inpainting Method with More Information Exploration [Paper]
      • Large Scale Image Completion via Co-Modulated Generative Adversarial Networks [[Paper]](Large Scale Image Completion via Co-Modulated Generative Adversarial Networks) [Code]
    • WACV
      • R-MNet: A Perceptual Adversarial Network for Image Inpainting [Paper] [Code]
      • Multi-Level Generative Chaotic Recurrent Network for Image Inpainting [Paper]
      • Hyperrealistic Image Inpainting with Hypergraphs [Paper] [Code]
    • TIP
      • Dynamic Selection Network for Image Inpainting [Paper] [Code]
      • Thanka Mural Inpainting Based on Multi-Scale Adaptive Partial Convolution and Stroke-Like Mask [Paper]
      • Image Inpainting by End-to-End Cascaded Refinement With Mask Awareness [Paper] [Code]
      • Texture Memory-Augmented Deep Patch-Based Image Inpainting [Paper]
    • TCSVT
      • Reference-Guided Landmark Image Inpainting With Deep Feature Matching [Paper] [Project]
      • Generative Memory-Guided Semantic Reasoning Model for Image Inpainting [Paper]

<🎯Back to Top>

  • Year 2020
    • CVPR
      • UCTGAN: Diverse Image Inpainting Based on Unsupervised Cross-Space Translation [Paper]
      • Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting [Paper] [Code]
      • RFR-Net: Recurrent Feature Reasoning for Image Inpainting [Paper] [Code]
      • Prior Guided GAN Based Semantic Inpainting [Paper]
      • Semantic Image Manipulation Using Scene Graph [Paper] [Code]
    • ECCV
      • MEDFE: Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizations [Paper] [Code]
      • High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling [Paper]
      • Guidance and Evaluation: Semantic-Aware Image Inpainting for Mixed Scenes [Paper]
      • Deep Generative Prior: Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation [Paper] [Code] [Video]
    • AAAI
      • Learning to Incorporate Structure Knowledge for Image Inpainting [Paper] [Code]
      • Region Normalization for Image Inpainting [Paper] [Code]
      • The Missing Data Encoder: Cross-Channel Image Completion with Hide-and-Seek Adversarial Network [Paper] [Code]
    • ACM MM
      • Image Inpainting Based on Multi-frequency Probabilistic Inference Model [Paper]

<🎯Back to Top>

  • Year 2019
    • CVPR
      • Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting [Paper] [Code]
      • Foreground-Aware Image Inpainting [Paper]
      • PEPSI: Fast Image Inpainting With Parallel Decoding Network [Paper] [Code]
      • Pluralistic Image Completion [Paper] [Code]
    • ICCV
      • StructureFlow: Image Inpainting via Structure-Aware Appearance Flow [Paper] [Code]
      • Coherent Semantic Attention for Image Inpainting [Paper] [Code]
      • LBAM: Image Inpainting With Learnable Bidirectional Attention Maps [Paper] [Code]
    • ICCVW
      • Deep Hyperspectral Prior: Single-Image Denoising, Inpainting, Super-Resolution. [Paper] [Code]
    • IJCAI
      • Generative Image Inpainting with Submanifold Alignment [Paper]
      • Coarse-to-Fine Image Inpainting via Region-wise Convolutions and Non-Local Correlation [Paper]
      • MUSICAL: Multi-Scale Image Contextual Attention Learning for Inpainting [Paper] [Code]
      • Single-shot Semantic Image Inpainting with Densely Connected Generative Networks [Paper]
      • GAIN: Gradient Augmented Inpainting Network for Irregular Holes [Paper]
      • Progressive Image Inpainting with Full-Resolution Residual Network [Paper] [Code]
      • Deep Fusion Network for Image Completion [Paper] [Code]

<🎯Back to Top>

  • Year 2018
    • CVPR
      • DeepFill-v1: Generative Image Inpainting With Contextual Attention [Paper] [Code] [Video]
      • Disentangling Structure and Aesthetics for Style-Aware Image Completion [Paper]
    • ECCV
      • Contextual-based Image Inpainting: Infer, Match, and Translate [Paper]
      • Shift-Net: Image Inpainting via Deep Feature Rearrangement [Paper] [Code]
      • Image Inpainting for Irregular Holes Using Partial Convolutions [Paper] [Code] [Video] [Demo]
    • ACM MM
      • Semantic Image Inpainting with Progressive Generative Networks [Paper] [Code]
    • NeurIPS
      • Image Inpainting via Generative Multi-column Convolutional Neural Networks [Paper] [Code]
    • Photo Realistic Image Completion via Dense Correspondence [Paper]
  • Year 2017
    • CVPR
      • High-Resolution Image Inpainting Using Multi-scale Neural Patch Synthesis [Paper] [Code]
      • Semantic Image Inpainting with Deep Generative Models [Paper] [Code]
      • Structural inpainting [Paper]

<🎯Back to Top>

  • Year 2016
    • CVPR
      • Context Encoders: Feature Learning by Inpainting [Paper] [Code]

<🎯Back to Top>

Blind Image Inpainting

  • Year 2023
    • CVPRW
      • Blind Image Inpainting via Omni-dimensional Gated Attention and Wavelet Queries [Paper] [Code]
    • TMM
      • FT-TDR: Frequency-Guided Transformer and Top-Down Refinement Network for Blind Face Inpainting [Paper]
    • TPAMI
      • Self-Prior Guided Pixel Adversarial Networks for Blind Image Inpainting [Paper]
  • Year 2022
    • ACM MM
      • TransCNN-HAE: Transformer-CNN Hybrid AutoEncoder for Blind Image Inpainting [Paper]
    • IPTA
      • ARIN: Adaptive Resampling and Instance Normalization for Robust Blind Inpainting of Dunhuang Cave Paintings [Paper]
  • Year 2020

<🎯Back to Top>

Edge-Based Image Inpainting

  • Year 2023
    • TPAMI
      • ZITS++: Image Inpainting by Improving the Incremental Transformer on Structural Priors [Paper] [Code] [Project]
  • Year 2022
    • CVPR
      • ZITS: Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding [Paper] [Code] [Project]
  • Year 2021
    • ICCV
      • Learning a Sketch Tensor Space for Image Inpainting of Man-made Scenes [Paper]
      • CTSDG: Image Inpainting via Conditional Texture and Structure Dual Generation [Paper] [Code]
    • TCSVT
  • Year 2019
    • ICCV
    • ICCVW
      • EdgeConnect: Structure Guided Image Inpainting using Edge Prediction [Paper] [Code]

<🎯Back to Top>

Sketch-Guided Image Inpainting

  • Year 2024
    • TMM
  • Year 2023
    • ACM MM
      • Draw2Edit: Mask-Free Sketch-Guided Image Manipulation [Paper] [Code]
  • Year 2022
  • Year 2021
    • CVPR
      • DeFLOCNet: Deep Image Editing via Flexible Low-Level Controls [Paper] [Code]
  • Year 2020
    • ECCV
      • Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketches [Paper] [Code] [Project]
  • Year 2019

<🎯Back to Top>

Face Image Inpainting

  • Year 2024
    • TCSVT
      • Learning from Text: A Multimodal Face Inpainting Network for Irregular Holes [Paper]
    • arXiv
      • Learning Position-Aware Implicit Neural Network for Real-World Face Inpainting [Paper]
      • E2F-Net: Eyes-to-Face Inpainting via StyleGAN Latent Space [Paper]
  • Year 2023
    • CVPRW
      • Unmasking Your Expression: Expression-Conditioned GAN for Masked Face Inpainting [Paper] [Code]
    • WACV
      • Nested Deformable Multi-head Attention for Facial Image Inpainting [Paper] [Code]
    • TMM
      • Pluralistic Face Inpainting With Transformation of Attribute Information [Paper]
    • TCSVT
      • Reference-Guided Large-Scale Face Inpainting With Identity and Texture Control [Paper]
  • Year 2022
    • TCSVT
      • SwapInpaint: Identity-Specific Face Inpainting With Identity Swapping [Paper]
      • Simultaneous Face Completion and Frontalization via Mask Guided Two-Stage GAN [Paper]
    • arXiv
  • Year 2021
    • ACM MM
      • When Face Completion Meets Irregular Holes: An Attributes Guided Deep Inpainting Network [Paper]
    • TMM
      • Recurrent Generative Adversarial Network for Face Completion [Paper]
  • Year 2020
    • CVPR
      • Learning Oracle Attention for High-Fidelity Face Completion [Paper]
    • AAAI
      • Geometry-Aware Face Completion and Editing [Paper]
    • IJCAI
      • r-FACE: Reference Guided Face Component Editing [Paper]
  • Year 2017

<🎯Back to Top>

Fashion Image Inpainting

  • Year 2024
    • TexFit: Text-Driven Fashion Image Editing with Diffusion Models [Paper]
  • Year 2019
    • TIP
      • Deep Portrait Image Completion and Extrapolation [Paper]
    • ICCV
      • FiNet: Compatible and Diverse Fashion Image Inpainting [Paper] [Code]

<🎯Back to Top>

Conventional Methods

  • Year 2024
    • arXiv
      • HySim: An Efficient Hybrid Similarity Measure for Patch Matching in Image Inpainting [Paper]
  • Year 2020
    • TIP
      • Image Inpainting Using Nonlocal Texture Matching and Nonlinear Filtering [Paper]
      • Truncated Low-Rank and Total p Variation Constrained Color Image Completion and its Moreau Approximation Algorithm [Paper]
  • Year 2019
    • Multiple Pyramids Based Image Inpainting Using Local Patch Statistics and Steering Kernel Feature [Paper]
  • Year 2018
    • TIP
      • A Group-Based Image Inpainting Using Patch Refinement in MRF Framework [Paper]
  • Year 2016
    • CVPR
      • Multiview Image Completion with Space Structure Propagation [Paper]
    • TIP
      • Image Inpainting Through Metric Labeling via Guided Patch Mixing [Paper]
  • Year 2011
    • ICIG
      • Structure-Aware Image Completion with Texture Propagation [Paper]
  • Year 2005
    • TIP
      • Edge-Based Image Restoration [Paper]
    • SIGGRAPH
      • Image Completion with Structure Propagation [Paper]
  • Year 2004
    • TIP
      • Region Filling and Object Removal by Exemplar-Based Image Inpainting [Paper]
  • Year 2000
    • SIGGRAPH

<🎯Back to Top>

Survey Papers

  • [IJCV 2024] Deep Learning-based Image and Video Inpainting: A Survey [Paper]
  • [Pattern Recognit. 2023] Deep Learning for Image Inpainting: A Survey [Paper]

<🎯Back to Top>

Datasets

  • Paris StreetView: Context Encoders: Feature Learning by Inpainting [Paper] [Code] (Since Google's terms of service has forbidden anyone from creating repositories of streetview data, the dataset is currently not available.)
  • Places: A 10 Million Image Database for Scene Recognition [Paper] [Dataset]
  • ImageNet: A Large-Scale Hierarchical Image Database [Paper] [Dataset]
  • CelebA-HQ: Progressive Growing of GANs for Improved Quality, Stability, and Variation [Paper] [Dataset]
  • FFHQ: A Style-Based Generator Architecture for Generative Adversarial Networks [Paper] [Dataset]

<🎯Back to Top>

Q&A

  • Q: The conference sequence of this paper list?
    • This paper list is organized according to the following sequence:
      • Conferences
        • CVPR
        • CVPRW
        • ICCV
        • ICCVW
        • ECCV
        • AAAI
        • IJCAI
        • WACV
        • NeurIPS
        • ICLR
        • ACM MM
        • SIGGRAPH
        • IPTA
      • Journals
        • TMM
        • TIP
        • TPAMI
        • TCSVT
      • arXiv

<🎯Back to Top>

References

The reference.bib file summarizes bibtex references of up-to-date image inpainting papers, widely used datasets, and toolkits. Based on the original references, I have made the following modifications to make their results look nice in the LaTeX manuscripts:

  • Refereces are normally constructed in the form of author-etal-year-nickname. Particularly, references of datasets and toolkits are directly constructed as nickname, e.g., imagenet.
  • In each reference, all names of conferences/journals are converted into abbreviations, e.g., Computer Vision and Pattern Recognition -> CVPR.
  • The url, doi, publisher, organization, editor, series in all references are removed.
  • The pages of all references are added if they are missing.
  • All paper names are in title case. Besides, I have added an additional {} to make sure that the title case would also work well in some particular templates.

If you have other demands of reference formats, you may refer to the original references of papers by searching their names in DBLP or Google Scholar.

<🎯Back to Top>

Star History

Star History Chart

<🎯Back to Top>

About

A collection of awesome image inpainting studies.

Topics

generative-adversarial-networks image-inpainting paper-list diffusion-models interactive-image-inpainting

Resources

Readme

License

MIT license
Activity

Stars

179 stars

Watchers

5 watching

Forks

13 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages