current position:Home>Python, image restoration in opencv - CV2 inpaint

Python, image restoration in opencv - CV2 inpaint

2022-01-30 12:38:13 Program yuan one_ Shining Xiaomei

Python,OpenCV Image restoration in ——cv2.inpaint

image inpainting The image changes

This blog will introduce how to pass OpenCV Image restoration technology in ——cv2.inpaint() Remove small noise from old photos 、 Strokes, etc . And provide an interactive program , utilize OpenCV Fast moving and fluid mechanics repair algorithms to repair their own pictures .

Most families have some old aging photos , There are some black spots on it 、 Strokes, etc . How to recover ?


Erase... In the paint tool : Will simply replace the black structure with a useless white structure , The effect is not ideal .OpenCV Image restoration technology in —— The basic idea is simple : Replace these bad marks with adjacent pixels , Make it look like a neighbor .

  • cv2.INPAINT_TELEA(Fast Marching Method Fast marching algorithm )

  • cv2.INPAINT_NS(Fluid Dynamics Method Fluid mechanics algorithm )

  • OpenCV Unrealized :Content-Aware Fill Content aware filling algorithm , This is a Adobe Photoshop An advanced repair technology used in .

cv2.inpaint() advantage : The repair effect is more natural ; shortcoming : When repairing, the original drawing and mask chart ( Consistent with the original image, only the contaminated pixel area has a value );

1. design sketch

Official original VS mask chart VS Fast marching algorithm repair effect VS Hydrodynamic repair effect as follows :  Insert picture description here Next, use an interactive example to realize your own image repair ;

The original picture VS Mask chart VS The repair effect diagram of fast travel algorithm is as follows : Move the drawing point freely with the left mouse button in the original drawing 、 Line , Right click to move the drawing rectangle to randomly add some contaminated areas ; And generate... According to the original diagram mask chart ,mask The graph is the same size as the original graph , And only the contaminated area is a picture of white pixels . You can see that the repair effect is still very good ~  Insert picture description here The original picture VS Mask chart VS The repair effect diagram of fluid mechanics algorithm is as follows : Move the drawing point freely with the left mouse button in the original drawing 、 Line , Right click to move the drawing rectangle to randomly add some contaminated areas ; mask The graph is the same size as the original graph , And only the contaminated area is a picture of white pixels . You can see that the repair effect is still very good ~  Insert picture description here There is not much difference between the effect picture repaired by fast marching algorithm and hydrodynamic algorithm ;

2. principle

  • cv2.INPAINT_TELEA (Fast Marching Method Fast marching algorithm ), Yes, near the point 、 Pixels near the boundary normal and on the boundary contour are given more weight . Once a pixel is repaired , It will move to the next nearest pixel using the fast marching method .

  • cv2.INPAINT_NS(Fluid Dynamics Method Fluid mechanics algorithm ), Some methods of hydrodynamics are used , The basic principle is heuristic . First, move from the known region to the unknown region along the edge ( Because the edges are continuous ). It matches the gradient vector at the boundary of the repair area , Continue the contour line ( A line connecting points of the same strength , Just as contour lines connect points with the same elevation ).

  • OpenCV Unrealized :Content-Aware Fill Content aware filling algorithm , This is a Adobe Photoshop An advanced repair technology used in .

3. Source code

#  Interactive case of image restoration —— The damaged image area is repaired by water flow filling algorithm ;
#  Use two methods to repair 
# cv2.INPAINT_TELEA (Fast Marching Method  Fast marching algorithm ), Yes, near the point 、 Pixels near the boundary normal and on the boundary contour are given more weight . Once a pixel is repaired , It will move to the next nearest pixel using the fast marching method .
# cv2.INPAINT_NS  Fluid mechanics algorithm , Some methods of hydrodynamics are used , The basic principle is heuristic , First, move from the known region to the unknown region along the edge ( Because the edges are continuous ). It matches the gradient vector at the boundary of the repair area , Continue the contour line ( A line connecting points of the same strength , Just as contour lines connect points with the same elevation ).

# USAGE 
# python inpaint.py D:/deepLearning/py-demo/20210808/images/ml.jpg

#  Press the left mouse button , Add some 、 Line , Press the right mouse button , Add a rectangle , To make contaminated images that need to be repaired 
#  Press the space bar : Perform the repair function 
#  Press down r key : Reset the to be repaired mask
#  Press down esc key , sign out 
import cv2
import numpy as np


class Sketcher:
    def __init__(self, windowname, dests, colors_func):
        self.prev_pt = None  #  The starting point of the line 
        self.drag_start = None  #  Start of rectangle 
        self.drag_rect = None  #  rectangular ( top left corner , The lower right corner ) coordinate 
        self.windowname = windowname
        self.dests = dests
        self.colors_func = colors_func
        self.dirty = False
        self.drawing = False
        self.mode = False
        self.show()
        cv2.setMouseCallback(self.windowname, self.on_mouse)

    def show(self):
        cv2.imshow(self.windowname, self.dests[0])

    def on_mouse(self, event, x, y, flags, param):
        pt = (x, y)
        if event == cv2.EVENT_LBUTTONDOWN:
            self.prev_pt = pt
            self.drawing = True
        elif event == cv2.EVENT_RBUTTONDOWN:
            #  Set... At the first initialization pt, Keep the previous point back as the starting point of the rectangle 
            if self.drag_start == None:
                self.drag_start = pt

        if self.prev_pt and flags & cv2.EVENT_FLAG_LBUTTON:
            for dst, color in zip(self.dests, self.colors_func()):
                cv2.line(dst, self.prev_pt, pt, color, 5)
            self.dirty = True
            self.prev_pt = pt
            self.show()

        if self.drag_start and flags & cv2.EVENT_FLAG_RBUTTON:
            xo, yo = self.drag_start
            x0, y0 = np.minimum([xo, yo], [x, y])
            x1, y1 = np.maximum([xo, yo], [x, y])
            self.drag_rect = None
            if x1 - x0 > 0 and y1 - y0 > 0:
                self.drag_rect = (x0, y0, x1, y1)
                for dst, color in zip(self.dests, self.colors_func()):
                    cv2.rectangle(dst, (x0, y0), (x1, y1), color, -1)
                self.dirty = True
                self.drag_start = None
                self.drag_rect = None
                self.show()
            else:
                self.drag_start = pt

 @property
    def dragging(self):
        return self.drag_rect is not None


def main():
    import sys
    try:
        fn = sys.argv[1]
    except:
        fn = 'images/ml_.jpg'

    img = cv2.imread(fn)
    if img is None:
        print('Failed to load image file:', fn)
        sys.exit(1)

    img_mark = img.copy()
    mark = np.zeros(img.shape[:2], np.uint8)
    sketch = Sketcher('img', [img_mark, mark], lambda: ((255, 255, 255), 255))

    while True:
        ch = cv2.waitKey()
        if ch == 27:
            break
        if ch == ord(' '):
            cv2.imshow('mask', mark)
            fmmres = cv2.inpaint(img_mark, mark, 3, cv2.INPAINT_TELEA)
            nsres = cv2.inpaint(img_mark, mark, 3, cv2.INPAINT_NS)
            cv2.imshow('inpaint fmm res', fmmres)
            cv2.imshow('inpaint ns res', nsres)
        if ch == ord('r'):
            img_mark[:] = img
            mark[:] = 0
            sketch.show()

    print('Done')


if __name__ == '__main__':
    main()
    cv2.destroyAllWindows()
 Copy code 

Reference resources

copyright notice
author[Program yuan one_ Shining Xiaomei],Please bring the original link to reprint, thank you.
https://en.pythonmana.com/2022/01/202201301238116936.html

Random recommended