Fourier transform of Python OpenCV image processing, lesson 52

Little knowledge , Great challenge ！ This article is participating in 「 A programmer must have a little knowledge 」 Creative activities

This article has participated in  「 Digging force Star Program 」 , Win a creative gift bag , Challenge creation incentive fund .

Python OpenCV 365 Day study plan , Go into the field of image with the eraser . This blog is the third in this series 52 piece .

Learning ahead

The Fourier transform （Fourier Transform,FT） For the first time today , Follow the old routine , We still apply it first , Then step by step iterative learning .

Fourier transform is right The same thing The change of viewing angle , No longer viewing from the time domain , View from the frequency domain . There are two new words mentioned here , Time domain and frequency domain , Let's briefly understand , Time domain , Changes in things over time , Frequency domain refers to the change frequency of things , Is it very winding , No problem , First use , First tune API.

Fourier principle omits , Let's talk about the impact on the image after applying it .

• After using the high pass filter , Will retain high-frequency information , Enhance image detail , For example, boundary enhancement ;
• After using low-pass filter , Low frequency information will be retained , The border is blurred .

Fourier transform application

Now that the principle has been put down first , Then apply it first , We will study separately numpy and OpenCV Fourier transform is realized in two ways .

Numpy Fourier transform

adopt numpy The function used to realize the Fourier transform is np.fft.fft2 , The function prototype is as follows ：

fft2(a, s=None, axes=(-2, -1), norm=None)
 Copy code 

The parameters are described as follows ：

• a： The input image ;
• s： Sequence of integers , Size of output array ;
• axex： Sequence of integers , Used to calculate FFT The optional shaft of ;
• norm： Normalization model .

Some parameters, if not actually used , It's hard to see the results , Of course, the description of the function , still The official manual Most reliable . The coding process at the application level is as follows ： adopt np.fft.fft2 The frequency distribution is obtained by Fourier transform of function , Call again np.fft.fftshift Function to transfer the center to the middle .

The test code is as follows ：

import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt

img = cv.imread('test.jpg', 0)

#  Fast Fourier transform algorithm to get the frequency distribution , Transform the spatial domain into the frequency domain 
f = np.fft.fft2(img)

#  The default result center point position is in the upper left corner , Transfer the center point to the middle position by the following code 
#  Move the low frequency part to the center of the image 
fshift = np.fft.fftshift(f)

# fft  The result is a complex number ,  The absolute result is the amplitude 
result = 20*np.log(np.abs(fshift))

plt.subplot(121)
plt.imshow(img, cmap='gray')
plt.title('original')
plt.axis('off')

plt.subplot(122)
plt.imshow(result, cmap='gray')
plt.title('result')
plt.axis('off')

plt.show()
 Copy code 

This is a case repeatedly mentioned in many places , In addition to the relevant contents . np.fft.fft2 Is a frequency conversion function , Its first parameter is the input image , Gray scale image . The second parameter is optional , It determines the size of the output array . If the size of the second parameter is larger than the size of the input image , It's calculating FFT Before zero fill input image ; If it's smaller than the input image , Input the cut into the image . If no parameters are passed , The size of the output array will be the same as the size of the input , Test the following ：

import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt

#  Create a picture 
src = np.ones((5, 5), dtype=np.uint8)*100

print(src)
print(src.shape)

f = np.fft.fft2(src,(7,7))
print(f.shape)
print(f)
 Copy code 

After the center point moves , You can refer to the running results in the figure below , And we can know np.fft.fft2 After the function is applied, the complex matrix is obtained .

Operations to be carried out later , Have some knowledge of Mathematics , I don't understand yet , Below are ： After frequency conversion , You can build the amplitude spectrum , Finally, the range is compressed by logarithmic transformation . Or it can be understood that the complex number is converted into floating-point number for Fourier spectrum display , The supplementary code is as follows ：

fimg = np.log(np.abs(fshift))
 Copy code 

The final result is as follows , Of course, this is a random graph , If you change to a gray image , It can be verified as follows .

The modified code is as follows ：

import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt

#  Create a picture 
# src = np.ones((5, 5), dtype=np.uint8)*100
src = cv.imread("./test.jpg", 0)
print("*"*100)
print(src)
print(src.shape)

f = np.fft.fft2(src)
print("*"*100)
print(f)

fshift = np.fft.fftshift(f)
print("*"*100)
print(fshift)
#  Convert complex numbers into floating-point numbers for Fourier spectrum display 
fimg = 20*np.log(np.abs(fshift))
print(fimg)

#  Image display 
plt.subplot(121), plt.imshow(src, "gray"), plt.title('origin')
plt.axis('off')
plt.subplot(122), plt.imshow(fimg, "gray"), plt.title('Fourier')
plt.axis('off')
plt.show()
 Copy code 

Basic conclusion ： On the left is the original grayscale image , On the right is the frequency distribution map , The closer it is to the center The lower the frequency , The higher the gray value, the brighter the brightness, and the greater the amplitude of the signal representing this frequency . Next, inverse transform it , That is, reverse operation back , adopt numpy Realize inverse Fourier transform , It's the inverse of Fourier transform , take Spectrum image Convert to original image . The core functions and prototypes used are as follows . np.fft.ifft2

#  Realize the inverse Fourier transform of image , Returns a complex array 
np.fft.ifft2(a, s=None, axes=(-2, -1), norm=None)
 Copy code 

np.fft.ifftshift

# fftshit() The inverse of a function , It moves the center low frequency part of the spectrum image to the upper left corner 
np.fft.ifftshift(x, axes=None)
 Copy code 

According to the above , Reverse the Fourier transform , The test code is as follows ：

import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt

#  Create a picture 
# src = np.ones((5, 5), dtype=np.uint8)*100
src = cv.imread("./test.jpg", 0)
print("*"*100)
print(src)
print(src.shape)

f = np.fft.fft2(src)
print("*"*100)
print(f)

fshift = np.fft.fftshift(f)
print("*"*100)
print(fshift)

#  Convert complex numbers into floating-point numbers for Fourier spectrum display 
fimg = np.log(np.abs(fshift))
print(fimg)

#  Inverse Fourier transform 
ifshift = np.fft.ifftshift(fshift)
#  Convert complex numbers into floating-point numbers for Fourier spectrum display 
ifimg = np.log(np.abs(ifshift))
if_img = np.fft.ifft2(ifshift)

origin_img = np.abs(if_img)

#  Image display 
plt.subplot(221), plt.imshow(src, "gray"), plt.title('origin')
plt.axis('off')
plt.subplot(222), plt.imshow(fimg, "gray"), plt.title('fourier_img')
plt.axis('off')
plt.subplot(223), plt.imshow(origin_img, "gray"), plt.title('origin_img')
plt.axis('off')
plt.subplot(224), plt.imshow(ifimg, "gray"), plt.title('ifimg')
plt.axis('off')
plt.show()
 Copy code 

The final result is as follows ：

If in the spectrum image after the above Fourier transform , Add a low-pass filter , We will get the following results .

import cv2 as cv
import numpy as np
from matplotlib import pyplot as plt

#  Realize low-pass filtering of Fourier transform 
src = cv.imread("./test.jpg", 0)
f = np.fft.fft2(src)
fshift = np.fft.fftshift(f)

rows, cols = src.shape
crow, ccol = rows//2, cols//2

#  Make a mask , It's the same size as the image ,np.zeros initialization 
mask = np.zeros((rows, cols), np.uint8)
#  Make the center position , Up and down, left and right 30, Set as 1
mask[crow-30:crow+30, ccol-30:ccol+30] = 1

#  Mask and  DFT  Only the middle region is preserved in the product multiplication 
fshift = fshift*mask

#  Inverse Fourier transform 
ifshift = np.fft.ifftshift(fshift)
#  Convert complex numbers into floating-point numbers for Fourier spectrum display 
ifimg = np.fft.ifft2(ifshift)
dft_img = np.abs(ifimg)


#  Image display 
plt.subplot(121), plt.imshow(src, "gray"), plt.title('origin')
plt.axis('off')
plt.subplot(122), plt.imshow(dft_img, "gray"), plt.title('dft_img')
plt.axis('off')
plt.show()
 Copy code 

If you want to achieve high pass filtering , Just modify the mask data .

mask = np.ones((rows, cols), np.uint8)
mask[crow-30:crow+30, ccol-30:ccol+30] = 0
 Copy code 

I've read a lot of articles , But an article is mentioned repeatedly , Here is also a note ：

zhuanlan.zhihu.com/p/19763358 Of course, the content of Fourier transform in the official manual also needs a long time to read , Official website address

OpenCV Fourier transform

OpenCV Realize Fourier transform and numpy Almost the same , The core difference is the use of functions , The differences are as follows ： Opencv Mainly through cv2.dif and cv2.idif（ Inverse Fourier transform ）, Before inputting an image, you need to convert the image from np.uint8 Convert to np.float32 Format , The frequency of the result is 0 It's going to be in the upper left corner , To switch to the center position , adopt shift Transform to achieve , And numpy Agreement ,cv2.dif The result returned is dual channel （ Real component , Imaginary part ）, It also needs to be converted into image format to show .

function cv2.dft() The prototype is as follows

dst = cv.dft(src[, dst[, flags[, nonzeroRows]]])
 Copy code 

Parameter description ：

• src： The input image , requirement np.float32 Format ;
• dst： Output image , Two channels （ Real component + Imaginary part ）, The size and type depend on the third parameter flags;
• flags： Represents a conversion token , The default is 0, There are multiple values , See later ;
• nonzeroRows： The default is 0, Not for now .

flags The values are as follows ：

• DFT_INVERSE： Replace the default forward transform with one-dimensional or two-dimensional inverse transform ;
• DFT_SCALE： Scale identifier , Calculate the scaling result according to the average number of data elements , if there be N Elements , Then the output result is in 1/N Zoom output , Often with DFT_INVERSE Use it with ;
• DFT_ROWS： Forward or reverse Fourier transform is performed on each row of the input matrix ; This identifier can be used to reduce the cost of resources when dealing with a variety of appropriate , These processes are often complex operations such as three-dimensional or high-dimensional transformation ;
• DFT_COMPLEX_OUTPUT： Forward transform one-dimensional or two-dimensional real arrays , Although this result is a complex array , But it has conjugate symmetry of the complex number （CCS）, It can be filled with an array of real numbers with the same size as the original array , This is the fastest choice and the default method of the function . You may want to get a full-size complex array （ Like simple spectral analysis and so on ）, By setting the flag bit, the function can generate a full-size complex output array ;
• DFT_REAL_OUTPUT： Inverse transformation of one-dimensional and two-dimensional complex arrays , The result is usually a complex matrix with the same size , But if the input matrix has conjugate symmetry of complex numbers （ For example, a with DFT_COMPLEX_OUTPUT Positive transformation result of identifier ）, Then the real matrix will be output .

The above content is excerpted from the Internet , The originator has been unable to find , embarrassed .

In summary, it is ：

• DFT_COMPLEX_OUTPUT： Get a matrix in complex form ;
• DFT_REAL_OUTPUT： Only the real part of the complex number is output ;
• DFT_INVERSE： Inverse Fourier transform ;
• DFT_SCALE： Is it divided by MxN （M That's ok N The pictures in the column , Co ownership MxN Pixels ）;
• DFT_ROWS： Each row of the input matrix is subjected to Fourier transform or inverse transform .

The last thing to note is this , The output spectrum result is a complex number , Need to call cv2.magnitude() Function converts the dual channel result of Fourier transform to 0 To 255 The scope of the .

This function is relatively simple , The prototype and description are as follows .

cv2.magnitude The function prototype ：cv2.magnitude(x, y)

• x Represents a floating point type X Coordinate value , The real part
• y Represents a floating point type Y Coordinate value , The imaginary part

The basics are simple to go through , Directly explain the case .

import cv2 as cv
import numpy as np
import matplotlib.pyplot as plt
plt.rcParams['font.sans-serif'] = ['SimHei']

src = cv.imread("test.jpg", 0)

# OpneCV Fourier transform function 
#  The image needs to be processed once float transformation 
result = cv.dft(np.float32(src), flags=cv.DFT_COMPLEX_OUTPUT)
#  Move the low frequency of the spectrum from the top left to the center 
dft_shift = np.fft.fftshift(result)
#  Spectrum image dual channel complex conversion to  0-255  Section 
result1 = 20 * np.log(cv.magnitude(dft_shift[:, :, 0], dft_shift[:, :, 1]))
#  Image display 
plt.subplot(121), plt.imshow(src, 'gray'), plt.title(' Original image ')
plt.axis('off')
plt.subplot(122), plt.imshow(result1, 'gray'), plt.title(' The Fourier transform ')
plt.axis('off')
plt.show()
 Copy code 

Running results and numpy Agreement .

Apply the above knowledge , Use OpenCV The function inside uses low-pass filtering for pictures .

import cv2 as cv
import numpy as np
import matplotlib.pyplot as plt
plt.rcParams['font.sans-serif'] = ['SimHei']

src = cv.imread("test.jpg", 0)

# OpneCV Fourier transform function 
#  The image needs to be processed once float transformation 
result = cv.dft(np.float32(src), flags=cv.DFT_COMPLEX_OUTPUT)
#  Move the low frequency of the spectrum from the top left to the center 
dft_shift = np.fft.fftshift(result)
#  Spectrum image dual channel complex conversion to  0-255  Section 
result = 20 * np.log(cv.magnitude(dft_shift[:, :, 0], dft_shift[:, :, 1]))

rows, cols = src.shape
crow, ccol = rows//2, cols//2

mask = np.zeros((rows, cols, 2), np.uint8)
mask[int(crow-30):int(crow+30), int(ccol-30):int(ccol+30)] = 1
# LPF（ Low pass filtering ）
fshift = dft_shift*mask
f_ishift = np.fft.ifftshift(fshift)
img_back = cv.idft(f_ishift)
img_back = cv.magnitude(img_back[:, :, 0], img_back[:, :, 1])

#  Image display 
plt.subplot(131), plt.imshow(src, 'gray'), plt.title(' Original image ')
plt.axis('off')
plt.subplot(132), plt.imshow(result, 'gray'), plt.title(' The Fourier transform ')
plt.axis('off')
plt.subplot(133), plt.imshow(img_back, 'gray'), plt.title(' Image after low-pass filtering ')
plt.axis('off')
plt.show()
 Copy code 

High pass filter code and operation effect , Just leave it to yourself .

Eraser bars

HPF（ High pass filtering ）,LPF（ Low pass filtering ） I hope today's 1 Hours （ It doesn't seem to be enough ） You get something , I'll see you on our next blog ~