current position:Home>Using OpenCV and python to identify credit card numbers
Using OpenCV and python to identify credit card numbers
2022-01-31 12:37:06 【AI Hao】
This is my participation 11 The fourth of the yuegengwen challenge 3 God .
Use OpenCV and Python Identify credit card number
In the previous post , We learned how to install Tesseract Binary file and use it for OCR. Then we learned how to use basic image processing techniques to clean up images to improve Tesseract OCR Output .
however , Should not Tesseract It is regarded as a general-purpose device that can obtain high-precision optical character recognition 、 Off the shelf solutions . In some cases , It will work well —— And in other cases , It will fail miserably . A good example of this use case is credit card recognition , Given input image , We hope :
Localize four groups of four digits , It's related to the sixteen digits on the credit card . application OCR Identify the sixteen digits on the credit card . Identify the type of credit card ( namely Visa、MasterCard、American Express etc. ).
In these cases ,Tesseract Library does not recognize numbers correctly ( This may be because Tesseract No training in credit card sample Fonts ). therefore , We need to OCR Credit card design our own custom solution . In today's blog post , I'll show you how to use template matching as OCR To help us create a solution to automatically identify credit cards and extract relevant credit card numbers from images .
Today's blog post is divided into three parts . In the first part , We will discuss OCR-A typeface , This is a font specially created to assist optical character recognition algorithm . Then we will design a computer vision and image processing algorithm , It can :
- Localize the four groups of four digits on the credit card .
- Extract each of the four groups , Then separate 16 Each of the numbers .
- Use templates to match and OCR-A Font recognition 16 Each of the credit card numbers .
Last , We'll look at some credit cards OCR Examples of the algorithm applied to real images .
Through and with OpenCV Template matching OCR
In this section , We will use Python + OpenCV Implement our template matching algorithm to automatically identify credit card numbers .
To achieve this , We need to apply many image processing operations , Including thresholds 、 Calculate the gradient amplitude representation 、 Morphological operation and contour extraction . These techniques have been used in other blog posts to detect barcodes in images and identify machine-readable areas in passport images .
Since many image processing operations will be applied to help us detect and extract credit card numbers , So I included a lot of intermediate screenshots when the input image passed through our image processing pipeline .
These additional screenshots will give you a deeper understanding of how we can link basic image processing technologies together to build solutions for computer vision projects . Let's get started .
Open a new file , Name it ocr_template_match.py , We'll start working :
# import the necessary packages
from imutils import contours
import numpy as np
import argparse
import imutils
import cv2
Copy code
To install / upgrade imutils , Just use pip :
pip install --upgrade imutils
Copy code
Be careful : If you use Python A virtual environment ( Like all my OpenCV Like the installation tutorial ), Make sure to use... First workon Command to access your virtual environment , Then install / upgrade imutils .
Now we have installed and imported the package , We can parse our command line arguments :
# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-i", "--image", required=True,
help="path to input image")
ap.add_argument("-r", "--reference", required=True,
help="path to reference OCR-A image")
args = vars(ap.parse_args())
Copy code
A parameter parser is established , Add two parameters , Then parse them , Store them as variables args . The two required command line arguments are :
--image : To carry out OCR The path of the processed image .
--reference : Reference resources OCR-A Image path . The image contains OCR-A The number in the font 0-9, This allows us to perform template matching later in the pipeline .
Next, let's define the type of credit card :
# define a dictionary that maps the first digit of a credit card
# number to the credit card type
FIRST_NUMBER = {
"3": "American Express",
"4": "Visa",
"5": "MasterCard",
"6": "Discover Card"
}
Copy code
Credit card type , For example, American Express 、Visa etc. , It can pass the inspection 16 The first digit in the credit card number to identify . We defined a dictionary FIRST_NUMBER , It maps the first number to the corresponding credit card type . Let's load the reference OCR-A Image to start our image processing pipeline :
# load the reference OCR-A image from disk, convert it to grayscale,
# and threshold it, such that the digits appear as *white* on a
# *black* background
# and invert it, such that the digits appear as *white* on a *black*
ref = cv2.imread(args["reference"])
ref = cv2.cvtColor(ref, cv2.COLOR_BGR2GRAY)
ref = cv2.threshold(ref, 10, 255, cv2.THRESH_BINARY_INV)[1]
Copy code
First , We load reference OCR-A Images , It is then converted to grayscale and threshold + reverse . In each of these operations , We store or overwrite ref , Our reference image .
Now let's be in OCR-A Locate the outline on the font image :
# find contours in the OCR-A image (i.e,. the outlines of the digits)
# sort them from left to right, and initialize a dictionary to map
# digit name to the ROI
refCnts = cv2.findContours(ref.copy(), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
refCnts = imutils.grab_contours(refCnts)
refCnts = contours.sort_contours(refCnts, method="left-to-right")[0]
digits = {}
Copy code
Found the outline in the reference image . then , because OpenCV 2.4、3 and 4 How versions store the returned contour information differently , We check the version and correct refCnts Make appropriate changes . Next , We sort the contours from left to right , And initialize a dictionary ,digits, It maps numeric names to regions of interest .
here , We should traverse the contour , extract ROI And associate it with its corresponding number :
# loop over the OCR-A reference contours
for (i, c) in enumerate(refCnts):
# compute the bounding box for the digit, extract it, and resize
# it to a fixed size
(x, y, w, h) = cv2.boundingRect(c)
roi = ref[y:y + h, x:x + w]
roi = cv2.resize(roi, (57, 88))
# update the digits dictionary, mapping the digit name to the ROI
digits[i] = roi
Copy code
Traverse the reference image contour .
In circulation , i Save numeric name / Number , c Save outline . We surround each contour c Calculate a bounding box , For storing rectangles (x, y) Coordinates and width / Height . Use the bounding rectangle parameter from ref( Reference image ) Extract from roi. The ROI Contains figures .
We will ROI Size to 57×88 Fixed size of pixels . We need to make sure that each number is adjusted to a fixed size , To apply template matching in number recognition later in this tutorial .
We will each number 0-9( Dictionary key ) With each roi Images ( The dictionary is worth ) Related to .
At this point , We have completed the work of extracting numbers from the reference image and associating them with the corresponding number names .
Our next goal is to isolate the input --image Medium 16 A credit card number . We need to find and isolate the numbers first , Then you can start template matching to identify each number . These image processing steps are very interesting and insightful , Especially if you've never developed an image processing pipeline before , Please pay close attention to .
Let's continue to initialize a few structured kernels :
# initialize a rectangular (wider than it is tall) and square
# structuring kernel
rectKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (9, 3))
sqKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5, 5))
Copy code
You can think of the kernel as a small matrix that we slide over the image , To carry out ( Convolution ) operation , For example, blur 、 sharpening 、 Edge detection or other image processing operations .
Two such cores are constructed —— A rectangle and a square . We will use a rectangle as Top-hat Morphological operator , Use square as closed operation . We'll see this soon . Now let's get ready for OCR Image :
# load the input image, resize it, and convert it to grayscale
image = cv2.imread(args["image"])
image = imutils.resize(image, width=300)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
Copy code
Loaded the command line parameter image containing the credit card photo . then , We adjust it to width=300 , Keep the aspect ratio , Then convert it to grayscale . Let's look at our input image :
Next is our resizing and grayscale operation :
Now our image has been grayed and the size is the same , Let's do morphological operations :
# apply a tophat (whitehat) morphological operator to find light
# regions against a dark background (i.e., the credit card numbers)
tophat = cv2.morphologyEx(gray, cv2.MORPH_TOPHAT, rectKernel)
Copy code
Use our rectKernel And our grayscale images , We execute Top-hat Morphological operation , Store the results as tophat.
Top-hat Operate on a dark background ( Credit card number ) The light colored area is displayed below , As shown in the figure below :
Given our high hat image , Let's calculate along x Directional gradient :
# compute the Scharr gradient of the tophat image, then scale
# the rest back into the range [0, 255]
gradX = cv2.Sobel(tophat, ddepth=cv2.CV_32F, dx=1, dy=0,
ksize=-1)
gradX = np.absolute(gradX)
(minVal, maxVal) = (np.min(gradX), np.max(gradX))
gradX = (255 * ((gradX - minVal) / (maxVal - minVal)))
gradX = gradX.astype("uint8")
Copy code
The next step in our effort to isolate the numbers is to calculate x High hat image in the direction Scharr gradient . Complete the calculation , Store the results as gradX .
In the calculation gradX After the absolute value of each element in the array , Let's take some steps to reduce the value to [0-255] Within the scope of ( Because the image is currently a floating point data type ). So , We calculated gradX Of minVal and maxVal, Then calculate the second 73 The scaling equation shown in line ( That is the smallest / Maximum normalization ). The last step is going to be gradX Convert to a range of [0-255] Of uint8. The results are shown in the following figure :
Let's continue to improve the credit card number search algorithm :
# apply a closing operation using the rectangular kernel to help
# cloes gaps in between credit card number digits, then apply
# Otsu's thresholding method to binarize the image
gradX = cv2.morphologyEx(gradX, cv2.MORPH_CLOSE, rectKernel)
thresh = cv2.threshold(gradX, 0, 255,
cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1]
# apply a second closing operation to the binary image, again
# to help close gaps between credit card number regions
thresh = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, sqKernel)
Copy code
To narrow the gap , We performed a close operation . Please note that , We used it again rectKernel. And then we went to gradX Image execution Otsu And binary threshold , Then there is another close operation . The results of these steps are as follows :
Next, let's find the outline and initialize the digital grouping location list .
# find contours in the thresholded image, then initialize the
# list of digit locations
cnts = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
cnts = imutils.grab_contours(cnts)
locs = []
Copy code
We found the contours and stored them in a list cnts in . then , We initialize a list to hold the number group location .
Now let's traverse the contour , At the same time, filter according to the aspect ratio of each contour , Allows us to trim the number group position from other unrelated areas of the credit card :
# loop over the contours
for (i, c) in enumerate(cnts):
# compute the bounding box of the contour, then use the
# bounding box coordinates to derive the aspect ratio
(x, y, w, h) = cv2.boundingRect(c)
ar = w / float(h)
# since credit cards used a fixed size fonts with 4 groups
# of 4 digits, we can prune potential contours based on the
# aspect ratio
if ar > 2.5 and ar < 4.0:
# contours can further be pruned on minimum/maximum width
# and height
if (w > 40 and w < 55) and (h > 10 and h < 20):
# append the bounding box region of the digits group
# to our locations list
locs.append((x, y, w, h))
Copy code
We loop through the contour in the same way as the reference image . After calculating the boundary rectangle of each contour c after , We calculate the aspect ratio by dividing the width by the height ar . Use aspect ratio , We analyze the shape of each contour . If ar Be situated between 2.5 and 4.0 Between ( Width greater than height ), as well as 40 To 55 Between pixels w and 10 To 20 Between pixels h, We attach the boundary rectangle parameter in a convenient tuple to locs.
The following figure shows the groups we found —— For demonstration purposes , I let OpenCV A bounding box is drawn around each group :
Next , We will sort the groups from left to right and initialize the list of credit card numbers :
# sort the digit locations from left-to-right, then initialize the
# list of classified digits
locs = sorted(locs, key=lambda x:x[0])
output = []
Copy code
We according to the x It's worth it locs Sort , So they will be sorted from left to right . We initialize a list output , It will save the credit card number of the image . Now we know the position of each group of four digits , Let's loop through the four sorted groups and determine the numbers .
This cycle is quite long , It is divided into three code blocks —— This is the first block :
# loop over the 4 groupings of 4 digits
for (i, (gX, gY, gW, gH)) in enumerate(locs):
# initialize the list of group digits
groupOutput = []
# extract the group ROI of 4 digits from the grayscale image,
# then apply thresholding to segment the digits from the
# background of the credit card
group = gray[gY - 5:gY + gH + 5, gX - 5:gX + gW + 5]
group = cv2.threshold(group, 0, 255,
cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1]
# detect the contours of each individual digit in the group,
# then sort the digit contours from left to right
digitCnts = cv2.findContours(group.copy(), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
digitCnts = imutils.grab_contours(digitCnts)
digitCnts = contours.sort_contours(digitCnts,
method="left-to-right")[0]
Copy code
In the first block of this loop , We extracted and filled groups on each side 5 Pixel , Apply threshold processing , And find and sort profiles . Please refer to the code for details . The extracted individual groups are shown below :
Let's continue the loop with nested loop for template matching and similarity score extraction :
# loop over the digit contours
for c in digitCnts:
# compute the bounding box of the individual digit, extract
# the digit, and resize it to have the same fixed size as
# the reference OCR-A images
(x, y, w, h) = cv2.boundingRect(c)
roi = group[y:y + h, x:x + w]
roi = cv2.resize(roi, (57, 88))
# initialize a list of template matching scores
scores = []
# loop over the reference digit name and digit ROI
for (digit, digitROI) in digits.items():
# apply correlation-based template matching, take the
# score, and update the scores list
result = cv2.matchTemplate(roi, digitROI,
cv2.TM_CCOEFF)
(_, score, _, _) = cv2.minMaxLoc(result)
scores.append(score)
# the classification for the digit ROI will be the reference
# digit name with the *largest* template matching score
groupOutput.append(str(np.argmax(scores)))
Copy code
Use cv2.boundingRect We get the extract that contains each number ROI Required parameters . In order to make template matching work with some degree of accuracy , We will roi The size is adjusted with us in the 144 Reference on line OCR-A Font digital image (57×88 Pixels ) The same size .
We initialized a score list . Consider it our confidence score —— It is higher , The more likely it is to be the right template .
Now? , Let's cycle through each reference number ( The third nested loop ) And perform template matching . This is where the heavy work is done for this script .
OpenCV There is one named cv2.matchTemplate Convenient function of , You can provide two images : One is the template , The other is the input image . take cv2.matchTemplate The purpose of applying to these two images is to determine their similarity .
under these circumstances , We provide reference digitROI Image and credit card with candidate numbers roi. Use these two images , We call the template matching function and store the results . Next , We extract scores from the results and append them to our score list . This completes the innermost cycle .
Use score ( Every number 0-9 One ), We take the maximum score —— The maximum score should be the number we correctly identify . We found the number with the highest score , adopt np.argmax Get a specific index . The integer name of the index represents the most likely number based on the comparison with each template ( Remember again , The index has been pre sorted as 0-9).
Last , Let's draw a rectangle around each group , And view the credit card number on the image in red text :
# draw the digit classifications around the group
cv2.rectangle(image, (gX - 5, gY - 5),
(gX + gW + 5, gY + gH + 5), (0, 0, 255), 2)
cv2.putText(image, "".join(groupOutput), (gX, gY - 15),
cv2.FONT_HERSHEY_SIMPLEX, 0.65, (0, 0, 255), 2)
# update the output digits list
output.extend(groupOutput)
Copy code
For the third and last block of this loop , Let's draw a... Around the group 5 Pixel fill rectangle , Then draw text on the screen .
The last step is to append the number to the output list . Pythonic The method is to use the extension function to set the iteratable object ( In this case, list ) Each element of is appended to the end of the list .
To view the execution of the script , Let's output the result to the terminal and display our image on the screen .
# display the output credit card information to the screen
print("Credit Card Type: {}".format(FIRST_NUMBER[output[0]]))
print("Credit Card #: {}".format("".join(output)))
cv2.imshow("Image", image)
cv2.waitKey(0)
Copy code
Print the credit card type to the console , Then on the following day 173 The bank prints the credit card number .
In the last few lines , We display the image on the screen and wait for any key to be pressed , Then exit the script 174 and 175 That's ok .
Take time to congratulate yourself —— You did . Take a look back. ( At a high level ), This script :
- Store credit card types in a dictionary .
- Obtain the reference image and extract the digital image .
- Store the number template in the dictionary .
- Localize four credit card number groups , Each group contains four digits ( in total 16 Digit number ).
- Extract to “ matching ” The number of .
- Perform template matching for each number , Put each individual ROI With each digital template 0-9 Compare , Store the score of each attempt to match at the same time .
- Find the highest score for each candidate number , And build a project called output A list of , It contains the credit card number .
- Output the credit card number and credit card type to our terminal , And display the output image on our screen .
Now it's time to look at the running script and check our results .
The credit card OCR result
Now we have credit cards OCR The system is coded , Let's try .
We obviously can't use a real credit card number in this example , So I used Google to collect some sample images of credit cards .
These credit cards are obviously fake , For demonstration purposes only . however , You can apply the same techniques in this blog post to identify the numbers on the actual credit card .
To check our credit cards OCR The operation of the system , Please open a terminal and execute the following command :
$ python ocr_template_match.py --reference ocr_a_reference.png \
--image images/credit_card_05.png
Credit Card Type: MasterCard
Credit Card #: 5476767898765432
Copy code
Our first result image ,100% correct :
Please note how we can correctly mark credit cards as MasterCard , Just check the first digit in the credit card number . Let's try the second picture , This is a visa :
$ python ocr_template_match.py --reference ocr_a_reference.png \
--image images/credit_card_01.png
Credit Card Type: Visa
Credit Card #: 4000123456789010
Copy code
Once again, , We can correctly match credit cards using template matching OCR.
$ python ocr_template_match.py --reference ocr_a_reference.png \
--image images/credit_card_02.png
Credit Card Type: Visa
Credit Card #: 4020340002345678
Copy code
summary
In this tutorial , We learned how to pass OpenCV and Python Perform optical character recognition using template matching (OCR).
say concretely , We apply our template matching OCR Method to identify the type of credit card and 16 A credit card number .
To achieve this , We divide the image processing pipeline into 4 A step :
- Detect four groups of four numbers on the credit card through various image processing technologies , Including morphological operations 、 Threshold and contour extraction .
- Extract each individual number from the four groups , Lead to the need for classification 16 A digital .
- By associating each number with OCR-A Compare Fonts , Apply template matching to each number to get our number classification .
- Check the first digit of the credit card number to determine the issuing company .
After evaluating our credit cards OCR After the system , We found it was 100% accurate , Provided that the issuing credit card company uses OCR-A Font as number .
To extend this application , You may want to collect real images of credit cards in the field , And may train machine learning models ( Through standard feature extraction or training or convolution neural network ) To further improve the accuracy of the system .
I hope you like this article about OCR 's blog post .
copyright notice
author[AI Hao],Please bring the original link to reprint, thank you.
https://en.pythonmana.com/2022/01/202201311237045770.html
The sidebar is recommended
- [algorithm learning] 1108 IP address invalidation (Java / C / C + + / Python / go / trust)
- Test platform series (71) Python timed task scheme
- Java AES / ECB / pkcs5padding encryption conversion Python 3
- Loguru: the ultimate Python log solution
- Blurring and anonymizing faces using OpenCV and python
- How fast Python sync and async execute
- Python interface automation test framework (basic) -- common data types list & set ()
- Python crawler actual combat, requests module, python realizes capturing video barrage comments of station B
- Python: several implementation methods of multi process
- Sword finger offer II 054 Sum of all values greater than or equal to nodes | 538 | 1038 (Java / C / C + + / Python / go / trust)
guess what you like
-
How IOS developers learn python programming 3-operator 2
-
How IOS developers learn python programming 2-operator 1
-
[Python applet] 8 lines of code to realize file de duplication
-
Python uses the pynvml tool to obtain the working status of GPU
-
Data mining: Python actual combat multi factor analysis
-
Manually compile opencv on MacOS and Linux and add it to Python / C + + / Java as a dependency
-
Use Python VTK to batch read 2D slices and display 3D models
-
Complete image cutting using Python version VTK
-
Python interface automation test framework (basic) -- common data types Dict
-
Django (make an epidemic data report)
Random recommended
- Python specific text extraction in actual combat challenges the first step of efficient office
- Daily python, Part 8 - if statement
- Django model class 1
- The same Python code draws many different cherry trees. Which one do you like?
- Python code reading (Chapter 54): Fibonacci sequence
- Django model class 2
- Python crawler Basics
- Mapping 3D model surface distances using Python VTK
- How to implement encrypted message signature and verification in Python -- HMAC
- leetcode 1945. Sum of Digits of String After Convert(python)
- leetcode 2062. Count Vowel Substrings of a String(python)
- Analysis of Matplotlib module of Python visualization
- Django permission management
- Python integrated programming -- visual hot search list and new epidemic situation map
- [Python data collection] scripy realizes picture download
- Python interface automation test framework (basic part) -- loop statement of process control for & while
- Daily python, Chapter 9, while loop
- Van * Python | save the crawled data with docx and PDF
- Five life saving Python tips
- Django frequency control
- Python - convert Matplotlib image to numpy Array or PIL Image
- Python and Java crawl personal blog information and export it to excel
- Using class decorators in Python
- Untested Python code is not far from crashing
- Python efficient derivation (8)
- Python requests Library
- leetcode 2047. Number of Valid Words in a Sentence(python)
- leetcode 2027. Minimum Moves to Convert String(python)
- How IOS developers learn Python Programming 5 - data types 2
- leetcode 1971. Find if Path Exists in Graph(python)
- leetcode 1984. Minimum Difference Between Highest and Lowest of K Scores(python)
- Python interface automation test framework (basic) -- basic syntax
- Detailed explanation of Python derivation
- Python reptile lesson 2-9 Chinese monster database. It is found that there is a classification of color (he) desire (Xie) monsters during operation
- A brief note on the method of creating Python virtual environment in Intranet Environment
- [worth collecting] for Python beginners, sort out the common errors of beginners + Python Mini applet! (code attached)
- [Python souvenir book] two people in one room have three meals and four seasons: 'how many years is it only XX years away from a hundred years of good marriage' ~?? Just come in and have a look.
- The unknown side of Python functions
- Python based interface automation test project, complete actual project, with source code sharing
- A python artifact handles automatic chart color matching