This is my participation 11 The fourth of the yuegengwen challenge 16 God , Check out the activity details ：2021 One last more challenge

Use dlib、OpenCV and Python Detect facial markers

today , We will use dlib and OpenCV To detect facial markers in the image .dlib Installation tutorial for ：

wanghao.blog.csdn.net/article/det…

Face detector model ：

download.csdn.net/download/hh…

Face markers have been successfully applied to face alignment 、 Head pose estimation 、 Face exchange 、 Blink detection and other fields .

In today's blog post , We will focus on the basics of facial markers , Include ：

1、 What exactly are facial markers and how they work .

2、 How to use dlib、OpenCV and Python Detect and extract facial markers from images .

This blog post is mainly divided into three parts ：

1、 Facial markers and why they are used in computer vision applications will be discussed .

2、 Show me how to use dlib、OpenCV and Python Detect and extract facial markers .

3、 We will look at some results of applying facial marker detection to images .

What are facial markers ？

Face marker detection is a subset of shape prediction problem . Given an input image （ It is usually a that specifies the object of interest ROI）, The shape predictor attempts to locate the key points of interest along the shape .

In the context of face markers , Our goal is to use shape prediction method to detect important face structures on human face .

therefore , Detecting facial markers is a two-step process ：

​ step 1： Locate the face in the image .

​ step 2： Face detection ROI Key facial structures on .

Face detection （ step 1） It can be realized in many ways .

We can use OpenCV Built in Haar cascade .

We can put the pre trained HOG+ linear SVM Object detector is specially used for face detection task .

Or we can even use an algorithm based on deep learning for face location .

In either case , The actual algorithms used to detect faces in images are irrelevant . contrary , It is important to , By some means , We get the face bounding box （ namely , Faces in the image （x,y）- coordinate ）.

Given face area , We can apply steps 2： Detect key facial structures in the facial region .

There are a variety of facial marker detectors , But all methods basically try to locate and mark the following facial areas ：

• mouth

• Right eyebrow

• Left eyebrow

• Right eye

• The left eye

• nose

• jaw

dlib The face marker detector included in the library is Kazemi and Sullivan（2014） The implementation of one millisecond face calibration of a series of regression trees is proposed .

This method first uses ：

• The training set of facial markers marked on the image . These images are manually marked , Specify a specific area around each facial structure （x,y） coordinate .

• transcendental , More specifically , Is the probability of the distance between input pixel pairs .

Given the training data , Training regression tree set , To estimate the face marker position directly from the pixel intensity itself （ namely , Don't make “ feature extraction ”）.

The end result is a facial marker detector , It can be used to detect facial markers with high-quality prediction in real time .

dlib Face marker detector

dlib The face marker detector pre trained in the library is used to estimate the number mapped to the face structure 68（x,y） Position of coordinates .

68 The index of coordinates can be shown in the following figure ：

These notes are 68 spot iBUG 300-W Part of a data set ,dlib The facial marker predictor is trained on this data set .

It is worth noting that , There are other styles of facial marker detectors , Including can be in HELEN Trained on the dataset 194 Point model .

No matter which dataset is used , Can use the same dlib The framework trains the shape predictor on the input training data - If you want to train the face marker detector or custom shape predictor , This is very useful .

In the rest of this blog post , I'll show you how to detect these facial markers in an image .

Use dlib、OpenCV and Python Detect facial markers

This blog post uses imutils library face_utils.py Two functions in .

The first utility function is rect_to_bb, yes “ Rectangle to border ” Abbreviation ：

def rect_to_bb(rect):
	# obtain dlib Predict the boundaries of and convert them 
	# According to the format we usually use （x,y,w,h）
	# Use OpenCV
	x = rect.left()
	y = rect.top()
	w = rect.right() - x
	h = rect.bottom() - y
	#  Returns a tuple  (x, y, w, h)
	return (x, y, w, h)
 Copy code 

This function accepts an argument rect, This parameter is assumed to be dlib detector （ Face detector ） Generated border rectangle .

rect The object includes the detected （x,y）- coordinate .

However , stay OpenCV in , We usually think of the bounding box as “（x,y,width,height）”, So for convenience ,rect_to_bb The function takes this rect Object to 4 Tuple coordinates .

second ,shape_to_np function ：

def shape_to_np(shape, dtype="int"):
	# initialization （x,y）- Coordinates the list 
	coords = np.zeros((68, 2), dtype=dtype)
	# stay 68 Cycle through the facial markers and convert them 
	# To （x,y）- Coordinate 2 Tuples 
	for i in range(0, 68):
		coords[i] = (shape.part(i).x, shape.part(i).y)
	# return （x,y） List of coordinates 
	return coords
 Copy code 

dlib The face marker detector will return an area containing face markers 68（x,y） Coordinate shape object .

Use shape_to_np function , We can convert this object into a NumPy Array .

With these two auxiliary functions , We can now detect facial markers in the image .

Open a new file , I'm going to call it facial_landmarks.py, Then insert the following code ：

# import the necessary packages
from imutils import face_utils
import numpy as np
import argparse
import imutils
import dlib
import cv2
# Construct a parameter parser and parse the parameters 
ap = argparse.ArgumentParser()
ap.add_argument("-p", "--shape-predictor", required=True,
	help="path to facial landmark predictor")
ap.add_argument("-i", "--image", required=True,
	help="path to input image")
args = vars(ap.parse_args())
 Copy code 

Import the required Python package .

Will use imutils Of face_utils The sub module accesses the helper functions detailed above .

Then import dlib.

Parse our command line arguments ：

--shape-predictor： This is the way to dlib The path of the pre trained facial marker detector . You can download the detector model here , You can also use the “ download ” Part to get the code + Sample image + A pre trained detector .

--image： We want to detect the path of the input image of the facial marker .

Now that our import and command line parameters have been processed , Let's initialize dlib Face detector and face marker predictor ：

#  initialization dlib Face detector （ be based on HOG） Then create 
#  Facial marker predictor 
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor(args["shape_predictor"])
 Copy code 

initialization dlib Pre trained face detector , The detector is based on a directional gradient standard histogram for object detection + linear SVM Method modification .

Then use the provided shape_predictor The path to load the face marker predictor .

however , Before we can actually detect facial marker points , We first need to detect the face in the input image ：

#  Load the input image , Resize , And convert it to gray 
image = cv2.imread(args["image"])
image = imutils.resize(image, width=500)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
#  Face detection in gray image 
rects = detector(gray, 1)
 Copy code 

adopt OpenCV Load our input image from disk , The image is then resized to have 500 The width of the pixel and convert it to grayscale to preprocess the image .

Process the bounding box of the face in the detection image .

The first parameter of the detector is our gray image （ Although this method can also be used for color images ）.

The second parameter is the number of image pyramid layers to be applied when enlarging the image before applying the detector （ This is equivalent to calculating... On the image cv2.N Time ）.

The advantage of improving the resolution of the input image before face detection is , It may allow us to detect more faces in the image - The disadvantage is that the larger the input image , The higher the computational cost of the detection process .

The number of faces in a given image （x,y）- coordinate , We can now apply face marker detection to each face region ：

#  Circular face detection 
for (i, rect) in enumerate(rects):
	#  Determine the facial sign of the facial area , then 
	#  Mark the face  (x, y)  The coordinates are converted to  NumPy Array 
	shape = predictor(gray, rect)
	shape = face_utils.shape_to_np(shape)
	#  take  dlib  Convert the rectangle to  OpenCV  Style bounding box 
	# [ namely (x, y, w, h)], Then draw the face bounding box 
	(x, y, w, h) = face_utils.rect_to_bb(rect)
	cv2.rectangle(image, (x, y), (x + w, y + h), (255, 255, 0), 2)
	#  Display face number 
	cv2.putText(image, "Face #{}".format(i + 1), (x - 10, y - 10),
		cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 0), 2)
	#  Cycle facial signs  (x, y)  coordinate 
	#  And draw them on the image 
	for (x, y) in shape:
		cv2.circle(image, (x, y), 1, (0, 255, 0), -1)
#  Display with face detection  +  Output image of face marker 
cv2.imshow("Output", image)
cv2.waitKey(0)
 Copy code 

Loop each face detection .

For each face detection , give 68（x,y）- coordinate , The coordinates are mapped to specific face features in the image .

And then dlib Convert shape objects to objects with shapes （68,2） Of NumPy Array .

Draw the bounding box around the detected face on the image , Draw the index of face .

Last , Loop over the detected facial markers and draw them separately .

Display the output image on the screen .

test result

Open terminal input ：

python facial_landmarks.py --shape-predictor shape_predictor_68_face_landmarks.dat --image 11.jpg
 Copy code 

summary

In today's blog post , We learned what facial markers are , And how to use it dlib、OpenCV and Python Test them .

Detecting facial markers in an image includes two steps ：

First , We must locate the face in the image . This can be achieved using many different technologies , But it usually involves Haar Cascade or HOG+ linear SVM detector （ But any method of generating a bounding box around a face is enough ）.

Apply shape predictor , Especially the face marker detector , To get the face ROI Middle facial region （x,y）- coordinate .

Given these facial markings , We can apply a variety of computer vision technologies , Include ：

Facial part extraction （ The nose 、 eyes 、 mouth 、 Mandibular line, etc ）

Face alignment

Head posture estimation

Face exchange

Blink detection

…...

I hope this article can help you ！！！

See... For complete code and documents ：

download.csdn.net/download/hh…