Object Detection
Today we’ll discuss about Object detection using pre-trained classifers
What we’ll be discussing :
- Haar Cascade Face Detector in OpenCV
- HoG Face Detector in Dlib
- Deep Learning based Face Detector in Dlib
The first thing to note : This tutorial is not a theoretical one, instead our focus will be on how to use and experiment with the pre trained classifers.
OpenCV provides pre-trained models for object detection. The pre-trained models are located in the data folder in the OpenCV installation.
All links are provided, including notebook link
Before we start, lets clear some things out of our way
- How do we download images from internet ?
- How do we view the images that are downloaded ?
let’s find out how to download images from internet and use it in our google colab notebook for object detection
# IMPORT OPEN CV
import cv2# IMPORT imshow TO DISPLAY IMAGES
from google.colab.patches import cv2_imshow# IMPORT TENSORFLOW TO DOWNLOAD IMAGES
import tensorflow as tf# HOW DO WE DOWNLOAD IMAGES USING TF ?url = " image url "
img_path = tf.keras.utils.get_file(fname='img.jpg', origin=img_url)# HOW TO VIEW IMAGES ?# this is done in two steps # step 1 # read the imageimg = cv2.imread(img_path)# step 2 # preview the imagecv2_imshow(img)
Haar-cascade classifier
What are Haar-cascade classifiers ?
It is a machine learning based approach where a cascade function is trained from a lot of positive and negative images. It is then used to detect objects in other images.
In this tutorial we will use the pre-trained Haar cascade models to detect faces in an image an much more.
link : OpenCV docs
link : Haar-cascade trained classifiers for detecting objects
Without wasting any time, lets jump right in and detect some faces using
haar-cascade frontal face detector
import cv2# frontal face detector
face_cascade = cv2.CascadeClassifier(
cv2.data.haarcascades +'haarcascade_frontalface_default.xml'
)
With haar-cascade classifier , it’s recommended to use gray scale images. The reason for using gray scale image is gray channel is easy to process and is computationally less intensive as it contains only 1-channel of black-white.
# Convert the image to gray-scale
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)# Preview the gray-scale image
cv2_imshow(img_gray)
Detect faces in the image
# we use gray_scale image here
detections = face_cascade.detectMultiScale(img_gray)# we got back the loaction of the faces in the image
# now we can use the color image here to draw bounding boxesfor x , y , w , h in detections:
#color-image #frame #color #border
cv2.rectangle( img, (x,y), (x+w,y+h), (34,139,34), 2)
# preview the image with boxes
cv2_imshow(img)
Steps in face detection using haar-cascade classifier
- Download the image
- Convert the image to gray scale
- Use the gray-scale image for detecting faces
- cv2.data.haarcascades +’haarcascade_frontalface_default.xml’
- returns values for [ x-position, y-position, width, height ]
- Draw boxes in the original image using cv2.rectangle(…)
We can simply create an utility function that does the job for us . We only give the image-url to this utility function and the utility will give us back the image with bounding boxes drawn, lets create one
# let's create an utility that will perform frontal face detection
def frontal_face_detection(image_url,**kwargs):
# download & read the image
image = read_image(image_url) # converting image to gray-scale
img_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) # classifier
face_cascade = cv2.CascadeClassifier(
cv2.data.haarcascades+'haarcascade_frontalface_default.xml') # detect faces
detections = face_cascade.detectMultiScale(img_gray,**kwargs)
for x,y,w,h in detections:
cv2.rectangle(image,(x,y),(x+w,y+h),(34,139,34),2) return cv2_imshow(image)
We use haarcascades frontal face pre-trained classifier to detect faces in the images. We haven’t set any parameters for detecting faces in the images.
we relied on the default values and for the image at hand the default values works fine.
But there are scenarios where we need something more than what default values can provide.
For example , Consider we have an image with some faces on it.The faces closer to camera appear larger than the faces that are further away from camera.
Harr-cascade classifier with default parameters gives a lot of false positives, also it may fail to detect faces in the image as well.
What can we do for improved detection using Haar-cascade trained classifiers ?
Lets adjust some parameters
- scaleFactor
- minNeighbours
- minSize
- maxSize
scaleFactor is a parameter that deal with scale of the image.
The value for scaleFactor should be greater than 1.
The default value for this parameter is 1.1
If we give a small value for scaleFactor, it will help us detect small faces in the image but caution if there are no small faces then it’ll give us lot of false positives
If we give a very big value for scaleFactor , it will not give us any detection.
The parameter scaleFactor depends on the size of the image
For each face we could draw a lot of bounding boxes. We need to select the best bounding box from all possible boxes. The parameter minNeighbours to the rescue. The minNeighbours parameter indicate the number of candidate bounding box that must exist for the final box to be generated
The minSize parameter help us set the size of bounding box. Any face that are smaller than minSize will not be detected. Default value of [30 , 30].
Similarly we’ve maxSize parameter that specify the maximum size of the bounding box , any face larger than maxSize will not be detected.
Now we have lot of false positives, this is the trade-off we have to make in-order to get what we want when using haarcascade classifiers
# if we set maxSize parameter to (40,40) all images bigger than (40,40) will not be detectedfrontal_face_detection(
image_url,
scaleFactor=1.02,
minNeighbors=4,
minSize=(1,1),
maxSize=(40,40))
Trade off with false positive
when we play around with these parameters we might see that a lot of false positives are being generated. It is the trade-off we have to make in-order to get what we want when using the haarcascade frontal face detector. In case you are not happy with the results , try adjusting the parameters or use a different classifier like face detection using dlib HOG.
Detecting Faces with HOG using dlib library
This algorithm give us better results than Haar cascade classifiers.
It’s very simple , we don’t need to convert the image into gray-scale as we did with haar-cascade classifier. We can read the image and send it directly in to the model for detection.
import dlibface_detector = dlib.get_frontal_face_detector()# higher the value of scale , smaller the boxes are
scale = 1
detection = face_detector(image, scale)
We will get a detection of each face in the image. The detection is a list of bounding box dimensions for each face in the image
Sample output :
rectangles[ [( left, top),(right, bottom) ], [ (left,top),(right,bottom) ], …]
for box in detection:
l,t,r,b = box.left(),box.top(),box.right(),box.bottom()
# draw boxes top-left , bottom-right, color , box-width
cv2.rectangle( (l,t) , (r,b) , (0,255,255,255) , 2 )# show the output
cv2_imshow(image)
when using HOG , if the faces in the image are small use a high value for the parameter ‘ scale ’. This will help us detect the small faces in the image
def HoG_face_detection(image_url, scale=1):
detector = dlib.get_frontal_face_detector()
# download the image
image_path = tf.keras.utils.get_file(origin=image_url)
# read the image
sample = cv2.imread(image_path)
# get detection
detections = detector(sample, scale)
# draw bounding boxes
for face in detections:
l,t,r,b = face.left(), face.top(), face.right(), face.bottom()
cv2.rectangle(sample, (l,t), (r,b), (128,250,114), 2)
# display the output
cv2_imshow(sample)let’s see an example
# https://unsplash.com/@benearlweberimage_url = 'https://images.unsplash.com/photo'
HoG_face_detection(image_url)
# https://unsplash.com/@naassomz1
image_url='https://images.unsplash.com/photo'
HoG_face_detection(image_url)
CNN based face detection
This repository contains trained models : Github
Download the face detection model mmod_human_face_detector.dat.bz2
CNN based face detection
- Import `dlib `
- Download the trained model
- Use `dlib.cnn_face_detection_model_v1(..path..)`
When using Dlib we don’t have to convert to gray scale, we can work with the color images itself
On Google Colab, click on Edit -> Notebook settings and change the runtime to GPU.
import dlib detector = dlib.cnn_face_detection_model_v1(..path..)scale = 1
detection = detector(image, scale)
Since this model is CNN based we will get a confidence value for the detection as well. Let’s check them
for box in detection:
l = box.rect.left()
t = box.rect.top()
r = box.rect.right()
b = box.rect.bottom() confidence = box.confidence
# draw boxes top-left , bottom-right, color , box-width
cv2.rectangle( (l,t) , (r,b) , (0,255,255,255) , 2 )# show the output
cv2_imshow(image)
Let’s create an utility for this
def cnn_face_detection(image_url, scale=1):
# read the image
image_path = tf.keras.utils.get_file(origin=image_url)
sample = cv2.imread(image_path)
# CNN based face detection
detector = dlib.cnn_face_detection_model_v1(
'/content/mmod_human_face_detector.dat')
detection = detector(sample, scale)
for box in detection:
l = box.rect.left()
t = box.rect.top()
r = box.rect.right()
b = box.rect.bottom()
confidence = box.confidence
# draw boxes image, top-left , bottom-right, color , box-width
cv2.rectangle(sample, (l,t) , (r,b) , (0,255,255,255) , 2 )
# show the output
cv2_imshow(sample)Let’s see it in action
# https://unsplash.com/@naassomz1
image_url='https://images.unsplash.com/photo'
cnn_face_detection(image_url)
Looks like we are not detecting all the faces !
Can we improve our detection ?
Yes !
Adjust the ‘ scale ’ parameter
Higher the values of scale parameter is , smaller the faces we can detect
# https://unsplash.com/@naassomz1
image_url='https://images.unsplash.com/photo'
cnn_face_detection(image_url, scale=3)
Other examples using haar casacade classifier
Eye detection using Haar-cascade
def eye_detection(image_url,**kwargs):
# download & read the image
image = read_image(image_url)
# converting image to gray-scale
img_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# classifier
face_cascade = cv2.CascadeClassifier(
cv2.data.haarcascades+'haarcascade_eye.xml')
# detect faces
detections = face_cascade.detectMultiScale(img_gray,**kwargs)
for x,y,w,h in detections:
cv2.rectangle(image,(x,y),(x+w,y+h),(128,228,114),2)
return cv2_imshow(image)
Full body detection using Haar-cascade
def full_body_detection(image_url,**kwargs):
# download & read the image
image = read_image(image_url)
# converting image to gray-scale
img_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# classifier
face_cascade = cv2.CascadeClassifier(
cv2.data.haarcascades+'haarcascade_fullbody.xml')
# detect faces
detections = face_cascade.detectMultiScale(img_gray,**kwargs)
for x,y,w,h in detections:
cv2.rectangle(image,(x,y),(x+w,y+h),(258,128,114),2)
return cv2_imshow(image)
Simle detection using Haar-cascade
def smile_detection(image_url,**kwargs):
# download & read the image
image = read_image(image_url)
# converting image to gray-scale
img_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# classifier
face_cascade = cv2.CascadeClassifier(
cv2.data.haarcascades+'haarcascade_smile.xml')
# detect faces
detections = face_cascade.detectMultiScale(img_gray,**kwargs)
for x,y,w,h in detections:
cv2.rectangle(image,(x,y),(x+w,y+h),(128,250,114),2)
return cv2_imshow(image)
Licence plate detection using Haar-cascade
# Russian licence plate detect
def russian_plate_detection(image_url,**kwargs):
# download & read the image
image = read_image(image_url)
# converting image to gray-scale
img_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# classifier
face_cascade = cv2.CascadeClassifier(
cv2.data.haarcascades+'haarcascade_russian_plate_number.xml')
# detect faces
detections = face_cascade.detectMultiScale(img_gray,**kwargs)
for x,y,w,h in detections:
cv2.rectangle(image,(x,y),(x+w,y+h),(128,250,114),2)
return cv2_imshow(image)
