Small businesses thrive on in-store customers. When they reopen post-lockdown, a major challenge would be ensuring the safety of their staff and customers. Sanitizing and limiting shop occupancy are important safety measures but so is social distancing. How can small shops, with their limited resources, monitor their customers and enforce social distancing?
Object detection in real-time is a potential solution.
Using object detection, shops can track customer location in real time and calculate the distance between customers on the shop floor. The system can be configured to generate alerts whenever customers break the social distancing protocol.
Most small shops have video surveillance systems in place. This solution can be incorporated into the existing framework, extending its capability.
Object Detection Technologies Used
To help small shop owners enforce social distancing, we created a Python-based solution that can detect customer movement and alert them when safe distance is not maintained. We used YOLO and OpenCV for object detection and distance measurement.
YOLO, “You Look Only Once,” is a neural network capable of detecting what is in an image and where it is, in one pass. Here, it is used to find the bounding boxes around all the people in each frame of the real-time video.
OpenCV preprocesses the images and prepares them for classification before passing them through YOLO. It is also used for drawing bounding boxes around the people detected in the image and for drawing lines between them showing safe and unsafe distance.
Steps Involved in Processing
- The frames are loaded and preprocessed.
- The frames are passed through the YOLO network for object detection and classification.
- The ones that come under the class “person” are filtered to identify all the people present in a frame. YOLO outputs all the information (center coordinates and bounding box dimensions for each person) required to calculate the bounding box coordinates for each person.
- The detected people are shown with the help of bounding boxes drawn on the frame using OpenCV (yellow-colored bounding boxes shown in the above video).
The Euclidean distances between all the possible pairs among the people in the frame are calculated using the center coordinates (x,y) returned by YOLO and the following operations are performed:
- People who do not maintain the minimum acceptable distance are highlighted using a red line between them to emphasize that they are unsafe.
- People who maintain the minimum acceptable distance are highlighted using a green line to emphasize that they are safe.
YOLO Object Detection Results
Here is a sequence of frames showing the people detected (highlighted in yellow bounding boxes) and the distance between them (highlighted in red if unsafe and green if safe).
Overcoming Limitations in Infrastructure
This YOLO Object Detection solution calculates only the 2D distance between people. For 3D distance measurement, multiple cameras will have to be deployed. The threshold distance (the minimum distance that needs to be maintained between two people) is relative. It can be calculated by considering the resolution and location of the camera.
Benefits of Object Detection Using YOLO and OpenCV
- No excessive overheads: Most small businesses already have video surveillance systems in place. The object detection solution using YOLO and OpenCV described in this post can be easily integrated with those systems, avoiding extra hardware overhead.
- Labor-saving: Monitoring customer movement manually is a tough proposition especially when shops are understaffed.
- Better at tracking: It can detect social distancing violations with greater precision than humans.
- Extended safety: It can be further enhanced to detect face mask usage and alert staff when people without face masks enter the shop.
Conclusion
Retail stores, cafes, etc., cannot afford to stay shut for long, their mainstay being in-store customers. At the same time, they cannot afford to be lax about social distancing when customers start streaming in. With the YOLO object detection solution, they can create an extra pair of eyes to look out for their staff and customers.
