Final Project-Person Counter and Display Device (Co-creator: Sannat)

Slide and Project Video

final slide

Components required

Microprocessor : Beagle bone black (BBB)
High resoution webcam
MicroSD card (for storage in BBB)

What are we building ?

Our aim was to build a proof of concept about a person counting and display device which would help people make decisions whether to board a crowded metro or not. As a part of testing, we wished to test this in a classroom and count students to display its accuracy. However, with lockdown and situation, that is not clearly possible but it also presented an opportunity to tweak our device a little bit and use it for checking social distancing

How are we building ?

Connect beagle bone black to system. Root into beagle bone black using ssh
Configure beagle bone with the webcam chosen. The reason why we chose beagle bone as our primary microprocessor board was due to its high processing power, USB, extra storage and ethernet ports and finally because it is a mobile Linux CPU with latest distribution of linux installed. Configuration is easy- just connect and check for USB devices in the terminal

Once the device is configured, install openCV (a python based image processing library in beagle bone) using pip -install opencv. (Before doing this ensure python is installed on beagle bone system).
For purpose of this project we will be using openCV libraries to detect human faces. In order to use them we have written customised programs for detection of face, profile and shoulders. (Code is attached below).
After codes are tested on laptop/computer using webcam and are showing no errors, copy these codes to beagle bone black using sftp (SSH file transfer protocol)
Write a script for running the code on beagle bone black. This script calls all codes in different languages, compiles them and updates the result on interface.
Basic scheme of how social distancing is measured and people are counted is attached below-

Script is run after every 10s and in a semi-continuous process, the number of people measured are updated in real time and the length of the latest array + Social distancing indicator is sent to the receiver email ID directly with encoded image.

Codes

For ease of viewing and browsing, codes are on github page here

Why are we building this ?

Broad use cases of person counter and display device include-

Real time attendance (training data base required)
Social distancing check in queues, marketplaces and offices. (Anywhere where positions can be calibrated)
Space availability in any place (Measure crowdedness)
Person analysis (training data base required)

Apart from these use cases the biggest functionality of this device is that it can run on any image feed grabbed from a CCTV, personal camera or random images using beagle bone or any processing device (standalone computer, Raspberry Pi, laptop etc.). This easy integration helps in attaching it to any place of interest.

Limitations and Future Scope

Social distancing indicator needs manual calibration for image detection right now. In cases where more people are in a 3D space, it is difficult to indicate and mark the images. Potential solutions can be working on a overhead image (essentially a 2D image of human heads) which detects human heads and calculates distances between the detected windows easily by finding the vector distance between coordinates on image.
A more innovative approach towards social distancing indicator would be to include AR (Augmented Reality) libraries from openCV and use them. Due to time restrictions, we couldn't do it but it is can be done with openCV.
Counting people in a given frame can be made more dynamic by capturing video. The current issue with video was it's size and we didn't have free memory on our microprocessor. If a SD card of size 128 GB or above is in beagle bone, it would provide more utility in recording the video. (After some time the video can be deleted)
We are doing face detection not recognition right now. It would be good to include some face data and carry out recognition as well (can be used for real time attendance as indicated in above section)
Microprocessor is doing all the processing work right now which can be pretty intensive for it's processor and may take some time. To reduce this latency, we can shift the processing work from microprocessor to a cloud computer. Microprocessor can just be used as an instructional device for sending/receiving data.

Work Division

Sannat was responsible for microcontroller programming, scripting and live tests whereas Ritika worked on all programming of image detection codes in python. (A lot of troubleshooting to look for errors was there which is still unaccounted for and both were involved in doing that)