Face and Person Detection in Images

Face detection, person detection and face recognition are related but distinct tasks. Detection locates a face or person, while recognition evaluates identity similarity between an observed face and stored representations.

Detection and alignment

Haar cascades and HOG-based methods historically provided low-cost solutions. Modern learned detectors are more robust to changes in pose, scale and occlusion.

Facial landmarks can align the eyes, nose and mouth before recognition, reducing variation caused by crop position and head pose.

Embedding-based recognition

A recognition network maps the aligned face to an embedding. Samples of the same identity are encouraged to lie close together, while different identities are separated. Open-set systems must also support a no-match decision for people not enrolled in the database.

Face detection quality constrains every later stage. Poor cropping or alignment can shift the resulting embedding.

Biometric Decision Structure

Face recognition is one example of a biometric system. During enrollment, reference samples are converted into a template or feature representation; later samples are compared with that representation.

In verification, the question is whether a person is who they claim to be, typically producing a one-to-one comparison. In identification, the question is which enrolled identity best matches the sample, requiring a one-to-many search. As the candidate population grows, identification behaves differently in both computational cost and false-match risk.

A face image is first detected, assessed for quality, aligned, and converted into a feature vector. The resulting similarity score is not an identity by itself; a decision threshold must be selected according to the cost of false acceptance and false rejection in the application.

Because biometric traits cannot be replaced as easily as passwords, data protection creates an additional engineering obligation. Retention and access policies for raw images, templates, and comparison logs belong to the system design.

Evaluation

False-match and false-non-match rates should be interpreted together with the decision threshold. Camera type, pose, illumination, age and data distribution can affect these rates. Access control and data minimisation are also architectural requirements for identity-processing systems.

Parent topic: Image and Audio Processing

References

• Anil K. Jain; Arun Ross; Salil Prabhakar. An Introduction to Biometric Recognition. IEEE Transactions on Circuits and Systems for Video Technology, 14(1), 4–20, 2004. doi:10.1109/TCSVT.2003.818349

• Rafael C. Gonzalez; Richard E. Woods. Digital Image Processing, Fourth Edition. Pearson, 2018.

• Richard Szeliski. Computer Vision: Algorithms and Applications, Second Edition. Springer, 2022.