From Gait to Gaze: Advanced Facial Recognition and Biometric Tools

Authors: Paul Wright and Neal Ysart (The Coalition of Cyber Investigators)

Facial recognition is a vital tool in many modern investigations, as demonstrated recently by Ukrainian Intelligence, which provided images of suspected North Korean soldiers working in Russia to South Korea. South Korea successfully ran those images through facial recognition, leading to crucial insights. But what happens when suspects attempt to go off-grid? Disguises such as masks, makeup, beards, or digitally altered photographs and videos can easily thwart older facial recognition systems. To counter these challenges and accurately identify targets, integrating Digital Intelligence (DIGINT) and Social Media Intelligence (SOCINT) offers both a solution and added assurance. This approach leverages technology-oriented methodologies that enhance identification efforts, effectively overcoming traditional facial recognition systems' limitations.

PART PLAYED BY DIGINT

DIGINT is essential within the intricate world of professional investigations, harnessing digital data and advanced investigative techniques to navigate new challenges. While DIGINT may not work against physical disguises, it excels in combining publicly accessible data and social media interactions to construct behavioral patterns. This approach effectively links individuals who attempt to conceal their true identity behind an alternate persona, thwarting attempts at concealment by constructing behavioural profiles.

Despite the advanced capabilities of professional investigators to identify individuals with altered appearances and recent advances in facial recognition software, no technology is flawless. Top systems such as Paliscope, FaceNet (Google) and DeepFace (Facebook) use deep learning and 3D mapping to identify defining characteristics like bone structure, allowing them to detect faces even when partially obscured by makeup or facial hair. However, these systems occasionally struggle with certain disguises or alterations. Nevertheless, they continually evolve and adapt, often requiring human expertise to refine their application.

Powerful commercial tools like Amazon Rekognition further enhance this capability by leveraging machine learning and pattern recognition to identify faces obscured by eyeglasses or beards. Its scalability suits large-scale surveillance systems where suspects might alter facial features to avoid detection. ClearviewAI, for instance, compares faces in partial masks against vast databases of public images, addressing even sophisticated identity-altering techniques. Additionally, infrared technology such as FLIR aids by identifying consistent heat signals, regardless of makeup or costumes, boosting confidence in investigative efforts when visual obstructions hinder traditional facial recognition.

However, human expertise remains essential. Forensic facial comparison experts analyze intrinsic facial factors like ear shape and bone density, which are challenging to change. This manual comparison supports face recognition technology when it encounters limitations.

GAIT ANALYSIS‍

Gait analysis, examining an individual's walking patterns, is another valuable tool in the investigative arsenal. Technologies like Sighthound and GaitWatch analyze walking habits, which are significantly more difficult to disguise than facial features. This approach proves especially effective in tracking individuals on Closed Circuit Television (CCTV) when facial disguises are in place.

Current studies frequently rely on multimodal biometrics, combining various data types like facial recognition, gait analysis, and speech analysis to enhance identification accuracy. Systems such as NEC's Bio-IDiom exemplify this approach, improving identification capacity even when certain physical features are obscured.

FACIAL RECOGNITION AND BEHAVIORAL BIOMETRICS

In situations involving heavy makeup or minor facial alterations, 3D facial recognition technology, available in systems like RealFace and FaceFirst, maps the face in three dimensions, allowing more advanced algorithms to detect disguises that only superficially change identity characteristics. The growing concern of digital identity manipulation, such as deepfakes, is met by tools like SensityAI, which detect these digital alterations, helping investigators avoid false identifications from altered images.

When physical features can be easily disguised to deceive these systems, behavioral biometrics adds a new dimension to identification. Law enforcement can track suspects based on individual behavioral patterns, which are far harder to change than physical appearances. Elements such as typing speed, voice modulation, and motor functions provide an additional identification layer when traditional methods fall short.

VOICE PATTERN ANALYSIS

Voice pattern analysis has aided identification since the early 20th century when “vocal portraits” were first incorporated into US criminal records. Today, AI advancements continue to transform digital investigations, with voice pattern analysis showing great promise by identifying unique acoustic characteristics like pitch, tone, rhythm, accent, dialect, and pronunciation. Established tools like Digilog’s DiVA (Digital Intelligence Voice Analysis) can identify stress levels in an individual’s voice, offering indications of potential untruths or doubts. This is especially valuable when facial characteristics in a video have been altered but the original audio remains. Interpol’s global voice database, the Speaker Identification Integrated Project (SiiP), now serves as its third-largest biometric database, illustrating the power of vocal fingerprinting as a highly effective investigative tool.

EXPANDING LINES OF INVESTIGATION 

When facial disguises and obstructions push facial recognition to its limits, traditional methods such as fingerprint and DNA analysis provide unwavering reliability. Biological markers remain stable despite bodily disguises, and systems like the Automated Fingerprint Identification System (AFIS) expedite identification processes, even allowing mobile fingerprint scanning for quick identification in the field.

Enhancing picture and video capture capabilities further expands investigative depth. Integrating CCTV collections with mobile surveillance and drones enables investigators to capture scenarios and angles where disguises are less effective. Video enhancement software like Amped FIVE can help clarify grainy footage, bringing hidden features to light.

Additional methods, such as vehicle registration checks, lend credibility to this comprehensive approach. License plate recognition systems like Automated Number Plate Recognition (ANPR) can connect suspects to vehicles seen near crime scenes. Eyewitness observations, including body language or distinct behavioral traits, can also provide valuable clues.

The roles of OSINT and SOCINT are indispensable in modern investigations. By mining publicly available data and following digital footprints, investigators create comprehensive profiles that support physical identification attempts. Tools like Maltego and Hunchly help reveal online relationships, track digital activities, and monitor social media behaviors.

CONCLUSION 

As disguises come across in investigations become more sophisticated, so must the approaches used to counter them. Facial recognition is advancing through robust AI systems, yet these advancements also present ethical considerations around privacy. To enhance accuracy, multimodal biometrics—incorporating thermal imaging, gait analysis, and voice pattern analysis—have become essential. OSINT, DIGINT, and SOCINT empower investigators to pursue comprehensive identification strategies with digital intelligence. The Coalition of Cyber Investigators remains committed to pushing the boundaries of investigation techniques and hopes this work sets a benchmark for future advancements.