Projects
Using Facial Recognition in a Sensitive Context
Lessons learned from our pilot to support the validation of missing person inquiries
One of the many tragedies of conflict and disaster is that people go missing, leaving their families to agonise over their fate, often for years. The International Commission on Missing Persons (ICMP) contributes to the search for these people through its Online Inquiry Centre (OIC). On the site, people can enter and retrieve information on individual missing person cases. One of the challenges presented by maintaining the OIC is the number of man-hours required to respond to inquiries that are received. Working with ICMP, the Centre for Innovation asked: would it be possible to support the verification of missing person inquiries by using facial recognition software? We developed a prototype to find out.
Image credit: ICMPValidating missing persons inquiries
When a person is reported missing via the OIC, ICMP asks for certain information such as name, age, gender, social media accounts, and a photo. To validate the inquiry, ICMP manually checks the information provided in the submitted form. One of these checks involves looking at the provided photos and assessing the likelihood that this person corresponds to the rest of the information, such as gender and age. This process is resource-heavy and currently takes up a lot of ICMP staff time.
Using facial recognition software
Facial recognition technology has become increasingly prevalent in recent years. The technology has not only shown potential in recognising a person by comparing photos, but also in estimating other characteristics of that person, such as age and gender. We used this capability of facial recognition technology, to estimate such characteristics, as a starting point to assess its potential as a solution for ICMP. We prototyped different facial recognition technologies using an anonymised dataset comparable to the data ICMP uses in their work. Keeping in mind the fact that we needed to work with sensitive data, we focused on data responsibility throughout the project.
What did we learn?
1. Facial recognition software can support inquiry verification. We used three types of facial recognition software, each of which demonstrated that age and gender assessment from an image was possible, although with varying degrees of accuracy. As we expected, age was much less reliable than gender. However, adding a threshold to the error (for example +/- 5 years to age) allowed for a useful form of flagging errors.
2. There are some specific errors you should take into account. Children, for example, were consistently classified as adults, making the software less reliable for facial recognition of young people. This is probably due to the algorithms being trained on datasets without photos of children, for obvious ethical reasons.
3. You might want to check where the software company stores its data. Commercial facial recognition software might require you to upload images to the software company’s servers. As they store those images for differing periods of time, this might be undesirable when using sensitive information. To overcome this, we used an open source version which matched, if not improved, the performance of the commercial products. This meant that ICMP could implement the technology on their own servers maintaining control of sensitive information.
The aim of our prototype was not to replace manual verification, but to help streamline the process and identify potential fake reports. Our approach ensures that flaws in the software do not negatively affect the verification of missing persons inquiries. If the prototype were applied to actual data from ICMP, it could help ICMP staff in their validation procedure. In the upcoming months, we will see how the Centre’s ongoing support for ICMP progresses and how it could be improved. We will keep you posted!
Get in touch
Did this article interest you? Email us at hello@llinc.leidenuniv.nl or take a look at our team of experts.
