Have you ever had a long conversation with someone who obviously knows you, and all the time you have no idea who they are?
Some of us struggle to recognise people we’ve met several times, while others can see a person once and remember them forever. These ‘super-recognisers’ can even match people to grainy photos on video footage, in a rare gift that makes them sought after for security jobs like passport control and forensic investigations.
Exactly how the brain identifies faces is a mystery that’s baffled cognitive psychologists, vision scientists and neuroscientists for decades, says Dr David White, the lead investigator in the Face Research Lab at the University of New South Wales (UNSW) in Sydney. “What we know is that the brain treats faces differently to other types of visual object, and appears to have dedicated circuitry for perceiving and recognising faces,” he says. “Although we all look at many faces each day, we do not all share the same abilities to process the important social information that they contain. On one end of the spectrum are people with prosopagnosia - or ‘face blindness’ - who can’t even recognise friends and family if they pass them on the street. On the other are super-recognisers who can recognise people they only encountered once, many years ago.”
About 2.5% of the population are super-recognisers, and about 2,5% are faceblind. When people with face blindness have scanners attached to measure brain activity, one subtle difference is that they seem to have less development in the ‘Fusiform Face Area’, a hub in the network of brain areas that supports our face recognition ability.
Accurately identifying people is crucial to governments, police and many other organisations, since wrong decisions can have profound social consequences such as identity theft, acts of terrorism or wrongful convictions, says White.
To test the range of recognition skills and identify super-recognisers whose gift could be going to waste, White’s team designed a challenging online test. In one trial, 30% of people incorrectly decided that two pictures of the same man were different people. In another test, when passport issuing officers compared a real person to a photo, 6% of valid photos were rejected, and 14% of fraudulent photos were accepted. The researchers found no link between how many years of experience the officers had and their accuracy, because the skill seems to be in your DNA.
Scotland Yard, the headquarters of London Metropolitan Police, has a super recogniser squad that has identified criminals in incredibly difficult circumstances. In some riots of 2011, CCTV footage showed a criminal whose hoodie and bandana left only his eyes visible. The super squad pinned him down by matching his eyes to a mugshot. “This is not a skill you can be taught, you’re either born with it or you’re not,” said the squad’s creator, former chief inspector Mick Neville.
Artificial intelligence expert Gianluca Mauro says recognising people is one of the most important human skills, yet we can’t really explain how we do it. It’s easier to explain how artificial intelligence does it. “Face detection algorithms are trained to transform faces into a list of numbers so that two pictures of the same person are turned into similar numbers,” he says. Those hundreds or thousands of numbers represent abstract but unique concepts like the shape of your nose or the width of your forehead. If the numbers generated for two different photographs are very similar, it means they’re probably the same person. If they’re different, it’s someone else.
In 2017 the Smithsonian published an article called How Your Brain Recognizes All Those Faces. Its author Ben Panko reported on experiments by neuroscientist Le Chang of the California Institute of Technology, who was studying six areas in the brain’s temporal lobe called face patches. Each patch contains neurons that appear to be much more active when a person is looking at a face rather than looking at any other objects.
Chang created nearly 2,000 artificial faces and broke down their component parts into 50 different characteristics, from skin colour to the amount of space between the eyes. Then he planted electrodes into two rhesus monkeys to monitor how the neurons in their face patches fired when they saw the artificial faces. Each neuron only responded to one feature of what makes faces different. “As far as your neurons are concerned, a face is a sum of separate parts, as opposed to a single structure,” Panko wrote.
Chang could then create faces that looked very different but produced the same patterns of neural firing because they shared key features. That let him to create an algorithm to analyse the firing patterns of 205 neurons, and draw the features suggested by the activity of each individual neuron. The faces they drew matched what the monkeys were seeing with nearly 70% accuracy. Chang’s colleague, neuroscientist Doris Tsao, said: “People always say a picture is worth a thousand words. But I like to say that a picture of a face is worth about 200 neurons.”
Can AI do it Better?
This may finally be the year that artificial intelligence overtakes even the super-gifted in the ability to recognise faces. In a 2017 study called Man Versus Machine: Who Wins When It Comes to Facial Recognition? deep learning algorithms were pitched against trained experts and super-recognisers, and showed similar levels of accuracy. But when AI was used in partnership with human intelligence, the accuracy was far better than when two human examiners worked together.
In 2018, Australian researchers also compared cutting-edge AI facial recognition technology to the top human performers and found that AI matched them. “If AI was as good as the best humans in 2018, Moore’s Law would suggest that AI is perhaps more accurate than the best humans in 2022,” says Dr White.
How do we do it?
Quite how the brain matches a face to a name stored in its database is another mystery, says Dr White. “What we do know is that a person’s ability to recognise faces is not the same ability as being able to name the face,” he says. “Even super-recognisers are bad at remembering names - they just know for certain that the person is familiar, and often express frustration that they can’t place them or remember their name.”
Our brains have more cortex devoted to processing visual information, such as a face, than for auditory information, like a name. However strong our fusiform area might be, putting a name to it is complex because that knowledge resides in a separate place, rather than in a single mental filing cabinet, and sometimes the attempts at integration fail.
Test your own skills here:
Are you a super-spotter or a social disaster? Try the UNSW Face Test
(First published in Brainstorm Magazine. Photos: UNSW, Sydney)
If you want travel & leisure writing that's sassy - but classy you've come to the right place. After 12 years as the IT Editor for Business Day I quit to follow my passion for writing opinionated pieces for any publication bold enough to run them. Contact me or read more.
