Face masks are breaking facial recognition algorithms, says new government study

Stopping the spread of disease and mass surveillance

By James Vincent Jul 28, 2020, 6:10am EDT

Face masks are one of the best defenses against the spread of COVID-19, but their growing adoption is having a second, unintended effect: breaking facial recognition algorithms.

Wearing face masks that adequately cover the mouth and nose causes the error rate of some of the most widely used facial recognition algorithms to spike to between 5 percent and 50 percent, a study by the US National Institute of Standards and Technology (NIST) has found. Black masks were more likely to cause errors than blue masks, and the more of the nose covered by the mask, the harder the algorithms found it to identify the face.

“With the arrival of the pandemic, we need to understand how face recognition technology deals with masked faces,” said Mei Ngan, an author of the report and NIST computer scientist. “We have begun by focusing on how an algorithm developed before the pandemic might be affected by subjects wearing face masks. Later this summer, we plan to test the accuracy of algorithms that were intentionally developed with masked faces in mind.”

[...]

NIST Launches Investigation of Face Masks’ Effect on Face Recognition Software

Algorithms created before the pandemic generally perform less accurately with digitally masked faces.

July 27, 2020



Credit: B. Hayes/NIST
NIST digitally applied mask shapes to photos and tested the performance of face recognition algorithms developed before COVID appeared. Because real-world masks differ, the team came up with variants that included differences in shape, color and nose coverage.

Now that so many of us are covering our faces to help reduce the spread of COVID-19, how well do face recognition algorithms identify people wearing masks? The answer, according to a preliminary study by the National Institute of Standards and Technology (NIST), is with great difficulty. Even the best of the 89 commercial facial recognition algorithms tested had error rates between 5% and 50% in matching digitally applied face masks with photos of the same person without a mask.

The results were published today as a NIST Interagency Report (NISTIR 8311), the first in a planned series from NIST’s Face Recognition Vendor Test (FRVT) program on the performance of face recognition algorithms on faces partially covered by protective masks.

“With the arrival of the pandemic, we need to understand how face recognition technology deals with masked faces,” said Mei Ngan, a NIST computer scientist and an author of the report. “We have begun by focusing on how an algorithm developed before the pandemic might be affected by subjects wearing face masks. Later this summer, we plan to test the accuracy of algorithms that were intentionally developed with masked faces in mind.”

The NIST team explored how well each of the algorithms was able to perform “one-to-one” matching, where a photo is compared with a different photo of the same person. The function is commonly used for verification such as unlocking a smartphone or checking a passport. The team tested the algorithms on a set of about 6 million photos used in previous FRVT studies. (The team did not test the algorithms’ ability to perform “one-to-many” matching, used to determine whether a person in a photo matches any in a database of known images).

[...]