Scientists have created a “digital mask” that will allow facial images to be stored in medical records while preventing the extraction and sharing of potentially sensitive personal biometric information.
In a study published today in natural medicinea team led by scientists from the University of Cambridge and Sun Yat-sen University in Guangzhou, China, used three-dimensional (3D) reconstruction and deep learning algorithms to erase identifiable features from images facial features while retaining disease-relevant features necessary for diagnosis.
Facial images can be helpful in identifying signs of disease. For example, features such as deep forehead wrinkles and wrinkles around the eyes are significantly associated with coronary heart disease, while abnormal changes in eye movements may indicate poor visual function and problems with visual cognitive development. However, facial images inevitably also record other biometric information about the patient, including race, gender, age, and mood.
With the increasing digitization of medical records comes the risk of data breaches. While most patient data can be anonymized, facial data is more difficult to anonymize while retaining essential information. Common methods, including blurring and cropping of identifiable areas, can lose important disease information, but cannot even completely escape facial recognition systems.
For privacy reasons, people are often reluctant to share their medical data for public medical research or electronic health records, which hampers the development of digital medical care.
Professor Haotian Lin of Sun Yat-sen University said, “During the COVID-19 pandemic, we have had to turn to consultations by telephone or video link rather than in person. Remote health care for eye diseases require patients to share a large amount of digital facial information. Patients want to know that their potentially sensitive information is secure and that their privacy is protected.”
Professor Lin and his colleagues have developed a “digital mask”, which inputs an original video of a patient’s face and produces a video based on the use of a deep learning algorithm and 3D reconstruction, while removing as much of the patient’s personal biometric information as possible – – and from which it was not possible to identify the individual.
Deep learning extracts features from different parts of the face, while 3D reconstruction automatically digitizes the shapes and movements of 3D faces, eyelids and eyeballs based on the extracted facial features. Converting digital mask videos to original videos is extremely difficult because most of the necessary information is no longer kept in the mask.
Next, the researchers tested the usefulness of the masks in clinical practice and found that diagnosis using the digital masks was consistent with that performed using the original videos. This suggests that the reconstruction was sufficiently accurate to be used in clinical practice.
Compared to the traditional method used to “de-identify” patients – image cropping – the risk of being identified was significantly lower in digitally masked patients. The researchers tested this by showing 12 eye doctors digitally masked or cropped images and asking them to identify the original from five other images. They correctly identified the original from the digitally masked image in just over a quarter (27%) of cases; for the cropped figure, they were able to do so in the overwhelming majority of cases (91%). However, this is probably an overestimate: in real situations, one would probably need to identify the original image from a much larger set.
The team interviewed randomly selected patients attending clinics to test their attitudes towards digital masks. Over 80% of patients believed the digital mask would alleviate their privacy concerns and they expressed an increased willingness to share their personal information if such a measure were implemented.
Finally, the team confirmed that digital masks can also evade AI-based facial recognition algorithms.
Professor Patrick Yu-Wai-Man from the University of Cambridge said: “Digital masking offers a pragmatic approach to protecting patient privacy while allowing information to be useful to clinicians. At the moment the only options available are crude, but our digital mask system is a much more sophisticated tool for anonymizing facial images.
“It could make telemedicine – telephone and video consultations – much more feasible, making healthcare delivery more efficient. If telemedicine is to be widely adopted, then we need to overcome the barriers and concerns around protecting privacy. Our digital mask is an important step in this direction.”
The research has been widely supported by science and technology planning projects of Guangdong province, China’s national key R&D program and the construction of high-level hospitals. Professor Yu-Wai-Man is supported by the UK’s National Institute for Health and Care Research (NIHR).