北京时间2022年3月15日，由IEEE生物识别联合会北京分会举办，中国科学院自动化研究所智能感知与计算研究中心NiCE团队承办的“生物特征识别与安全”学术论坛第5期活动圆满落幕！此次报告由中国科学院自动化研究所赫然研究员主持，邀请美国密歇根州立大学Arun Ross教授带来主题为“Altered Biometric Data:The Good and the Bad”的学术分享，观众通过线上会议、直播平台两种方式参与了此次活动。

On March 15, 2022, the 5th Session of “Trustworthy Biometrics Webinar” was successfully concluded, which was hosted by the IEEE Beijing Section Biometrics Council Chapter and organized by NiCE team from the Center for Research on Intelligent Perception and Computing of the Institute of Automation, Chinese Academy of Sciences. This report was chaired by Prof. Ran He, prominent researcher from the Institute of Automation, Chinese Academy of Sciences, invited Arun Ross, professor in the Department of Computer Science and Engineering at Michigan State University, to share on the themes of “Altered Biometric Data:The Good and the Bad”. Viewers participated in this event on online conferences and live streaming platform.

生物识别技术使用指纹、人脸、虹膜、声音和步态等身体和行为特征来识别个体，然而获取到的生物特征数据可能出于某些原因而被修改。虽然一些修改旨在提高生物识别系统的性能，但也存在许多恶意的修改行为，例如欺骗或混淆身份。此外，数据可能在经历一系列更改操作后生成一组与原始数据近似重复（Near-duplicate）的数据。在本次讲座中，我们讨论以下问题和方法：(a) 如何检测被修改的生物特征数据；(b) 如何确定近似重复生物特征数据之间的关系，并构建表示它们变换顺序的系统发育树（Phylogeny Tree）；(c) 怎样利用更改的生物特征数据实现增强隐私。

Biometrics refers to the use of physical and behavioral traits such as fingerprints, face, iris, voice and gait to recognize an individual. The biometric data (e.g., a face image) acquired from an individual may be modified for several reasons. While some modifications are intended to improve the performance of a biometric system (e.g., image enhancement), others may be intentionally adversarial (e.g., spoofing or obfuscating an identity). Furthermore, the data may be subjected to a sequence of alterations resulting in a set of near-duplicate data (e.g., applying a sequence of image filters to an input face image). In this talk, we will discuss methods for (a) detecting altered biometric data; (b) determining the relationship between near-duplicate biometric data and constructing a phylogeny tree denoting the sequence in which they were transformed; and (c) using altered biometric data to enhance privacy.

PRNU提取是一种传统的相机溯源方法，通过提取图像中的非均匀响应噪声（Photo-Response Nonuniformity Noise, PRNU），取其高频部分与噪声模板匹配，判断图像与模板是否出自同一相机或传感器。

PRNU(photo-response nonuniformity noise) extraction is a traditional camera traceability method. By extracting the PRNU in the image and matching high-frequency part with the noise template, judge whether the image and the template come from the same camera or sensor.

然而PRNU会暴露相机或传感器型号，可能造成用户隐私泄漏。Ross团队提出一种Sensor De-identification的方法，通过调整DCT coefficients修改图像的PRNU噪声，在保证图像生物特征识别准确率的前提下，降低了传感器识别的准确率，实现了隐私保护的功能。

However, PRNU will expose camera or sensor models, which may cause user privacy leakage. Ross team proposed a Sensor De-identification method, which modifies the PRNU noise of the image by adjusting DCT coefficients, reduces the accuracy of sensor recognition while ensuring the accuracy of image biometric recognitionand, and realizes the function of privacy protection.

随着多媒体合成技术的发展，出现了将两个甚至多个不同人脸合成为一张人脸图像（Morphed face）的技术，合成人脸可能与数据库中多个人脸有着高度的相似性，对人脸识别任务提出挑战。

With the development of multimedia synthesis technology, the technology of combining two or more different human faces into a morphed face appears. The morphed face may have high similarity with multiple human faces in the database, which poses a challenge to the task of face recognition.

Ross教授等人以检测身份证件中的合成人脸图像为背景，提出一种使用条件生成网络cGAN检测合成人脸的方法。将证件图像和参考图像输入条件生成网络，输出图像与参考图像进行生物特征比较，可以通过相似度判别证件图像是否为合成图像。此方法还能用于发掘组成合成图像的其他人脸信息。

Prof. Ross et al. proposed a method of detecting morphed face in ID documents using conditional generation networks. Inputing the document image and the reference image into cGAN, and comparing biological characteristics of the output image with the reference image, whether the document image is a morphed image can be judged by similarity measure. This method can also be used to discover other face information that makes up the morphed face.

将光度变换（例如亮度和对比度调整）应用于人脸图像，可以创建一组近似重复的图像（Near Duplicates）。这些近似的图像在视觉上可能无法辨认。在数字图像取证中，从一组近似重复的图像中识别原始图像，并推断它们之间的变换关系非常重要。

Photometric transformations, such as brightness and contrast adjustment, can be applied to a face image repeatedly creating a set of near-duplicate images. These near-duplicates may be visually indiscernible. Identifying the original image from a set of near-duplicates, and deducing the relationship between them, is important in the context of digital image forensics.

通常通过构建一个图像系统发育树（Phylogeny Tree, IPT）来推断一组近似重复图像间的关系。Ross教授等人利用三种不同的基函数族来模拟重复图像之间的成对关系，通过对函数参数的似然比进行评估，得到两两图像间不对称的变换指标，再利用深度优先搜索构建系统发育树。该方法在溯源和系统发育树重建上都具备优异的性能。

This is commonly done by generating an image phylogeny tree. Ross et al. utilize three different families of basis functions to model pairwise relationships between near-duplicate images, compute the likelihood ratio of the estimated parameters of the function above to obtain the asymmetric measure, finally use depth first search to construct IPT. This method works well in both the root identification and IPT reconstruction.

万能指纹（MasterPrints）指能够和大量指纹匹配的真实或合成的指纹。尤其对于许多设备上的小型传感器，往往不能捕获完整的指纹信息，这时万能指纹可以与较多局部指纹（Partial Fingerprints）进行匹配，攻击者可以利用它们发动针对特定主体的字典攻击，从而破坏指纹识别系统的安全性。

Masterprints are real or synthetic fingerprints that can match a large number of fingerprints in the database. Especially for the small sensors on many devices, they often can not capture the complete fingerprint information. At this time, the masterprints can be matched with a lot partial fingerprints, then attackers can use them to launch dictionary attacks against specific subjects, so as to break the security of fingerprint recognization system.

DeepMasterPrints基于WGAN生成指纹图像，使用潜变量进化技术（Latent Variable Evolution）生成能够与数据库中大量指纹匹配的万能指纹。

DeepMasterPrints generates fingerprints based on WGAN, using LVE(Latent Variable Evolution) to generate masterprints that can match with a large amount of prints in the database.

利用生物特征和数字取证技术，人们可以通过人脸图像来得到年龄，性别和种族等软生物特征（Soft-biometric Attributes），这可能会导致隐私的泄漏。Ross等人提出了一种基于GAN的半对抗网络（Semi-Adversarial Networks, SAN），通过图像扰动方法将软生物特征隐私赋予人脸图像，称为PrivacyNet。该网络修改输入人脸图像，使其可由人脸匹配器用于匹配目的，但不能由属性分类器可靠使用。此外，PrivacyNet允许用户选择模糊输入人脸图像中的特定属性（例如，年龄和种族），同时允许提取其他类型的属性。

Using biometircs and digital forensics techniques, we can deduce soft-biometric attributes such as age, gender and race from a face image, which can lead to privacy leakage. Ross et al. develop a technique for imparting soft biometric privacy to face images via an image perturbation methodology, using a GAN-based Semi-Adversarial Network (SAN) – referred to as PrivacyNet. The network modifies an input face image such that it can be used by a face matcher for matching purposes but cannot be reliably used by an attribute classifier. Further, PrivacyNet allows a person to choose specific attributes that have to be obfuscated in the input face images (e.g., age and race), while allowing for other types of attributes to be extracted (e.g., gender).

本次论坛Arun Ross教授围绕修改的生物特征数据（Altered Biometric Data）的优点与缺点，具体针对数据生成，图像取证和隐私保护的几项工作进行了分享。最后，非常感谢来自各研究机构和高校的老师和同学踊跃参与，活动圆满成功。

In this Webniar, Professor Arun Ross shared the good and the bad of altered biometric data, specifically on several works of altered and synthetic data, digital image forensics and privacy protection. Finally, thank you very much for the active participation of teachers and students from research institutions and universities, the activity concluded successfully.

下一期“生物特征识别与安全”学术论坛我们邀请了IEEE Biometric Council的VP（vice president），来自University of Ljubljana, Slovenia的Vitomir Struc教授带来学术分享，Vitomir Struc教授目前在University of Ljubljana, Slovenia电气工程系担任副教授，Vitomir的研究围绕人工智能、模式识别、深度学习、生物特征识别、人脸识别、情感识别、信号处理、机器学习、计算机视觉和其他相关领域。希望大家能继续支持与关注我们后续的系列活动，期待大家加入微信群一起讨论。我们欢迎更多来自世界各地的人加入我们，在未来一起探索更值得信赖的生物识别技术。

Next session of “Trustworthy Biometrics Webinar”, we invited the vice president of IEEE Biometric Council, Prof. Vitomir Struc to bring an academic sharing. Vitomir Struc is currently working as an Associate Professor at the Faculty of Electrical Engineering at the University of Ljubljana Slovenia, Vitomir’s research centers around artificial intelligence, pattern recognition, deep learning, biometrics, face recognition, emotion recognition, signal processing, machine learning, computer vision and other related areas. I hope you can continue to support and pay attention to our follow-up activities, and look forward to your joining the wechat group for discussion. We welcome more people from all over the world to join us and explore more reliable biometric technology in the future.