北京时间2022年5月6日下午，由IEEE生物识别联合会北京分会举办，中国科学院自动化研究所智能感知与计算研究中心NiCE团队承办的“生物特征识别与安全”学术论坛第6期活动圆满落幕！此次报告由中国科学院自动化研究所彭勃副研究员主持，邀请斯洛文尼亚卢布尔雅那大学Vitomir Štruc副教授带来主题为“Generative models in biometrics”的学术分享，观众通过线上会议、直播平台两种方式参与了此次活动。

On May 6, 2022, the 6th Session of “Trustworthy Biometrics Webinar” was successfully concluded, which was hosted by the IEEE Beijing Section Biometrics Council Chapter and organized by NiCE team from the Center for Research on Intelligent Perception and Computing of the Institute of Automation, Chinese Academy of Sciences. This report was chaired by Dr. Bo Peng, associate researcher from the Institute of Automation, Chinese Academy of Sciences, invited Vitomir Štruc, associate professor at the university of Ljubljana, Slovenia, to share on the themes of “Generative models in biometrics”. Viewers participated in this event on online conferences and live streaming platform.

随着生成建模领域的发展以及Gnerative Adversarial Networks(GAN)等强大的模型框架的出现，近期涌现了各种各样的新技术和创新性算法，以解决包括生物识别技术在内的各种计算机视觉问题。在这次报告中，Vitomir Štruc首先简要概述生成模型的最新进展以及其团队在生成模型上的研究情况，然后展示如何将StyleGAN2生成器使用在生物识别技术相关研究上，并深入介绍以下内容：（1）基于GAN inversion的面部图像编辑技术-MaskFaceGAN，该技术允许对照片真实感进行操作并明确地解决了在基于隐空间编辑技术中常见的属性纠缠问题；（2）使用其团队的双支StyleGAN2(DB-StyleGAN2)模型完成bimodal ocular图像生成（及标注）任务。最后，Vitomir Štruc阐述了生成模型现有的一些问题与挑战及其未来的发展方向。

With the developments in the field of generative modeling and with the appearance of powerful model architectures, such as Generative Adversarial Networks (GAN), a wide range of new techniques and inventive algorithms has emerged recently to solve diverse computer vision problems, including many problems in the area of biometrics. In this talk, Prof. Vitomir Štruc first provided a short overview of recent advances in generative modeling and described some of their research efforts that focus explicitly on generative models. Next, Prof. Vitomir Štruc presented their recent biometrics-related research utilizing the powerful StlyGAN2 generator and talk about: (i) face image editing with their GAN inversion based MaskFaceGAN technique that allows for photo realistic image manipulation and explicitly addresses the problem of attribute entanglement seen with many latent-space based editing solutions, and (ii) bimodal ocular image generation (and annotation) with their Dual-Branch StlyGAN2 (DB-StyleGAN) model.  Finally, Prof. Vitomir Štruc elaborated on some of the existing challenges with generative models and highlight future research directions.

Vitomir Štruc等人希望通过添加属性标签的方式实现目标图像特定属性的变化，而不改变其他属性。现有的基于latent space的方法能够生成真实感较好的图像，但是通过GAN inversion在隐空间编辑隐向量的方法往往不能实现很好的细节控制和属性解耦。Vitomir Štruc团队设计了MaskFaceGAN，首先使用人脸解析器face parser根据给定属性确定需要调整和保持的面部区域，然后设计损失函数约束隐码优化，最后借助人脸解析得到的mask，通过混合重组的方式得到最终的人脸图像。此外，通过调整损失函数细节，还能够实现多属性控制及细节调整。MaskFaceGAN实现了很好的属性解耦性能，并且在不同数据集上具备较好的普适性。

Prof. Vitomir Štruc and his team hope to change the specific attribute of the target image while keeping other attributes unchanged by adding target attribute label. So far, techniques based on latent space have achieved photo realistic images, but still can not achieve fine control and attribute disentanglement. Prof. Vitomir Štruc et al. design MaskFaceGAN, first use a face parser to make sure which part of the face to change and which part to maintain, then design the loss function to constrain the latent code optimization, and finally blend the generated image and the origin image to get the final output using masks. In addtion, by fine control of the loss function, they can also achieve multi-attributes and intensity control. MaskFaceGAN is able to disentangle the attributes well and adapt to different datasets generally.

现代生物特征识别系统往往基于深度学习模型，需要大量带有标注的数据，目前比较常见的做法是使用人工合成的数据集。Vitomir Štruc团队设计了BiOcularGAN，来生成逼真的人眼图像及其分割掩码。BiOcularGAN主要包括两部分，第一部分是Dual-Branch StyleGAN2，利用网络中分支结构，生成同一人眼分别在VIS和NIR域下的两张图像，作者假设如果能够以dual-branch的方式训练模型，模型就能具备更好的捕捉图像语义结构的能力；第二部分是一个语义分割图生成器，受DatasetGAN启发，语义分割图生成器利用Feature Maps和Ensemble Classifiers预测每个像素的语义标签，生成语义分割图。相较DatasetGAN，受益于dual-branch的训练方式，该模型能够更好地捕捉语义信息，从而生成更加精准的语义分割图。

Modern biometric systems always base on deep learning models, which need a lot of annotated data. Right now usual methods are using artificially synthesized dataset. Vitomir Štruc and his team design BiOcularGAN to generate realistic eye image and its segmentation mask. BiOcularGAN has mainly two parts, one is called Dual-Branch StyleGAN2, by using the Dual-Branch structure, it can generate two images of the same eye in VIS domin and NIR domin respectively. The author assume that if the model can be trained in a dual-branch manner, the model can have a better ability to capture the semantic structure of the image. The second part is a semantic mask generator, inspired by DatasetGAN, the semantic mask generator use feature maps and ensemble classifiers to predict the semantic label of each pixel to generate the semantic segmantation mask. Compared with DatasetGAN, benefited from the dual-branch training method, the model can capture the semantic information better and generate more precise semantic segmentation mask.

最后，Vitomir Štruc教授简单介绍了其团队的项目和研究方向，包括生物特征识别、人脸隐私保护和对抗攻击等，希望感兴趣的同学和老师能在未来与Vitomir Štruc团队开展相关方向上的合作。

Finally, Prof. Vitomir Štruc simply introduced the recent projects and research directions of his laboratory, not only biometrics, but also privacy protection, adversarial defenses and so on, and hoped to cooperate with the audience who are interested in the relevant directions in the future.

本次论坛Vitomir Štruc教授围绕生物特征识别中的生成模型（Generative models in biometrics），具体针对MaskFaceGAN和BiOcularGAN两项工作进行了分享，并与参会老师和同学进行交流探讨。最后，非常感谢来自各研究机构和高校的老师和同学踊跃参与，活动圆满成功。希望大家能继续支持与关注我们后续的系列活动，期待大家加入微信群一起讨论。我们欢迎更多来自世界各地的人加入我们，在未来一起探索更值得信赖的生物识别技术。

In this Webniar, Prof. Vitomir Štruc shared generative models in biometrics, specifically on MaskFaceGAN and BiOcularGAN. Finally, thank you very much for the active participation of teachers and students from research institutions and universities, the activity concluded successfully. I hope you can continue to support and pay attention to our follow-up activities, and look forward to your joining the wechat group for discussion. We welcome more people from all over the world to join us and explore more reliable biometric technology in the future.