image: Fig.1 Concept of pluggable diffractive neural networks (P-DNN) for multiple tasks. P-DNN is composed of common layers (marked in red) and pluggable layers (marked in blue and green, respectively). view more
Credit: OEA
A new publication from Opto-Electronic Advances, 10.29026/oea.2024.230005 discusses Pluggable multitask diffractive neural networks based on cascaded metasurfaces.
Deep learning algorithms based on artificial neural networks are completely changing information processing methods in many scientific and engineering fields. Currently, it has been applied to many specific tasks, including image classification, image encryption, speech recognition, and language translation. However, With the rapid development of artificial intelligence algorithms, there is a huge contradiction between the computing power required by artificial neural networks and the computing power provided by electronic chips. The slowdown of Moore's Law in the "post-Moore era" and the limitations of von Neumann architecture have resulted in high energy consumption and time consumption issues for existing computing architecture.
With the advantages of high computing speed, high parallelism, and low energy consumption during transmission, the diffractive neural networks with light as the medium can simulate the artificial neural networks for high-speed computing, providing a solution to the problem of computing power and energy consumption. Recently, all-optical diffractive neural networks have been implemented and validated in imaging processing and object recognition. Diffractive neural networks are typically constructed by several diffractive layers, with each unit structure inside being viewed as a neuron, and the interconnections between neurons across different layers are realized through light diffractive. Diffractive neural networks have been widely researched in various tasks, including image recognition, linear matrix operations, logical operations, and beam shaping.
Currently, diffractive neural networks are usually implemented in the terahertz and microwave bands, but it is still difficult to achieve integration miniaturization and lack of reconfigurability. Compared with traditional diffractive optical elements, metasurfaces are more compact in the optical band and can be used to simultaneously control the amplitude and phase of light by changing the shape, size, and arrangement of the meta-atoms inside the metasurfaces. The implementation of diffractive neural networks using metasurfaces helps to obtain miniaturized intelligent integrated optical devices.
The authors of this article propose the pluggable diffractive neural networks (P-DNN) to address the reconfigurability issue of diffractive neural networks. Combined with the transfer learning algorithm, the phase parameters of internal plug-ins can be trained, and multiple recognition tasks, such as handwritten digit recognition and fashion recognition, can be achieved by switching the pluggable components in the networks. As shown in Fig. 1, the modulation layer of the pluggable diffractive neural networks can be divided into two parts: the shared layer for preprocessing input information and the pluggable layer for switching multiple tasks. The input information is modulated using a specific shape mask, and the incident light is focused onto a sub-detecting area on the detecting plane after passing through the pluggable diffractive neural networks, and the object classification is determined by the energy distribution. In the case of fixed shared layer plug-ins, other recognition tasks can be achieved, such as switching from handwritten digit recognition to fashion recognition.
To validate the effectiveness of the proposed method, the research team designed a two-layer cascaded metasurfaces operating at 800 nm near-infrared band, demonstrating the ability of the pluggable diffractive neural networks for handwritten digit and fashion recognition tasks. The experimental classification accuracies of handwritten digits and fashions classification tasks exceed 91.3% and 90.0%, respectively (Fig.2a, b). By analyzing the energy distribution in the sub-detecting areas of the detecting plane, when the shared layer plug-in is fixed, the handwritten digit plug-in and the fashion plug-in are used respectively, and the handwritten digits and fashions can be accurately identified according to the energy distribution. (Fig. 2c, d).
The pluggable diffractive neural networks are a universal model that can be applied to various classification tasks, improving the flexibility of networks design while effectively reducing the consumption of computing resources and training time. The good reconfigurability of the pluggable diffractive neural networks provides a solution to the problem of the lack of reconfigurability in diffractive neural networks, and the use of metasurfaces helps to achieve device integration and miniaturization. In the future, the pluggable diffractive neural networks based on metasurfaces can be used as optical integrated components for different functional AI systems, providing low-energy and high-speed computing for specific tasks, such as real-time object detecting in autonomous driving systems and intelligent optical filtering in microscope imaging.
# # # # # #
The research group of Prof. Lingling Huang from Beijing Institute of Technology is committed to the frontier and applied research in the fields of micro-nano optics, diffractive optics, and holographic imaging, and has undertaken the National Key R&D Program of China, National Natural Science Foundation of China program, and Beijing Outstanding Young Scientist Program, etc.
Professor Lingling Huang is a professor in Beijing Institute of Technology, China. She received double B.S. degree (both Science and Engineering B.S. degree) in Optoelectronics from Tianjin University and Nankai University, in 2009. And she received Ph. D. degree in Optical Engineering from Tsinghua University, in 2014. Her research activities are focused on nanophotonics and optical metasurfaces. She is a Most Cited Chinese Researcher of Elsevier. She has been selected as Young Scholar of Changjiang River, Beijing Outstanding Young Scientist, et al. And she has been granted with Friedrich Wilhelm Bessel Research Award from Alexander von Humboldt Foundation, Germany, Youth Science and Technology Award from Ministry of Education, China, etc.
He Cong is a master's student in Beijing Institute of Technology. His research direction mainly includes diffractive neural networks, micro-nano optical metasurfaces, and related research work was published in Opto-Electronic Advances.
# # # # # #
Opto-Electronic Advances (OEA) is a high-impact, open access, peer reviewed monthly SCI journal with an impact factor of 8.933 (Journal Citation Reports for IF2021). Since its launch in March 2018, OEA has been indexed in SCI, EI, DOAJ, Scopus, CA and ICI databases over the time and expanded its Editorial Board to 36 members from 17 countries and regions (average h-index 49).
The journal is published by The Institute of Optics and Electronics, Chinese Academy of Sciences, aiming at providing a platform for researchers, academicians, professionals, practitioners, and students to impart and share knowledge in the form of high quality empirical and theoretical research papers covering the topics of optics, photonics and optoelectronics.
# # # # # #
More information: http://www.oejournal.org/oea
Editorial Board: http://www.oejournal.org/oea/editorialboard/list
All issues available in the online archive (http://www.oejournal.org/oea/archive).
Submissions to OEA may be made using ScholarOne (https://mc03.manuscriptcentral.com/oea).
ISSN: 2096-4579
CN: 51-1781/TN
Contact Us: oea@ioe.ac.cn
Twitter: @OptoElectronAdv (https://twitter.com/OptoElectronAdv?lang=en)
WeChat: OE_Journal
# # # # # #
He C, Zhao D, Fan F, Zhou HQ, Li X et al. Pluggable multitask diffractive neural networks based on cascaded metasurfaces. Opto-Electron Adv 7, 230005 (2024). doi: 10.29026/oea.2024.230005
# # # # # #
Journal
Opto-Electronic Advances