＂HD＂ imaging of non-line-of-sight targets

Written by | Tsinghua University Team

Description | This article is submitted by the author (research group).

In daily life, because light travels in a straight line, we can’t observe the scene out of sight.(Non-visual field), such as objects hidden behind corners, obstacles, etc.

Emerging in recent yearsNon-visual field imaging technology(noun explanation > > >) broke this restriction, and made the non-visual scene "come into view", and our vision was expanded unprecedentedly.

The principle of this technology to make light "turn" is:Laser is emitted to the middle interface (such as the wall in Figure 1) in the field of view, and photons are diffusely reflected at the middle interface (the first time), the non-visual target surface (such as the rabbit in Figure 1) and the middle interface (the second time) in turn. After the echo signal is collected by the detector, the non-visual scene is reconstructed through advanced algorithms.

Fig. 1 schematic diagram of non-visual field imaging principle

Non-line-of-sight imaging, a revolutionary technology, has important application prospects in many fields, such as automatic driving, disaster rescue, security and anti-terrorism, remote sensing reconnaissance and so on.However, in the process of non-line-of-sight imaging, the echo of laser after multiple diffuse reflections is extremely weak (only photon level), and the signal-to-noise ratio is extremely low. At the same time, the non-line-of-sight imaging problem has multiple solutions, which seriously affects the imaging quality. Especially for fast detection scenes, the objects reconstructed by existing algorithms often have very fuzzy boundaries, accompanied by strong noise, which brings great challenges to the development and future application of non-visual detection technology.

In view of this, fromTsinghua University Department of Precision Instruments, Tsinghua University Qiu Chengtong Mathematics Center.The research team of, proposed a method based on.Joint regularization of signal and target(Signal-object collaborative regularization,　SOCR) ofNon-horizon reconstruction method.

The result is "Non-line-of-sight reconstruction with signal–object collaborative regularization"published in the title.　Light: Science & Applications.

This method is based on the following assumptions:

(1) The reconstructed surface is sparse in the solution area.

(2) The local structure of the object to be measured is repeated many times in the whole target.

(3) The ideal echo signal is smooth.

By solving the optimization problem, the new algorithm can simultaneously obtain two-dimensional information of non-visual target: reflectivity and surface normal.

The author uses two goal-based adaptive learning methods.dictionary(noun explanation > > >), respectively describing the local structure and non-local correlation of objects, and the extracted object features can be used for further intelligent recognition and classification. Figure 2 shows the algorithm framework and the reflectivity reconstruction results of the pyramid.

The left side of the algorithm flow diagram in Figure 2 shows the local structure of the object obtained from the estimated signal and dictionary learning in the pyramid example, and the right side shows the initial solution, sparse solution and final reconstruction result of reflectivity.

In the photos taken by the new method, the non-visual objects have clear local details and boundary contours, and there is almost no background noise in the imaging area. In indoor/outdoor, confocal/non-confocal, long/short-term, multi-measuring points/few-measuring points, high-precision and high-definition imaging results have been obtained (see Figure 3). In addition, the framework has strong compatibility, and can be combined with other non-visual detection physical models to meet the needs of different imaging tasks.

Fig. 3 is a schematic diagram of partial reconstruction results. The first column is the object to be measured, the second column is the reflectance of reconstruction, and the third to fifth columns are the components of reconstruction results in horizontal direction, vertical direction and depth direction respectively.

This method can not only "shoot" high-definition photos, but also is obviously superior to the existing methods in error control. Fig. 4 compares the reconstruction error between the SOCR method proposed in this paper and the previous mainstream method (D-LCT), in which the white area is the distorted pixel. It can be seen that the number of distorted pixels is greatly reduced by 85%, the mean square error in depth direction is reduced by 41%, the maximum normal angle error is reduced by 56%, and the average normal angle error is reduced by 44%.

Fig. 4 Schematic diagram of pyramid depth and normal error A Wall view depth error C Wall view normal angle error of object B to be measured

Aiming at the problem that the noise of observation signal seriously affects the reconstruction quality in non-visual field imaging, this paper proposes an innovative reconstruction method based on the joint regularization of signal and detection target. By comprehensively considering the sparsity, non-local self-similarity and signal smoothness of the target to be detected, an important breakthrough has been made in the reconstruction quality of reflectivity and surface normal information of non-visual field target, which significantly improves the imaging accuracy of contour, reconstruction depth and normal angle of non-visual field target, especially for detection scenes with strong background light interference.

The "high definition" reconstruction method of non-line-of-sight targets proposed in this paper is universal and compatible, and it is expected to accelerate the practical application and promotion of non-line-of-sight imaging technology in the fields of automatic driving, disaster rescue, security and anti-terrorism, remote sensing reconnaissance and so on, which is expected!

Paper information:

Liu, X., Wang, J., Li, Z. et al. Non-line-of-sight reconstruction with signal–object collaborative regularization. Light Sci Appl 10, 198 (2021).

The correspondent of this paper is Fu Xing, associate professor of Tsinghua University Precision Instrument Department and Key Laboratory of Photonic Measurement and Control Technology, Ministry of Education, and Qiu Lingyun, assistant professor of Mathematical Science Center in Qiu Chengtong, Tsinghua University. The first authors of this paper are Tsinghua University doctoral students Liu Xintong and Wang Jianyu. This project is supported by the National Natural Science Foundation of China.

Paper address:

https://doi.org/10.1038/s41377-021-00633-3

The paper portal is here, please come in > > >

Editor | Shien Ding, Zhao Yang

Welcome the submission of the research group-press release

Article Reprint/Business Cooperation/Submission by Research Group, WeChat:447882024

Take you to read 1 document every day!Join >Light reading club