Scientists develop a new light field sensor for 3D scene construction with unprecedented angular resolution

(Nanowerk News) A research team from the National University of Singapore (NUS) Faculty of Science, led by Professor Liu Xiaogang from the Department of Chemistry, has developed a 3D imaging sensor that has extremely high angular resolution, which is the capacity of optical instruments to distinguish points of an object that separated by a small angular distance, namely 0.0018o. This innovative sensor operates on a unique angle-to-color conversion principle, which allows it to detect 3D light planes across X-rays to the visible light spectrum.

The light field includes a combination of the intensity and direction of the light beam, which can be processed by the human eye to precisely detect the spatial relationship between objects. However, traditional light sensing technologies are less effective. Most cameras, for example, can only produce two-dimensional images, which is sufficient for casual photography but insufficient for more sophisticated applications, including virtual reality, self-driving cars and biological imaging. This application requires the construction of a precise 3D scene from a certain space.

For example, a self-driving car can use light field sensing to see the road and more accurately assess road hazards so it can adjust its speed accordingly. Light field sensing also allows surgeons to accurately depict a patient’s anatomy at varying depths, allowing them to make more precise incisions and better assess a patient’s risk of injury.

“Currently, light field detectors use a series of lenses or photonic crystals to obtain multiple images of the same space from multiple angles. However, integrating these elements into a semiconductor for practical use is very complicated and expensive,” explained Prof Liu. “Conventional technologies can detect light fields only in the wavelength range of ultraviolet to visible light, leading to limited application in X-ray sensing.”

In addition, compared to other light field sensors such as the microlens array, the NUS team’s light field sensor has an angular measurement range greater than 80 degrees, a high angular resolution that is potentially less than 0.015 degrees for smaller sensors, and a wide spectral response range. wider between 0.002 nm and 550 nm. This specification makes the novel sensor capable of capturing 3D images with higher depth resolution. The large-scale angle sensing structure consisting of phosphors nanocrystals, a key component of the sensor, is illuminated under ultraviolet light. The three light-emitting phosphors producing red, green, and blue light are arranged in a pattern to capture detailed angular information which is then used for the construction of the 3D image. The team is also considering using other materials for the structure. (Image: NUS)

The breakthrough was published in a journal Natural (“X-ray-to-visible light field detection via pixelated color conversion”).

Made possible by perovskite nanocrystals

At the heart of the new light field sensors are inorganic perovskite nanocrystals – compounds that have excellent optoelectronic properties. Due to their controllable nanostructure, perovskite nanocrystals are efficient light emitters, with an excitation spectrum that spans X-rays to visible light. The interaction between the perovskite nanocrystals and the light beam can also be adjusted by carefully changing their chemical properties or by introducing small amounts of impurity atoms.

NUS researchers have patterned perovskite crystals onto a transparent thin film substrate and integrated it into a color charge-coupled device (CCD), which converts incoming light signals into color-coded outputs. This crystal converter system comprises the basic light field sensor functional unit.

When the incident light hits the sensor, the nanocrystals get excited. In turn, the perovskite unit emits its own light in different colors depending on the angle at which the incident light beam is emitted. The CCD captures the emitted colors, which can then be used for 3D image reconstruction.

“However, a single angle value is not sufficient to determine the absolute position of objects in three-dimensional space,” said Dr Yi Luying, Research Associate in the NUS Chemistry Department and first author of the paper. “We found that adding another base crystal converter unit perpendicular to the first detector and combining it with the designed optical system can provide more spatial information about the object in question.”

They then tested their light-field sensor in proof-of-concept experiments and found that their approach was indeed able to capture 3D images — with accurate reconstruction of depth and dimensions — of objects placed 1.5 meters away.

Their experiments also demonstrated the capacity of the new light field sensor to resolve even very fine details. For example, a precise image of a computer keyboard was created that captures even the shallow protrusions of each key. The image shows the design (left) and output (right) of a 3D light field sensor. The designed device (left) encodes light fields as color output. An array of patterned perovskite nanocrystals converts different directions of light into different colors, which can be detected by the device’s camera coupled with a color charge. The right image shows a reconstructed 3D depth image of the camera-generated Merlion model. (Image: Yi Luying)

Future discoveries

Prof Liu and his team are looking for methods to improve the spatial accuracy and resolution of their light field sensor, such as using high-end color detectors. The team has also filed an international patent for the technology.

“We will also explore more advanced technologies to fabricate denser perovskite crystal patterns onto transparent substrates, which can result in better spatial resolution. Using materials other than perovskite can also expand the detection spectrum of light field sensors,” said Prof Liu.