State of the art

In the last decades various technological principles have been exploited to retrieve the 3D image of direct LOS scenes. The most promising approaches are based on stereo-vision, holography or ToF techniques, providing a better depth resolution when the three approaches are combined [202]. As demonstrated for medical imaging [203], underwater 3D reconstruction [67] and urban traf- fic [185], the stereo-vision 3D imaging is based on combining two 2D images seen from a different

(a)

(b) (c)

Figure 6.1: Schematic of stereovision approach for 3D imaging. a) Since the human eyes are spatially separated, the brain extrapolates the 3D information of the scene combining the 2D im- ages simultaneously seen from two different prospectives by the left and the right eye (b-c) by a triangulation process.

prospective by two sensors spatially separated. In more details, this approach relies on computing the disparity map D(xr, xl) given by the difference of the image coordinates between two corre-

sponding pixels xr and xlof the left and of the right image [204]. Figure 6.1 shows the holographic

approach.

Widely demonstrated in 3D microscopy [205], 3D scenes visualization [175] and retrieval of large scale objects [206], 3D holography is based on recovering the 3D image of the scene by the inter- ference pattern of a reference and a probe beam scattered from the object [186, 187]. In this case the 3D information of the scene is encoded in the phase of the intensity hologram captured using a Charged Coupled Device (CCD) or a Complementary Metal Oxide Semiconductor (CMOS) sen- sor. Figure 6.2 shows the holographic approach .

Widely investigated in radar imaging systems [207], recent results demonstrated the 3D recov- ery of LOS and NLOS scenes exploiting the ToF information of the return. Known as LiDAR, this approach combines the temporal information provided by the ToF of the return and the spa- tial information provided either using a pixelated sensor or scanning the scene. Each pixel of the

laser _BS M M lens lens CCD target

Figure 6.2: Schematic of holography approach for 3D imaging. A beam splitter (BS) divides the light beam in two separated beams, known as the object beam and the reference beam. The two beams are then expanded and the object beam illuminates the object to be retrieved. The light scattered by the object travels towards a recording medium where the reference and the object beam are recombined in an interference pattern. In digital holography, the hologram produced by the interference of the two beams is then recorded and stored by a CCD or CMOS sensor rather than a photographic film, as happens for classical holography.

sensor collects the return signal scattered back from a specific (x-y) portion of the transverse plan, encoding the (x-y) information of the scene. At the same time, the depth information is inferred by the time the light takes to go from the laser source to the target and then back to the sensor. A single-pixel is instead used in the scanning approach. Here, a laser spot scans the investigated scene and a single-pixel detector confocally collects the return signal.

The current 3D imaging technologies based on the ToF information rely on either a pulsed or con- tinuous wave (CW) modulated illumination. As reported in Fig. 6.3, the pulsed light approach recovers the ToF information measuring the temporal difference between the arrival-time of the return signal and the arrival-time of a reference. In the CW modulation approach, the ToF infor- mation is instead recovered measuring the phase difference between the emitted and the received signal, as discussed in Chapter 3. The current CW modulation ToF cameras technologies available on the market can reach a rate of 120 frames per second with a centimetre resolution and a distance range between 0.5 and 10 metres [208].

Due to the recent advances in single photon detection and TCSPC techniques [2], the pulsed modulation ToF approach is one of the most common method used to retrieve the full 3D im- age of LOS and NLOS scenes, providing a better depth resolution when combined with the other suggested approaches [202]. This approach can be further improved exploiting the advantage of

pulsed laser

single-photon sensitive camera

PC TCSPC

Figure 6.3: Schematic of ToF approach for optical 3D imaging by pulsed light modulation. In order to recover the 3D image of a scene, a pulsed light source flood illuminates the scene to be recovered. A single-photon sensitive camera operating in time-correlated single-photon counting (TCSPC) mode collects the return signal scattered back from the objects. The depth of the scene is then inferred by the ToF of the return.

emitted signal received light m m m m 0 1 2 3 distance light emitter CMOS sensor

Figure 6.4: Schematic of ToF approach for optical 3D imaging by CW modulation. In order to recover the 3D image of a scene, an infra-red sinusoidal modulated beam is flood illuminating the scene to be recovered. A pixelated CMOS sensor collects the return signal. The phase difference is then obtained measuring the amount of light reflected at four points m0, m1, m2, m3equally disposed

using a single-pixel camera [57,69]. The current 3D imaging technologies can be further improved applying deep learning techniques, as demonstrated in 3D microscopy [144], super-resolved fluo- rescence microscopy [209] and lensless imaging [163].

Despite the recent advances in computational imaging algorithms and single-photon sensitive de- tectors, the ToF approach requires either many consecutive measurements as it happens for scan- ning systems, or the use of a pixelated sensor in order to recover the spatial information on the transverse plane, providing prior knowledge about the scene to be recovered. Therefore, the cur- rent 3D imaging sensors are usually made by many-pixels or require a scanning imaging system not compatible with compact and portable systems for remote sensing applications. Moreover, the 3D retrieval of a scene requires as many consecutive measurements as the transverse spatial resolution of the image, leading to lower data acquisition and data transfer rate. We describe the ToF detection methods that can be used to retrieve the 3D image of scenes by pulsed illumination, focusing the attention on pixelated detector systems.