In this paper, we focused on SPCI and proposed a technique to solve the limitation of resolution in IP. Experimental verification based on the proposed method showed that the limitation could be eliminated. In addition, the possibility of application to IP was demonstrated by examining refocus image generation and colorization.
However, on the other hand, measurement time is a major issue in the field of SPCI. Conventional IP technology can capture a 3D image with a single shot, similar to a general imaging system. On the other hand, the proposed method uses SPCI, which requires many patterns to reconstruct one image. From this background, with the development of technology, digital micro mirror device (DMD), a device that can display patterns at high speed, or methods using compressive sensing and deep learning that can reconstruct images with a small number of images are being studied. Despite these studies, it is currently difficult to obtain high quality images in real time.
Under these circumstances, our laboratory is developing a high-speed correlation system using an optical correlation system. This technology is a system that enables correlation between the data recorded on the hologram disc and the input data at a very high speed, and we are currently studying the application of this to SPCI. By combining these technologies, we can expect a breakthrough in the field of SPCI as well as the proposed method.
References
[1] R. Raskar and J. Tumblin. Computational Photography Mastering New Techniques for Lenses,
Lighting, and Sensors. A. K. Peter Press, 2012.
[2] M. G. Lippmann. ´Epreuves, r´eversibles donnant la sensation du relief. Physical Review A, 4:821, 1908.
[3] J. H. Park, K. Hong, and B. Lee. Recent progress in three-dimensional information processing based on integral imaging. Applied Optics, 48:H77, 2009.
[4] X. Xiao, B. Javidi, M. M. Corral, and A. Stem. Advances in three-dimensional integral imaging: sensing, display, and applications. Applied Optics, 52:546, 2013.
[5] F. Okano, H. Hoshino, J. Arai, and I. Yuyama. Real-time pickup method for a three-dimensional image based on integral photography. Applied Optics, 36:1593, 1997.
[6] H. Hoshino, F. Okano, H. Isono, and I. Yuyama. Analysis of resolution limitation of integral photography. Journal of the Optical America A, 15:2059, 1998.
[7] J. Arai, F. Okano, H. Hoshino, and I. Yuyama. Gradient-index lens-array method based on real-time integral photography for three-dimensional images. Applied Optics, 37:2034, 1998. [8] J. Arai, M. Kawakita, T. Yamashita, H. Sasaki, M. Miura, H. Hiura, M. Okui, and F. Okano.
Integral three-dimensional television with video system using pixel-offset method. Optics Ex-
press, 21:3474, 2013.
[9] J. Arai, E. Nakasu, T. Yamashita, H. Hiura, M. Miura, T. Nakamura, and R. Funatsu. Progress overview of capturing method for integral 3-d imaging displays. Proceedings of the IEEE, 105:837, 2017.
[10] M. Kawakita, H. Sasaki, N. Okaichi, H. Watanabe, M. Kano, J. Arai, and T. Mishina. 3d tv based on integral photography. Proceedings of SPIE, 10666:166604, 2018.
[11] H. Sasaki, N. Okaichi, H. Watanabe, M. Kano, M. Miura, M. Kawakita, and T. Mishina. Color moir´e reduction and resolution enhancement of flat-panel integral three-dimensional display.
Optics Express, 27:8488, 2019.
[12] D. Abookasis and J. Rosen. Computer-generated holograms of three-dimensional objects syn- thesized from their multiple angular viewpoints. Journal of the Optical America A, 20:1537, 2003.
[13] J. H. Park, G. Baasantseren, N. Kim, G. Park, J. M. Kang, and B. Lee. View image generation in perspective and orthographic projection geometry based on integral imaging. Optics Express, 16:8800, 2008.
[14] M. F. Duarte, M. A. Davenport, D. Takhar, J. N. Laska, T. Sun, K. F. Kelly, and R. G. Baraniuk. Single-pixel imaging via compressive sampling. IEEE Signal Processing Magazine, 25:83, 2008.
[15] S. S. Welsh, M. P. Edgar, R. Bowman, P. Jonathan, B. Sun, and M. J. Padgett. Fast full-color computational imaging with single-pixel detectors. Optical Society of America, 21:23068, 2013. [16] Z. Zhang, S. Liu, J. Peng, M. Yao, G. Zheng, and J. Zhong. Simultanuous spatial, spectral, and
3d compressive imaging via efficient fourier single-pixel measurements. Optica, 5:315, 2018. [17] G. M. Gibson, B. Sun, M. P. Edgar, D. B. Phillips, N. Hempler, G. T. Maker, G. P. A. Malcolm,
and M. J. Padgett. Real-time imaging of methane gas leaks using a single-pixel camera. Optics
Express, 25:2998, 2017.
[18] R. I. Stantchev, B. Sun, S. M. Hornett, P. A. Hobson, G. M. Gibson, M. J. Padgett, and E. Hendry. Non-invasive, near-field terahertz imaging of hidden objects using a single pixel detector. Science advances, 2:e1600190, 2016.
[19] J. Greenberg, K. Krishnamurthy, and D. Brady. Compressive single-pixel snapshot x-ray diffrac- tion imaging. Optics Letters, 39:111, 2014.
[20] E. Tajahuerce, V. Dur´an, P. Clemente, E. Irles, F. Soldevila, P. Andr´es, and J. Lancis. Image transmission through dynamic scattering media by single-pixel photodetection. Optics Express, 22:16945, 2014.
[21] J. Peng, M. Yao, J. Cheng, Z. Zhang, S. Li, G. Zheng, and J. Zhong. Micro-tomography via single-pixel imaging. Optics Express, 26:31094, 2018.
[22] M. Yao, J. Cheng, Z. Huang, Z. Zhang, S. Li, J. Peng, and J. Zhong. Reflection light-field microscope with a digitally tunable aperture by single-pixel imaging. Optics Express, 27:33040, 2019.
[23] T. B. Pittman, Y. H. Shih, D. V. Strekalov, and A. V. Sergienko. Optical imaging by means of two-photon quantum entanglement. Physical Review A, 52(5):R3429, 1995.
[24] A. Gatti, E. Brambilla, and L. A. Lugiato. Entangled imaging and wave-particle duality: from the microscopic to the macroscopic realm. Physical Review Letters, 90:133603, 2003.
[25] A. Gatti, D. Magatti, M. Bache, E. Brambilla, F. Ferri, and L. A. Lugiato. High-resolution ghost image and ghost diffraction experiments with incoherent pseudo-thermal light. Proceedings of
SPIE, 5893:58930D, 2005.
[26] J. H. Shapiro. Computational ghost imaging. Physical Review A, 78:061802, 2008.
[27] K. Shibuya, K. Nakae, Y. Mizutani, and T. Iwata. Comparison of reconstructed images between ghost imaging and hadamard transform imaging. Optical Review, 22:897, 2015.
[28] Z. Zhang, X. Wang, G. Zheng, and J. Zhong. Hadamard single-pixel imaging versus fourier single-pixel imaging. Optics Express, 25:19619, 2017.
[29] O. Katz, Y. Bromberg, and Y. Silberberg. Compressive ghost imaging. Applied Physics Letters, 95:131110, 2009.
[30] R. Zhu, G. Li, and Y. Guo. Compressed-sensing-based gradient reconstruction for ghost imag- ing. International Journal of Theoretical Physics, 58:1215, 2019.
[31] M. P. Edgar, G. M. Gibson, and M. J. Padgett. Principles and prospects for single-pixel imaging. nature photonics, 13:13, 2019.
[32] L. Bian, J. Suo, Q. Dai, and F. Chen. Experimental comparison of single-pixel imaging algo- rithms. Journal of the Optical Society of America A, 35:78, 2018.
[33] B. Sun, M. P. Edgar, R. Bowman, L. E. Vittert, S. Welsh, A. Bowman, and M. J. Padgett. 3d computational imaging with single-pixel detectors. Science, 340:844, 2013.
[34] E. S. Balaguer, P. L. Carmona, C. Chabert, F. Pla, J. Lancis, and E. Tajahuerce. Low-cost single-pixel 3d imaging by using an led array. Optics Express, 26:15623, 2018.
[35] M. J. Sun, M. P. Edgar, G. M. Gibson, B. Sun, N. Radwell, R. Lamb, and M. J. Padgett. Single- pixel three-dimensional imaging with time-based depth resolution. nature communications, 7:12010, 2016.
[36] Z. Zhang and J. Zhong. Three-dimensional single-pixel imaging with far fewer measurements than effective image pixels. Optics Letters, 41:2497, 2016.
[37] E. S. Balaguer, P. Clemente, E. Tajahuerce, F. Pla, and J. Lancis. Full-color stereoscopic imaging with a single-pixel photodetector. Journal of Display Technology, 12:417, 2016. [38] L. M. Le´on, P. Clemente, Y. Mori, V. Climent, J. Lancis, and E. Tajahuerce. Single-pixel
digital holography with phase-encoded illumination. Optics Express, 25:4975, 2017.
[39] H. Gonz´alez, L. M. Le´on, F. Soldevila, M. A. Esquivel, J. Lancis, and E. Tajahuerce. High sam- pling rate single-pixel digital holography system employing a dmd and phase-encoded patterns.
Optics Express, 26:20342, 2018.
[40] R. J. Woodham. Photometric method for determining surface orientation from multiple images.
Optical Engineering, 19(1):191139, 1980.
[41] M. Matsumoto and T. Nishimura. Mersenne twister: A 623-dimensional equidistributed uni- form pserudorandom number generator. ACM Transactions on Modeling and Computer Sim-
ulation, 8:3, 1998.
[42] W. K. Pratt, J. Kane, and H. C. Andrews. Hadamard transform image coding. Proceedings of
the IEEE, 57:58, 1969.
[43] S. A. Martucci. Symmetric convolution and the discrete sine and cosine transforms. IEEE
Transactions of Signal Processing, 42:1038, 1994.
[44] F. Wang, H. Wang, H. Wang, G. Li, and G. Situ. Learning from simulation: An end-to-end deep-learning approach for computational ghost imaging. Optics Express, 27(18):25560, 2019. [45] M. Lyu, W. Wang, H. Wang, H. Wang, G. Li, N. Chen, and G. Situ. Deep-learning-based
ghost imaging. Scientific Reports, 7:17865, 2017.
[46] Y. He, G. Wang, G. Dong, S. Zhu, H. Chen, A. Zhang, and Z. Xu. Ghost imaging based on deep learning. Scientific Reports, 8:6469, 2018.
[47] Y. Wang, Y. Liu, J. Suo, G. Situ, C. Qiao, and Q. Dai. High speed computational ghost imaging via spatial sweeping. Scientific Reports, 87:45325, 2017.
[48] A. Woods, T. Docherty, and R. Koch. Image distortions in stereoscopic video systems. Pro-
[49] S. H. Hong, J. S. Jang, and Baharm Javidi. Three-dimensional volumetric object reconstruction using computational integral imaging. Optics Express, 12:483, 2004.
[50] B. Cho, P. Kopycki, M. M. Corral, and M. Cho. Computational volumetric reconstruction of integral imaging with improved depth resolution considering continuously non-uniform shifting pixels. Optics and Lasers in Engineering, 111:114, 2018.
Acknowledgments
I am grateful to many people for their help with this study. First, I would like to thank my research advisor, Professor Eriko Watanabe. She provided not only research guidance and support, but also guidance on what was necessary after coming to society and advice on future life. In addition, I was able to have a variety of experiences that I could not easily do, including presentations at conferences overseas and researches with other research institutions. I was really happy to work on my research in such a wonderful environment. Although it was a short period of three years, I was able to spend very densely and have a very useful experience in my future life. Thank you from the bottom of my heart.
I would also like to thank Dr. Tetsuhiko Muroi, Dr. Norihiko Ishii, Dr. Masato Miura, and Dr. Teruyoshi Nobukawa of Science & Technology Research Laboratories, Japan broadcasting corporation (NHK). They have been providing all kinds of support for about two years from the first year of the master’s program when this research started. At the time of the launch of this research, I was able to proceed with my research because I was able to have an active discussion even with almost no knowledge of this field. Thanks to these support, I was able to achieve achievements such as conference presentations and paper submissions.
As a senior in the lab, Dr. Kanami Ikeda, now an associate professor at Osaka Prefecture University, spent one year in the lab when I was in my fourth year of undergraduate study. I was given a lot of learning and supports. Even after graduating, I met at the conference venue and occasionally contacted me and talked about various things. I appreciate you sincerely.
I have been blessed with really good lab members for three years in this laboratory. Not only discussing about research, but also shared the painful and enjoyable things with these members, and they supported me in various aspects. Among them, Mr. Keisuke Saito, Mr. Katsutoshi Inomoto, and Ms. Ayano Inoue of the same period, who have been together for a long time, influenced me a lot. They each had an interesting personality, and I never get tired talking with them for a long time. In addition, many discussions were made in the field of research, and a variety of knowledge was gained. I was able to spend three years in the lab in a truly wonderful environment. I thank them.
Finally, I would like to thank my parents and brother for their understanding of progressing to the master’s degree, always warmly watching, sometimes giving advice, and for their support. This work would not be possible without their support. Thank you very much for everything.
This study was able to be summarized as it was really supported by so many people. I am immensely grateful for the supports to all of you.
Research achievements
Paper
[First author]
1. Ren Usami, Teruyoshi Nobukawa, Masato Miura, Norihiko Ishii, Eriko Watanabe, and Tet- suhiko Muroi: “Dense parallax image acquisition method using single-pixel imaging for inte- gral photography,” Optics Letters, Vol. 45, 25-28 (2020).
[Co-author]
1. Ayano Inoue, Ren Usami, Keisuke Saito, Yasunobu Honda, Kanami Ikeda, and Eriko Watan- abe: “Optical correlator-based computational ghost imaging towards high-speed computa- tional ghost imaging,” Japanese Journal of Applied Physics, Vol. 58, SKKA02 (2019).
International Conference
[First author]
1. Ren Usami, Hidenori Suzuki, Kanami Ikeda, and Eriko Watanabe: “Convolutional neural network based video identification system for content management system with optical corre- lator,” The Irago Conference (Tokyo, Japan, 2017. 11).
2. Ren Usami, Ayano Inoue, Keisuke Saito, Yasunobu Honda, Kanami Ikeda: “Reference Pat- terns for Optical Correlator-based Computational Ghost Imaging, ” 23rd Microoptics Confer- ence (Taipei, Taiwan, 2018. 10).
3. Ren Usami, Teruyoshi Nobukawa, Masato Miura, Norihiko Ishii, Eriko Watanabe, and Tet- suhiko Muroi: “Acquisition of Dense Parallax Images Using a Two-Dimensional Image Sensor by Applying Single-Pixel Imaging to Integral Photography,” International Symposium on Imaging, Sensing, and Optical Memory (ISOM) 2019 (Niigata, Japan, 2019. 10). Student
4. Ren Usami, Teruyoshi Nobukawa, Masato Miura, Norihiko Ishii, Eriko Watanabe, and Tet- suhiko Muroi: “Refocused image acquisition from high angular resolution parallax images in integral photography applying single-pixel imaging,” Photonics West 2020 (California, U.S.A., 2020. 2).
[Co-author]
1. Ayano Inoue, Ren Usami, Keisuke Saito, Yasunobu Honda, Kanami Ikeda, and Eriko Watan- abe: “Towards high-speed computational ghost imaging using holographic optical correlator,” International Symposium on Imaging, Sensing, and Optical Memory (ISOM) 2019 (Fukuoka, Japan, 2018. 10).
2. Yasunobu Honda, Ren Usami, Ayano Inoue, Keisuke Saito, and Eriko Watanabe: “Improve- ment of reconstructed image quality by optimization of binary random pattern in optical correlator-based computational ghost imaging ,” Information Phontonics 2019 (Kanagawa, Japan, 2019. 4). Domestic Conference [First author] 1. 宇佐美廉,鈴木秀典,池田佳奈美,渡邉恵理子, “Residual Networkを用いた改ざん動画検出,”平 成29年度第1回コンテンポラリーオプティクス研究会 (東京, 2017. 10). 優秀発表賞 2. 宇佐美廉, 鈴木秀典, 池田佳奈美, 渡邉恵理子, “光相関演算による動画識別システム-Residual Networkを用いた改ざん動画識別-,” 情報フォトニクス研究グループ第12回関東学生研究論文 講演会(東京, 2018. 3).
3. Ren Usami, Teruyoshi Nobukawa, Masato Miura, Norihiko Ishii, Eriko Watanabe, and Tet- suhiko Muroi, “Investigation of dense parallax color image acquisition using single-pixel imag- ing,” 日本光学会年次学術講演会 Optics & Photonics Japan (OPJ) 2019 (大阪, 2019. 12).
SPIE Student Prize
[Co-author]
1. 本多康伸,宇佐美廉,斎藤圭佑,猪上綾乃,池田佳奈美,渡邉恵理子, “高速計算機ゴーストイメー ジングに向けた光相関器の実験評価,” 日本光学会年次学術講演会 Optics & Photonics Japan (OPJ) 2018 (東京, 2018. 10).
2. 児玉周太朗,宇佐美廉, 井元克駿,渡邉恵理子, “位相シフトデジタルホログラフィによる拡散板
背後の顕微3次元イメージング,”情報フォトニクス研究グループ第13回関東学生研究論文講演
3. 本多康伸,宇佐美廉,斎藤圭佑,猪上綾乃,渡邉恵理子, “光相関器による計算機ゴーストイメージ ングの基礎評価,”情報フォトニクス研究グループ第13回関東学生研究論文講演会(東京, 2019. 3). Patent 1. 「光相関システムおよび光相関用データの記録方法」,渡邉恵理子,菅谷寿鴻,池田佳奈美,斎藤 圭佑,宇佐美廉(特願2018-060719) Awards 1. 平成29年度第1回コンテンポラリーオプティクス研究会 優秀発表賞【宇佐美廉,他, ”Residual Networkを用いた改ざん動画検出”】2017. 10優秀発表賞
2. International Symposium on Imaging, Sensing, and Optical Memory (ISOM) 2019 The Stu- dent Paper Award【Ren Usami, et al., “Acquisition of Dense Parallax Images Using a Two- Dimensional Image Sensor by Applying Single-Pixel Imaging to Integral Photography”】2019. 10 Student Award
3. 日本光学会年次学術講演会Optics & Photonics Japan (OPJ) 2019 Best Student Presentation Award【Ren Usami, et al., “Investigation of dense parallax color image acquisition using single-pixel imaging”】2019. 12 SPIE Student Prize