A linearization-based hybrid approach for 3D reconstruction of objects in a single image

Kotan, Muhammed ; Öz, Cemil ; Kahraman, Abdulgani

International Journal of Applied Mathematics and Computer Science, Tome 31 (2021) no. 3, pp. 501-513.

Résumé

The shape-from-shading (SFS) technique uses the pattern of shading in images in order to obtain 3D view information. By virtue of their ease of implementation, linearization-based SFS algorithms are frequently used in the literature. In this study, Fourier coefficients of central differences obtained from gray-level images are employed, and two basic linearization-based algorithms are combined. By using the functionally generated surfaces and 3D reconstruction datasets, the hybrid algorithm is compared with linearization-based approaches. Five different evaluation metrics are applied on recovered depth maps and the corresponding gray-level images. The results on defective sample surfaces are also included to show the effect of the algorithm on surface reconstruction. The proposed method can prevent erroneous estimates on object boundaries and produce satisfactory 3D reconstruction results in a low number of iterations.

Détail

Keywords: 3D reconstruction, shape from shading, shape recovery, image comparison, surface inspection
Mots-clés : rekonstrukcja 3D, odzyskiwanie kształtu, porównywanie obrazów, inspekcja powierzchni

Bibliographie

[1] [1] Abada, L. and Aouat, S. (2015). Tabu search to solve the shape from shading ambiguity, International Journal on Artificial Intelligence Tools 24(05): 1550035.

[2] [2] Abada, L. and Aouat, S. (2016). A machine learning approach for shape from shading, arXiv 1607.03284.

[3] [3] Ascher, U.M. and Carter, P.M. (1993). A multigrid method for shape from shading, SIAM Journal on Numerical Analysis 30(1): 102–115.

[4] [4] Barron, J.T. and Malik, J. (2011). High-frequency shape and albedo from shading using natural image statistics, Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 11), Colorado Springs, USA, pp. 2521–2528.

[5] [5] Bednarık, J., Fua, P. and Salzmann, M. (2018). Learning shape-from-shading for deformable surfaces, arXiv 1803.08908.

[6] [6] Breuß, M., Cristiani, E., Durou, J.-D., Falcone, M. and Vogel, O. (2010). Numerical algorithms for perspective shape from shading, Kybernetika 46(2): 207–225.

[7] [7] Breuß, M. and Yarahmadi, A.M. (2020). Perspective shape from shading, in J.D. Durou et al. (Eds), Advances in Photometric 3D-Reconstruction, Springer, Cham, pp. 31–72.

[8] [8] Cadavid, S. and Abdel-Mottaleb, M. (2008). 3-D ear modeling and recognition from video sequences using shape from shading, IEEE Transactions on Information Forensics and Security 3(4): 709–718.

[9] [9] Chen, Z.-M., Cao, J.-Z. and Huang, J.-Q. (2010). A novel 3D reconstruction algorithm based on hybrid immune particle swarm optimization, Proceedings of the 29th Chinese Control Conference, Beijing, China, pp. 5228–5231.

[10] [10] Ciaccio, E.J., Bhagat, G., Lewis, S.K. and Green, P.H. (2017). Use of shape-from-shading to characterize mucosal topography in celiac disease videocapsule images, World Journal of Gastrointestinal Endoscopy 9(7): 310.

[11] [11] Ciecierski, K.A. (2020). Mathematical methods of signal analysis applied in medical diagnostic, International Journal of Applied Mathematics and Computer Science 30(3): 449–462, DOI: 10.34768/amcs-2020-0033.

[12] [12] Durou, J.-D., Falcone, M. and Sagona, M. (2008). Numerical methods for shape-from-shading: A new survey with benchmarks, Computer Vision and Image Understanding 109(1): 22–43.

[13] [13] Fanany, M.I. and Kumazawa, I. (2004). A neural network for recovering 3D shape from erroneous and few depth maps of shaded images, Pattern Recognition Letters 25(4): 377–389.

[14] [14] Fanany, M.I., Ohno, M. and Kumazawa, I. (2002). A scheme for reconstructing face from shading using smooth projected polygon representation NN, International Conference on Image Processing, Rochester, USA, Vol. 2, pp. II–II.

[15] [15] Franchini, S., Gentile, A., Vassallo, G. and Vitabile, S. (2020). Implementation and evaluation of medical imaging techniques based on conformal geometric algebra, International Journal of Applied Mathematics and Computer Science 30(3): 415–433, DOI: 10.34768/amcs-2020-0031.

[16] [16] Frankot, R.T. and Chellappa, R. (1988). A method for enforcing integrability in shape from shading algorithms, IEEE Transactions on Pattern Analysis and Machine Intelligence 10(4): 439–451.

[17] [17] Gallen, R., Eastop, D., Bozia, E. and Barmpoutis, A. (2015). Digital imaging: The application of shape-from-shading to lace, seals and metal objects, Journal of the Institute of Conservation 38(1): 41–53.

[18] [18] Ghayourmanesh, S. and Zahng, Y. (2007). Shape from shading of SAR imagery in Fourier space, 2007 IEEE International Geoscience and Remote Sensing Symposium, Barcelona, Spain, pp. 835–837.

[19] [19] Grosse, R., Johnson, M.K., Adelson, E.H. and Freeman, W.T. (2009). Ground truth dataset and baseline evaluations for intrinsic image algorithms, 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan, pp. 2335–2342.

[20] [20] Haefner, B., Quéau, Y., Möllenhoff, T. and Cremers, D. (2018). Fight ill-posedness with ill-posedness: Single-shot variational depth super-resolution from shading, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, pp. 164–174.

[21] [21] Han, F. and Zhu, S.-C. (2005). Cloth representation by shape from shading with shading primitives, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, USA, Vol. 1, pp. 1203–1210.

[22] [22] Horn, B.K. (1970). Shape from Shading: A Method for Obtaining the Shape of a Smooth Opaque Object from One View, PhD thesis, Massachusetts Institute of Technology, Cambridge.

[23] [23] Hu, Q., Shou, Z., He, L., Cai, Q., Qu, M. and Zhang, Y. (2019). Three-dimensional characterization method of pile-rock interface roughness based on fractal geometry, Arabian Journal of Geosciences 12(18): 599.

[24] [24] Kazmi, I.K., You, L. and Zhang, J.J. (2016). A hybrid approach for character modeling using geometric primitives and shape-from-shading algorithm, Journal of Computational Design and Engineering 3(2): 121–131.

[25] [25] Kemelmacher-Shlizerman, I. and Basri, R. (2010). 3D face reconstruction from a single image using a single reference face shape, IEEE Transactions on Pattern Analysis and Machine Intelligence 33(2): 394–405.

[26] [26] Kong, F.-H. (2008). A new method of inspection based on shape from shading, 2008 Congress on Image and Signal Processing, Sanya, China, Vol. 2, pp. 291–294.

[27] [27] Kotan, M. and Öz, C. (2017). Surface inspection system for industrial components based on shape from shading minimization approach, Optical Engineering 56(12): 123105.

[28] [28] Lu, J., Zhang, S., Shi, L., Hou, D. and Wang, X. (2018). Automatic correction of the adverse effects of light on fruit surfaces using the shape-from-shading method, Czech Journal of Food Sciences 36(1): 37–43.

[29] [29] Maurer, D., Ju, Y.C., Breuß, M. and Bruhn, A. (2018). Combining shape from shading and stereo: A joint variational method for estimating depth, illumination and albedo, International Journal of Computer Vision 126(12): 1342–1366.

[30] [30] Pentland, A. (1989). Shape information from shading: A theory about human perception, Spatial Vision 4(2–3): 165–182.

[31] [31] Ping-Sing, T. and Shah, M. (1994). Shape from shading using linear approximation, Image and Vision Computing 12(8): 487–498.

[32] [32] Pradhan, R., Ghose, M. and Jeyaram, A. (2010). Extraction of depth elevation model (DEM) from high resolution satellite imagery using shape from shading approach, International Journal of Computer Applications 7(12): 40–46.

[33] [33] Quéau, Y., Durou, J.-D. and Aujol, J.-F. (2018). Variational methods for normal integration, Journal of Mathematical Imaging and Vision 60(4): 609–632.

[34] [34] Quéau, Y., Mélou, J., Castan, F., Cremers, D. and Durou, J.-D. (2017). A variational approach to shape-from-shading under natural illumination, International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition, Venice, Italy, pp. 342–357.

[35] [35] Sakarya, U. and Erkmen, İ. (2003). An improved method of photometric stereo using local shape from shading, Image and Vision Computing 21(11): 941–954.

[36] [36] Salary, R.R., Lombardi, J.P., Rao, P.K. and Poliks, M.D. (2017). Online monitoring of functional electrical properties in aerosol jet printing additive manufacturing process using shape-from-shading image analysis, Journal of Manufacturing Science and Engineering 139(10): 101010.

[37] [37] Song, K. and Yan, Y. (2013). A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects, Applied Surface Science 285(B): 858–864.

[38] [38] Tozza, S. and Falcone, M. (2016). Analysis and approximation of some shape-from-shading models for non-Lambertian surfaces, Journal of Mathematical Imaging and Vision 55(2): 153–178.

[39] [39] Turan, M., Almalioglu, Y., Araujo, H., Konukoglu, E. and Sitti, M. (2017). A non-rigid map fusion-based direct SLAM method for endoscopic capsule robots, International Journal of Intelligent Robotics and Applications 1(4): 399–409.

[40] [40] Wang, C., Wang, C., Qin, H. and Zhang, T.-y. (2017). Video-based fluid reconstruction and its coupling with SPH simulation, The Visual Computer 33(9): 1211–1224.

[41] [41] Wang, G., Zhang, X. and Cheng, J. (2020). A unified shape-from-shading approach for 3D surface reconstruction using fast eikonal solvers, International Journal of Optics 2020(8): 1–12.

[42] [42] Wilson, D. and Laxminarayan, S. (2006). Handbook of Biomedical Image Analysis: Volume 1: Segmentation Models, Springer Science Business Media, New York.

[43] [43] Worthington, P.L. and Hancock, E.R. (2001). Surface topography using shape-from-shading, Pattern Recognition 34(4): 823–840.

[44] [44] Wu, B., Li, F., Hu, H., Zhao, Y.,Wang, Y., Xiao, P., Li, Y., Liu, W.C., Chen, L., Ge, X., Yang, M., Xu, Y., Ye, Q., Wu, X. and Zhang, H. (2020). Topographic and geomorphological mapping and analysis of the Chang’e-4 landing site on the far side of the moon, Photogrammetric Engineering Remote Sensing 86(4): 247–258.

[45] [45] Xiong, Y., Chakrabarti, A., Basri, R., Gortler, S.J., Jacobs, D.W. and Zickler, T. (2014). From shading to local shape, IEEE Transactions on Pattern Analysis and Machine Intelligence 37(1): 67–79.

[46] [46] Yamany, S.M. and Farag, A.A. (1998). A system for human jaw modeling using intra-oral images, Proceedings of the 20th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Hong Kong, China, Vol. 2, pp. 563–566.

[47] [47] Yang, D. and Deng, J. (2018). Shape from shading through shape evolution, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, pp. 3781–3790.

[48] [48] Yang, L., Li, E., Long, T., Fan, J., Mao, Y., Fang, Z. and Liang, Z. (2018). A welding quality detection method for arc welding robot based on 3D reconstruction with SFS algorithm, International Journal of Advanced Manufacturing Technology 94(1–4): 1209–1220.

[49] [49] Yuen, S.Y., Tsui, Y.Y. and Chow, C.K. (2007). A fast marching formulation of perspective shape from shading under frontal illumination, Pattern Recognition Letters 28(7): 806–824.

[50] [50] Zhang, R., Tsai, P.-S., Cryer, J.E. and Shah, M. (1999). Shape-from-shading: A survey, IEEE Transactions on Pattern Analysis and Machine Intelligence 21(8): 690–706.

[51] [51] Zheng, Q. and Chellappa, R. (1991). Estimation of illuminant direction, albedo, and shape from shading, IEEE Transactions on Pattern Analysis and Machine Intelligence 13(7): 680–702.