In our shape clustering method, the cluster number K and scaling parameter σ
are manually set. In future work, we can design an algorithm analyzing the initial shape S0, so that the cluster number and scaling parameter can be automatically
5.2 Future work 50
estimated. Furthermore, we wish to take rotation into account when perform- ing clustering, in order to get a more accurate presentation of reconstructability improvement in each subsequence.
In our spatial-temporal method, we have already noticed that a temporal smooth initialization is sometimes inferior to a rigid initialization. In future work, we would like to estimate the difficulty of recovering the correct rotation matrix, so that it can be automatically determined whether using a rigid initialization is a better option. We could also add rotation into the optimization process, thus to further improving the result.
References
[1] N. van der Aa, X. Luo, G. Giezeman, R. Tan, and R. Veltkamp, “Umpm benchmark: A multi-person dataset with synchronized video and motion cap- ture data for evaluation of articulated human motion and interaction,” in
Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference on, Nov 2011, pp. 1264–1269.
[2] M. Lee, J. Cho, C.-H. Choi, and S. Oh, “Procrustean normal distribution for non-rigid structure from motion,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2013, pp. 1280–1287.
[3] R. Garg, A. Roussos, and L. Agapito, “Dense variational reconstruction of non-rigid surfaces from monocular video,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2013, pp. 1272–1279.
[4] M. Brand, “A direct method for 3D factorization of nonrigid motion observed in 2D,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005, pp. 122–128.
[5] A. Del Bue, “A factorization approach to structure from motion with shape priors,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008, pp. 1–8.
[6] Y. Jingyu and M. Pollefeys, “A factorization-based approach for articulated nonrigid shape, motion and kinematic chain recovery from video,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 5, pp. 865–877, 2008.
[7] Y. Dai, H. Li, and M. He, “A simple prior-free method for non-rigid structure- from-motion factorization,” International Journal of Computer Vision, vol. 107, no. 2, pp. 101–122, 2014.
[8] M. Paladini, A. D. Bue, M. Stosic, M. Dodig, J. Xavier, and L. Agapito, “Fac- torization for non-rigid and articulated structure using metric projections,”
References 52
in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009, pp. 2898–2905.
[9] I. Akhter, Y. Sheikh, S. Khan, and T. Kanade, “Nonrigid structure from motion in trajectory space,” in Advances in Neural Information Processing Systems, 2008, pp. 41–48.
[10] P. Gotardo and A. Martinez, “Non-rigid structure from motion with comple- mentary rank-3 spaces,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011, pp. 3065–3072.
[11] C. Russell, J. Fayad, and L. Agapito, “Dense non-rigid structure from mo- tion,” in 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), 2012 Second International Conference on, Oct 2012, pp. 509– 516.
[12] C. Russell, R. Yu, and L. Agapito, “Video pop-up: Monocular 3d recon- struction of dynamic scenes,” in European Conference on Computer Vision. Springer, 2014, pp. 583–598.
[13] K. Fragkiadaki, M. Salas, P. Arbelaez, and J. Malik, “Grouping-based low-rank trajectory completion and 3d reconstruction,” in Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2014, pp. 55–63. [Online]. Available: http://papers.nips.cc/paper/ 5414-grouping-based-low-rank-trajectory-completion-and-3d-reconstruction. pdf
[14] C. Bregler, A. Hertzmann, and H. Biermann, “Recovering non-rigid 3D shape from image streams,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2000, pp. 690–696.
[15] J. Xiao, J. Chai, and T. Kanade, “A closed-form solution to non-rigid shape and motion recovery,” Int’l J. Computer Vision, vol. 67, no. 2, pp. 233–246, 2006.
[16] A. Del Bue, J. a. Xavier, L. Agapito, and M. Paladini, “Bilinear factoriza- tion via augmented lagrange multipliers,” in Proc. European Conf. Computer Vision, 2010, pp. 283–296.
[17] T. Simon, J. Valmadre, I. Matthews, and Y. Sheikh, Separable Spatiotemporal Priors for Convex Reconstruction of Time-Varying 3D Point Clouds. Cham:
References 53
Springer International Publishing, 2014, pp. 204–219. [Online]. Available: http://dx.doi.org/10.1007/978-3-319-10578-9 14
[18] I. Akhter, Y. Sheikh, and S. Khan, “In defense of orthonormality constraints for nonrigid structure from motion,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009, pp. 1534–1541.
[19] C. Tomasi and T. Kanade, “Shape and motion from image streams under orthography: a factorization method,” Int’l J. Computer Vision, vol. 9, no. 2, pp. 137–154, 1992.
[20] L. Torresani, A. Hertzmann, and C. Bregler, “Nonrigid structure-from-motion: Estimating shape and motion with hierarchical priors,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 5, pp. 878–892, 2008.
[21] H. S. Park, T. Shiratori, I. Matthews, and Y. Sheikh, “3d reconstruction of a moving point from a series of 2d projections,” in Proc. European Conf. Computer Vision, 2010, pp. 158–171.
[22] J. Valmadre and S. Lucey, “General trajectory prior for non-rigid reconstruc- tion,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2012, pp. 1394–1401.
[23] X. Llado, A. Del Bue, and L. Agapito, “Euclidean reconstruction of deformable structure using a perspective camera with varying intrinsic parameters,” in
ICPR, 2006, pp. 139–142.
[24] R. Vidal and D. Abretske, “Nonrigid shape and motion from multiple perspec- tive views,” in Proc. European Conf. Computer Vision, 2006, pp. 205–218. [25] R. Hartley and R. Vidal, “Perspective nonrigid shape and motion recovery,”
in Proc. European Conf. Computer Vision, 2008, pp. 276–289.
[26] X. Llad´o, A. D. Bue, and L. Agapito, “Non-rigid metric reconstruction from perspective cameras,” IVC, vol. 28, no. 9, pp. 1339–1353, 2010.
[27] Y. Zhu, D. Huang, F. De La Torre, and S. Lucey, “Complex non-rigid motion 3d reconstruction by union of subspaces,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2014, pp. 1542–1549.
[28] R. Yu, C. Russell, N. D. F. Campbell, and L. Agapito, “Direct, dense, and deformable: Template-based non-rigid 3d reconstruction from rgb video,” in
References 54
The IEEE International Conference on Computer Vision (ICCV), December 2015.
[29] J. Xiao, J. Chai, and T. Kanade, “A closed-form solution to non-rigid shape and motion recovery,” in Proc. European Conf. Computer Vision, vol. 3024, 2004, pp. 573–587.
[30] S. Vicente, J. Carreira, L. Agapito, and J. Batista, “Reconstructing pascal voc,” in Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Con- ference on, June 2014, pp. 41–48.
[31] L. Torresani, A. Hertzmann, and C. Bregler, “Learning non-rigid 3d shape from 2d motion,” in Advances in Neural Information Processing Systems 16, S. Thrun, L. Saul, and B. Sch¨olkopf, Eds. MIT Press, 2004, pp. 1555–1562. [Online]. Available: http://papers.nips.cc/paper/ 2509-learning-non-rigid-3d-shape-from-2d-motion.pdf
[32] Y. Zhu and S. Lucey, “Convolutional sparse coding for trajectory reconstruc- tion,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. PP, no. 99, pp. 1–1, 2013.
[33] H. S. Park, T. Shiratori, I. Matthews, and Y. Sheikh, “3D reconstruction of a moving point from a series of 2D projections,” in Proc. European Conf. Com- puter Vision, ser. Lecture Notes in Computer Science, K. Daniilidis, P. Mara- gos, and N. Paragios, Eds., vol. 6313. Springer Berlin / Heidelberg, 2010, pp. 158–171.
[34] A. Y. Ng, M. I. Jordan, and Y. Weiss, “On spectral clustering: Analysis and an algorithm,” in Advances in Neural Information Processing Systems, 2002, pp. 849–856. [Online]. Available: http://papers.nips.cc/paper/ 2092-on-spectral-clustering-analysis-and-an-algorithm.pdf
[35] R. Ranftl, V. Vineet, Q. Chen, and V. Koltun, “Dense monocular depth es- timation in complex dynamic scenes,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.