Journal of Zhejiang University SCIENCE A
ISSN 1673-565X(Print), 1862-1775(Online), Monthly

2009   Vol. 10   No. 6   p. 810~819

Bi-dimension decomposed hidden Markov models for multi-person activity recognition

Wei-dong ZHANG^†, Feng CHEN, Wen-li XU

(Department of Automation, Tsinghua University, Beijing 100084, China)
^†E-mail: zwd03@mails.tsinghua.edu.cn
Received May 22, 2008; revision accepted Oct. 18, 2008; Crosschecked Apr. 10, 2009

Abstract: We present a novel model for recognizing long-term complex activities involving multiple persons. The proposed model, named ‘decomposed hidden Markov model’ (DHMM), combines spatial decomposition and hierarchical abstraction to capture multi-modal, long-term dependent and multi-scale characteristics of activities. Decomposition in space and time offers conceptual advantages of compaction and clarity, and greatly reduces the size of state space as well as the number of parameters. DHMMs are efficient even when the number of persons is variable. We also introduce an efficient approximation algorithm for inference and parameter estimation. Experiments on multi-person activities and multi-modal individual activities demonstrate that DHMMs are more efficient and reliable than familiar models, such as coupled HMMs, hierarchical HMMs, and multi-observation HMMs.

Key words: Multi-channel setting, Hierarchical modeling, Hidden Markov model, Activity recognition
doi:10.1631/jzus.A0820388             CLC number: TP391.4

References:

[1] Brand, M., Oliver, N., Pentland, A., 1997. Coupled Hidden Markov Models for Complex Action Recognition. Proc. CVPR, p.994-999.

[2] Bui, H.H., Venkatesh, S., West, G., 2002. Policy recognition in the abstract hidden Markov model. J. Artif. Intell. Res., 17:451-499.

[3] Du, Y., Chen, F., Xu, W., 2007. Human interaction representation and recognition through motion decomposition. IEEE Signal Processing Lett., 14(12):952-955.

[4] Du, Y., Chen, F., Xu, W., Zhang, W., 2008. Activity recognition through multi-scale motion detail analysis. Neurocomputing, 71:3561-3574.

[5] Fine, S., Singer, Y., Tishby, N., 1998. The hierarchical hidden Markov model: analysis and applications. Mach. Learning, 32(1):41-62.

[6] Forster, M., 2000. Key concepts in model selection performance and generalizability. J. Math. Psychol., 44:205-231.

[7] Ghahramani, Z., 2001. An introduction to hidden Markov models and Bayesian networks. Int. J. Pattern Recogn. Artif. Intell., 15(1):9-42.

[8] Gong, S., Xiang, T., 2003. Recognition of Group Activities Using Dynamic Probabilistic Networks. Proc. ICCV, p.742-749.

[9] Intille, S.S., Bobick, A.F., 2001. Recognizing planned, multi-person action. Comput. Vis. Image Underst., 81(3):414- 445.

[10] Liu, X.H., Chua, C.S., 2006. Multi-agent activity recognition using observation decomposed hidden Markov models. Image Vis. Comput., 24:166-175.

[11] Moeslund, T.B., Hilton, A., Krüger, V., 2006. A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst., 104(2-3):90-127.

[12] Murphy, K.P., 2002. Dynamic Bayesian Networks: Representation, Inference and Learning. PhD Thesis, University of California, Berkeley, USA.

[13] Murphy, K.P., Paskin, M., 2001. Linear Time Inference in Hierarchical HMMs. Proc. NIPS, p.833-840.

[14] Nguyen, N., Phung, D., Venkatesh, S., Bui, H.H., 2005. Learning and Detecting Activities from Movement Trajectories Using the Hierarchical Hidden Markov Model. Proc. CVPR, p.955-960.

[15] Oliver, N., Garg, A., Horvitz, E., 2004. Layered representations for learning and inferring office activity from multiple sensory channels. Comput. Vis. Image Underst., 96(2):163-180.

[16] Schwarz, G., 1978. Estimating the dimension of a model. Ann. Statist., 6(2):461-464.

[17] Wada, T., Matsuyama, T., 2000. Multiobject behavior recognition by event driven selective attention method. IEEE Trans. PAMI, 22(8):873-887.

[18] Zacks, J.Z., Tversky, B., 2001. Event structure in perception conception. Psychol. Bull., 127(1):3-21.

[19] Zhang, D., Gatica-Perez, D., Bengio, S., McCowan, I., 2006. Modeling individual and group actions in meetings with layered HMMs. IEEE Trans. Multim., 8(3):509-520.

[20] Zhang, W., Chen, F., Xu, W., Zhang, E., 2006. Real-time Video Intelligent Surveillance System. Proc. ICME, p.1021-1024.

[21] Zhang, W., Chen, F., Xu, W., Cao, Z., 2007. Decomposition in Hidden Markov Models for Activity Recognition. Proc. MCAM, p.232-241.

[22] Zhang, W., Chen, F., Xu, W., Du, Y., 2008. Hierarchical group process representation in multi-agent activity recognition. Signal Processing: Image Commun., 23(10):739-753.