Engineering Research Seminar 11, 2012: Bellman's Principle with Inequality Constraints

Professor Edwin Chong, Department of Electrical and Computer Engineering, Colorado State University

Date: 2012-06-06
Time: 11:00 to 12:00
Venue: 5-4-22, Engineering meeting room 1


We consider an example by Haviv (1996) of a constrained Markov decision process that, in some sense, violates Bellman's principle. We resolve this issue by showing how to preserve a form of Bellman's principle that accounts for a change of constraint at states that are reachable from the initial state. We illustrate some key ideas using examples, including the following. Suppose you set a goal of saving for retirement with some specific constraint on how much you will need for a comfortable post-career life. If you win the lottery, how should your subsequent spending and saving decisions change?

About the Speaker

Professor Edwin K. P. Chong was born in Kuala Lumpur. He received the B.E.(Hons.) degree with First Class Honors from the University of Adelaide, South Australia, in 1987; and the M.A. and Ph.D. degrees in 1989 and 1991, respectively, both from Princeton University, where he held an IBM Fellowship. He joined the School of Electrical and Computer Engineering at Purdue University in 1991, where he was named a University Faculty Scholar in 1999. Since August 2001, he has been a Professor of Electrical and Computer Engineering and a Professor of Mathematics at Colorado State University. His current interests are in stochastic models and optimization methods. He co-authored the recent best-selling book, An Introduction to Optimization, 3rd Edition, Wiley-Interscience, 2008. He is an inaugural Senior Editor of the IEEE Transactions on Automatic Control, and is also on the editorial board of Computer Networks and the Journal of Control Science and Engineering. Professor Chong is a Fellow of the IEEE, and served as an IEEE Control Systems Society Distinguished Lecturer. He received the NSF CAREER Award in 1995 and the ASEE Frederick Emmons Terman Award in 1998. He was a co-recipient of the 2004 Best Paper Award for a paper in the journal Computer Networks. In 2010 he received the IEEE Distinguished Member Award from the Control Systems Society.