Risk-sensitive Markov Decision Process for Underground Construction Planning and Estimating
This paper presents an application of dynamic decision making under uncertainty in planning and estimating underground construction. The application of the proposed methodology is illustrated by its application to an actual tunneling project—The Hanging Lake Tunnel Project in Colorado, USA. To encompass the typical risks in underground construction, tunneling decisions are structured as a risk-sensitive Markov decision process that reflects the decision process faced by a contractor in each tunneling round. This decision process consists of five basic components: (1) decision stages (locations), (2) system states (ground classes and tunneling methods), (3) alternatives (tunneling methods), (4) ground class transition probabilities, and (5) tunneling cost structure. The paper also presents concepts related to risk preference that are necessary to model the contractor’s risk attitude, including the lottery concept, utility theory, and the delta property. The optimality equation is formulated, the model components are defined, and the model is solved by stochastic dynamic programming. The main results are the optimal construction plans and risk-adjusted project costs, both of which reflect the dynamics of subsurface construction, the uncertainty about geologic variability as a function of available information, and the contractor’s risk preference.