Metacognition is seen as the human’s capability to introspect their thought process and report their level of uncertainty/confidence in the course of learning. The metacognitive ability can be extremely useful in guiding behaviour during learning, in deciding whether to explore a new alternative or stick with the current one. In the past few years, the neuroscientific community has made some progress in understanding the neural basis of uncertainty/confidence representation. However, little is known about how uncertainty/confidence arises at the computational level during reinforcement learning. Here we propose to combine machine learning with behavioural data to characterise the exact computational steps that underlie the psychological construction of uncertainty during learning in complex environments, also aim to design a formal model for human’s state space learning process based on metacognition. The central aim of this work is to provide a mechanistic understanding of how uncertainty is constructed at the algorithmic level by the human brain and how it is used to drive learning.