Most of the complexity derives from the fact that, because the benefits of information are only indirect, computing its value requires planning across a sequence of steps. Moreover, this planning requires not only a simple knowledge of the order of various steps, but a sophisticated model of the task structure that specifies the hidden (causal) relationships between consecutive steps. Consider for example the simple act of directing gaze to the water faucet while preparing

a tea (Figure 2A). To generate this apparently trivial act, the brain must know not only that the faucet is associated with the task (after all, so are the kitchen floor and the walls) but that lifting the handle will cause the water to flow, which in turn will have selleck chemicals llc Selleck Talazoparib a determining influence on preparing the tea. In other words, to determine which sources of uncertainty should be optimally resolved, the brain must know which steps are causal or predictive of the future outcome ( Gershman and Niv, 2010). In a simple scenario such as making a tea this computation may be greatly aided by extensive practice. In other behaviors, however, it requires much more difficult inferences on longer time scales. It can be prohibitively complex for example, to determine which one of the available stimuli is informative if one lands on Mars, or which economic

indicator is truly consequential for a future outlook. Converging evidence shows that humans indeed infer hidden models of complex tasks (Acuña and Schrater, 2010; Braun et al., 2010; Daw et al., 2011; Gershman and Niv, 2010;

Yakushijin and Jacobs, 2011), and indirect evidence from tasks involving schemas or contextual associations suggests that lower animals may also possess this capacity (Balan and Gottlieb, 2006; Braun et al., 2010; Johnson et al., 2012). Building internal models that identify the relevant steps is critical for specifying what subset of a very high-dimensional PDK4 information stream should be considered at a given time. Such models, in other worlds, are necessary for deciding to what to attend. As mentioned above in relation with the associability equation (Equation 2), this process entails an executive mechanism that learns how to learn—that is, decides how to organize the moment by moment sampling of sensory information. The need for hierarchical learning has been discussed in relation to motor control and cognitive tasks ( Braun et al., 2010; Johnson et al., 2012) and, as it is clear from this discussion, is also at the heart of attention control. Given an appropriate model of a task structure, informative options (stimuli or actions) may be identified through a prediction error mechanism as those options which, by reducing uncertainty, increase the expected future reward.