- Discusses options, HAM, and MAXQ algs. The latter two work around the notion that the whole task is decomposed into subtasks, each of which has a subgoal.
- These hierarchical methods are able to solve two types of problems:
- Abstract actions given: Abstract actions for achieving subgoals are given in terms of the actions that are lower in the hierarchy
- Subgoals given: Concrete realization of abstract actions in terms of subordiante actions isn’t know but a decomposition into subtasks is given
- The options approach is very general so it isn’t restricted to subgoal related abstract actions
- Main idea of this paper is to allow agent to choose actions for time scales, so in hillcar, only 2 commands are needed to solve the domain.
- Not really relevant to what I care about, so ignoring the rest.