Speeding-up Reinforcement Learning with Multi-step Actions. Schoknecht, Riedmiller. Lecture Notes in Computer Science. 2002.


  1. Discusses options, HAM, and MAXQ algs.  The latter two work around the notion that the whole task is decomposed into subtasks, each of which has a subgoal.
  2. These hierarchical methods are able to solve two types of problems:
    1. Abstract actions given: Abstract actions for achieving subgoals are given in terms of the actions that are lower in the hierarchy
    2. Subgoals given: Concrete realization of abstract actions in terms of subordiante actions isn’t know but a decomposition into subtasks is given
    3. The options approach is very general so it isn’t restricted to subgoal related abstract actions
  3. Main idea of this paper is to allow agent to choose actions for time scales, so in hillcar, only 2 commands are needed to solve the domain.
  4. Not really relevant to what I care about, so ignoring the rest.
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: