An Optimal Approximate Dynamic Programming Algorithm for the Economic Dispatch Problem with Grid-Level Storage. Nasciento, Powell.

  1. Proves convergence of an approx DP algo for high-dimensional control problems linked by a scalar storage device
  2. Problem is formulated as a stochastic DP where value of stored energy is done with a piecewise linear value func approximator
  3. Problem is concave, so exploration isn’t needed, and a proof is provided for convergence
  4. Discrete state, vector valued, but low-dimensional
  5. Actions are also discrete
  6. “For our problem, xt (action at time t) can easily have hundreds or thousands of dimensions, depending on the level of aggregation.  In addition, we are unable to compute the expectation analytically, introducing the additional complication of maximizing an expectation that is itself computationally intractable.”
  7. Because the action space is so large, thorough exploration is not possible.  They find optimal solutions by concavity
  8. They say there are three possible curses of dimensionality: in finding a mapping over S, selecting A, and predicting S’
  9. “Our strategy represents a form of approximate value iteration where we focus on finding the slopes of a function rather than the value”
  10. Although approximate value functions have poor convergence and diverge, concavity allows a result that works well in practice, and in this application
  11. This paper, however, has the first convergence proof (for approx VI on concave value functions?)
  12. Assumptions made here are also weaker than in many similar papers
  13. Although some state values (such as energy in storage) are continuous, the method will only allow those continuous variables to reach a discrete number of  values

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: