Clinical Data Based Optimal STI Strategies for HIV: a Reinforcement Learning Approach. Ernst, Stan, Goncalves, Wehenkel


  • Are insterested in STI (structured treatment interruption) where patients are taken off drugs for periods of time during treatment, say it is helpful for patients to have a period without drug side-effects, and that in some cases repeated sessions of STI have somehow caused some peoples’ immune systems to be able to control the virus
  • Discuss use of control theory to schedule drug treatment, but this requires assummed knowledge of the type of system it is, along with the relevant parameters
    • Authors want to devise strategies that come right from clinical data instead without assuming an infection model
  • They don’t use actual clinical data in their experiments; the data comes from a simulation of HIV mutation dynamics
  • Use fitted-Q with extra-trees for dynamic programming
  • Check Banks et. al. Dynamic multidrug therapies for HIV: Optimal and STI control approaches
  • They have a seemingly strange reward function, I wonder what the justification is
  • They have a pretty ad-hoc method of data generation (totally random, then epsilon greedy, then greedy)
  • The RL result pulls the system into a good state, as is verifiable given it is a simulation and solvable from the known dynamics
  • Discuss issues of using simulation:
    • Dynamics vary from person to person
    • The reward function, time discretization, and decay factor may not allow for an optimal (obviously in another sense) solution
    • In reality it is difficult to distinguish between two values used in the feature vector; for that reasons and others there is significant partial observability
    • In real life, there is more noise in terms of patients ignoring medication, or incorrect transcriptions of values to data set
  • “In this paper, we have considered the problem of computing structured treatment interruption strategies for HIV infected patients from clinical data only.”
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: