Classes | Public Member Functions | Public Attributes | Protected Types | Protected Member Functions | Private Attributes
Dyna Class Reference

#include <Dyna.hh>

Inheritance diagram for Dyna:
Inheritance graph

List of all members.


struct  dynaExperience

Public Member Functions

 Dyna (int numactions, float gamma, float initialvalue, float alpha, int k, float epsilon, Random rng=Random())
 Dyna (const Dyna &)
virtual int first_action (const std::vector< float > &s)
int getBestAction (const std::vector< float > &s)
double getSeconds ()
float getValue (std::vector< float > state)
virtual void last_action (float r)
void logValues (ofstream *of, int xmin, int xmax, int ymin, int ymax)
virtual int next_action (float r, const std::vector< float > &s)
void printState (const std::vector< float > &s)
std::vector< float >::iterator random_max_element (std::vector< float >::iterator start, std::vector< float >::iterator end)
virtual void savePolicy (const char *filename)
virtual void seedExp (std::vector< experience >)
virtual void setDebug (bool d)
virtual ~Dyna ()

Public Attributes

float epsilon

Protected Types

typedef const std::vector
< float > * 

Protected Member Functions

void addExperience (float r, state_t s, bool term)
state_t canonicalize (const std::vector< float > &s)

Private Attributes

const float alpha
float * currentq
std::vector< dynaExperienceexperiences
const float gamma
const float initialvalue
const int k
int lastact
state_t laststate
const int numactions
std::map< state_t, std::vector
< float > > 
Random rng
std::set< std::vector< float > > statespace

Detailed Description

Agent that uses straight Q-learning, with no generalization and epsilon-greedy exploration.

Definition at line 17 of file Dyna.hh.

Member Typedef Documentation

typedef const std::vector<float>* Dyna::state_t [protected]

The implementation maps all sensations to a set of canonical pointers, which serve as the internal representation of environment state.

Definition at line 61 of file Dyna.hh.

Constructor & Destructor Documentation

Dyna::Dyna ( int  numactions,
float  gamma,
float  initialvalue,
float  alpha,
int  k,
float  epsilon,
Random  rng = Random() 

Standard constructor

numactionsThe number of possible actions
gammaThe discount factor
initialvalueThe initial value of each Q(s,a)
alphaThe learning rate
epsilonThe probability of taking a random action
rngInitial state of the random number generator to use

Definition at line 7 of file

Dyna::Dyna ( const Dyna )

Unimplemented copy constructor: internal state cannot be simply copied.

Dyna::~Dyna ( ) [virtual]

Definition at line 21 of file

Member Function Documentation

void Dyna::addExperience ( float  r,
state_t  s,
bool  term 
) [protected]

Definition at line 93 of file

Dyna::state_t Dyna::canonicalize ( const std::vector< float > &  s) [protected]

Produces a canonical representation of the given sensation.

sThe current sensation from the environment.
A pointer to an equivalent state in statespace.

Definition at line 146 of file

int Dyna::first_action ( const std::vector< float > &  s) [virtual]

Implements Agent.

Definition at line 23 of file

int Dyna::getBestAction ( const std::vector< float > &  s)

Definition at line 35 of file

double Dyna::getSeconds ( )

Definition at line 317 of file

float Dyna::getValue ( std::vector< float >  state)

Definition at line 251 of file

void Dyna::last_action ( float  r) [virtual]

Implements Agent.

Definition at line 133 of file

void Dyna::logValues ( ofstream *  of,
int  xmin,
int  xmax,
int  ymin,
int  ymax 

Definition at line 235 of file

int Dyna::next_action ( float  r,
const std::vector< float > &  s 
) [virtual]

Implements Agent.

Definition at line 106 of file

void Dyna::printState ( const std::vector< float > &  s)

Definition at line 185 of file

std::vector< float >::iterator Dyna::random_max_element ( std::vector< float >::iterator  start,
std::vector< float >::iterator  end 

Definition at line 160 of file

void Dyna::savePolicy ( const char *  filename) [virtual]

Reimplemented from Agent.

Definition at line 285 of file

void Dyna::seedExp ( std::vector< experience seeds) [virtual]

Reimplemented from Agent.

Definition at line 193 of file

void Dyna::setDebug ( bool  d) [virtual]

Implements Agent.

Definition at line 180 of file

Member Data Documentation

bool Dyna::ACTDEBUG [private]

Definition at line 103 of file Dyna.hh.

const float Dyna::alpha [private]

Definition at line 95 of file Dyna.hh.

float* Dyna::currentq [private]

Definition at line 99 of file Dyna.hh.

Definition at line 55 of file Dyna.hh.

std::vector<dynaExperience> Dyna::experiences [private]

Definition at line 89 of file Dyna.hh.

const float Dyna::gamma [private]

Definition at line 92 of file Dyna.hh.

const float Dyna::initialvalue [private]

Definition at line 94 of file Dyna.hh.

const int Dyna::k [private]

Definition at line 96 of file Dyna.hh.

int Dyna::lastact [private]

Definition at line 101 of file Dyna.hh.

Definition at line 100 of file Dyna.hh.

const int Dyna::numactions [private]

Definition at line 91 of file Dyna.hh.

std::map<state_t, std::vector<float> > Dyna::Q [private]

The primary data structure of the learning algorithm, the value function Q. For state_t s and int a, Q[s][a] gives the learned maximum expected future discounted reward conditional on executing action a in state s.

Definition at line 87 of file Dyna.hh.

Random Dyna::rng [private]

Definition at line 98 of file Dyna.hh.

std::set<std::vector<float> > Dyna::statespace [private]

Set of all distinct sensations seen. Pointers to elements of this set serve as the internal representation of the environment state.

Definition at line 81 of file Dyna.hh.

The documentation for this class was generated from the following files:

Author(s): Todd Hester
autogenerated on Thu Jun 6 2019 22:00:14