#include <ExplorationModel.hh>
Public Member Functions | |
| bool | addStateToSet (const std::vector< float > &s) |
| bool | checkForState (const std::vector< float > &s) |
| ExplorationModel (MDPModel *model, int modelType, int exploreType, int predType, int nModels, float m, int numactions, float rmax, float qmax, float rrange, int nfactors, float v, float n, const std::vector< float > &featmax, const std::vector< float > &featmin, Random rng) | |
| ExplorationModel (const ExplorationModel &) | |
| virtual ExplorationModel * | getCopy () |
| float | getFeatDistToVisitedSA (const std::vector< float > &s) |
| virtual float | getStateActionInfo (const std::vector< float > &state, int act, StateActionInfo *retval) |
| virtual bool | updateWithExperience (experience &e) |
| virtual bool | updateWithExperiences (std::vector< experience > &instances) |
| virtual | ~ExplorationModel () |
Public Attributes | |
| bool | MODEL_DEBUG |
Private Attributes | |
| int | exploreType |
| std::vector< float > | featmax |
| std::vector< float > | featmin |
| float | M |
| MDPModel * | model |
| int | modelType |
| const float | n |
| int | nfactors |
| int | nModels |
| int | numactions |
| int | predType |
| float | qmax |
| float | rmax |
| Random | rng |
| float | rrange |
| std::set< std::vector< float > > | statespace |
| const float | v |
Detailed Description
This model wraps an another model and adds reward bonuses based on model confidence, # of visits, or other metrics.
Definition at line 21 of file ExplorationModel.hh.
Constructor & Destructor Documentation
| ExplorationModel::ExplorationModel | ( | MDPModel * | model, |
| int | modelType, | ||
| int | exploreType, | ||
| int | predType, | ||
| int | nModels, | ||
| float | m, | ||
| int | numactions, | ||
| float | rmax, | ||
| float | qmax, | ||
| float | rrange, | ||
| int | nfactors, | ||
| float | v, | ||
| float | n, | ||
| const std::vector< float > & | featmax, | ||
| const std::vector< float > & | featmin, | ||
| Random | rng | ||
| ) |
Default contstructor
- Parameters:
-
model The underlying MDP Model being used. modelType the type of model being used. exploreType type of reward bonuses to be added on top of model predType the way in which ensemble models combine their models nModels # of models to use for ensemble models (i.e. random forests) m # of visits for a given state-action to be considered known numactions # of actions in the domain rmax maximum one-step reward in the domain qmax maximum possible q-value in a domain rrange range of one-step rewards in the domain nfactors # of state features in the domain b/v coefficient to determine magnitude of variance reward n coefficient to determine magnitude of novelty reward featmax the maximum value of each state feature featmin the minimum value of each state feature rng Random Number Generator
Definition at line 14 of file ExplorationModel.cc.
| ExplorationModel::ExplorationModel | ( | const ExplorationModel & | em | ) |
Copy constructor
Definition at line 38 of file ExplorationModel.cc.
| ExplorationModel::~ExplorationModel | ( | ) | [virtual] |
Definition at line 57 of file ExplorationModel.cc.
Member Function Documentation
| bool ExplorationModel::addStateToSet | ( | const std::vector< float > & | s | ) |
Add state to a set of visited states
Definition at line 286 of file ExplorationModel.cc.
| bool ExplorationModel::checkForState | ( | const std::vector< float > & | s | ) |
Check if the given state is in the set of visited states
Definition at line 294 of file ExplorationModel.cc.
| ExplorationModel * ExplorationModel::getCopy | ( | ) | [virtual] |
Implements MDPModel.
Definition at line 51 of file ExplorationModel.cc.
| float ExplorationModel::getFeatDistToVisitedSA | ( | const std::vector< float > & | s | ) |
Find distance in feature space to nearest visited state-action
Definition at line 299 of file ExplorationModel.cc.
| float ExplorationModel::getStateActionInfo | ( | const std::vector< float > & | state, |
| int | act, | ||
| StateActionInfo * | retval | ||
| ) | [virtual] |
Implements MDPModel.
Definition at line 113 of file ExplorationModel.cc.
| bool ExplorationModel::updateWithExperience | ( | experience & | e | ) | [virtual] |
Implements MDPModel.
Definition at line 88 of file ExplorationModel.cc.
| bool ExplorationModel::updateWithExperiences | ( | std::vector< experience > & | instances | ) | [virtual] |
Implements MDPModel.
Definition at line 63 of file ExplorationModel.cc.
Member Data Documentation
int ExplorationModel::exploreType [private] |
Definition at line 90 of file ExplorationModel.hh.
std::vector<float> ExplorationModel::featmax [private] |
Definition at line 86 of file ExplorationModel.hh.
std::vector<float> ExplorationModel::featmin [private] |
Definition at line 87 of file ExplorationModel.hh.
float ExplorationModel::M [private] |
Definition at line 93 of file ExplorationModel.hh.
MDPModel* ExplorationModel::model [private] |
Underlying MDP model that we've wrapped and that we add bonus rewards onto.
Definition at line 84 of file ExplorationModel.hh.
Definition at line 71 of file ExplorationModel.hh.
int ExplorationModel::modelType [private] |
Definition at line 89 of file ExplorationModel.hh.
const float ExplorationModel::n [private] |
Definition at line 97 of file ExplorationModel.hh.
int ExplorationModel::nfactors [private] |
Definition at line 95 of file ExplorationModel.hh.
int ExplorationModel::nModels [private] |
Definition at line 92 of file ExplorationModel.hh.
int ExplorationModel::numactions [private] |
Definition at line 94 of file ExplorationModel.hh.
int ExplorationModel::predType [private] |
Definition at line 91 of file ExplorationModel.hh.
float ExplorationModel::qmax [private] |
Definition at line 94 of file ExplorationModel.hh.
float ExplorationModel::rmax [private] |
Definition at line 94 of file ExplorationModel.hh.
Random ExplorationModel::rng [private] |
Definition at line 99 of file ExplorationModel.hh.
float ExplorationModel::rrange [private] |
Definition at line 94 of file ExplorationModel.hh.
std::set<std::vector<float> > ExplorationModel::statespace [private] |
Set of all distinct sensations seen. This way we can know what we've visited.
Definition at line 81 of file ExplorationModel.hh.
const float ExplorationModel::v [private] |
Definition at line 96 of file ExplorationModel.hh.
The documentation for this class was generated from the following files: