Public Member Functions | Private Attributes
FactoredModel Class Reference

#include <FactoredModel.hh>

Inheritance diagram for FactoredModel:
Inheritance graph
[legend]

List of all members.

Public Member Functions

void addFactorProb (float *probs, std::vector< float > *next, std::vector< float > x, StateActionInfo *retval, int index, std::vector< std::map< float, float > > predictions, float *confSum)
std::vector< float > addVec (const std::vector< float > &a, const std::vector< float > &b)
 FactoredModel (int id, int numactions, int M, int modelType, int predType, int nModels, float treeThreshold, const std::vector< float > &featRange, float rRange, bool needConf, bool dep, bool relTrans, float featPct, bool stoch, bool episodic, Random rng=Random())
 FactoredModel (const FactoredModel &)
virtual FactoredModelgetCopy ()
float getSingleSAInfo (const std::vector< float > &state, int act, StateActionInfo *retval)
virtual float getStateActionInfo (const std::vector< float > &state, int act, StateActionInfo *retval)
bool initMDPModel (int nfactors)
void setTreeParams (float margin, float forestPct, float minRatio)
std::vector< float > subVec (const std::vector< float > &a, const std::vector< float > &b)
virtual bool updateWithExperience (experience &e)
virtual bool updateWithExperiences (std::vector< experience > &instances)
virtual ~FactoredModel ()

Private Attributes

bool COPYDEBUG
const bool dep
const bool episodic
float EXP_PCT
const float FEAT_PCT
const std::vector< float > featRange
int id
const int M
bool MODEL_DEBUG
const int modelType
const int nact
const bool needConf
int nfactors
const int nModels
std::vector< Classifier * > outputModels
const int predType
const bool relTrans
ClassifierrewardModel
Random rng
const float rRange
const bool stoch
ClassifierterminalModel
const int treeBuildType
const float treeThresh

Detailed Description

Builds an mdp model consisting of a tree (or ensemble of trees) to predict each feature, reward, and termination probability. Thus forming a complete model of the MDP.

Definition at line 23 of file FactoredModel.hh.


Constructor & Destructor Documentation

FactoredModel::FactoredModel ( int  id,
int  numactions,
int  M,
int  modelType,
int  predType,
int  nModels,
float  treeThreshold,
const std::vector< float > &  featRange,
float  rRange,
bool  needConf,
bool  dep,
bool  relTrans,
float  featPct,
bool  stoch,
bool  episodic,
Random  rng = Random() 
)

Default constructor

Parameters:
ididentify the model
numactions# of actions in the domain
M# of visits for a given state-action to be considered known
modelTypeidentifies which type of model to use
predTypeidentifies how to combine multiple models
nModels# of models to use for ensemble models (i.e. random forests)
treeThresholddetermines the amount of error to be tolerated in the tree (prevents over-fitting with larger and larger trees)
featRangerange of each feature in the domain
rRangerange of reward values in domain
needConfdo we need confidence measures?
depassume dependent transitions between features (or indep)
relTransmodel relative transitions of features (or absolute)
featPctpct of features to remove from set used for each tree split
stochif the domain is stochastic or deterministic
episodicif the domain is episodic
rngRandom Number Generator

FactoredModel.cc Implements the FactoredModel class, which uses separate supervised learners to model each feature of an MDP. Please cite: Hester and Stone, "Real Time Targeted Exploration in Large Domains", ICDL 2010.

Author:
Todd Hester

Definition at line 10 of file FactoredModel.cc.

Copy Constructor for MDP Tree

Definition at line 39 of file FactoredModel.cc.

Definition at line 78 of file FactoredModel.cc.


Member Function Documentation

void FactoredModel::addFactorProb ( float *  probs,
std::vector< float > *  next,
std::vector< float >  x,
StateActionInfo retval,
int  index,
std::vector< std::map< float, float > >  predictions,
float *  confSum 
)

Combines predictions for each separate state feature into probabilities of the overall state vector

Definition at line 696 of file FactoredModel.cc.

std::vector< float > FactoredModel::addVec ( const std::vector< float > &  a,
const std::vector< float > &  b 
)

Helper function to add two vectors together

Definition at line 763 of file FactoredModel.cc.

Implements MDPModel.

Definition at line 70 of file FactoredModel.cc.

float FactoredModel::getSingleSAInfo ( const std::vector< float > &  state,
int  act,
StateActionInfo retval 
)

Method to get a single sample of the predicted next state for the given state-action, rather than the full distribution given by getStateActionInfo

Definition at line 385 of file FactoredModel.cc.

float FactoredModel::getStateActionInfo ( const std::vector< float > &  state,
int  act,
StateActionInfo retval 
) [virtual]

Implements MDPModel.

Definition at line 496 of file FactoredModel.cc.

bool FactoredModel::initMDPModel ( int  nfactors)

Initialize the MDP model with the given # of state features

Definition at line 90 of file FactoredModel.cc.

void FactoredModel::setTreeParams ( float  margin,
float  forestPct,
float  minRatio 
)

Set some parameters of the subtrees

std::vector< float > FactoredModel::subVec ( const std::vector< float > &  a,
const std::vector< float > &  b 
)

Helper function to subtract two vectors

Definition at line 780 of file FactoredModel.cc.

Implements MDPModel.

Definition at line 297 of file FactoredModel.cc.

bool FactoredModel::updateWithExperiences ( std::vector< experience > &  instances) [virtual]

Implements MDPModel.

Definition at line 192 of file FactoredModel.cc.


Member Data Documentation

Definition at line 113 of file FactoredModel.hh.

const bool FactoredModel::dep [private]

Definition at line 103 of file FactoredModel.hh.

const bool FactoredModel::episodic [private]

Definition at line 107 of file FactoredModel.hh.

float FactoredModel::EXP_PCT [private]

Definition at line 110 of file FactoredModel.hh.

const float FactoredModel::FEAT_PCT [private]

Definition at line 105 of file FactoredModel.hh.

const std::vector<float> FactoredModel::featRange [private]

Definition at line 99 of file FactoredModel.hh.

int FactoredModel::id [private]

Definition at line 89 of file FactoredModel.hh.

const int FactoredModel::M [private]

Definition at line 92 of file FactoredModel.hh.

Definition at line 112 of file FactoredModel.hh.

const int FactoredModel::modelType [private]

Definition at line 93 of file FactoredModel.hh.

const int FactoredModel::nact [private]

Definition at line 91 of file FactoredModel.hh.

const bool FactoredModel::needConf [private]

Definition at line 102 of file FactoredModel.hh.

int FactoredModel::nfactors [private]

Definition at line 90 of file FactoredModel.hh.

const int FactoredModel::nModels [private]

Definition at line 95 of file FactoredModel.hh.

std::vector<Classifier*> FactoredModel::outputModels [private]

Classifier to predict each feature

Definition at line 81 of file FactoredModel.hh.

const int FactoredModel::predType [private]

Definition at line 94 of file FactoredModel.hh.

const bool FactoredModel::relTrans [private]

Definition at line 104 of file FactoredModel.hh.

Classifier to predict reward

Definition at line 84 of file FactoredModel.hh.

Definition at line 108 of file FactoredModel.hh.

const float FactoredModel::rRange [private]

Definition at line 100 of file FactoredModel.hh.

const bool FactoredModel::stoch [private]

Definition at line 106 of file FactoredModel.hh.

Classifier to prediction termination probability

Definition at line 87 of file FactoredModel.hh.

const int FactoredModel::treeBuildType [private]

Definition at line 96 of file FactoredModel.hh.

const float FactoredModel::treeThresh [private]

Definition at line 97 of file FactoredModel.hh.


The documentation for this class was generated from the following files:


rl_agent
Author(s): Todd Hester
autogenerated on Thu Jun 6 2019 22:00:14