#include <FactoredModel.hh>

Inheritance diagram for FactoredModel:

Public Member Functions
void	addFactorProb (float probs, std::vector< float > next, std::vector< float > x, StateActionInfo retval, int index, std::vector< std::map< float, float > > predictions, float confSum)
std::vector< float >	addVec (const std::vector< float > &a, const std::vector< float > &b)
	FactoredModel (int id, int numactions, int M, int modelType, int predType, int nModels, float treeThreshold, const std::vector< float > &featRange, float rRange, bool needConf, bool dep, bool relTrans, float featPct, bool stoch, bool episodic, Random rng=Random())
	FactoredModel (const FactoredModel &)
virtual FactoredModel *	getCopy ()
float	getSingleSAInfo (const std::vector< float > &state, int act, StateActionInfo *retval)
virtual float	getStateActionInfo (const std::vector< float > &state, int act, StateActionInfo *retval)
bool	initMDPModel (int nfactors)
void	setTreeParams (float margin, float forestPct, float minRatio)
std::vector< float >	subVec (const std::vector< float > &a, const std::vector< float > &b)
virtual bool	updateWithExperience (experience &e)
virtual bool	updateWithExperiences (std::vector< experience > &instances)
virtual	~FactoredModel ()
Private Attributes
bool	COPYDEBUG
const bool	dep
const bool	episodic
float	EXP_PCT
const float	FEAT_PCT
const std::vector< float >	featRange
int	id
const int	M
bool	MODEL_DEBUG
const int	modelType
const int	nact
const bool	needConf
int	nfactors
const int	nModels
std::vector< Classifier * >	outputModels
const int	predType
const bool	relTrans
Classifier *	rewardModel
Random	rng
const float	rRange
const bool	stoch
Classifier *	terminalModel
const int	treeBuildType
const float	treeThresh

Detailed Description

Builds an mdp model consisting of a tree (or ensemble of trees) to predict each feature, reward, and termination probability. Thus forming a complete model of the MDP.

Definition at line 23 of file FactoredModel.hh.

Constructor & Destructor Documentation

FactoredModel::FactoredModel	(	int	id,
		int	numactions,
		int	M,
		int	modelType,
		int	predType,
		int	nModels,
		float	treeThreshold,
		const std::vector< float > &	featRange,
		float	rRange,
		bool	needConf,
		bool	dep,
		bool	relTrans,
		float	featPct,
		bool	stoch,
		bool	episodic,
		Random	rng = `Random()`
	)

Default constructor

Parameters:

id	identify the model
numactions	# of actions in the domain
M	# of visits for a given state-action to be considered known
modelType	identifies which type of model to use
predType	identifies how to combine multiple models
nModels	# of models to use for ensemble models (i.e. random forests)
treeThreshold	determines the amount of error to be tolerated in the tree (prevents over-fitting with larger and larger trees)
featRange	range of each feature in the domain
rRange	range of reward values in domain
needConf	do we need confidence measures?
dep	assume dependent transitions between features (or indep)
relTrans	model relative transitions of features (or absolute)
featPct	pct of features to remove from set used for each tree split
stoch	if the domain is stochastic or deterministic
episodic	if the domain is episodic
rng	Random Number Generator

FactoredModel.cc Implements the FactoredModel class, which uses separate supervised learners to model each feature of an MDP. Please cite: Hester and Stone, "Real Time Targeted Exploration in Large Domains", ICDL 2010.

Author:: Todd Hester

Definition at line 10 of file FactoredModel.cc.

FactoredModel::FactoredModel ( const FactoredModel & m )

Copy Constructor for MDP Tree

Definition at line 39 of file FactoredModel.cc.

FactoredModel::~FactoredModel ( ) [virtual]

Definition at line 78 of file FactoredModel.cc.

Member Function Documentation

void FactoredModel::addFactorProb	(	float *	probs,
		std::vector< float > *	next,
		std::vector< float >	x,
		StateActionInfo *	retval,
		int	index,
		std::vector< std::map< float, float > >	predictions,
		float *	confSum
	)