Functions
def	rendertrial (maxiter=100)

Variables
float	DECAY_RATE = 0.99

	env = DPendulum()

list	h_rwd = []

float	LEARNING_RATE = 0.85

int	NEPISODES = 500

int	NSTEPS = 50

	NU = env.nu

	NX = env.nx

	Q = np.zeros([env.nx, env.nu])

float	Qref = reward + DECAY_RATE * np.max(Q[x2, :])

	RANDOM_SEED = int((time.time() % 10) * 1000)

	reward

float	rsum = 0.0

	u

	x = env.reset()

	x2

Detailed Description

Example of Q-table learning with a simple discretized 1-pendulum environment.

Function Documentation

◆ rendertrial()

def qtable.rendertrial ( maxiter = 100 )

Roll-out from random state using greedy policy.

Definition at line 31 of file qtable.py.

Variable Documentation

◆ DECAY_RATE

float qtable.DECAY_RATE = 0.99

Definition at line 21 of file qtable.py.

◆ env

qtable.env = DPendulum()

Definition at line 24 of file qtable.py.

◆ h_rwd

list qtable.h_rwd = []

Definition at line 46 of file qtable.py.

◆ LEARNING_RATE

float qtable.LEARNING_RATE = 0.85

Definition at line 20 of file qtable.py.

◆ NEPISODES

int qtable.NEPISODES = 500

Definition at line 18 of file qtable.py.

◆ NSTEPS

int qtable.NSTEPS = 50

Definition at line 19 of file qtable.py.

◆ NU

qtable.NU = env.nu

Definition at line 26 of file qtable.py.

◆ NX

qtable.NX = env.nx

Definition at line 25 of file qtable.py.

◆ Q

qtable.Q = np.zeros([env.nx, env.nu])

Definition at line 28 of file qtable.py.

◆ Qref

float qtable.Qref = reward + DECAY_RATE * np.max(Q[x2, :])

Definition at line 58 of file qtable.py.

◆ RANDOM_SEED

qtable.RANDOM_SEED = int((time.time() % 10) * 1000)

Definition at line 13 of file qtable.py.

◆ reward

qtable.reward

Definition at line 55 of file qtable.py.

◆ rsum

float qtable.rsum = 0.0

Definition at line 50 of file qtable.py.

◆ u

qtable.u

Initial value:

 =  np.argmax(
             Q[x, :] + np.random.randn(1, NU) / episode
         )

Definition at line 52 of file qtable.py.

◆ x

qtable.x = env.reset()

Definition at line 49 of file qtable.py.

◆ x2

qtable.x2