pomdp_problems.tiger package¶
pomdp_problems.tiger.tiger_problem module¶
The classic Tiger problem.
This is a POMDP problem; Namely, it specifies both the POMDP (i.e. state, action, observation space) and the T/O/R for the agent as well as the environment.
The description of the tiger problem is as follows: (Quote from POMDP: Introduction to Partially Observable Markov Decision Processes by Kamalzadeh and Hahsler )
A tiger is put with equal probability behind one of two doors, while treasure is put behind the other one. You are standing in front of the two closed doors and need to decide which one to open. If you open the door with the tiger, you will get hurt (negative reward). But if you open the door with treasure, you receive a positive reward. Instead of opening a door right away, you also have the option to wait and listen for tiger noises. But listening is neither free nor entirely accurate. You might hear the tiger behind the left door while it is actually behind the right door and vice versa.
States: tigerleft, tigerright Actions: openleft, openright, listen Rewards:
+10 for opening treasure door. 100 for opening tiger door. 1 for listening.
Observations: You can hear either “tigerleft”, or “tigerright”.
Note that in this example, the TigerProblem is a POMDP that also contains the agent and the environment as its fields. In general this doesn’t need to be the case. (Refer to more complicated examples.)

class
pomdp_problems.tiger.tiger_problem.
ObservationModel
(probs)[source]¶ Bases:
pomdp_py.framework.basics.ObservationModel
This problem is small enough for the probabilities to be directly given externally

probability
(self, observation, next_state, action, **kwargs)[source]¶ Returns the probability of \(\Pr(os',a)\).
 Parameters
observation (Observation) – the observation \(o\)
next_state (State) – the next state \(s'\)
action (Action) – the action \(a\)
 Returns
the probability \(\Pr(os',a)\)
 Return type
float

sample
(self, next_state, action, **kwargs)[source]¶ Returns observation randomly sampled according to the distribution of this observation model.
 Parameters
 Returns
the observation \(o\)
 Return type


class
pomdp_problems.tiger.tiger_problem.
TransitionModel
(probs)[source]¶ Bases:
pomdp_py.framework.basics.TransitionModel
This problem is small enough for the probabilities to be directly given externally

probability
(self, next_state, state, action, **kwargs)[source]¶ Returns the probability of \(\Pr(s's,a)\).


class
pomdp_problems.tiger.tiger_problem.
RewardModel
(scale=1)[source]¶ Bases:
pomdp_py.framework.basics.RewardModel

probability
(self, reward, state, action, next_state, **kwargs)[source]¶ Returns the probability of \(\Pr(rs,a,s')\).

sample
(self, state, action, next_state, **kwargs)[source]¶ Returns reward randomly sampled according to the distribution of this reward model.


class
pomdp_problems.tiger.tiger_problem.
PolicyModel
[source]¶ Bases:
pomdp_py.algorithms.po_uct.RandomRollout
This is an extremely dumb policy model; To keep consistent with the framework.

class
pomdp_problems.tiger.tiger_problem.
TigerProblem
(obs_probs, trans_probs, init_true_state, init_belief)[source]¶ Bases:
pomdp_py.framework.basics.POMDP

STATES
= {State(tigerright), State(tigerleft)}¶

ACTIONS
= {Action(listen), Action(openright), Action(openleft)}¶

OBSERVATIONS
= {Observation(tigerright), Observation(tigerleft)}¶


pomdp_problems.tiger.tiger_problem.
test_planner
(tiger_problem, planner, nsteps=3)[source]¶ Runs the actionfeedback loop of Tiger problem POMDP
 Parameters
tiger_problem (TigerProblem) – an instance of the tiger problem.
planner (Planner) – a planner
nsteps (int) – Maximum number of steps to run this loop.