pomdp_py.representations.belief package

pomdp_py.representations.belief.histogram module

pomdp_py.representations.belief.histogram.abstraction_over_histogram(current_histogram, state_mapper)[source]
pomdp_py.representations.belief.histogram.update_histogram_belief(current_histogram, real_action, real_observation, observation_model, transition_model, oargs={}, targs={}, normalize=True, static_transition=False, next_state_space=None)[source]
update_histogram_belief(current_histogram, real_action, real_observation,

observation_model, transition_model, oargs={}, targs={}, normalize=True, deterministic=False)

This update is based on the equation: \(B_{new}(s') = n O(z|s',a) \sum_s T(s'|s,a)B(s)\).

Parameters:

  • current_histogram (Histogram) – is the Histogram that represents current belief.

  • real_action (Action) –

  • real_observation (Observation) –

  • observation_model (ObservationModel) –

  • transition_model (TransitionModel) –

  • oargs (dict) Additional parameters for observation_model (default {}) –

  • targs (dict) Additional parameters for transition_model (default {}) –

  • normalize (bool) –

  • static_transition (bool) – This basically means Pr(s’|s,a) = Indicator(s’ == s). This then means that sum_s Pr(s’|s,a)*B(s) = B(s’), since s’ and s have the same state space. This thus helps reduce the computation cost by avoiding the nested iteration over the state space; But still, updating histogram belief requires iteration of the state space, which may already be prohibitive.

  • next_state_space (set) – this parameter is None and the state space given by current_histogram will be directly considered as the state space of the updated belief. This is useful for space and time efficiency in problems where the state space contains parts that the agent knows will deterministically update, and thus not keeping track of the belief over these states.

Returns:

the histogram distribution as a result of the update

Return type:

Histogram

pomdp_py.representations.belief.particles module

pomdp_py.representations.belief.particles.abstraction_over_particles(particles, state_mapper)
pomdp_py.representations.belief.particles.particle_reinvigoration(particles, num_particles, state_transform_func=None)

Note that particles should contain states that have already made the transition as a result of the real action. Therefore, they simply form part of the reinvigorated particles. At least maintain num_particles number of particles. If already have more, then it’s ok.

pomdp_py.representations.belief.particles.update_particles_belief(current_particles, real_action, real_observation=None, observation_model=None, transition_model=None, blackbox_model=None, state_transform_func=None)
update_particles_belief(Particles current_particles,

Action real_action, Observation real_observation=None, ObservationModel observation_model=None, TransitionModel transition_model=None, BlackboxModel blackbox_model=None, state_transform_func=None)

This is the second case (update particles belief explicitly); Either BlackboxModel is not None, or TransitionModel and ObservationModel are not None. Note that you DON’T need to call this function if you are using POMCP. |TODO: not tested|

Parameters:

state_transform_func (State->State) – the reinvigorated particles.