Existing POMDP Solvers

The library currently contains an implementation of POMCP, POUCT, and basic ValueIteration.

po_rollout	PO-rollout: Baseline algorithm in the POMCP paper [2].
po_uct	This algorithm is PO-UCT (Partially Observable UCT).
pomcp	We implement POMCP as described in the original paper Monte-Carlo Planning in Large POMDPs https://papers.nips.cc/paper/4031-monte-carlo-planning-in-large-pomdps
ValueIteration	This algorithm is only feasible for small problems where states, actions, and observations can be explicitly enumerated.

The library also currently interfaces with pomdp-solve, developed by Anthony R. Cassandra, and sarsop, developed by NUS. See Using External Solvers for details and examples.

sarsop	SARSOP, using the binary from https://github.com/AdaCompNUS/sarsop This is an anytime POMDP planning algorithm
vi_pruning	Value Iteration with pruning, using the software pomdp-solve https://www.pomdp.org/code/ developed by Anthony R.

Note

A pomdp_py Agent with enumerable state \(S\), action \(A\), and observation spaces \(\Omega\), with explicitly defined probability for its models (\(T,O,R\)) can be directly used as input to the above functions that interface with the solvers’ binaries programs.