Existing POMDP Solvers

The library currently contains an implementation of POMCP, POUCT, and basic ValueIteration.


PO-rollout: Baseline algorithm in the POMCP paper [2].


This algorithm is PO-UCT (Partially Observable UCT).


We implement POMCP as described in the original paper Monte-Carlo Planning in Large POMDPs https://papers.nips.cc/paper/4031-monte-carlo-planning-in-large-pomdps


This algorithm is only feasible for small problems where states, actions, and observations can be explicitly enumerated.

The library also currently interfaces with pomdp-solve, developed by Anthony R. Cassandra, and sarsop, developed by NUS. See Using External Solvers for details and examples.


SARSOP, using the binary from https://github.com/AdaCompNUS/sarsop This is an anytime POMDP planning algorithm


Value Iteration with pruning, using the software pomdp-solve https://www.pomdp.org/code/ developed by Anthony R.


A pomdp_py Agent with enumerable state \(S\), action \(A\), and observation spaces \(\Omega\), with explicitly defined probability for its models (\(T,O,R\)) can be directly used as input to the above functions that interface with the solvers’ binaries programs.