pomdp_py Documentation


pomdp_py is a general purpose POMDP library written in Python and Cython. It features simple and comprehensive interfaces to describe POMDP or MDP problems. Originally written to support POMDP planning research, the interfaces also allow extensions to model-free or model-based learning in (PO)MDPs, multi-agent POMDP planning/learning, and task transfer or transfer learning.

Why pomdp_py? It provides a POMDP framework in Python with clean and intuitive interfaces. This makes POMDP-related research or projects accessible to more people. It also helps sharing code and developing a community.

POMDP stands for Partially Observable Markov Decision Process [2].

The code is available on github. We welcome contributions to this library in:

  1. Implementation of additional POMDP solvers (see Existing POMDP Solvers)

  2. Implementation of additional POMDP domains (see Examples)

  3. Interfacing with existing POMDP libraries (majority in other languages).

  4. Extension of pomdp_py beyond planning (see Use Case Extensions).

Getting Started


If you find this library helpful to your work, please cite the following paper

   title = {pomdp\_py: A Framework to Build and Solve POMDP Problems},
   author = {Zheng, Kaiyu and Tellex, Stefanie},
   booktitle = {ICAPS 2020 Workshop on Planning and Robotics (PlanRob)},
   year = {2020},
   url = {https://icaps20subpages.icaps-conference.org/wp-content/uploads/2020/10/14-PlanRob_2020_paper_3.pdf},
   note = {Arxiv link: "\url{https://arxiv.org/pdf/2004.10099.pdf}"}



Leslie Pack Kaelbling, Michael L Littman, and Anthony R Cassandra. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101(1-2):99–134, 1998.

Brown University AI