pomdp_py Documentation

Overview

pomdp_py is a general purpose POMDP library written in Python and Cython. It features simple and comprehensive interfaces to describe POMDP or MDP problems. Originally written to support POMDP planning research, the interfaces also allow extensions to model-free or model-based learning in (PO)MDPs, multi-agent POMDP planning/learning, and task transfer or transfer learning.

Why pomdp_py? It provides a POMDP framework in Python with clean and intuitive interfaces. This makes POMDP-related research or projects accessible to more people. It also helps sharing code and developing a community.

POMDP stands for Partially Observable Markov Decision Process [1].

The code is available on github. We welcome contributions to this library in:

1. Implementation of additional POMDP solvers (see Existing POMDP Solvers)

2. Implementation of additional POMDP domains (see Examples)

3. Interfacing with existing POMDP libraries (majority in other languages).

4. Extension of pomdp_py beyond planning (see Use Case Extensions).

Getting Started

• Installation

• What’s New? (Changelog)

• Examples:

  1. Tiger: The Tiger domain shows how one could quickly implement a simple POMDP.

  2. Multi-Object Search (MOS) The Multi-Object Search domain is an example for how one may structure their project (as a convention) when using pomdp_py for more complicated domains.

• Design Principles

• Existing POMDP Domains

• Existing POMDP Solvers

• Using External Solvers

• Debugging Tools

Further

• Use Case Extensions

• Other POMDP Libraries

• Building Documentations

Citation

If you find this library helpful to your work, please cite the following paper

@inproceedings{zheng2020pomdp_py,
   title = {pomdp\_py: A Framework to Build and Solve POMDP Problems},
   author = {Zheng, Kaiyu and Tellex, Stefanie},
   booktitle = {ICAPS 2020 Workshop on Planning and Robotics (PlanRob)},
   year = {2020},
   url = {https://icaps20subpages.icaps-conference.org/wp-content/uploads/2020/10/14-PlanRob_2020_paper_3.pdf},
   note = {Arxiv link: "\url{https://arxiv.org/pdf/2004.10099.pdf}"}
}

API References

API References

• pomdp_py
  • pomdp_py package
    • pomdp_py.algorithms package
    • pomdp_py.framework package
    • pomdp_py.representations package
    • pomdp_py.utils package
  • pomdp_py.problems package
    • Subpackages
    • Module contents

Tools

• Index

• Module Index

• Search Page

[1]

Leslie Pack Kaelbling, Michael L Littman, and Anthony R Cassandra. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101(1-2):99–134, 1998.