Changelog

Best viewed on the website.

Version 1.3.5.1 (03/30/2024)

  • Hot fixes for WeightedParticles.

  • Agent can be created with a name.

  • Bug fixes for release script and GHA workflow.

Version 1.3.5 (03/12/2024)

  • Refactored _search in POUCT to be more modular

  • More fields made accessible for POUCT

  • Allows initializing Agent and Environment without all models ready, and provide those models later (or update them) by set_models

  • Minor function renaming for Option

  • Added DictState

  • Improved modularity and documentation for the RockSample example

  • WeightedParticles now hashable if frozen.

  • Changed det_dict_hash to not use sorting

  • Added a Github Action workflow for automatically building, releasing wheels and uploading them to PyPI

  • Default branch renamed to main, and dev-latest is the development branch. Before a release, dev-<version> should be created based off of dev-latest.

Version 1.3.4 (01/26/2024)

  • Modernize repo without introducing new features.

  • Migrate to pyproject.toml to specify dependencies, with minimal setup.py to build Cython extensions.

  • Separate core and optional dependencies. Optional dependencies are needed only for pomdp_py.problems.

  • Added CI testing to repo.

  • Added pre-commit checks to repo.

Version 1.3.3 (07/25/2023)

  • Bumped minimum Python requirement from 3.7 to 3.8 due to EOL of 3.7.

  • Fix cpdef -> cdef to avoid installation failure after Cython 3.0.0 release (pomdp-py#30).

  • Added float_precision argument to to_pomdp_file (pomdp-py#29)

  • Add __init__ signature for Environment in comments to be visible in docs

  • Fix s -> sp in ValueIteration (pomdp-py#20)

  • Allow updating rollout policy for POUCT and POMCP

  • Fix in setup.py so that wheel builds properly.

  • Change set to list in pomdp_problems.tiger.tiger_problem.py to tame error regarding random.sample in Python 3.11.

  • Minor bug fixes and documentation.

Version 1.3.2 (04/03/2022)

  • Fix in histogram (in __str__): Print all of histogram as is instead of printing top 5 to avoid confusion.

  • Improve documentation for the tiger tutorial; Specifically, clarified PolicyModel and gave a reference to ActionPrior.

  • Built documentation with new Sphinx version (4.5.0)

Version 1.3.1 (11/03/2021)

Version 1.3.0.1 (09/30/2021)

  • Removed dependency on pygraphviz;

  • Added utils.debugging.Treedebugger, which makes it easier to inspect the search tree. See debugging.

  • Added WeightedParticles; Refactored Particles. (particles)

  • Optionally show progress bar while simulating in POUCT/POMCP.

  • Added a CLI interface to simplify running example domains, e.g. python -m pomdp_py -r tiger runs Tiger.

  • Can initialize Environment with BlackboxModel.

  • For the OOBelief class in oopomdp.pyx, now mpe and random can take an argument return_oostate (default True), which returns a sampled state as type OOState. This can be useful if you would like to inherit OOBelief and return a state of your own type when implementing its mpe and random functions.

  • Added __ne__ methods to framework classes.

  • Reorganized util by breaking it into different modules.

  • Code refactoring: Remove unnecessary *args, **kwargs in interface signature. Backwards compatible.

  • Bug fix regarding hashing and pickling.

  • Verified installation on Windows (TODO)

Version 1.2.4.6 (canceled)

  • Fix setup.py so that pip install -e . works.

Version 1.2.4.5 (07/05/2021)

  • Edit setup.py file so that Cython modules in pomdp-py can be :code:`cimport`ed.

(skipped versions due to attempting pypi release)

Version 1.2.4.1 (06/02/2021)

  • Fix documentation (external solver examples).

  • Update tiger_problem.py to match documentation

Version 1.2.4 (06/01/2021)

  • pomdp_py/algorithms/value_function.py:

    • zero-probability observation should be skipped.

    • refactored so that value() can take either a dict or a sequence of arguments.

  • Available on PyPI

  • .value field of VNode is instead changed to be a property, computed by finding the maximum value of the children Q-Nodes.

Version 1.2.3 (03/22/2021)

  • Bug fix in solvers.py and conversion.py (18fc58e0, cfc88e8d8)

  • Bug fix in the MOS domain’s observation model (719c2edf5)

  • Linked docker image in documentation issue #13.

  • Updated documentations

Version 1.2.2.1 (01/25/2021)

  • Updated documentation for external library interfacing. Added citation.

Version 1.2.2 (01/17/2021)

  • Resolved issue #10. Set value in V-Node to be the max among its children Q-Nodes. Initial V-Node value set to negative infinity.

  • Avoid search tree building during rollout (thanks Jason)

  • Documentation clarification about .sample and .argmax functions in the RewardModel.

  • Small pomdps (with enumerable state, action, observation spaces) defined in pomdp_py can be converted to .pomdp and .pomdpx file formats.

  • Added interfacing with pomdp_solve and tested.

  • Added interfacing with sarsop and tested.

  • Added utils/templates.py that contains some convenient implementations of the POMDP interface.

  • Bug fixes (in histogram.pyx)

Version 1.2.1 (12/23/2020)

  • Fixed preferred rollout and action prior implementation; Previously the initial visits and values were not applied.

  • Fixed UCB1 value calculation when number of visits is 0; Previously a divide by zero error will be thrown. But it should have infinite value.

  • Fixed another potential math domain error due to log(0) in UCB1 value calculation when initial number of visit set to 0.

  • Fixed bug in particle belief update (minor type error)

  • Simplified the Tiger example code, updated the Tiger example in documentation.

  • Fixed bug in ValueIteration and verified in Tiger that it’s able to differentiate differentiate between listen/stay actions when horizon = 3, but not so when horizon = 1 or 2. The same behavior is observed using the pomdp solver by POMDP.org.

  • Added an exact value function in pomdp_py.algorithms.value_function. It is a simpler exact value iteration algorithm.

  • Added Load/Unload domain Pull request #9

  • Pull request #11

Pull Request #3 (08/01/2020)

  • Added num_sims parameter to POMCP/POUCT that allows specifying the number of simulations per planning step (Previously only planning_time was available.

  • Added cythonized versions of tiger and rocksample domains which are much faster.

Pull Request #1 (06/02/2020)

  • Added continuous light-dark domain. A solver (BLQR) is attempted but not ready yet.

  • Bug fix in 2D MOS domain rollout; action step size changeable

  • Added Tag domain, tested with POUCT random rollout

  • Documentation

Version 1.0 - 1.2 (02/08/2020 - 02/16/2020)

  • Rewritten with cleaner interfaces and Cython integration

  • Refactored POMCP: Now it extends POUCT which does not assume particle belief representation.

  • Included Tiger, RockSample, and a 2D multi-object search (MOS) domain.

  • Using Sphinx Documentation

Version 0.0

  • Implementation of POMCP and OO-POMDP