Skip to content

brohrer/cartographer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Fuzzy Naive Cartographer

The "model" part of a model-based reinforcement learning algorithm.

Installation

Pull down the FNC repository. At the command line:

git clone git@codeberg.org:brohrer/cartographer.git

Install it.

python3 -m pip install -e cartographer

Usage

The FNC class Naive Cartographer is intended to be used in tandem with a planner (not included) to build a reinforcement learning agent. At each time step

  1. The FNC makes a prediction
  2. The planner uses that prediction to choose an action
  3. The FNC registers the action selection

The outline of an agent class looks like this:

from cartographer import NaiveCartographer as Model 

class Agent
    def __init__(self, n_sensors, n_actions):
        model = Model(n_sensors=n_sensors, n_actions=n_actions)
        planner = ...

    def step(self, sensors, reward):
        predictions, rewards, uncertainties = model.predict(sensors, reward)
        actions = planner.choose_actions(predictions, rewards, uncertainties)
        model.update_actions(actions)
        return actions

About

A Fuzzy Naive Cartographer (FNC) builds a Markov Decision Process-like model of its world in the form of a set of sequences. Sequences are of the form of state-action-state as in MDPs. Here they're also referred to as feature-action-outcome sequences, to disambiguate the before- and after-state.

FNC builds a value function too. It creates a set of feature-action pairs and associates a reward with each. This is analogous to the state-action value functions of Q-learning.

A model and a value function allow for prediction and planning. Knowing the current active features and recent actions, both the reward and the resulting features can be anticipated.

Technically FNC is just the "model" part of a model-based RL algorithm. It needs to paired with a planner, some element that will choose an action or goal, to be a complete RL algorithm. There are some rudimentary planners included with this module to get you up and running.

There is more detail in the white paper at https://brandonrohrer.com/cartographer

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages