Environments

Model the dynamics and output of a reinforcement learning environment

In a reinforcement learning scenario, the environment models the world with which the agent interacts.

Reinforcement Learning Toolbox™ provides predefined objects that implement different benchmark environments. You can also create your own environments using custom functions for the environment dynamics, modifying an existing environment template class, or using a Simulink^® model.

For an introduction to reinforcement learning environments, see Reinforcement Learning Environments.

Functions

expand all

Environment Interface

`rlFiniteSetSpec`	Create specifications object for a finite-set action or observation channel
`rlNumericSpec`	Create specifications object for a numeric action or observation channel
`getActionInfo`	Obtain action data specifications from reinforcement learning environment, agent, or experience buffer
`getObservationInfo`	Obtain observation data specifications from reinforcement learning environment, agent, or experience buffer
`validateEnvironment`	Validate custom reinforcement learning environment
`bus2RLSpec`	Create reinforcement learning data specifications for elements of a Simulink bus

Grid World and MDP Environments

`createGridWorld`	Create a two-dimensional grid world for reinforcement learning
`createMDP`	Create Markov decision process model
`rlMDPEnv`	Create Markov decision process environment for reinforcement learning

Predefined Environments

rlPredefinedEnv Create a predefined reinforcement learning environment

Reward Computation

`generateRewardFunction`	Generate a reward function from control specifications to train a reinforcement learning agent (Since R2021b)
`exteriorPenalty`	Exterior penalty value for a point with respect to a bounded region (Since R2021b)
`hyperbolicPenalty`	Hyperbolic penalty value for a point with respect to a bounded region (Since R2021b)
`barrierPenalty`	Logarithmic barrier penalty value for a point with respect to a bounded region (Since R2021b)

Custom Environments

`rlFunctionEnv`	Create custom reinforcement learning environment using your reset and step functions
`rlMultiAgentFunctionEnv`	Create custom multiagent reinforcement learning environment (Since R2023b)
`rlTurnBasedFunctionEnv`	Create custom turn-based multiagent reinforcement learning environment (Since R2023b)
`rlCreateEnvTemplate`	Create custom reinforcement learning environment template
`rlSimulinkEnv`	Create environment object from a Simulink model already containing agent and environment
`createIntegratedEnv`	Create environment object from a Simulink environment model that does not contain an agent block
`SimulinkEnvWithAgent`	Reinforcement learning environment with a dynamic model implemented in Simulink
`bus2RLSpec`	Create reinforcement learning data specifications for elements of a Simulink bus
`validateEnvironment`	Validate custom reinforcement learning environment

Neural Network Environments

`rlNeuralNetworkEnvironment`	Environment model with deep neural network transition models (Since R2022a)
`rlContinuousDeterministicTransitionFunction`	Deterministic transition function approximator object for neural network-based environment (Since R2022a)
`rlContinuousGaussianTransitionFunction`	Stochastic Gaussian transition function approximator object for neural network-based environment (Since R2022a)
`rlContinuousDeterministicRewardFunction`	Deterministic reward function approximator object for neural network-based environment (Since R2022a)
`rlContinuousGaussianRewardFunction`	Stochastic Gaussian reward function approximator object for neural network-based environment (Since R2022a)
`rlIsDoneFunction`	Is-done function approximator object for neural network-based environment (Since R2022a)
`predict`	Predict next observation, next reward, or episode termination given observation and action input data (Since R2022a)
`evaluate`	Evaluate function approximator object given observation (or observation-action) input data (Since R2022a)
`accelerate`	(Not recommended) Option to accelerate computation of gradient for approximator object based on neural network (Since R2022a)

Setup, Reset, and Cleanup Environments

`reset`	Reset environment, agent, experience buffer, or policy object (Since R2022a)
`setup`	Set up reinforcement learning environment or initialize data logger object (Since R2022a)
`cleanup`	Clean up reinforcement learning environment or data logger object (Since R2022a)

Blocks

RL Agent

Reinforcement learning agent

Topics

Introduction to Reinforcement Learning Environments

Reinforcement Learning Environments
Model environment dynamics using a MATLAB^® object that generates rewards and observations in response to agents actions.

Grid World Environments

Load Predefined Grid World Environments
Load grid world environments in which the actions, observations, and rewards are already defined.
Create Custom Grid World Environments
Create custom grid world environments by defining your own grid size, rewards and obstacles.

Predefined Control System Environments

Load Predefined Control System Environments
Load predefined environments used as benchmarks for control systems design.

Custom MATLAB Environments

Define Reward and Observation Signals in Custom Environments
Create a reward signal that measures how successfully the agent actions are achieving a goal.
Create Custom Environment Using Step and Reset Functions
Create reinforcement learning environments by supplying custom step and reset functions.
Create Custom Environment from Class Template
Create a custom reinforcement learning environment by modifying a template environment class.

Custom Simulink Environments

Define Reward and Observation Signals in Custom Environments
Create a reward signal that measures how successfully the agent actions are achieving a goal.
Create Custom Simulink Environments
Create a custom environment using a Simulink model that generates rewards and observations in response to agents actions.
Create and Simulate the Same Environment in both MATLAB and Simulink
Understand differences between reinforcement learning loops implemented in MATLAB and Simulink.
Water Tank Reinforcement Learning Environment Model
Create a reinforcement learning Simulink environment that contains an RL Agent block in place of a controller for the water level in a tank.

Load Environments in Reinforcement Learning Designer

Load MATLAB Environments in Reinforcement Learning Designer
Load a MATLAB environment in the reinforcement designer app.
Load Simulink Environments in Reinforcement Learning Designer
Load a Simulink environment in the reinforcement designer app.