Actions

Similarly to observations, several types of actions can be used in every environment. They are defined in the action module. Each environment comes with a default action type, which can be changed or customised using environment configurations. For instance,

import gym
import highway_env

env = gym.make('highway-v0')
env.configure({
    "action": {
        "type": "ContinuousAction"
    }
})
env.reset()

Continuous Actions

The ContinuousAction type allows the agent to directly set the low-level controls of the vehicle kinematics, namely the throttle \(a\) and steering angle \(\delta\).

Note

The control of throttle and steering can be enabled or disabled through the longitudinal and lateral configurations, respectively. Thus, the action space can be either 1D or 2D.

Discrete Actions

The DiscreteAction is a uniform quantization of the ContinuousAction above.

The actions_per_axis parameter allows to set the quantization step. Similarly to continuous actions, the longitudinal and lateral axis can be enabled or disabled separately.

Discrete Meta-Actions

The DiscreteMetaAction type adds a layer of speed and steering controllers on top of the continuous low-level control, so that the ego-vehicle can automatically follow the road at a desired velocity. Then, the available meta-actions consist in changing the target lane and speed that are used as setpoints for the low-level controllers.

The full corresponding action space is defined in ACTIONS_ALL

ACTIONS_ALL = {
        0: 'LANE_LEFT',
        1: 'IDLE',
        2: 'LANE_RIGHT',
        3: 'FASTER',
        4: 'SLOWER'
    }

Some of these actions might not be always available (lane changes at the edges of the roads, or accelerating/decelrating beyond the maximum/minimum velocity), and the list of available actions can be accessed with get_available_actions() method. Taking an unavailable action is equivalent to taking the IDLE action.

Similarly to continuous actions, the longitudinal (speed changes) and lateral (lane changes) actions can be disabled separately through the longitudinal and lateral parameters. For instance, in the default configuration of the intersection environment, only the speed is controlled by the agent, while the lateral control of the vehicle is automatically performed by a steering controller to track a desired lane.

Manual control

The environments can be used as a simulation:

env = gym.make("highway-v0")
env.configure({
    "manual_control": True
})
env.reset()
done = False
while not done:
    env.step(env.action_space.sample())  # with manual control, these actions are ignored

The ego-vehicle is controlled by directional arrows keys, as defined in EventHandler

API

class highway_env.envs.common.action.ActionType(env: AbstractEnv, **kwargs)[source]

A type of action specifies its definition space, and how actions are executed in the environment

space() gym.spaces.space.Space[source]

The action space.

property vehicle_class: Callable

The class of a vehicle able to execute the action.

Must return a subclass of highway_env.vehicle.kinematics.Vehicle.

act(action: Union[int, numpy.ndarray]) None[source]

Execute the action on the ego-vehicle.

Most of the action mechanics are actually implemented in vehicle.act(action), where vehicle is an instance of the specified highway_env.envs.common.action.ActionType.vehicle_class. Must some pre-processing can be applied to the action based on the ActionType configurations.

Parameters

action – the action to execute

property controlled_vehicle

The vehicle acted upon.

If not set, the first controlled vehicle is used by default.

class highway_env.envs.common.action.ContinuousAction(env: AbstractEnv, acceleration_range: Optional[Tuple[float, float]] = None, steering_range: Optional[Tuple[float, float]] = None, speed_range: Optional[Tuple[float, float]] = None, longitudinal: bool = True, lateral: bool = True, dynamical: bool = False, clip: bool = True, **kwargs)[source]

An continuous action space for throttle and/or steering angle.

If both throttle and steering are enabled, they are set in this order: [throttle, steering]

The space intervals are always [-1, 1], but are mapped to throttle/steering intervals through configurations.

ACCELERATION_RANGE = (-5, 5.0)

Acceleration range: [-x, x], in m/s².

STEERING_RANGE = (-0.7853981633974483, 0.7853981633974483)

Steering angle range: [-x, x], in rad.

space() gym.spaces.box.Box[source]

The action space.

property vehicle_class: Callable

The class of a vehicle able to execute the action.

Must return a subclass of highway_env.vehicle.kinematics.Vehicle.

act(action: numpy.ndarray) None[source]

Execute the action on the ego-vehicle.

Most of the action mechanics are actually implemented in vehicle.act(action), where vehicle is an instance of the specified highway_env.envs.common.action.ActionType.vehicle_class. Must some pre-processing can be applied to the action based on the ActionType configurations.

Parameters

action – the action to execute

class highway_env.envs.common.action.DiscreteAction(env: AbstractEnv, acceleration_range: Optional[Tuple[float, float]] = None, steering_range: Optional[Tuple[float, float]] = None, longitudinal: bool = True, lateral: bool = True, dynamical: bool = False, clip: bool = True, actions_per_axis: int = 3, **kwargs)[source]
space() gym.spaces.discrete.Discrete[source]

The action space.

act(action: int) None[source]

Execute the action on the ego-vehicle.

Most of the action mechanics are actually implemented in vehicle.act(action), where vehicle is an instance of the specified highway_env.envs.common.action.ActionType.vehicle_class. Must some pre-processing can be applied to the action based on the ActionType configurations.

Parameters

action – the action to execute

class highway_env.envs.common.action.DiscreteMetaAction(env: AbstractEnv, longitudinal: bool = True, lateral: bool = True, target_speeds: Optional[Union[numpy.ndarray, Sequence[float]]] = None, **kwargs)[source]

An discrete action space of meta-actions: lane changes, and cruise control set-point.

ACTIONS_ALL = {0: 'LANE_LEFT', 1: 'IDLE', 2: 'LANE_RIGHT', 3: 'FASTER', 4: 'SLOWER'}

A mapping of action indexes to labels.

ACTIONS_LONGI = {0: 'SLOWER', 1: 'IDLE', 2: 'FASTER'}

A mapping of longitudinal action indexes to labels.

ACTIONS_LAT = {0: 'LANE_LEFT', 1: 'IDLE', 2: 'LANE_RIGHT'}

A mapping of lateral action indexes to labels.

space() gym.spaces.space.Space[source]

The action space.

property vehicle_class: Callable

The class of a vehicle able to execute the action.

Must return a subclass of highway_env.vehicle.kinematics.Vehicle.

act(action: int) None[source]

Execute the action on the ego-vehicle.

Most of the action mechanics are actually implemented in vehicle.act(action), where vehicle is an instance of the specified highway_env.envs.common.action.ActionType.vehicle_class. Must some pre-processing can be applied to the action based on the ActionType configurations.

Parameters

action – the action to execute

class highway_env.envs.common.action.MultiAgentAction(env: AbstractEnv, action_config: dict, **kwargs)[source]
space() gym.spaces.space.Space[source]

The action space.

property vehicle_class: Callable

The class of a vehicle able to execute the action.

Must return a subclass of highway_env.vehicle.kinematics.Vehicle.

act(action: Union[int, numpy.ndarray]) None[source]

Execute the action on the ego-vehicle.

Most of the action mechanics are actually implemented in vehicle.act(action), where vehicle is an instance of the specified highway_env.envs.common.action.ActionType.vehicle_class. Must some pre-processing can be applied to the action based on the ActionType configurations.

Parameters

action – the action to execute