Observations

For all environments, several types of observations can be used. They are defined in the observation module. Each environment comes with a default observation, which can be changed or customised using environment configurations. For instance,

import gym
import highway_env

env = gym.make('highway-v0')
env.configure({
    "observation": {
        "type": "OccupancyGrid",
        "vehicles_count": 15,
        "features": ["presence", "x", "y", "vx", "vy", "cos_h", "sin_h"],
        "features_range": {
            "x": [-100, 100],
            "y": [-100, 100],
            "vx": [-20, 20],
            "vy": [-20, 20]
        },
        "grid_size": [[-27.5, 27.5], [-27.5, 27.5]],
        "grid_step": [5, 5],
        "absolute": False
    }
})
env.reset()

Note

The "type" field in the observation configuration takes values defined in observation_factory() (see source)

Kinematics

The KinematicObservation is a \(V\times F\) array that describes a list of \(V\) nearby vehicles by a set of features of size \(F\), listed in the "features" configuration field. For instance:

Vehicle

\(x\)

\(y\)

\(v_x\)

\(v_y\)

ego-vehicle

5.0

4.0

15.0

0

vehicle 1

-10.0

4.0

12.0

0

vehicle 2

13.0

8.0

13.5

0

vehicle V

22.2

10.5

18.0

0.5

Note

The ego-vehicle is always described in the first row

If configured with normalize=True (default), the observation is normalized within a fixed range, which gives for the range [100, 100, 20, 20]:

Vehicle

\(x\)

\(y\)

\(v_x\)

\(v_y\)

ego-vehicle

0.05

0.04

0.75

0

vehicle 1

-0.1

0.04

0.6

0

vehicle 2

0.13

0.08

0.675

0

vehicle V

0.222

0.105

0.9

0.025

If configured with absolute=False, the coordinates are relative to the ego-vehicle, except for the ego-vehicle which stays absolute.

Vehicle

\(x\)

\(y\)

\(v_x\)

\(v_y\)

ego-vehicle

0.05

0.04

0.75

0

vehicle 1

-0.15

0

-0.15

0

vehicle 2

0.08

0.04

-0.075

0

vehicle V

0.172

0.065

0.15

0.025

Note

The number \(V\) of vehicles is constant and configured by the vehicles_count field, so that the observation has a fixed size. If fewer vehicles than vehicles_count are observed, the last rows are placeholders filled with zeros. The presence feature can be used to detect such cases, since it is set to 1 for any observed vehicle and 0 for placeholders.

Feature

Description

\(presence\)

Disambiguate agents at 0 offset from non-existent agents.

\(x\)

World offset of ego vehicle or offset to ego vehicle on the x axis.

\(y\)

World offset of ego vehicle or offset to ego vehicle on the y axis.

\(vx\)

Velocity on the x axis of vehicle.

\(vy\)

Velocity on the y axis of vehicle.

\(heading\)

Heading of vehicle in radians.

\(cos_h\)

Trigonometric heading of vehicle.

\(sin_h\)

Trigonometric heading of vehicle.

\(cos_d\)

Trigonometric direction to the vehicle’s destination.

\(sin_d\)

Trigonometric direction to the vehicle’s destination.

\(long_{off}\)

Longitudinal offset to closest lane.

\(lat_{off}\)

Lateral offset to closest lane.

\(ang_{off}\)

Angular offset to closest lane.

Example configuration

import gym
import highway_env

config = {
    "observation": {
        "type": "Kinematics",
        "vehicles_count": 15,
        "features": ["presence", "x", "y", "vx", "vy", "cos_h", "sin_h"],
        "features_range": {
            "x": [-100, 100],
            "y": [-100, 100],
            "vx": [-20, 20],
            "vy": [-20, 20]
        },
        "absolute": False,
        "order": "sorted"
    }
}
env = gym.make('highway-v0')
env.configure(config)
obs = env.reset()
print(obs)
[[ 1.          1.          0.04        1.          0.          1.
   0.        ]
 [ 1.          0.1931498   0.         -0.13476129  0.          1.
   0.        ]
 [ 1.          0.40495145  0.         -0.18964133  0.          1.
   0.        ]
 [ 1.          0.61508363  0.08       -0.07828975  0.          1.
   0.        ]
 [ 1.          0.82810324  0.         -0.15318432  0.          1.
   0.        ]
 [ 1.          1.         -0.04       -0.15132266  0.          1.
   0.        ]
 [ 1.          1.          0.04       -0.09136145  0.          1.
   0.        ]
 [ 1.          1.          0.04       -0.06644491  0.          1.
   0.        ]
 [ 1.          1.          0.         -0.10128364  0.          1.
   0.        ]
 [ 1.          1.          0.08       -0.18424626  0.          1.
   0.        ]
 [ 0.          0.          0.          0.          0.          0.
   0.        ]
 [ 0.          0.          0.          0.          0.          0.
   0.        ]
 [ 0.          0.          0.          0.          0.          0.
   0.        ]
 [ 0.          0.          0.          0.          0.          0.
   0.        ]
 [ 0.          0.          0.          0.          0.          0.
   0.        ]]

Grayscale Image

The GrayscaleObservation is a \(W\times H\) grayscale image of the scene, where \(W,H\) are set with the observation_shape parameter. The RGB to grayscale conversion is a weighted sum, configured by the weights parameter. Several images can be stacked with the stack_size parameter, as is customary with image observations.

Example configuration

   from matplotlib import pyplot as plt
   %matplotlib inline
config = {
       "observation": {
           "type": "GrayscaleObservation",
           "observation_shape": (128, 64),
           "stack_size": 4,
           "weights": [0.2989, 0.5870, 0.1140],  # weights for RGB conversion
           "scaling": 1.75,
       },
       "policy_frequency": 2
   }
   env.configure(config)
   obs = env.reset()

   _, axes = plt.subplots(ncols=4, figsize=(12, 5))
   for i, ax in enumerate(axes.flat):
       ax.imshow(obs[i, ...].T, cmap=plt.get_cmap('gray'))
   plt.show()
../_images/index_1_0.png

Illustration of the stack mechanism

We illustrate the stack update by performing three steps in the environment.

for _ in range(3):
    obs, _, _, _ = env.step(env.action_type.actions_indexes["IDLE"])

    _, axes = plt.subplots(ncols=4, figsize=(12, 5))
    for i, ax in enumerate(axes.flat):
        ax.imshow(obs[i, ...].T, cmap=plt.get_cmap('gray'))
plt.show()
../_images/index_2_0.png ../_images/index_2_1.png ../_images/index_2_2.png

Occupancy grid

The OccupancyGridObservation is a \(W\times H\times F\) array, that represents a grid of shape \(W\times H\) discretising the space \((X,Y)\) around the ego-vehicle in uniform rectangle cells. Each cell is described by \(F\) features, listed in the "features" configuration field. The grid size and resolution is defined by the grid_size and grid_steps configuration fields.

For instance, the channel corresponding to the presence feature may look like this:

presence feature: one vehicle is close to the north, and one is farther to the east.

0

0

0

0

0

0

0

1

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

The corresponding \(v_x\) feature may look like this:

\(v_x\) feature: the north vehicle drives at the same speed as the ego-vehicle, and the east vehicle a bit slower

0

0

0

0

0

0

0

0

0

0

0

0

0

0

-0.1

0

0

0

0

0

0

0

0

0

0

Example configuration

"observation": {
    "type": "OccupancyGrid",
    "vehicles_count": 15,
    "features": ["presence", "x", "y", "vx", "vy", "cos_h", "sin_h"],
    "features_range": {
        "x": [-100, 100],
        "y": [-100, 100],
        "vx": [-20, 20],
        "vy": [-20, 20]
    },
    "grid_size": [[-27.5, 27.5], [-27.5, 27.5]],
    "grid_step": [5, 5],
    "absolute": False
}

Time to collision

The TimeToCollisionObservation is a \(V\times L\times H\) array, that represents the predicted time-to-collision of observed vehicles on the same road as the ego-vehicle. These predictions are performed for \(V\) different values of the ego-vehicle speed, \(L\) lanes on the road around the current lane, and represented as one-hot encodings over \(H\) discretised time values (bins), with 1s steps.

For instance, consider a vehicle at 25m on the right-lane of the ego-vehicle and driving at 15 m/s. Using \(V=3,\, L = 3\,H = 10\), with ego-speed of {\(15\) m/s, \(20\) m/s and \(25\) m/s}, the predicted time-to-collisions are \(\infty,\,5s,\,2.5s\) and the corresponding observation is

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

Example configuration

"observation": {
    "type": "TimeToCollision"
    "horizon": 10
},

API

class highway_env.envs.common.observation.GrayscaleObservation(env: AbstractEnv, observation_shape: Tuple[int, int], stack_size: int, weights: List[float], scaling: Optional[float] = None, centering_position: Optional[List[float]] = None, **kwargs)[source]

An observation class that collects directly what the simulator renders.

Also stacks the collected frames as in the nature DQN. The observation shape is C x W x H.

Specific keys are expected in the configuration dictionary passed. Example of observation dictionary in the environment config:

observation”: {

“type”: “GrayscaleObservation”, “observation_shape”: (84, 84) “stack_size”: 4, “weights”: [0.2989, 0.5870, 0.1140], # weights for RGB conversion,

}

space() gym.spaces.space.Space[source]

Get the observation space.

observe() numpy.ndarray[source]

Get an observation of the environment state.

class highway_env.envs.common.observation.KinematicObservation(env: AbstractEnv, features: List[str] = None, vehicles_count: int = 5, features_range: Dict[str, List[float]] = None, absolute: bool = False, order: str = 'sorted', normalize: bool = True, clip: bool = True, see_behind: bool = False, observe_intentions: bool = False, **kwargs: dict)[source]

Observe the kinematics of nearby vehicles.

space() gym.spaces.space.Space[source]

Get the observation space.

normalize_obs(df: pandas.core.frame.DataFrame) pandas.core.frame.DataFrame[source]

Normalize the observation values.

For now, assume that the road is straight along the x axis. :param Dataframe df: observation data

observe() numpy.ndarray[source]

Get an observation of the environment state.

class highway_env.envs.common.observation.OccupancyGridObservation(env: AbstractEnv, features: Optional[List[str]] = None, grid_size: Optional[Tuple[Tuple[float, float], Tuple[float, float]]] = None, grid_step: Optional[Tuple[float, float]] = None, features_range: Dict[str, List[float]] = None, absolute: bool = False, align_to_vehicle_axes: bool = False, clip: bool = True, as_image: bool = False, **kwargs: dict)[source]

Observe an occupancy grid of nearby vehicles.

space() gym.spaces.space.Space[source]

Get the observation space.

normalize(df: pandas.core.frame.DataFrame) pandas.core.frame.DataFrame[source]

Normalize the observation values.

For now, assume that the road is straight along the x axis. :param Dataframe df: observation data

observe() numpy.ndarray[source]

Get an observation of the environment state.

pos_to_index(position: Union[numpy.ndarray, Sequence[float]], relative: bool = False) Tuple[int, int][source]

Convert a world position to a grid cell index

If align_to_vehicle_axes the cells are in the vehicle’s frame, otherwise in the world frame.

Parameters
  • position – a world position

  • relative – whether the position is already relative to the observer’s position

Returns

the pair (i,j) of the cell index

fill_road_layer_by_lanes(layer_index: int, lane_perception_distance: float = 100) None[source]

A layer to encode the onroad (1) / offroad (0) information

Here, we iterate over lanes and regularly placed waypoints on these lanes to fill the corresponding cells. This approach is faster if the grid is large and the road network is small.

Parameters
  • layer_index – index of the layer in the grid

  • lane_perception_distance – lanes are rendered +/- this distance from vehicle location

fill_road_layer_by_cell(layer_index) None[source]

A layer to encode the onroad (1) / offroad (0) information

In this implementation, we iterate the grid cells and check whether the corresponding world position at the center of the cell is onroad/offroad. This approach is faster if the grid is small and the road network large.

class highway_env.envs.common.observation.KinematicsGoalObservation(env: AbstractEnv, scales: List[float], **kwargs: dict)[source]
space() gym.spaces.space.Space[source]

Get the observation space.

observe() Dict[str, numpy.ndarray][source]

Get an observation of the environment state.

class highway_env.envs.common.observation.ExitObservation(env: AbstractEnv, features: List[str] = None, vehicles_count: int = 5, features_range: Dict[str, List[float]] = None, absolute: bool = False, order: str = 'sorted', normalize: bool = True, clip: bool = True, see_behind: bool = False, observe_intentions: bool = False, **kwargs: dict)[source]

Specific to exit_env, observe the distance to the next exit lane as part of a KinematicObservation.

observe() numpy.ndarray[source]

Get an observation of the environment state.