What you do in this lab

You start one real training run on a cloud GPU, and you read it while it runs. Before the button, this page walks you through every piece the run is made of: the body, what it sees, how it is scored, when a try ends, the recipe, and the knobs you may turn.

Read the cards in order. Each one asks you to do one thing. The last card is the button.

Isaac Lab makes 1024 copies of the twin, each on its own course of 10 waypoints about 5 m apart. All 1024 cars live in one tensor. One sim step moves every car at once. Headless: numbers come back, not pictures.

The body you train

Do this: drive the car below. This is the twin as the physics engine sees it: a box for the chassis, a nose, four cylinders for the wheels, with the real masses and sizes. That is all training ever touches. Grip, the steering lock, what happens in a collision: all from these shapes. Full steer is 26.37 degrees, the angle the real car's servo reaches at the end of its calibrated travel: the same number in the sim, in SIL, and on the floor.

Then do this: switch the body to the scan mesh. Nothing about the driving changes. The mesh is a skin: we filmed the real car, built a splat from the video (a 3D Gaussian splat: a cloud of coloured blobs that renders like a photo from any side), turned the splat into a mesh (a surface of triangles), and laid the mesh over the same joints: the chassis mesh on the chassis body, a wheel mesh on each wheel.

Training never sees the mesh. Physics wants few, simple shapes: fast, stable, thousands of copies. Eyes want the real car. Keeping the two apart means the pretty one can change any time without training again.

What the car sees

Not a camera. Not lidar. Each step, the policy gets three numbers: the distance to the next waypoint, and the cos and sin of the heading error. You watched them in The Twin.

// one angle as cos and sin: no jump at +/-180 degrees. No perception, on purpose.

The reward: how a step is scored

Every step, every car gets a number. Training pushes the policy toward actions that got bigger numbers. Four terms, with the weights from the env's cfg:

• progress x 1.0 - how much closer to the waypoint than last step. Going backwards costs.
• heading x 0.05 - a small bonus for facing the waypoint; it fades fast with the angle (coefficient 0.25).
• goal reached + 10.0 - once, when the car is within 0.4 m. The next waypoint lights up.
• steer rate - 0.05 x (change in steer)^2 - a tax on jerking the wheel. Without it the policy bang-bangs the steering.

Do this: read the lines below top to bottom and find the four terms in the reward = (...) sum. This is the file the GPU runs, not a copy.

sim/isaaclab/GoatRacer/GoatRacer/tasks/goatracer_fiesta_3obs_env.py

    def _get_rewards(self) -> torch.Tensor:
        c = self.cfg
        position_progress = self._previous_position_error - self._position_error
        heading_rew = torch.exp(-self._target_heading_error.abs() / c.heading_coefficient)
        goal_reached = self._position_error < c.position_tolerance
        self._target_index = self._target_index + goal_reached.long()
        self.task_completed = self._target_index > (c.num_goals - 1)
        self._target_index = self._target_index % c.num_goals

        # squared steer change — _actions2/_last_actions2 are maintained by the
        # chassis' _pre_physics_step
        dsteer = self._actions2[:, 0] - self._last_actions2[:, 0]
        self._ep3_dsteer += dsteer.abs()
        self._ep3_steps += 1.0
        reward = (
            position_progress * c.position_progress_weight
            + heading_rew * c.heading_progress_weight
            + goal_reached.float() * c.goal_reached_bonus
            - dsteer.pow(2) * c.steer_rate_weight
        )

        one_hot = torch.nn.functional.one_hot(self._target_index, num_classes=c.num_goals)
        self.waypoints.visualize(marker_indices=one_hot.view(-1).tolist())
        return torch.nan_to_num(reward, nan=0.0)

When a try ends

A try (an episode) ends three ways: the car finished all 10 waypoints (success), the clock ran out (30.0 s), or the car flipped. Then that copy resets to the start of a fresh course and goes again. 1024 copies, each on its own clock: at any moment some are starting and some are finishing.

sim/isaaclab/GoatRacer/GoatRacer/tasks/goatracer_fiesta_3obs_env.py

    def _get_dones(self) -> tuple[torch.Tensor, torch.Tensor]:
        # original contract (+ physics reality: a flipped car has failed).
        # NOTE: timeout deliberately lands in the *terminated* slot — the v3
        # chassis fixed that order (see its _get_dones); this env keeps the
        # original layout on purpose. Don't "fix" either one.
        flipped = self.robot.data.projected_gravity_b[:, 2] > -0.5
        task_failed = (self.episode_length_buf > self.max_episode_length) | flipped
        if hasattr(self, "_nan_envs") and self._nan_envs.any():
            task_failed = task_failed | self._nan_envs
            self.extras["nan_resets"] = self._nan_envs.sum()
        # eval_v3_speed reads these: "laps" = completed courses; no crash concept here
        self.extras["lap_completed"] = self.task_completed.clone()
        self.extras["crashed"] = torch.zeros_like(self.task_completed)
        return task_failed, self.task_completed

Do this: find flipped. It reads the gravity vector in the car's own frame: when "down" is no longer below the car, the car is on its roof.

The loop and the network

One iteration: every car drives 32 steps with the current policy, the rewards are summed up, and PPO (the training algorithm) pulls the weights toward what scored well. Then again. Your run is 60 iterations.

The network it pulls:

// 3 obs -> dense 32 -> ELU -> dense 32 -> ELU -> dense 2 · 1,250 params

• Input - the three numbers.
• Layers - each dense layer multiplies by learned weights, adds a bias, and bends through ELU.
• Output - steer and drive.
• Parameters - 1,250 weights. Training moves these toward a higher reward.

The recipe: every hyperparameter

Everything PPO is told, from the recipe file the GPU reads. Nothing here is typed into this page; it is the file. The ones you may turn in the Train card are marked.

sim/isaaclab/GoatRacer/GoatRacer/tasks/agents.py - class GoatRacerFiesta3ObsPPORunnerCfg

Do this: pick one knob and decide what you expect if you double it. Then, after your first run, do a second with that one change and the same seed, and compare the curves.

How to read the curve

A real run of this task: reward starts near zero, passes 10 by iteration 10, 70 by iteration 20, and levels off near 80. That climb and flatten means it learned.

Where it flattens, more iterations stop buying skill. Remember that spot. The next lab abuses it.

More worlds learns faster per iteration, same plateau. Past a full GPU, more worlds buys smoother, not better.

What you get back, and the button

The run ends with the curve, a replay of the policy driving, and policy.onnx, the exported policy file in the format the real car loads. The Jetson on the car runs it with no PyTorch and no GPU.

Do this: in the Train card below, leave every knob where it is, press the button, and watch the curve. Your first run is the baseline every later run is compared to.

// You are reading this with no account. The Train card shows a run that is already done; an account starts your own.

Follow the White Rabbit

• Spinning Up: PPO - the algorithm, with the maths behind clip_param and gamma.
• Isaac Lab docs - the simulator.
• Netron - open your policy.onnx in it and see the same picture.