The Automated-Driving Software Stack

Automated-driving software converts uncertain sensor data into physical motion by estimating location, understanding the scene, predicting how it may develop, choosing a lawful path and controlling the vehicle. The familiar pipeline is perception, localization, prediction, planning and control; machine learning can merge some stages, but the safety questions remain.

Perception builds a world model

Perception software detects lanes, road edges, free space, signs, signals, vehicles, pedestrians, cyclists and other relevant objects. It tracks them over time and estimates position, velocity, orientation and uncertainty.

Sensor fusion combines camera images, radar returns, lidar point clouds and vehicle-motion data. Fusion is not a vote in which the majority wins. The software must account for timing, calibration, sensor confidence and correlated failure. A reflection, occlusion or blocked lens can make one source misleading.

A useful world model also represents what is not visible. If a pedestrian is partly hidden behind a van, the system should carry that uncertainty into prediction and planning rather than treating the unseen area as empty.

Localization estimates position and confidence

Localization combines GNSS, inertial sensing, wheel motion, maps and observed road features. The output is an estimated pose—position and orientation—plus uncertainty.

That uncertainty changes what maneuvers are safe. A vehicle that cannot confidently identify its lane should not begin a close lane change merely because the route planner wants one. In tunnels, urban canyons or changed road layouts, the system must rely less on a single positioning source.

Prediction represents several possible futures

Other road users do not follow scripts. A cyclist may avoid a pothole, a child may emerge from an occlusion and a driver may turn without signaling.

Prediction models estimate possible trajectories and intentions, ideally with probabilities rather than one certain future. Planning then needs to remain safe across credible alternatives. This is different from trying to guess the single most likely action.

Uncertainty should change behavior. More uncertainty can justify lower speed, greater separation or postponing a maneuver. Excessive caution can also create hazards by blocking traffic or behaving unpredictably, so the system must make progress without pretending uncertainty is gone.

Planning chooses behavior and motion

Planning operates at several scales:

Route planning selects roads and destinations.
Behavior planning decides whether to follow, yield, merge, change lane or stop.
Motion planning creates a collision-avoiding trajectory with speed and curvature.

The plan must respect traffic rules, physical limits, passenger comfort and the behavior other road users can understand. A mathematically collision-free path can still be poor if it cuts too close to a cyclist or hesitates ambiguously in an intersection.

Planning also needs a fallback path. The software should know what it can do if a lane closes, a sensor degrades or the destination becomes unreachable.

Control turns a trajectory into movement

The controller translates the planned path into steering, motor torque and braking. It must account for speed, tire grip, road gradient, payload and actuator delay.

EVs add brake blending: regenerative braking and friction braking must deliver the requested deceleration predictably as battery state, temperature and traction change. Smoothness is not cosmetic. Abrupt control can destabilize passengers, reduce grip and make the vehicle’s intentions harder for others to read.

Independent monitors can check whether commanded and actual motion diverge, whether the vehicle remains inside a safe envelope and whether the primary stack is healthy.

Machine learning is a component, not an assurance argument

Machine learning is widely used for perception, prediction and increasingly planning. Some systems use a modular architecture; others use learned models that map more directly from sensor input to trajectories. Real implementations can sit anywhere between those descriptions.

Neither “end-to-end AI” nor a hand-engineered pipeline proves capability. The questions are:

What data and scenarios define the intended behavior?
How are rare, biased or mislabeled cases found?
How does the system represent uncertainty?
Which constraints sit outside the learned model?
How is a software change evaluated for regression?
What independent monitor can contain an unsafe output?

ISO/PAS 8800 addresses safety risks from AI output insufficiencies, systematic errors and random hardware errors in road vehicles. It reflects the central problem: high average model accuracy is not the same as an acceptable safety argument.

Training, testing and validation are different jobs

Training improves the model. Testing looks for failures. Validation asks whether the complete system is suitable for its intended domain.

Developers combine:

recorded-data replay;
software- and hardware-in-the-loop testing;
simulation with controlled variations;
closed-course scenarios;
supervised public-road testing; and
monitored driverless operation after deployment.

Simulation can generate dangerous and rare scenarios without exposing the public, but it is only useful to the extent that vehicle, sensor and road-user models represent reality. Public-road mileage provides operational evidence but is inefficient for proving performance in rare, precisely defined hazards. A credible program needs both scenario-based and exposure-based evidence.

Passing a fixed test set is not enough. Software can overfit the benchmark, and one improvement can create a regression elsewhere.

Fallback and remote assistance

The stack continuously checks the operating domain, sensor health, localization quality and vehicle state.

At Level 2, a limit returns the complete task to a driver who should already be supervising. At Level 3, the system manages a request to intervene and a risk-reduction response. At Level 4, it must reach a minimal-risk condition without depending on a driver.

Driverless fleets may use remote assistance for context, such as confirming that an unusual object is a temporary barrier. Remote assistance is different from continuous remote driving. The vehicle must remain responsible for safe motion unless its operating model explicitly uses a remote driver.

Updates change a safety-critical product

Over-the-air updates can improve perception, expand a domain or alter driving behavior. They can also create regressions. A new version therefore needs traceability, security, staged deployment, monitoring and a rollback or containment strategy.

An update does not change the SAE level by marketing declaration. The complete feature—including hardware, software, operating domain, fallback, approval and user instructions—must support the new role.

For buyers, the most visible software behavior is often the interface. The vehicle should clearly communicate which feature is active, what it is doing, what it expects from the human and why it is approaching a limit. Mode confusion is a system defect, not merely user error.