How to Validate Sensor and Perception Claims

Senast ändrad: juli 29, 2026

Sensor specifications describe components under stated conditions; they do not establish that a vehicle will perceive, decide and act safely in the real world. Validation must connect measurements to the complete driving function.

Four different questions

A useful evaluation separates four layers:

  1. Measurement: How accurately does the sensor report light, range, velocity or temperature contrast?
  2. Perception: How well does software detect, classify and track relevant objects and road features?
  3. Driving policy: Does the system predict motion and choose an appropriate response?
  4. Vehicle behavior: Does steering or braking achieve the required result without creating unacceptable false interventions?

A strong result at one layer cannot be substituted for the next. A lidar can measure a pedestrian-shaped surface precisely while the classifier labels it incorrectly. A camera can recognize a red light while the route logic associates it with the wrong lane. Radar can track a vehicle accurately while the planner leaves insufficient braking margin.

How to read a range claim

“Detects objects at 250 metres” is incomplete unless it identifies:

  • the target's size, material and optical reflectivity or radar cross-section;
  • the field of view and angular location;
  • probability of detection and false-alarm rate;
  • weather, lighting and sensor-cover condition;
  • minimum range, resolution, update rate and latency;
  • the algorithm and software version;
  • whether “detection” means one return, a classified object or a stable track.

Supplier numbers can be useful engineering inputs, but they are not directly comparable when their definitions differ. NIST's work on methods for evaluating 3D lidar emphasizes measurement uncertainty, latency, environmental conditions and target properties. See NIST: methods to evaluate 3D lidars used in automated driving.

Conditions must be representative

Good-weather averages hide the cases most likely to separate architectures. Test matrices should cover illumination, glare, darkness, precipitation, fog, spray, contaminated covers, road geometry, unusual objects, occlusion and degraded calibration. They should also cover combinations rather than changing one variable at a time only.

The required set comes from the function's operational design domain and foreseeable misuse. A highway pilot restricted to divided roads has different evidence needs from an urban point-to-point Level 2 feature or a driverless Level 4 service.

Both misses and false actions matter

A low missed-detection rate can be purchased with many false positives. For a warning system that may be annoying; for steering or automatic braking, repeated false interventions can create risk and erode trust. Reports should therefore include precision and recall, false interventions per distance or time, severity, time-to-collision distributions and confidence intervals where appropriate.

Rare events require substantial exposure or targeted scenario generation. Simulation and closed-course tests improve coverage, but their models and scenario selection also need validation. Public-road distance alone can contain little relevant exposure.

Comparison tests need controls

A fair comparison should use:

  • production-equivalent hardware and current software;
  • the same initial conditions, targets and success criteria;
  • repeated runs and disclosed failures;
  • both favorable and adverse conditions;
  • a complete description of feature settings and driver intervention;
  • independent observation when commercial claims are involved.

A supplier demonstration against an unnamed “camera-only” vehicle cannot identify why one run differed. Hardware generation, software, speed, target, braking policy and tuning may all vary.

Compliance, consumer tests and safety evidence

Regulations and consumer programs establish defined performance requirements; they do not certify universal safe driving. UNECE Regulation No. 152 addresses automatic emergency braking for specified vehicle categories and scenarios. See UNECE Regulation No. 152 — Advanced Emergency Braking Systems. NHTSA's FMVSS No. 127 sets pedestrian AEB and related requirements for new light vehicles in the United States. See NHTSA: FMVSS No. 127 automatic emergency braking. Euro NCAP evaluates selected safety-assist scenarios and updates its protocols over time at Euro NCAP Safety Assist.

ISO 21448 addresses unreasonable risk arising from functional insufficiencies and foreseeable misuse when the intended functionality depends on complex sensing and algorithms. See ISO 21448:2022 — Safety of the intended functionality. It complements, rather than replaces, vehicle-level scenario testing and the safety case described in automated-driving safety case.

A practical evidence ladder

Treat claims with increasing confidence as evidence moves through these stages:

  • physical principle or simulation;
  • laboratory component test;
  • controlled track test;
  • repeatable vehicle test across conditions;
  • independent consumer or regulatory assessment;
  • monitored field performance with defined exposure and reporting;
  • evidence that the complete system remains safe through degradation and fallback.

No single rung answers every question. Field data can miss rare scenarios; controlled tests can simplify the road; a standard can lag deployment. Converging evidence is stronger than one spectacular result.

The standard EVKX will use

EVKX will describe which measurements a sensor can provide, the conditions that change its performance, and the architecture in which it is used. We will not infer that a vehicle is safer merely because it has more sensors, or that one sensor is unnecessary because a feature worked without it.

For each driver-assistance function, the relevant statement is configuration-specific: which sensors supply evidence, what the software does with it, what the driver must supervise, and which limitations the manufacturer declares.

Sources

Mer information