Sensors and Compute: How Automated Vehicles See
An automated vehicle must measure a world that is partly hidden, constantly changing and often hostile to sensors; cameras, radar and lidar produce different data with different failure modes. The useful question is whether the complete architecture—including compute, positioning, cleaning, power and actuation—supports the feature’s operating domain and safety case.
Cameras: rich meaning, dependent on light
Cameras capture colour, texture and fine visual detail. They are central to recognizing lane markings, traffic-light state, signs, brake lights, road users, gestures and free space.
Their output is information-rich but indirect. Software must infer distance and motion from image sequences, multiple viewpoints or learned depth cues. Performance can degrade with glare, darkness, fog, rain, snow, dirt, low contrast and an obstructed lens. High dynamic range, exposure control, lens heating and cleaning help, but they do not remove every limitation.
Surround coverage also matters. A forward camera that supports lane keeping is not equivalent to a multi-camera system designed to monitor all directions without a human fallback.
Radar: direct range and relative velocity
Automotive radar transmits radio waves and measures their return. It is particularly useful for distance, relative speed and motion, and can remain effective in conditions that challenge visible-light cameras.
Traditional radar has lower angular resolution than cameras or lidar, which can make closely spaced objects and exact shapes harder to separate. Imaging radar increases spatial detail, but capability still depends on frequency, aperture, placement, signal processing and interference handling.
Radar is not a universal bad-weather guarantee. Heavy precipitation, reflections, multipath effects and installation geometry can still complicate interpretation.
Lidar: active three-dimensional measurement
Lidar sends laser pulses and measures their return time to build a three-dimensional point cloud. It can provide accurate geometry and distance without relying on ambient light.
The trade-offs include cost, packaging, power, contamination and performance in rain, snow, spray or highly reflective scenes. Wavelength, scanning method, range, resolution and placement vary widely, so counting lidar units says little by itself.
Level 4 systems often use overlapping camera, radar and lidar coverage because each modality supplies different evidence. Some consumer systems follow a camera-led strategy. Neither architecture can be judged from a parts list alone; the supported domain, fallback design and validation evidence are decisive.
Short-range and non-visual sensing
Ultrasonic sensors measure short distances and remain useful for parking, curb detection and close-range maneuvering. They are not designed for long-range road perception.
Wheel-speed sensors, steering-angle sensing and inertial measurement units track the vehicle’s own motion. Microphones can help detect sirens and other audible cues. Driver-monitoring cameras observe gaze, head position and availability in systems that depend on a human.
Every additional modality creates integration work. Time synchronization, calibration and confidence estimation are as important as the nominal sensor specification.
Positioning and maps
GNSS provides a global position but can be degraded by tunnels, buildings, interference or atmospheric effects. Automated systems therefore combine it with inertial measurements, wheel odometry and observed landmarks.
HD maps can add lane geometry, traffic controls and expected road structure. They are a prior, not ground truth. Construction and temporary traffic management can invalidate stored information, so the system must reconcile maps with the live scene.
Localization is not one coordinate. The software needs an estimate and an uncertainty: which lane the vehicle occupies, how it is oriented and whether the confidence is sufficient for the next maneuver.
Cleaning, calibration and self-diagnosis
A sensor suite that works only when freshly cleaned is not production-ready. Cameras and lidar may need washers, wipers, air jets or heaters. Enclosures must manage water, ice, vibration, thermal expansion and minor impacts.
Calibration aligns each sensor with the vehicle and with every other sensor. A small change in camera angle or body repair can corrupt fused geometry. The system needs manufacturing calibration, service procedures and diagnostics that detect drift or blockage.
The vehicle also needs to understand degraded capability. If one sensor is obscured, the correct response may be to reduce speed, narrow the domain, request a takeover or reach a minimal-risk condition. Continuing with unjustified confidence is the dangerous failure.
Compute and data movement
Onboard computers time-align sensor streams, run perception and prediction models, plan motion and issue control commands under real-time deadlines. Memory bandwidth, latency and thermal management can matter as much as peak processing figures.
Higher automation also needs safety monitoring that is sufficiently independent from the main driving stack. Power converters, wiring, networks and cooling become part of the safety architecture because the driving system depends on them.
Compute is an energy load. In an EV, that energy comes from the traction battery through low-voltage conversion, and heat must be rejected by the thermal system. Efficient accelerators and compact sensor suites can reduce the penalty, but it should be included in range and fleet planning.
Actuation and redundancy
Perception is useful only if the vehicle can steer, accelerate and brake predictably. Automated operation depends on electronically controllable steering, propulsion, friction braking, regenerative braking, gear selection and stability systems.
For Level 3 and Level 4, the architecture may need alternative paths for power, communication, braking, steering, localization or compute. Redundancy is not simply duplicating a component. Two identical sensors behind the same dirty windscreen share a failure cause. A meaningful design separates power, placement, sensing principle or control path where the safety analysis requires it.
The objective is controlled degradation: detect a fault, retain enough capability for the situation and reach the defined fallback state.
How to read a hardware claim
Do not rank systems by camera, radar, lidar or processor count. Ask:
- What is the feature’s operating domain and automation level?
- Which directions, ranges and failure modes have overlapping coverage?
- How are sensors cleaned, heated, calibrated and diagnosed?
- What happens after blockage, loss of localization or compute failure?
- Are steering, braking, communication and power paths fault-tolerant?
- Has the integrated system been validated, rather than only the components?
Hardware enables capability. It does not prove safe behavior on its own.