Camera-Only vs Multimodal Sensing

“Camera-only” and “multimodal” describe engineering architectures, not automation levels or safety ratings. Either can support useful functions, and either can fail if its requirements, operating limits or validation are weak.

Why the argument is often misleading

Public debate frequently starts with a preferred sensor and works backward to a verdict. The more disciplined sequence is:

Define the driving function and its operational design domain.
Define the objects, road features and adverse conditions the system must handle.
Allocate sensing, computation and fallback requirements.
Validate the complete vehicle against those requirements.

A system that follows a route, changes lanes and responds to traffic controls can still be SAE Level 2 when the human must supervise continuously. Conversely, fitting lidar does not make a vehicle autonomous. Automation level concerns the allocation of the dynamic driving task and fallback, not the bill of materials.

What a camera-centric design can do

Cameras carry dense angular detail, color and texture. They are well suited to lane markings, traffic-light state, sign text, gestures, road-edge interpretation and object classification. Multiple cameras, temporal sequences, learned priors and vehicle motion can also estimate depth.

This means the claim “a camera cannot measure distance” is too broad. Stereo geometry and motion parallax can recover metric depth when their assumptions hold; monocular networks can estimate depth from learned visual structure. The qualification matters: these are generally indirect estimates whose uncertainty changes with texture, lighting, occlusion, calibration and scene novelty. Radar and lidar obtain range from returned signals more directly.

Camera-centric systems also have practical attractions: cameras are inexpensive, compact and already necessary for semantic road understanding. One perception stack can learn from large video fleets. Fewer sensor modalities can reduce packaging, calibration and fusion complexity.

What camera-centric does not prove

Humans primarily use vision, but “people drive with two eyes” is not a safety proof for a vehicle. Human perception includes active gaze, a movable head, contextual reasoning, memory, physical intuition and the ability to stop when uncertain. Humans also crash.

Camera performance can deteriorate with glare, darkness, low contrast, lens contamination, precipitation, condensation and partial occlusion. Tesla's own owner information warns that blocked or degraded cameras can make driver-assistance features unavailable or impaired. See Tesla owner's manual: camera limitations and troubleshooting.

Removing radar or ultrasonic sensors can still be a rational product decision if the remaining architecture is validated for the promised functions. It does not establish that the removed modality lacks useful information.

Tesla Vision and Musk's lidar criticism

Tesla documents a transition of several vehicle configurations to the camera-based Tesla Vision architecture, including software that disabled the front radar on specified Model S vehicles. See Tesla service bulletin: transition to Tesla Vision and Tesla: Autopilot and Full Self-Driving Capability. At Tesla's 2019 Autonomy Investor Day, Elon Musk argued strongly that lidar was unnecessary for Tesla's autonomous-driving approach; the event is preserved in Tesla's investor materials at Tesla: 2019 Autonomy Investor Day.

Those statements should be read as Tesla's architecture thesis:

solve the road scene principally through vision and neural networks;
use fleet data and vertically integrated software to improve the stack;
avoid the cost and integration burden of lidar, and on some configurations radar and ultrasonic sensing.

They are not controlled comparative evidence that lidar or radar is useless in every driving system. Tesla's approach must be judged by the behavior, limitations and safety evidence of each released feature and software version. The company also states that current Autopilot features require active driver supervision and do not make the vehicle autonomous.

Why other developers retain radar and lidar

Radar adds direct range and radial-velocity measurements and is comparatively insensitive to ambient illumination. Lidar supplies explicit 3D geometry at high angular resolution. These signals can constrain vision estimates, improve observability in selected conditions and enable independently developed perception paths.

Waymo's sixth-generation automated-driving hardware uses cameras, radar and lidar with overlapping fields of view. See Waymo: sixth-generation Waymo Driver. Mercedes-Benz describes DRIVE PILOT as using lidar, camera, radar and ultrasonic sensors together with high-precision positioning and an HD map. See Mercedes-Benz: DRIVE PILOT technology. Mobileye describes a camera subsystem and a separately developed radar-lidar subsystem intended to provide independent perception channels. See Mobileye: True Redundancy.

These examples do not prove that “more sensors are always safer.” They show that developers with different products, operating domains and fallback responsibilities have made different allocations.

The costs of adding modalities

Every additional sensor can introduce:

packaging, heating, cleaning and repair requirements;
calibration and time-synchronization errors;
electromagnetic or optical interference;
more data, compute and thermal load;
false associations and correlated training errors;
common dependencies that undermine apparent redundancy.

Fusion can improve perception only if the system understands the reliability of each input and handles disagreement. See Sensor fusion.

Claims that need qualification

“Lidar sees perfectly in darkness and weather.” Darkness is a strength because lidar is active, but fog, rain, snow, spray, contamination and target reflectivity can reduce usable range or create returns. The effect depends on the sensor and condition.

“Radar works in all weather.” Radar is often less affected than visible-light sensors, but rain clutter, multipath, interference, angular resolution and target reflectivity still matter.

“More sensors equal redundancy.” Only if the architecture can meet a defined requirement after a failure, detects that failure, and avoids critical common causes.

“Camera-only can never be safe.” Sensor modality alone cannot establish impossibility. The claim must be tested against a defined function and domain.

“A successful demonstration proves the architecture.” A demonstration is one observation. It rarely reveals false-positive rates, rare-event performance, degraded modes or changes across software versions.

How to compare real systems

Ask for evidence at the system level:

Which features and roads are included, and who performs fallback?
Which hardware and software version was tested?
What happens with a blocked or miscalibrated sensor?
Are daylight, darkness, glare, rain, fog, snow and road spray represented?
Are results reported with both missed detections and false interventions?
Does the test include complete braking or steering behavior, not just object detection?
Is the comparison independent, repeatable and representative of the deployed operational design domain?

ISO 21448 provides a framework for considering hazards from perception and algorithm insufficiencies even without a conventional hardware fault. See ISO 21448:2022 — Safety of the intended functionality. The wider evidentiary standard is discussed in Safety and evidence for automated driving.

The defensible conclusion

Camera-only is a coherent architecture with real advantages and hard limitations. Multimodal sensing is a coherent architecture with different advantages and hard limitations. Neither label is a shortcut to safety.

The strongest system is the one whose sensing architecture is sufficient for its assigned task, whose uncertainty and failures are controlled, and whose claims survive representative vehicle-level testing.