The iPhone 12 – LiDAR At Your Fingertips

The LiDAR in Apple’s new iPhone moves 3D imaging from the esoteric to the ordinary. It is a significant boost for the technology, similar to photography’s transition in the mid-2000s from a specialized world of cameras and lenses to a simple finger-click that can be relished and shared via a general-purpose smartphone by billions of consumers. LiDAR is similarly positioned to find applications in the consumer and business domain – for photography, AR/VR, entertainment, socialization, collaborations, autonomy, security, and gaming.

There are significant applications for LiDAR beyond the consumer realm, some of which demand higher performance in terms of range and resolution than offered by the iPhone 12 LiDAR. This includes robotics and movement automation where this revolutionary mode of imaging is being positioned for use as a “safety” rather than as a “comfort” or “enjoyment” feature. Will deployment of the iPhone 12 LiDAR provide a boost to these other applications?

One reason it will is that like cameras, the entire semiconductor ecosystem will gear up to produce billions of electronic and optical chips to support the consumer smartphone and personal computing LiDAR market – and create a robust and mature infrastructure for other applications to leverage.

As discussed in a previous article, there are many ways to construct a LiDAR for automotive and robotics applications (Table 1). Over 75 automotive LiDAR companies are pursuing different designs based on combinations of wavelengths, Field of View (FoV) scanning approaches, and the physical principle used (Time of Flight or ToF with linear detectors, ToF with Photon Counting Detectors and Frequency Modulated Continuous Wave or FMCW).

The iPhone 12 LiDAR falls in cell ❼ – (flash illumination and no scanning, although the figure above calls out a LiDAR scanner). It operates at the 8XX nm wavelength, and uses Photon Counting detectors (also known as SPADs or Single Photon Avalanche Photodiodes) and Vertical Cavity Surface Emitting Lasers (or VCSELs). Generally, LiDAR companies in other cells use EEL (Edge Emitting Lasers) or fiber lasers.

The architecture choices for the iPhone 12 LiDAR are driven primarily by size, packaging constraints and cost. VCSELs and flash illumination (with no moving parts to scan the laser) enable miniaturization. In terms of performance, this LiDAR has a range of ~ 5m and a limited FoV – perfectly adequate for the types of consumer applications that it is meant to promote.

2 questions for deploying the architecture for automotive safety applications are:

1) Can it support higher performance requirements?

• 200 m range at low reflectivity
• 120° X 30° FoV (may be lower for ADAS, 90° X 30°)
• Good angular resolution (0.2° X 0.2°)
• High point cloud density (~ 1M points/second) ?

2) Is higher performance achievable at an affordable cost point?

VCSELs are an attractive illumination choice at the 8XX-9XX nm wavelengths – one advantage is that it is a planar illumination device (as opposed to EELs which require significantly higher wafer-level processing and complex packaging). Other features include easier integration of driver circuitry across large 2D arrays and 5X lower temperature variation of wavelength as compared to EELs (allows use over automotive temperature ranges without active cooling or temperature stabilization).

Traditionally, VCSELs have suffered from lower brightness (energy emitted/area) as compared to EELs. This leads to large emitter sizes – impacting cost and system size. VCSELs are processed on 6″ diameter (150 mm) Gallium Arsenide (GaAs) wafers and cost about $10,000/wafer at modest volumes. With a usable area of ~ 15,000 mm², this translates to ~$0.6/mm². This metric is important to consider since scaling to higher performance will require larger VCSEL chip areas and impact system pricing.

Another drawback of VCSELs is that they are realizable only at the shorter wavelengths (high power 1550 nm high power VCSELs do not exist). This can be a constraining factor because of eye-safety. To date, the only credible LiDAR data sets showing 200m range performance are based on 1550 nm systems using EELs or fiber lasers (cells ❷, ❸, ❻ in Table 1). Many of the 8XX-9XX nm players (cells ❶, ❹, ❺ and ❼ in Table 1) make claims about achieving this performance – not backed up by publicly available data. This may be the reason why Continental recently chose to invest in Aeye (even though it has an independent LiDAR effort at 1064 nm wavelength) and Daimler/TORC announced a partnership with Luminar (Aeye and Luminar LiDARs operate at 1550 nm and have publicly demonstrated the long-range performance required for L3 and L4 trucking). It will be interesting to see if the VCSEL-SPAD LiDAR architectures can demonstrate this level of performance. If they do, the more expensive 1550 nm ToF solutions will become irrelevant (Cells ❸, ❻ in Table 1).

VCSEL chip size and cost, and the ability to provide high range performance at lower wavelengths are critical factors that will drive scalability of the iPhone LiDAR architecture to the higher performance automotive safety applications.

Once VCSELs are used as the illumination choice, it almost dictates the choice of SPAD detectors. This is because SPADs are single photon sensitive – this means that instead of a large energy pulse required with a traditional linear detector to extract 3D information via ToF approaches (Cells ❶, ❸ and ❺ in Table 1), a series of lower power pulses can be used to extract the same information. Given the lower peak power capability of a VCSEL (as opposed to an EEL), this is a natural fit. SPADs need narrow spectral filtering to reduce false returns due to solar background – and the lower VCSEL wavelength variation with temperature allows for practical deployment of SPAD filters and overall manufacturing yields. Additionally, at the 8XX-9XX nm wavelengths, SPADs and readout electronics can be monolithically fabricated on silicon wafers using CMOS processing, leading to cost-effective solutions (generally not true for linear mode APDs).

Table 2 compares EELs and VCSELs with different combinations of detectors and scanning and its impact on the required laser power and semiconductor chip area (based on typical system designs):

EELs typically achieve power densities of 100 W/mm². Flash solutions using EELs with linear APDs (Row 1) are impractical for long-range and large FOV since they demand high peak power and large chip area. Some companies like Continental have a flash solution (Row 1), although it is for lower range and FoV. Using mechanical scanning (Row 2) alleviates this issue, and is the direction chosen by Velodyne, Hesai, and Innoviz (Cell ❶ in Table 1).

VCSELs traditionally have had 10X lower brightness and power density than EELs (Rows 3 and 5). The use of SPADs dramatically reduces the required peak power (by 100X over the use of a linear mode APD) although multiple pulses are required to create adequate signal/background noise performance. Recent advances in VCSEL technology have increased power density through the use of multiple emitting junctions and higher quality wafer processing. As VCSELs start approaching EELs (100 W/mm²), flash or semi-flash solutions start becoming more feasible (Rows 4 and 6). The challenge for VCSELs in terms of scaling performance is to surpass the 10W/mm² power density and move or exceed EEL benchmarks of 100 W/mm².   At the current costs of $0.6/mm², a VCSEL chip would cost ~$500 – this needs to drop by ~3-5X to support LiDAR system pricing of < $1,000/unit. Achieving this requires higher VCSEL brightness, lower wafer processing costs and more efficient system designs.

Current Status and Future Directions

Some automotive LiDAR companies are pursuing similar architectures as the iPhone 12 (8XX/9XX nm VCSELs and SPADs). This includes Sense Photonics, Ouster, Opsys Tech, and ZF/Ibeo. Lumentum reportedly supplies VCSELs for the iPhone 12 LiDAR and is also making a focused effort to penetrate the automotive LiDAR market. Trilumina, AMS, Leonardo-LaserTel, and II-VI are also addressing the VCSEL market for automotive and other applications.

Lumentum will certainly play an important role in improving the power density performance and costs of VCSEL chips as they scale for the smartphone market. They recently announced VCSELs approaching power densities of ~ 800W/mm² at 125°C, surpassing EEL performance.   The spectral shift over a -40°C to 125°C temperature range (typical automotive operating temperature ranges) is ~10 nm, which allows for uncooled operation, manufacturability, and less demanding solar filters. According to Matt Everett, Director of Automotive Product Management at Lumentum, these high power VCSELs have been designed and tested, and are currently in the product development stage. Sampling has begun and scale-up for production and automotive deployment will occur in 2023. Lumentum believes the smartphone demand for VCSELs creates significant volumes and GaAs processing infrastructure, which will be leveraged to produce cost-effective VCSELs for automotive deployments.

Sense Photonics is pursuing pure flash operation with SPADs and VCSELs, similar to the iPhone 12 architecture. The company’s long-range product for ADAS and AVs is in development and uses a 940 nm VCSEL array and SPAD-based silicon TOF image sensor that can operate at a 200 m range. As discussed in Table 2 (Row 4),  flash systems typically need to generate ~ 90 kW of optical power and require 900 mm² of chip area. It will be interesting to see how Sense overcomes that barrier. In general, such high peak power can cause eye-safety issues, but Sense’s unique IP enables mounting of individual VCSEL emitters on a flexible and curved substrates over a larger area. This allows for eye-safe performance and superior thermal performance. VCSEL costs will be managed either by a combination of improved VCSEL brightness and superior SPAD designs. “Stretching” the laser power over larger areas increases system size, although Sense’s approach to physical separation of the transmit and receive portion of the LiDAR is an advantage. Current Sense products use classical indirect-ToF detectors rather than SPADs and have a more limited range (65m for the 80° x 30° variant).  Hod Finkelstein (Sense’s CTO) indicates the shift to in 2021 to long-range products based on their proprietary SPAD designs. The unique SPAD design allows for acquiring photons and digitizing the histogram data of more than 100,000 pixels simultaneously. In addition, the chip can perform edge level processing  to reduce transmission bandwidth and deliver useful data to the automotive perception stack. This significantly reduces latency and computing resources which are attractive to ADAS and AV customers.

Ouster has announced the ES2 LiDAR (expected to launch with prototypes in 2022 and serial production in 2024). It promises 200 m range over a 26° X 13° FoV, has no moving parts, and uses a VCSEL array that flashes the VFoV and electronically scans across the HFoV (relies on proprietary driver circuits than can selectively illuminate columns of pixels in the VFoV). Per Angus Pacala (Ouster CEO), a pure 2D flash approach is difficult – primarily because of the complexity of designing a laser driver to pulse a large current through thousands of VCSELs (the ES2 has 10,000 VCSEL elements, constructed in a 2D format). The ES2 leverages the experience from the mature OS series rotating LiDAR product line (like the SPADs and VCSEL chips). A key piece of IP that it brings to bear is the ability to manage solar background effectively through the use of filters integrated into the SPAD chip. The ES2 is targeted at a $600 price tag (uses roughly 90 mm² of VCSEL chip area for the smaller FoV compared to Row 6 in Table 1). The company expects to expand the FoV of future ES2 generations as VCSELs become brighter and other innovations lead to a more efficient system design (reducing the VCSEL and SPAD semiconductor costs).

Opsys Tech was founded by ex-Finisar (now part of II-VI) management, with a deep background in optics and VCSELs. The company advertises a pure solid-state LiDAR solution using VCSELs and SPADs and claims to be the first LiDAR company to use matrix-addressable VCSELs which enable system and performance advantages as compared to traditional 1D/2D flash LiDAR. Mark Donovan, Vice President of Product Development believes there are limitations to operating with traditional flash – eye-safety becomes an issue, there is more potential for interference from other LiDARs, and ROI operation is not possible. Electronic addressability across the VFoV and the HFoV allows operation with higher optical power density compared to 1D/2D flash, allowing longer range. The current SP2 product achieves 120 m range over a 23° X 13° FoV, with an impressive angular resolution (0.1° X 0.1°) and a raw frame rate of 1000 Hz. A total of 16 averages per point to reduce solar background and detector dark noise yields a processed frame rate of 30 Hz. Opsys Tech claims that it achieves the 100W/mm² VCSEL power density today, and needs ~100 mm² of VCSEL semiconductor area to meet this performance. Each VCSEL requires 20 A of peak current to generate 100W at short pulses and low duty cycles. In the future, the company believes that 500W/mm² is possible. Future iterations of the product (SP3) will achieve a 200m range, with the same FoV.  Getting to the larger FoVs will require multiple LiDARs – this can create cost and integration issues. Opsys claims that improvements in the VCSEL power density, as well as their experience using 8″ wafer platforms will allow them to deliver high performance and compact system sizes at competitive price points.

Apple’s decision to introduce LiDAR into the iPhone will mature the semiconductor ecosystem to produce the required optics and electronics more efficiently, and accelerate higher performance and lower cost. These advances will favorably impact more demanding LiDAR applications for automotive safety in terms of maturity, cost and performance. Very importantly, it will also promote consumer familiarity with LiDAR and acceptance of this critical technology for automotive safety and autonomy.