How Does LiDAR Sensor in iPhone Work and How to Use It?

Did you know that your iPhone might have an advanced LiDAR sensor?

LiDAR? What is a LiDAR? How can I use it? And I have it on my iPhone?

What is LiDAR?

LiDAR stands for “Light Detection and Ranging.” It is a remote sensing method that creates high-resolution three-dimensional models of topography and constructed features. If we simplify it enough, it’s like a tape measure. Except LiDAR sensor uses the speed of light to measure distance. It measures how long light takes to reflect off an object and return to its source. The sensor sends that data to a processor that generates a point cloud.

LiDAR scanners were traditionally used primarily in surveying and nowadays are commonly used in the automotive industry. LiDAR is especially helpful for autonomous vehicles to “understand” the traffic by detecting the edge of a road, people, and other cars.

Thanks to Apple, we now have LiDAR technology’s power in our phones.

Does my Apple iPhone have a LiDAR sensor?

All Apple’s latest iPhone Pro models from 12 onward (iPhone 12 Pro, 13 Pro, 14 Pro, and their Pro Max counterparts), plus the iPad Pros, feature a built-in LiDAR scanner. The sensor can create 3D models of close-range objects up to 5 m. The 3D model is generated by combining a LiDAR sensor and a camera on the back of the iPhone. The LiDAR generates a point cloud, and the camera adds photogrammetry and textures for the 3D model.

Where is the LiDAR sensor on my iPhone?

LiDAR sensor is placed on the back of the iPhone Pro models. It’s the dark circle opposite the flash, sitting next to the three camera lenses. Please remember that the sensor only works with the rear-facing camera.

How to use the LiDAR sensor?

Currently, there is no native Apple application to use LiDAR on iPhones. However, you’ll notice improvements in the apps that utilize it. Namely, apps Measure and Photos now work faster, as they sense their surroundings more accurately.

There are third-party applications available on the Apple App Store, like 3D Scanner App, Polycam, and PIX4Dcatch. They are free in the sense that you can use them to collect the data and build point clouds, but more advanced options are locked behind a paywall, like model generation (in a cloud) and advanced export options.

I use Polycam, as their free model without a subscription is enough to play around with.

LiDAR 3D Models Made with iPhone

You will get the best results in scanning simple objects with solid plane shapes—for example, a gravestone. Note: You can load the entire 3D model on that link, poke around and change the camera view.

The point cloud was accurate as I could scan the object from all sides, including from the top. The applied textures from the camera added a lovely touch, especially the shadows.

However, as you move to larger objects, you will notice the most significant downside—what you cannot scan will be missing from the final model. You can move around the item and extend as high as your arm can, but that is it.

Here is a 3D model of the birch house.

iPhone could collect all the flat surfaces and apply birch, wood, and moss textures. It even captured graffiti on the benches. However, it failed to build a wooden fence on the front side, as the surface needs to be thick enough to reflect the light back to the sensor. There is a hole in the roof because I could not fly over it to complete the full scan. Still, you can move inside the house and poke around.

The true power of LiDAR in an iPhone is its integration with other sensors. Namely the GPS, which helps track the trajectory of the scan. However, be aware that GPS accuracy could be more precise if you scan and move in small areas. If you move slowly, you can collect elevation changes and capture a model of city stairs (Stube Veselka Senjaka).

The Bottom Line and Final Thoughts

LiDAR technology still needs improvement, particularly in integrating with GPS, where accuracy is an issue, especially in urban areas. One more area of improvement is sensor resolution to enhance the detection of thin objects like wires or tree branches, which are still too narrow for the iPhone sensor to correctly “see” and reflect the light off. The sensor is also helpful if you avoid giant 3D models.

As it is now, it is already excellent for consumer-grade and general-purpose use cases.

Another practical use case is to use your iPhone to map rooms or measure medium-sized objects. You can do this even with things you cannot scan from the top, as you do not need to get all the dimensions to do the measurements.