Welcome to the Lidar Age: 11 Ways Laser Sensors Are Taking Over Everything
“Hit me with those laser beams,” sang Frankie Goes to Hollywood in 1984. Lidar pioneers? Maybe not. But the British New Wave band did unknowingly predict a time when, just decades later, laser beams would be hitting everything in sight.
Welcome to the Lidar Age, when devices ranging from smartphones to vacuum cleaners to automobiles are all using the technology to determine the precise location of objects in space. Invented in the 1960s by the Hughes Aircraft Company and used by NASA’s Apollo 15 to map the surface of the moon, lidar sensors work by firing millions of energy pulses per second to create a “point cloud,” a 3D image that results from the location of the laser beams’ returns.
Though most people now associate lidar with self-driving vehicles, lidar has been turning up everywhere lately, driven by advances in sensor technology, reliability, and miniaturization. These new applications are helping lidar move another step closer to widespread acceptance, driving further innovation, increasing performance in range and resolution, and bringing down cost. Lidar is now powerful, small, and affordable enough to make its way into every corner of everyday life.
Here, then, are some surprising non-automotive applications of lidar technology.
Smartphones and Tablets
Apple was one of the first smartphone manufacturers to use lidar on its devices, introducing it on the 2020 iPad Pro, and then fitting it on the iPhone 12 Pro and Pro Max to enhance camera functionality and strengthen augmented reality (AR) applications. Although lidar itself doesn’t improve the quality of photos and videos, it speeds up the autofocus, improving accuracy and low light performance. Couple that with the ability to accurately measure distances and dimensions, and the stage is set for a new generation of AR functionality. The latest version of Snapchat’s Lens Studio uses smartphone lidar to bring avatars to life, the spatial awareness enabling CGI characters on your phone screen to look like they’re standing on your furniture. TikTok also jumped at the opportunity to incorporate lidar functionality to enhance videos with a new generation of effects, “visually bridging the digital and physical worlds.”
Smart Vacuums and Delivery Bots
The days of the wall-crashing, furniture-banging robot vacuum may be nearing their end. At CES 2021, Samsung introduced a smarter “smart vacuum” equipped with lidar to detect and avoid floor-level objects, and a 3D sensor with object recognition algorithms to analyze those objects. That same technology—minus the floor cleaning systems—is being deployed outdoors too, in the small delivery robots being trialled by parcel and food delivery companies. In one pilot by Starship—a company founded by some of the brains behind Skype—students on campuses across the US, and residents of towns such as Milton Keynes and Northampton in the UK, are being served by these small, autonomous “bots.” Serve Robotics, Amazon, and FedEx are among those racing to develop the technology that powers these low-speed sidewalk crawlers equipped with lidar, cameras, and GPS. The bots’ larger siblings, road-going vehicles designed to carry groceries, are being trialed by companies such as Walmart.
Anyone who’s seen Game of Thrones has already experienced the power of lidar. The ancient walled city of Dubrovnik in Croatia was famously used as the setting for the fictional city of King’s Landing, but much of what you see on screen is a 3D representation created using data from lidar scans of Dubrovnik enhanced with digital imagery. Likewise, Joker and John Wick 3 also used lidar in the creation of powerful visual effects.
In 2016, an aerial lidar survey discovered several hidden buildings near the famous Sutton Hoo site (dramatized most recently in the Netflix movie The Dig), including a possible Anglo-Saxon palace. A year later, researchers from the University of Bradford explored the area further using ground penetrating radar and drone-mounted lidar. This enabled them to see through the vegetation to the surface of the ground below, and create a digitally-enhanced model of the site. In recent years, lidar has also enabled the discovery of tens of thousands of previously unknown Mayan pyramids and other remains in Guatemala and in Mexico, the presence of an ancient island settlement off the Florida coast, and Roman ruins and an Iron Age fort in the UK.
The city of Hoi An is often referred to as the Savile Row of Vietnam, famous for its high quality, affordable tailoring. But these old-school artisans have a new trick up their sleeve: lidar-based 3D scanners, which enable tailors to quickly size up a generously proportioned tourist in need of a new suit. A few awkward minutes in a scanning booth is all a Hoi An tailor needs; the data is then used to produce a custom-made suit ready to wear 24 hours later.
Real Estate Demos
“Honey, it’s a little smaller than the agent suggested.” What if there were a device which could turn shady rental brokers into honest ones by producing reliable room dimensions, down to the most intricate detail? Step in, lidar. Devices that combine lidar and high-definition wide-angled cameras are making their way into the real estate sales process. The realtor saves time measuring, and the customer gets an accurate virtual tour of their dream home and an exact 3D representation of the property’s interior and exterior.
Assigning a precise head count to a very large crowd has long perplexed event organizers and security professionals. Lidar can detect the position and outline of people and, crucially, can be programmed to block out personally identifiable features, ensuring anonymity. Great news for developers of security systems, as well as for retail, sports, and leisure venue management, and anyone else reliant on crowd safety and monitoring.
The Atlas initiative in Greenland uses two remote lidar systems to capture 3D time-lapse data of the Helheim Glacier. Lidar is also used in forestry, as it can distinguish between coniferous and deciduous trees, and can be used to measure the height and density of tree cover. Lidar scans from the air provide information about specific land features that are useful not only for forestry organizations, but also for mining companies. So-called bathymetric lidar, which can penetrate water, enables the mapping and monitoring of sea and riverbed features.
Urban planners are hoping that by installing fixed lidar units, they can help keep traffic on the move. Using lidar’s real-time 3D traffic data, and vehicle speed and classification, analysts can predict vehicle trajectories at intersections. And by examining traffic data, city planners can optimize traffic flow by installing dynamic traffic-light phasing and variable speed limits, for example, helping improve road safety, especially for vulnerable road users such as pedestrians and cyclists.
The widespread availability of increasingly hi-tech unmanned aerial vehicles (UAVs), or drones, is creating new commercial applications, and adding a new dimension for professional and amateur photography and film-making. We’ve seen how lidar-equipped drones can be used for aerial monitoring of activity on the ground. But drones have infamously been used to photograph privacy-seeking celebrities, deliver contraband into prisons, and even illegally fly near airports. Enter drone-detecting lidar, strategically-positioned devices that can be used to spot the presence and track the motion of one or more drones up to several hundred meters away.
Lidar scanning isn’t only being used to create 3D models of cities and buildings for movies and gaming—it’s also being used to scan ancient buildings and structures for real-life reconstruction and restoration. The late art historian Professor Andrew Tallon used lidar in the cathedral of Notre Dame in Paris to create highly detailed scans made up of more than a billion data points—and given the destruction caused by the fire in 2019, his work represents the best record of the pre-disaster cathedral.
Lidar has also been used to digitally map the 114-year-old Ribblehead viaduct in the UK as part of a major restoration program to prolong the life of the Victorian structure. And in Venice, lidar is proving to be instrumental in preservation efforts. In 2020, researchers combined digital photography with lidar scans to create a highly detailed 3D model of the entire city of canals.
Thanks to advances in R&D and product design, improved technology, falling component prices, and new manufacturing solutions, lidar use is projected to grow even further in the years ahead. Though it is still not an off-the-shelf commodity, but rather an intricate, application-specific, purpose-designed system, that may change someday soon. And when it does, the laser beams will really be hitting everywhere.