Exploring New Frontiers in Computer Vision with Dr. Laura Leal-Taixé
Dr. Laura Leal-Taixé pushes through the doors of a Munich rock climbing gym, her baby on one hip and a gaggle of PHD students behind her. They head towards a bouldering area and take stock of the wall.
“Bouldering is something I never thought I would do,” Leal-Taixé says. “But one of my students described it as an optimization problem where you have pieces of rock and you have to decide how to make your way up the wall. That got my attention,” she laughs.
Leal-Taixé is a professor at the Technical University of Munich, where she leads a computer vision research team called the Dynamic Vision and Learning Group.
“In our lab, we look at the world through a series of cameras,” Leal-Taixé says. “The goal is to understand the moving objects captured by the cameras, which could be anything that moves or has moved—people, cars, a child’s toy ball.”
Inside the lab
The team uses videos—scenes of people and cars navigating train stations, music festivals, rainy side streets—to train computer systems to detect moving objects, predict where they are going, and follow them throughout the frame.
Leal-Taixé’s team works on Multiple Object Tracking (following many moving objects at once), Segmentation (defining precise boundaries around a moving object, down to a person’s hand or foot), and Visual localization (identifying an object’s location within a space at the centimeter level).
“From our research, we build algorithms that can then be used by autonomous vehicles and robots to help them navigate spaces without interfering with moving objects around them,” Leal-Taixé says.
A unique journey
For many researchers, careers in robotics and artificial intelligence stem from lifelong fascinations with these subjects. But Leal-Taixé’s path to computer vision came later in life. Leal-Taixé is originally from Barcelona. “In college in Spain, I studied Telecommunications Engineering, which is super far away from what I’m doing now,” she says. “I became interested in computer vision when I went to Boston for my Masters Thesis.”
On a whim, Leal-Taixé signed up for a course on computer vision. “The professor was amazing,” Leal-Taixé says. “I was enjoying my other classes, but this one really caught my attention. I loved the coding involved—the process of inventing something and then seeing its results. It felt really exciting.”
In the summer of 2020, Leal-Taixé received a phone call. It was from Deva Ramanan, a Principal Scientist at Argo AI, a global self-driving products and services company headquartered in Pittsburgh, Pennsylvania. He was familiar with her work, and was hoping to bring her on board the Argo team to help with tracking.
“I had heard of Argo,” Leal-Taixé says, “But I didn’t know much about the research the company was doing. I thought, ‘Ok, let’s see what this is all about.’”
Leal-Taixé remembers meeting with the team and feeling an immediate sense of exhilaration. “I couldn’t believe some of the projects they were taking on,” Leal-Taixé says. “And the caliber of scientists on their team. I just thought, ‘Wow, I would love to collaborate with these people.’”
Looking closer with Lidar
Leal-Taixé’s work at Argo, which she will take on in addition to her work at the University of Munich, will focus on Lidar processing. “Lidar is a sensor that scans an environment and gives you a three-dimentional map of the world,” Leal-Taixé says. “The problem is that the map is raw; it’s a series of points in space and you don’t know what the points belong to—a car? A road? A person? My work involves training an autonomous vehicle to analyze these points, take meaning from them, and then react in an appropriate manner.”
Leal-Taixé explains that the goal of her research is for a self-driving car to be able to read the Lidar points and recognize what they are indicating. If the vehicle determines that there is a car in front, a car behind, and a car to the side, it can then track each individual car. This tracking allows the autonomous vehicle to know when to brake so it doesn’t bump into the car in front, or to speed up so that it can pass the car to its right.
Opening new doors
Of the projects Leal-Taixé will be tackling at Argo, one of the ones she is most excited about is open-world tracking with Lidar. “There is closed-world tracking, which takes place in a fixed environment, where you know exactly who and what you’ll be tracking,” Leal-Taixé says. “But in open-world tracking, you have no idea what will step into your path.”
Leal-Taixé says that while open-world tracking with two-dimensional video has been explored, open-world tracking with three-dimensional Lidar is novel. “Training a vehicle to react to unfamiliar objects detected on Lidar hasn’t been done before,” Leal-Taixé says. “This will be breaking new ground.”
Leal-Taixé’s dive into open-world tracking with Argo will begin this summer, just around the time of her daughter’s first birthday. She acknowledges that leading the Munich lab, working at Argo, and parenting a soon-to-be toddler means a very full plate.
“I’ve had to become much more efficient,” she laughs. Leal-Taixé says one of her daughter’s favorite things to do is go on walks and look at trees. “She loves examining the different leaves,” Leal-Taixé says.
And so after a long day working on computer vision, Leal-Taixé breaks to take in the joy and revelation of a child’s way of processing the world.