Autonomous parking involves complex perception algorithms. We present an AI-based parking sign assist system relying on live perception that can fuse to map systems.
Here’s the latest video in the NVIDIA DRIVE Labs series. These videos take an engineering-focused look at individual autonomous vehicle challenges and how the NVIDIA DRIVE team is mastering them. Catch up on more NVIDIA DRIVE posts.
Autonomous parking involves an array of complex perception and decision-making algorithms and traditionally relies on high-definition (HD) maps to retrieve parking information.
However, map coverage and poor or outdated localization information can limit such systems. Adding to this complexity, the system must understand and interpret parking rules that vary from region to region.
In this DRIVE Labs post, we show how AI-based live perception can help scale autonomous parking to regions across the globe.
Autonomous parking system overview
Understanding and interpreting parking rules can be more nuanced than it appears.
Different parking rules within the effective region can be overridden. For example, “No Stopping” can overwrite “No Parking.”
In addition, nonparking-related signs can infer parking rules. For example, in Germany, parking is not allowed within 15 meters of any bus stop signs. In the U.S., parking is illegal within 30 feet before a stop sign.
Finally, besides explicit clues like a physical sign, there are implicit signs that carry parking information. For example, in many areas, an intersection indicates the end of the previous active parking rule.
An advanced algorithm-based parking sign assist (PSA) system is critical for autonomous vehicles to understand the complexity of parking rules and react accordingly.
Traditional PSA systems rely on input from HD maps alone. However, the NVIDIA DRIVE AV software stack leverages state-of-the-art deep neural networks (DNNs) and computer vision algorithms to improve the coverage and robustness of autonomous parking in real-world scenarios. These techniques can detect, track, and classify a wide variety of parking traffic signs and road intersections in real time.
- The WaitNet DNN detects traffic signs and intersections.
- The wait perception stack tracks individual signs and intersections to provide 3D positions through triangulation.
- SignNet DNN identifies traffic sign types.
The results from the modules are then fed into the PSA system, which uses the data to determine whether the car is in a parking strip, what the restrictions are, and whether the car is allowed to stop or park within the region.
Parking sign assist overview
After the PSA system receives the detected parking signs and road intersections, it abstracts the object into a Start Parking Sign or an End Parking Sign. This level of abstraction allows the system to scale worldwide.
A Start Parking Sign marks a potential start of a new parking strip and an End Parking Sign may close one or more existing parking strips. Figures 1 and 2 show how parking strips are formed.
Figure 1 abstracts signs and road intersections to form parking strips. The diagram shows that a single sign can generate multiple virtual signs. For example, the sign in the middle serves as the “end” sign for the leftmost sign and it serves as the “start” for the rightmost sign.
In addition to forming a parking strip, the PSA system uses the semantic meaning of signs to classify a parking strip as no-parking, no-stopping, parking-allowed, and unknown states. Then this information can be provided to the driver or any autonomous parking system.
Figure 3 shows the main function workflow of the PSA system. In Frame A, the “Parking Area Start” sign is detected and a new parking strip is created. After the car drives a while, a “Parking Area End” sign is detected, which matches the start sign of that parking strip.
Finally, the PSA system stores all active parking strips in its memory and signals the driver the current parking status based on traffic rules implied by the parking strip in effect.
Conclusion
The PSA system achieves complex decision-making with remarkable accuracy, running in just a few milliseconds on NVIDIA DRIVE AGX. It is also compatible with any perception-only autonomous vehicle stack that uses live camera sensor input.
Our current SignNet DNN supports more than 20 parking signs in Europe, including bus stop signs, no parking signs, and no stopping signs, with coverage continuing to expand. We are also adding optical character recognition (OCR) and natural language processing (NLP) modules into the system to handle complex information carried by written texts on the signs.
To learn more about the software functionality that we are building, see the rest of the NVIDIA DRIVE Lab video series.