A breakthrough in the simulation and learning of contact-rich interactions provides tools and methods to accelerate robotic assembly and simulation research.
NVIDIA robotics and simulation researchers presented Factory: Fast Contact for Robotic Assembly at the 2022 Robotics: Science and Systems (RSS) conference. This work is a novel breakthrough in the simulation and learning of contact-rich interactions, which are ubiquitous in robotics research. Its aim is to greatly accelerate research and development in robotic assembly, as well as serve as a powerful tool for contact-rich simulation of any kind.
Robotic assembly: What, why, and challenges
Assembly is essential across the automotive, aerospace, electronics, and medical industries. Examples include tightening nuts and bolts, soldering, peg insertion, and cable routing.
However, robotic assembly remains one of the oldest and most challenging tasks in robotics. It has been exceptionally difficult to automate because of the physical complexity, high reliability, part variability, and high accuracy requirements.
In industry, robotic assembly methods may achieve high precision, accuracy, and reliability but often require expensive equipment and custom fixtures that can be time-consuming to set up and maintain (preprogrammed trajectories and careful tuning, for example). Tasks that involve robustness to variation (part types, appearance, and locations) and complex manipulation are frequently done using manual labor.
Research methods may achieve lower cost, higher adaptivity, and improved robustness but are often less reliable and slower.
Simulation: A tool for solving the challenges in robotic assembly
Simulation has been used for decades to verify, validate, and optimize robot designs and algorithms in robotics. This includes ensuring the safety of deploying these algorithms. It has also been used to generate large-scale datasets for deep learning, perform system identification, and develop planning and control methods.
In reinforcement learning (RL) research, we have recently seen how simulation results can be transferred to a real system. The importance of accurate physics simulation for robotics development cannot be overemphasized.
Physics-based simulators like MuJoCo and NVIDIA Isaac Gym have been used to train virtual agents to perform manipulation and locomotion tasks, such as solving a Rubik’s Cube or walking on uneven terrain using ANYmal. The policies have successfully transferred to real-world robots.
However, the power of a fast and accurate simulator has not substantially impacted robotic assembly. Developing such simulators for complex bodies with different variations and motions is a difficult task.
For example, a simple nut-and-bolt assembly requires more than pure helical motion. There are finite clearances between the threads of the nut and bolt, which allow the nut to move with six degrees of freedom. Even humans require some level of carefulness to ensure that the nut has proper initial alignment with the bolt and does not get stuck during tightening.
However, simulating the task with traditional methods may require meshes with tens of thousands of triangles. Detecting collisions between these meshes, generating contact points and normals, and solving non-penetration constraints are major computational challenges.
Despite the fact that there is an abundance of threaded fasteners in the world, no existing robotics simulator is able to simulate even a single nut-and-bolt assembly in real time at the same rate as the underlying physical dynamics.
In Factory, the researchers developed methods to overcome the challenges in robotic assembly and other contact-rich interactions.
What is Factory?
Factory (Fast Contact for Robotic Assembly) is a set of physics simulation methods and robot learning tools for achieving real-time and faster simulation of a wide range of contact-rich interactions. One of the Factory applications is robotic assembly.
Factory offers the following central contributions:
- A set of methods for fast, accurate physical simulation of contact-rich interactions through a novel GPU-based synthesis of signed distance function (SDF)-based collisions, contact reduction, and a Gauss-Seidel solver.
- A robot learning suite consisting of:
- 60 high-quality assets, including a Franka robot and all rigid-body assemblies from the NIST Assembly Task Board 1, the established benchmark for robotic assembly
- Three Isaac Gym-style learning environments for robotic assembly
- Seven classical robot controllers
- Proof-of-concept reinforcement learning policies for robots performing contact-rich tasks (a simulated Franka Robot solving the most contact-rich task on the NIST board, nut-and-bolt assembly)
The physics simulation methods in the Factory paper have been integrated into the PhysX physics engine used by Isaac Gym. The asset suite and reinforcement learning policies are available with the latest version of Isaac Gym and the Isaac Gym Environments GitHub repo. The simulation methods are also available in the Omniverse Isaac Sim simulator, with reinforcement learning examples coming later this summer.
Simulation methods and results
Using the fast GPU-based implementations of SDF collisions for objects, contact reduction algorithms for reducing contacts from the SDF collisions, and custom numerical solvers, the researchers were not only able to simulate a single M16 nut and bolt in real time but 1,024 in parallel environments and real time. This is essentially 20,000x faster than the prior state-of-the-art.
The researchers demonstrated the simulator’s performance in a wide range of challenging scenes, including the following:
- 512 bowls falling into a pile in the same environment
- A pile of nuts fed into a feeder mechanism vibrating at 60 Hz
- A Franka robot executing a hand-scripted trajectory to grasp and tighten a nut onto a bolt, with 128 instances of this environment executing in real time
Robot learning tools
The most established benchmark for robotic assembly is the NIST assembly task board, the focus of an annual robotics competition since 2017. The NIST Task Board 1 consists of 38 unique parts. However, the CAD models provided are not ideal for physics simulations due to a lack of real-world clearances, interferences between parts, hand-derived measurements, and so on. Realistic models are hard to find.
Factory uses 60 high-quality, simulation-ready part models, each with an Onshape CAD model, one or more OBJ meshes, a URDF description, and estimated material properties that conform to international standards (ISO 724, ISO 965, and ISO 286) or which are based on models sourced from manufacturers. These models include all parts on the NIST assembly Task Board 1 with dimensional variations that span real-world tolerance bands. Clearance between parts ranges from 0 to a maximum of 2.66 mm, with many parts within the 0.1-0.5 mm range.
Factory provides three robotic assembly scenes for Isaac Gym that can be used for developing planning and control algorithms, collecting simulated sensor data for supervised learning, and training RL agents. Each scene contains a Franka robot and disassembled assemblies from the NIST Task Board 1.
The assets can be randomized in types and locations across all environments. All scenes have been tested with up to 128 simultaneous environments on an NVIDIA RTX 3090 GPU. The scenes are shown below:
The seven robot controllers available in the learning environments include a joint-space inverse differential kinematics (IK) motion controller, a joint-space inverse dynamics (ID) controller, a task-space impedance controller, an operational space motion controller, an open-loop force controller, a closed-loop proportional force controller, and a hybrid force-motion controller.
The researchers intend that the models, environments, and controllers continuously grow with contributions from them and the community.
Proof-of-concept RL policies
Factory employs GPU-accelerated on-policy RL to solve the most contact-rich task on NIST Task Board 1: assembling a nut onto a bolt. Like many assembly tasks, such a procedure is long-horizon and challenging to learn end-to-end. The problem was separated into three phases:
- Pick: The robot grasps the nut with a parallel-jaw gripper from a random location on a work surface.
- Place: The robot transports the nut to the top of a bolt fixed to the surface.
- Screw: The robot brings the nut into contact with the bolt, engages the mating threads, and tightens the nut until it contacts the base of the bolt head.
The training was done on a single GPU. Large randomizations were applied to the initial position and orientation of the objects with a batch of 3-4 policies trained simultaneously using proximal policy optimization (PPO). Each batch takes 1-1.5 hours to train and each subpolicy is trained in over 128 environments with a maximum of 1,024 policy updates for rapid experimentation. The success rate was 98.4% at test time.
Finally, to evaluate the potential for sim-to-real transfer (transferring the policy learned in simulation to real-world robotics systems), the researchers compared the contact forces generated during these interactions in simulation to contact forces measured in the real world by humans performing the same task with a wrench. For more information, see the R-PAL Daily Interactive Manipulation (DIM) dataset.
The figure below shows that the histogram of the simulation Fasten Nut lies in the middle of the histogram of the Real Fasten Nut, which shows a strong consistency with the real-world values.
Conclusion and future directions
Although Factory was developed with robotic assembly as a motivating application, there are no limitations on using the methods for entirely different tasks within robotics, such as grasping complex non-convex shapes in home environments, locomotion on uneven outdoor terrain, and non-prehensile manipulation of aggregates of objects.
The future direction of this work is to realize full end-to-end simulation for complex physical interactions, including techniques for efficiently transferring the trained policies to real-world robotic systems. This can potentially minimize cost and risk, improve safety, and achieve efficient behaviors.
One day, every advanced industrial manufacturing robot might be trained in simulation using such techniques for seamless transfer to the real world.
Towards this end, NVIDIA developers are working to refine the physics simulation methods used by the Factory research so that they can be used within Omniverse Isaac Sim. Limited functionality is already present, and will become more robust over time.
Get started with Factory
- Visit the project page
- Download the standalone Isaac Gym Preview Release
- Download the Isaac Gym Environments for the Isaac Gym Preview release, (includes Factory RL environments)
- Download the next-generation Isaac Sim and explore the Omniverse Preview Samples to learn how to use SDF collisions in Omniverse and Isaac Sim
- Watch Physics Simulation and Deep Reinforcement Learning for Contact-Rich Robotic Assembly with Kier Storey and Yashraj Narang (GTC Fall 2021)
The Factory RL environments will also be available in future versions of the Omniverse Isaac Gym Environments for Isaac Sim.