Aircraft exterior requires frequent inspection from assembly to regular maintenance, which is manually detect anomalies such as dents, cracks, leaking, etc., and determine the parameters of each anomaly. The process is not only time-consuming but also error-prone if a human inspector is not fully focused. This situation can be alleviated by 3D scanning technology which generates sub-millimeter precision 3D point cloud model of the scanned surfaces in real time and thus any defects can be identified from the point cloud model. Furthermore, the robotics and image processing technologies can be leveraged to automate the aircraft surface inspection for saving time and relieving human workers from such repetitive and stressful tasks. However, the robot-assisted aircraft surface inspection remains challenging due to the large inspection area, high accuracy requirement, and often lacking a 3D geometry model for planning the robot trajectory.

This paper presents a two-stage approach to autonomously inspect aircraft surface in a static environment, such as an assembly station or a MRO (Maintenance, Repair and Overhaul) shop, by a mobile manipulator equipped with a high-precision 3D laser scanner following an optimal inspection trajectory learned from the low-resolution point cloud model of the aircraft when the 3D CAD model of the aircraft is unavailable. In the first stage, a low-cost consumer-grade RGBD camera is used to generate a rough 3D point cloud model of the surface area with aligning multiple frames of points together, then a full coverage discrete scanning trajectory is learned based on the rough model by using the reinforcement learning technique, finally the mobile manipulator follows the trajectory to scan and create a full coverage and high-resolution point cloud model of the target aircraft surface area with the same RGBD camera. In the second stage, a continuous inspection trajectory is generated based on the high-resolution point cloud model for the mobile manipulator to follow and inspect. An iGPS system is used for tracking the mobile manipulator to ensure the accuracy of the generated point cloud model for the autonomous robotic inspection. Our work focuses on the optimal trajectory learning based on a rough point cloud model of the aircraft. The approaches of full coverage surface area scanning and autonomous manipulator movement for inspection are simulated in Gazebo and the mobile manipulator is controlled through ROS operation system. Due to the limited height of mobile manipulator, the approach can only be applied to the inspection of the middle and lower portions of an aircraft, but the proposed learning-based trajectory planning technology can be extended to whole aircraft inspection when a UAV becomes available for scanning the top portion of the aircraft.


Automating Aircraft Exterior Inspection With Robotics and Machine Learning Technologies

Division / Sponsors: 
Aerospace, Bioengineering, Applied Mechanics, Design Engineering Division, Advanced Energy Systems, Manufacturing Engineering Division, Noise Control and Acoustics, Nondestructive Evaluation, Diagnosis, and Prognosis (NDPD), Nuclear Engineering Division, Process Industries Division, Fluids Engineering, Heat Transfer, Materials, Microelectromechanical Systems, Safety Engineering and Risk Analysis, Computers & Information in Engineering


Paper Proceedings: https://asmedigitalcollection.asme.org/IMECE/IMECE2020/volume/84478

IMECE2020 - Virtual Conference

Supporting the need among researchers worldwide for additional online and multimedia content, ASME now offers the ASME Conference Video Collection. This platform will collect and provide long-term access to conference presentation videos, encompassing all technology areas within ASME's technical conference and events program. About 30,000-40,000 engineers across all mechanical engineering and related disciplines participate in ASME conferences each year. This includes authors of approximately 6,000 to 8,000 technical papers published by ASME, the majority of whom also submit video presentations for our virtual conferences. The video collection will be available as an add-on to existing product packages or as a separate subscription.