A Robust Methodology for Rig Equipment Automation Leveraging Computer Vision

Introducing an advanced artificial intelligence (AI) system leveraging computer vision technology to automate rig operations in the oil and gas industry. Our approach uniquely addresses the challenges of deploying computer vision technology in harsh rig environments, ensuring consistent operation under variable conditions. The system reduces manual labor, enhances safety, and improves operational efficiency with a 95% accuracy rate. Tested over two years across diverse geographical locations, it demonstrates robust performance. We detail the process of data preprocessing and constructing a training dataset with images of diverse pipes and environmental conditions to enhance model robustness. We evaluate computer vision models such as YOLOs (You Only Look Once) [Redmon 2016] and RetinaNet [Ross 2017], employing heuristics and statistical smoothing to stabilize results. Additionally, we review camera selection, calibration methods, and settings adaptation for various rig conditions and high-speed operations. The paper discusses the server setup and inference methodology needed at the edge to achieve high-speed inference, high accuracy, and continuous model improvement. One specific application of our approach is automating iron roughneck positioning during the tripping phase—a sector previously limited to trip-in operations due to challenges related to trip-out operations such as mud-covered pipes, thin joints, and variable lighting conditions. The proposed methodology achieves approximately 95% accuracy, with 95% of connections automated without human input, underscoring its efficiency and reducing the workload on drillers. This automation enhances safety, accelerates operations, and maintains high precision across diverse environmental conditions, including adverse weather and day-night cycles. Trained and tested across various rig environments, the system overcomes traditional challenges such as mud interference with camera operations and the detection of mud-covered pipes. Our findings highlight the system's robust performance and adaptability, using budget-friendly cameras that contribute to its scalability and cost-effectiveness. The deployment strategies discussed ensure efficient scaling and maintenance, enhancing system resilience against operational failures. Overall, the system not only streamlines operations but also provides consistent and faster connection times, demonstrating the transformative impact of digital technologies on enhancing operational efficiencies and safety in oil and gas operations. This paper highlights the techniques needed to make computer vision AI a practical approach for rig automation. It explores strategies for adapting to diverse environmental and lighting conditions, introduces automated labeling for thousands of images to streamline training dataset collection, and presents a comprehensive pipeline for end-to-end AI, from data collection to accuracy enhancements. We will also discuss efficient deployment, scaling, and maintenance of real-time AI system, alongside methods to enhance system robustness and future automation advancements in the oil and gas industry.