Application of Artificial Intelligence in Coal Mine Robotics

Abstract With the rapid development of artificial intelligence technology, its application in coal mines has become more and more extensive. In the process of coal mine production, the urgency of the demand for robot replacement has accelerated the industrial application of coal mine robots and the application of artificial intelligence technology in coal mine robots. The application of artificial intelligence technology in coal mine robots is analyzed and explored, the main research contents of artificial intelligence technology and its application in industry are introduced, the current situation of the application of artificial intelligence in coal mine production is analyzed, the concept of applying artificial intelligence technology to coal mine robots effectively is elaborated, and the prospect of the development of artificial intelligence in coal mine robots is prospected.

Keywords artificial intelligence, coal mine robot, intelligent perception, intelligent decision making, intelligent monitoring，material removal robot

0 Introduction

The underground production and operation process of coal mine has the problems of many people going down the shaft, high risk of disaster, high accident rate, harsh operating environment and serious environmental pollution [1]. Facing the high-risk underground operations, coal mine robots become one of the important ways to achieve the goal of safe and efficient underground coal mine production. Coal mine robots can assist or replace people to complete some dangerous mining operations and achieve safe and efficient production in coal mines. In order to achieve "no one is safe", robots are the trend to replace miners in underground operations.

With the strategy of "Made in China 2025", "German Industry 4.0" and "American Industrial Internet", 5G communication, Internet of Things, big data, cloud computing and artificial intelligence The gradual maturity of technologies such as 5G communication, Internet of Things, big data, cloud computing and artificial intelligence has greatly promoted the transformation and upgrading of China's traditional manufacturing industry [2]. As an emerging science and technology, artificial intelligence can make computer technology more accurate, fast, and convenient to complete complex scientific calculations that the human brain is incapable of undertaking, and achieve partial replacement, extension, and enhancement of the human brain, thus creating intelligent machines that can complete complex and dangerous operations instead of humans [3].

Future coal mine production will develop toward unmanned, autonomous, intelligent, and efficient, in which artificial intelligence technology will play an irreplaceable role and diverse artificial intelligence technologies will be applied to coal mine robots [4]. Although the current application of artificial intelligence in the field of industrial coal mines is still in the fumbling period, however, with the increasingly widespread application of artificial intelligence technology in the field of coal mines, the construction of unmanned operation mines is inevitable [5].

1 The urgent problems in the coal industry

China's coal industry has experienced more than 40 years of development, and the mining of coal mineral resources gradually tends to be intelligent, but there are still some bottlenecks that need to be solved.

1.1 Technology and equipment need to be upgraded

Although the mining and transportation of coal in China have gone through the stages of digitalization, automation and informatization, the overall technical level and production equipment are still lower than those of developed countries [6].In 2019, the former State Administration of Coal Mine Safety Supervision proposed to accelerate the industrialization and application of coal mine robots for digging, coal mining, transportation, safety control, support and rescue. The current coal mine robot is no longer just completing simple repetitive operations, it can sense the surrounding environment and give real-time feedback to the outside world, but it does not yet have independent thinking, identification, reasoning, judgment and decision-making capabilities, and still needs human participation to complete some complex work tasks.

1.2 Serious safety hazards

The coal industry is a high-risk industry, and there are various hazards in every step of production, water, fire, gas, coal dust, geological formations and other disasters are frequent, and the unknown complex underground environment seriously threatens the life safety of underground operators. Although the intelligent monitoring and early warning technology of coal mines based on the Internet of Things, big data and cloud computing has largely reduced the occurrence of accidents and guaranteed the safe production of coal mines, there are still many problems. The poor accuracy and sensitivity of sensors lead to incomplete and untimely collection of precursor information; monitoring systems are independent of each other and have a single function, and the integration and application integration depth of the cloud platform is not deep enough; monitoring system database security is weak; monitoring equipment lacks deep learning as well as self-adaptive capability [7].

1.3 Serious environmental pollution

Coal mines produce coal dust during the mining process, and also produce harmful gases such as carbon monoxide and carbon dioxide to pollute the atmosphere [8]. At the same time, the production effluent from coal mining contains a large amount of heavy metals and acidic substances, which can easily seep into the soil or enter the groundwater to pollute the geology and water sources. Coal mining projects will encroach on a large amount of vegetation and agricultural land, and the land is prone to collapse after mining leading to the destruction of the surface layer [9].

2 The main research content of artificial intelligence

2.1 Pattern recognition

Pattern recognition in artificial intelligence technology uses the powerful data collection, analysis and processing functions of advanced computer technology to simulate human perception and recognition of the external environment by setting the corresponding programs in advance. Intelligent robots incorporating pattern recognition can better simulate human sensory abilities, recognize characters, sounds, images, scenes and their fused information with high accuracy, and accurately perceive and model the surrounding environment through the acquisition of multi-source information [10].

Machine vision in artificial intelligence technology, as one of the most important environmental perception modalities, simulates human visual capabilities to improve the robot's understanding of the downhole environment, operational processes, and feedback phenomena. Intelligent robots incorporating machine vision are, first, able to adapt well to the downhole operating environment and collaborate well with other artificial devices; second, able to capture more information about the external landscape and to understand and dig deeper into the content of images through stereo vision, visual inspection, and dynamic image analysis techniques; and third, able to judge the underground feedback phenomena of the operating process and feed back information about the robot's status to the motion control system [11].

2.2 Expert system

Expert systems are technologies that model the knowledge and experience of human experts and are used to solve problems such as system decisions, processes, and failures. Through artificial intelligence techniques, knowledge systems are created for downhole systems that simulate humans to solve practical problems encountered during operations. Human experts can predict system failures, determine failure points and generate troubleshooting solutions based on the current state of the system, such as equipment displays and sounds, operational data parameters, and the state of the product, when solving real-world problems. Therefore, expert systems are commonly used for fault prediction, diagnosis and troubleshooting. In addition, in the manufacturing industry, expert systems are also used for production planning decisions, production process optimization, production coordination, and optimization of equipment parameters.

2.3 Machine Learning

Machine learning in artificial intelligence technologies mimics human learning capabilities through model frameworks and algorithms to automatically extract intrinsic laws through training data, environmental information, and feedback to improve system performance and enhance environmental adaptation and robustness. Robots that incorporate machine learning have human-like law extraction and knowledge summarization capabilities to identify effective information from the large amount of information resources collected and learn to improve their own intelligence. Machine learning technology can effectively solve a series of problems in unexpected situations and largely reduce labor and production costs [12].

2.4 Distributed Artificial Intelligence

Distributed artificial intelligence system coordinates the scheduling and control of heterogeneous multi-intelligent body systems by scientifically and rationally combining artificial intelligence and computer technology, so as to enhance the performance of the artificial intelligence system, improve the task execution capability, and increase the efficiency of the cooperative work of each independent system in the intelligent robot. When the intelligent robot encounters some unexpected situations, it can still guarantee each subsystem to carry out normal work. The current distributed artificial intelligence system is still in the initial stage of research and development, and the technical difficulty lies in how to coordinate the operation rules of different systems [13].

3 Status of application of artificial intelligence in coal mine robots

3.1 Application of artificial intelligence in the motion control of coal mine robots

In order to ensure that coal mine robots can operate properly in complex underground environments, research scholars have applied artificial intelligence technologies such as expert systems and artificial neural networks to robot motion control methods, algorithms and collaborative operations. By simulating human expert thinking and knowledge level, coal mine robots can solve some complex multidimensional nonlinear problems, reduce the amount of operations for dynamical system analysis, parameter setting and data processing, and improve control efficiency and accuracy.

Wang Nian et al [14] researchers designed an intelligent mine robot based on embedded ucos and used GSM network to realize remote control of the device; Zhang Chuancai et al [15] researchers used BP neural network to establish a measurement method to determine the robot turning angle based on motor speed and running time, which can provide angle parameters for robot path planning; Wang Xuesong et al [16] researchers personnel approximated the kinetic uncertain parameters based on the improved Elman neural network and sent control commands for the coal mine robot servo system using a neuro-fuzzy controller; Song Xin et al [17] researchers applied neural networks in the field of robot control to accomplish actions such as robotic arm multi-joint coupling control, end trajectory planning, and hydraulic valve control.

3.2 Application of Artificial Intelligence in Intelligent Perception and Danger Prediction of Coal Mine Robot

Mining inspection robots realize all-round perception of underground environment information by carrying various sensors, real-time monitoring of instrument and equipment failure, personnel safety and disaster information such as gas, coal dust, water and fire, and timely issuance of early warning to reduce the occurrence of coal mine accidents. For several technical difficulties such as inaccurate identification and untimely monitoring in complex underground environments, researchers use deep learning, pattern recognition, and expert system technologies to further enhance the robot's accurate identification and real-time monitoring of underground emergent hazards.

Researchers in Lu Wanjie et al [18] used deep learning algorithms based on convolutional neural networks to model and train coal mine equipment so that the underground inspection robot could accurately identify the type of coal mine equipment; researchers in Zhang Fan et al [19] proposed a mining image reconstruction method based on residual neural networks for the disturbing effects of underground noise on the visualized operating environment, which effectively improved the clarity of monitoring images and Nie Zhen et al [20] used a genetic algorithm based on BP neural network to build a tunnel gas environment intelligent detection system and obtain real-time data of gas concentration distribution on different tunnel sections in the path of coal mine inspection robots; Pan Yue et al [21] used BP neural network to establish a diagnostic model for fan faults and establish a mapping between fan fault types and fan rotor vibration frequency bands, thus realizing fan fault diagnosis. relationship, and then achieve fan fault diagnosis; Yan Junjie et al [22] researchers based on artificial neural network to establish a diagnostic model for coal mine machinery gear faults, using the input signal to train the neural network model, classify the output signal, and then determine the gear fault.

3.3 Application of artificial intelligence in autonomous positioning navigation and map construction for coal mine robots

Achieving autonomous positioning and navigation in complex unstructured coal mine environments requires consideration of both the inability of GPS technology to be applied directly downhole and the need to overcome interference from external factors such as dust, temperature, humidity, noise, and airflow, which places higher demands on autonomous and accurate positioning and navigation technology for robots in restricted and closed environments downhole. Map construction, positioning navigation, path planning, and real-time obstacle avoidance of coal mine robots based on artificial intelligence technology have become hot spots for applied research.

Bai Yun [23] proposed variable structure fuzzy neural network and applied it to the environment sensing process of snake underground rescue robots, fusing multi-source sensor data to achieve obstacle recognition and environment modeling of snake robots in harsh environments; Fu Hua et al [24] researchers used artificial neural network model to model and dynamically describe the workspace of intelligent coal mine monitoring system, using neural network model for robot obstacle avoidance path planning; Zhang Yaofeng et al [25] researchers used Elman network-based compensation for ultrasonic sensor measurement error of underground robot, which greatly improved the accuracy of ultrasonic ranging and obstacle detection; Zhai Guodong et al [26] researchers summarized binocular vision technology in coal mine rescue robot to obtain accident scene information and achieve autonomous obstacle avoidance and path planning, including pattern classification and recognition, visual measurement and 3D reconstruction, combined measurement and localization, and visual servo control; Ma Hongwei et al [27] researchers constructed a depth camera-based machine vision system and proposed a depth vision-based navigation method, in which the robot is equipped with an RGB-D depth camera for data acquisition to achieve map creation and autonomous navigation.

4 Research on intelligent coal mine robots

There are various kinds of artificial intelligence technologies, and the main research contents applied to the field of coal mine robots include multimodal fusion intelligent perception, knowledge learning and intelligent decision making, and intelligent control cooperative operation. Through perception, learning, decision making, and collaborative control, the intelligent development of coal mine robots is realized.

4.1 Multi-modal fusion intelligent perception

The coal mine robot is equipped with various explosion-proof, high-precision and high-reliability sensors to build an intelligent perception system with multimodal fusion of vision, hearing, smell, touch, etc., to complete intelligent recognition and analysis, abnormal sound recognition, abnormal temperature monitoring, smoke detection, harmful gas concentration detection, autonomous obstacle avoidance, autonomous grasping and other operations.

(1) Research on machine vision recognition and visual detection technologies in coal mine application scenarios. Through image processing and understanding, the robot is able to, firstly, identify and monitor equipment digital meters, LCD screens, indicators, valves, etc.; secondly, detect pipeline liquid dripping, tape running and cracking; thirdly, carry out personnel intrusion, personnel on duty, personnel dressing detection; fourthly, identify and track foreign objects such as gangue, anchor rods, road logs, iron pipes, etc. that appear on the tape.

(2) Research on technologies such as robot hearing i.e. sound detection and recognition in coal mine application scenarios. Using high-sensitivity sound pickup sensor, high-speed DSP digital signal processor, combined with adaptive dynamic noise reduction processing technology, audio feature extraction and detection model algorithm recognition technology to identify abnormal sound in the mine.

(3) Research on intelligent recognition technology for robotic olfaction i.e. gas detection in coal mine application scenarios. Accurate detection of methane, hydrogen sulfide, carbon monoxide, oxygen and other gases in the environment and whether smoke exceeds the limit, timely detection of gas leaks and early warning of fires.

(4) Research on haptic technology for robots in coal mine application scenarios. Collect the temperature of objects such as motors, pumps, bearings, rollers, tapes, etc. through contact or non-contact and analyze the data; through force sensing equipment, real-time monitoring of contact force, gripping force, operating force, internal stress, and realize force sensing and safety control.

4.2 Knowledge learning and intelligent decision making

In view of the current problems of incompatible coal mine robot system protocols and lack of information sharing and integration, we will deeply integrate coal mine robots with new generation information technology, build a generalized, standard and flexible system for mutual learning and knowledge sharing of coal mine robots, and break through the technical bottlenecks of coal mine robot scene understanding, safety detection, precise positioning, autonomous perception and efficient navigation. Realize cloud-based online services for common technologies of coal mine robots to solve the limitations of individual robots and improve the intelligent decision making of coal mine robots.

(1) Establish a learning and generalization framework that integrates the individual and the whole. At the individual level, a single robot integrates sensing, decision-making, control, collaboration, and human-robot interaction information during operation, and conducts incremental, real-time, online training through an artificial intelligence learning framework represented by neural networks to dynamically adjust the robot's operational state and achieve full-cycle optimal control and decision-making. At the overall level, multi-robots upload and distribute their learned knowledge among themselves through the new generation of information technology, so that when a robot faces a brand-new operational task, it can quickly familiarize itself with the operational characteristics with the knowledge results of other robots, reduce re-learning time, and enhance the overall system's task flexibility and adaptability.

(2) Establish an operation mode in which the robot body and the cloud are integrated. Breakthrough the traditional robot R&D and integration model, and realize a new robot R&D and integration route that integrates the local lightweight robot body with the high-performance data processing capability in the cloud with the help of "5G + cloud computing". The algorithms that require strong computing power, such as intelligent environment perception, pattern recognition, map construction, and autonomous navigation, are moved to the cloud, and the local robot uploads the data of on-board sensors and actuators to the cloud in real time, and optimizes the calculation of perception, modeling, and execution through the powerful data processing and computing power of the cloud. The result is sent to the local robot in real time, which reduces the computational burden of the local robot and shifts more hardware resources to the sensor and execution side to achieve a lightweight, streamlined, and high-performance operational robot design.

4.3 Intelligent control of cooperative operation

Integrating deep learning and laser/visual SLAM technologies into coal mine robots, combined with intelligent sensing system of multimodal fusion, realizes the functions of autonomous movement, precise positioning, position adjustment, intelligent operation planning, autonomous operation and intelligent disaster sensing of coal mine robots in complex mine environment, and realizes intelligent collaborative control of detection, digging and support operation processes.

(1) Integrating neural network technology into the collaborative operation control and planning of multiple coal mine robots. Self-organization, self-grouping and self-coordination of mobile robots in mines to achieve integration of heterogeneous equipment. Through intelligent task decomposition, task assignment, and load balancing technologies, a swarm of robots in complex environments in mines is formed, and technologies such as autonomous navigation in underground space, multi-sensor state sensing, intelligent operation planning, and multi-robot collaborative control are applied to realize efficient collaborative operations among multiple robots in workface digging, drilling, extraction, transportation, and support.

(2) Extending the mode of human interaction with a single robot to human interaction with multiple robot groups, and realizing the intervention and collaboration of operators on robot groups. During the operation of coal mine robots, each heterogeneous robot with different functions forms a complex multi-robot collaborative swarm. At the same time, the multi-robot collaborative swarm needs to be able to collaborate with the operator in depth. Through AI technology, we can break through the simple "command-execution-display" mode of existing human-robot interaction technology, and integrate human intervention into the control cycle to realize a new mode of human-robot interaction with "human in the loop", and realize the "unmanned underground system group + unmanned underground system group + unmanned underground system group". underground unmanned system group + underground operator" mode of operation, to improve the overall system's operational efficiency, task flexibility and robustness.

In order to achieve the goal of smart coal mine, we will carry out research on "coal mine robot +", "coal mine robot + 5G" to realize comprehensive sensing and interconnection, full domain information sharing and multi-channel human-robot interaction; "coal mine robot + cloud computing "Coal mine robot + cloud computing" realizes the compatibility of lightweight and low-cost robot ontology and high-performance learning computing capability; "coal mine robot + big data" realizes dynamic prediction, information integration, and provides data basis for robot evolutionary learning; "coal mine robot + AI "Mine robot + AI" realizes intelligent autonomous perception, optimal analysis and decision making, and knowledge learning evolution, thus forming a complete intelligent system of three-dimensional perception, autonomous learning and cooperative control in the mine.

5 Future Outlook

Artificial intelligence has been widely applied in the field of coal mine robotics, and more research results have been achieved. However, as an emerging frontier technology, artificial intelligence still has limitations.

(1) The current AI technology is mainly oriented to a single task, and a general AI framework that can face multiple tasks has not yet been realized. For example, the models trained for image recognition cannot be used for sound detection and recognition; the algorithm framework for recognizing a specific target object cannot be extended to the recognition of arbitrary target objects, and the data set needs to be constructed and retrained when a new classification target appears. This feature limits the application of AI in complex task scenarios.

(2) Artificial intelligence algorithms need to rely on a large amount of data, and operations such as data collection, processing, calibration and alignment need to be done manually, which is less efficient. How to use a smaller amount of data to achieve higher performance has become one of the current research hotspots of artificial intelligence methods.

(3) There are many types of coal mine robots, and there exist a large number of sensing devices, driving devices and actuating devices. The data formats of each device are diverse, and it is difficult to form a unified data interface, making the data between each system independent of each other. The incompatible data makes it difficult for the AI system to coordinate the robots in each part of the coal mine production process, and to obtain enough data to form a closed-loop unified plan for the entire production process.

(4) The environment in which coal mine robots operate is extremely dangerous, so current AI systems alone cannot guarantee a high level of safety and stability. How to integrate the AI system with the operator's manual intervention and integrate human intervention into the whole AI system's operation loop becomes one of the key elements to be addressed in the next step.

In the future, AI systems applied to coal mine robots will develop toward generalization, low overhead, unification, and human-machine collaboration, with the emergence of a general AI algorithm framework for multiple tasks that continuously learns and evolves online using small amounts of data and low-cost training methods, capable of integrating key data from all aspects of coal mine production for integrated computing and scheduling, and able to collaborate with each other and humans to achieve It is capable of collaborating with humans to achieve efficient, safe, and autonomous coal mine production.

6 Conclusion

With the development of artificial intelligence technology, the coal mining industry will see a major change. With the efficient model building, parallel computing, and planning capabilities of AI, the intelligence and automation of coal mine robots will reach a new level, truly realizing the unmanned and safe requirements of coal mine production. At the same time, artificial intelligence will enable a significant increase in coal mine production efficiency and promote the safe, healthy and sustainable development of the coal mine industry.