Hybrid Approach for UAV-Based Video Streaming in Wind Farm Inspections via Digital Twins

The operation and maintenance activities on wind farms are complex and extremely expensive [1] due to the large number of units to be controlled, the difficulties on accessing them [2], and the specialized equipment [6] required for these operations. To reduce these costs and provide contextualized remote information and automatic evaluation, technologies such as Digital Twins (DTs) and Unmanned Aerial Vehicles (UAVs) have recently emerged [3][4], which can be used together to fully digitize these infrastructures and reduce their maintenance costs [5]. The concept of Digital Twins (DTs) has gained significant traction in recent years, particularly within the fields of industrial IoT and smart manufacturing. A DT refers to a virtual representation of a physical object, system, or process, which mirrors its real-world counterpart using real-time data from sensors, data-models and/or simulations. Bearing this in mind the integration of UAVs (e.g. drones) with digital twin technology has opened new possibilities for real-time monitoring, inspection, and decision-making across various industries as they can be used in conjunction to monitor assets, structures, and environments in real-time. As such, UAVs equipped with cameras, LiDAR, and other sensors collect real-time data on physical assets, which is then fed into a digital twin model. This allows for continuous, up-to-date visualization of the monitored asset, letting this way to remotely monitor real-time data collected by UAVs and make betterinformed decisions about asset management or respond to issues faster, as this data helps enhance the decision-making process, offering a comprehensive view of the asset's health and operational efficiency. In this context and thanks to the execution of ERROTAID project (ZL-2022/00317) funded within the HAZITEK programme by the Basque Government and co-financed by the European Regional Development Fund (FEDER), this article aims to describe how DTs and UAVs can work altogether to perform both predictive maintenance and issue detection labours in wind farms’ assets, by the usage of a hybrid strategy for video streaming processes based on the combination of RTSP and MQTT on inspection activities, and a sample full and operative integration of this DT system is implemented for demonstration. By the application of this video streaming strategy, there can be performed both a potential real time tracking of UAV’s position and identification of several failures on those wind farm assets by streaming this video data to a remote inspector or, in the next phases of this development, into AI algorithms specifically trained to detect anomalies [7][8][9][10] on this kind of assets (dirt, erosion, pitting, deformation, damage, lightning strikes, cracking, corrosion, etc.). By analysing frame-by-frame this video data received from the UAV's IP cameras, it is also reliable to predict potential operational failures, as asset deterioration can be assessed and maintenance orders generated automatically [11][12][13][14], improving maintenance and operation activities. Additionally, this work aims to show the capabilities of using MQTT as the driver to autonomously control the UAV during its inspection flight, eliminating somehow the necessity of having a human supervisor in the field, so that the whole inspection flight can be managed throughout the DT in a holistic 3D integration of wind energy infrastructures, their location and information of interest in a real-time interactive digital environment.