Marine Photovoltaic Module Salt Detection via Semantic-Driven Feature Optimization in Mask R-CNN

2.1. Framework Overview This study proposes a semantic information-guided recognition framework to tackle the unique challenges of salt crystallization detection on offshore floating PV modules—particularly the visual interference caused by surface grid lines. The methodology is structured into three sequential stages: (1) construction of a marine salt deposition dataset, (2) semantic thresholding for low-level feature enhancement, in the output, the red area represents the salt analysis zone, the green area represents the surface of the photovoltaic module, and the purple area represents the grid lines of the module, and (3) semantic-integrated salt detection via Mask R-CNN, as illustrated in Figure 1. Each stage is designed to improve the model’s capacity for accurate and robust salt deposition recognition in complex marine environments. 2.1.1. Part 1: Construction of Marine Salt Deposition Dataset The first stage focuses on constructing a reliable and diverse marine salt deposition dataset, which serves as the foundation for the entire framework. Due to the limited availability and high acquisition cost of offshore on-site images, a comparison of on-site and custom datasets was performed, revealing that self-constructed images can effectively replicate the edge morphology and crystallization texture of real salt deposits. To obtain the dataset, a custom dataset capture frame was developed, comprising a mobile guide rail, a sliding imaging device, and a configurable camera module, enabling controlled image acquisition under various lighting conditions, viewing angles, and salt coverage scenarios. to further enrich the dataset, a series of augmentation operations—including trimming, rotation, and flipping—were applied to simulate diverse deposition patterns and improve the generalization ability of the detection model. Following dataset expansion, each image was manually annotated using labelme for annotation, ensuring pixel-level precision of salt deposition boundaries. this three-step process—obtaining, expanding, and annotating datasets—yields a well-balanced and high-quality dataset tailored to support accurate salt crystallization detection in real-world offshore applications. 2.1.2. Part 2: Semantic Thresholding for Low-Level Feature Enhancement In the second stage, the emphasis shifts to enhancing image features that distinguish salt deposits from visually similar grid lines [23,24]. This is achieved through a semantic thresholding approach designed to amplify salt-specific visual cues while minimizing irrelevant background interference. Initially, a median filtering operation [25] is applied to the raw images for smoothing, effectively suppressing noise and enhancing edge continuity [26]. Subsequently, the preprocessed images are divided into local subregions, and Otsu’s algorithm is employed as the threshold determination method to compute optimal thresholds within each region. This adaptive thresholding process ensures more accurate separation of salt crystallization from grid structures. The resulting binary maps are then used to extract semantic information, guiding the model toward correct classification of salt regions. These semantic representations capture salt-specific patterns and textures, serving as enriched priors that effectively improve the model’s ability to recognize and isolate salt deposits in later stages. 2.1.3. Part 3: Semantic-Integrated Salt Detection via Mask R-CNN The final stage integrates the semantic information generated from thresholding into an enhanced Mask R-CNN model to perform precise instance-level segmentation of salt deposits. A composite input is constructed by merging the original RGB image with the semantic binary map, forming a four-channel representation that combines both visual and semantic features. The Mask R-CNN architecture is adapted to accommodate this input structure, allowing the network to leverage semantic guidance during training. This integration significantly boosts the model’s performance, resulting in a notable improvement in segmentation recognition rate, especially in complex backgrounds where salt deposits are difficult to distinguish. A series of evaluation indicators for applications is used to assess practical performance. Experimental results confirm that the semantic-enhanced Mask R-CNN demonstrates superior robustness, generalization, and detection accuracy, validating the framework’s potential for deployment in intelligent offshore PV maintenance systems. 2.2. Construction of the Marine Salt Deposition Dataset A complete and well-annotated dataset is essential for training robust and generalizable models for salt crystallization detection in offshore PV systems. The variability in salt morphology, lighting conditions, and background interference—such as grid lines—requires diverse and high-quality image data to ensure the detection model can adapt to real-world scenarios. In this study, the dataset construction process is grounded in practical engineering experience and motivated by real environmental conditions observed in operational marine PV installations. Specifically, this study is based on an actual marine engineering project—the TJUT-U2 floating PV power station, independently developed and deployed by our research team. During long-term operation of this offshore floating PV plant, periodic inspections were conducted using unmanned aerial vehicles (UAVs). The UAV-based visual monitoring system captured high-resolution images of PV modules under different operational and environmental states. These aerial images revealed significant instances of salt crystallization forming on the surfaces of PV panels, particularly along the lower edges and near areas of water splash exposure, as shown in Figure 2. Such salt accumulation directly affects the transmittance of sunlight and degrades the power generation efficiency of the PV modules [27]. Over time, the presence of salt patches not only reduces system output but also accelerates surface wear and thermal imbalance, posing potential risks to the long-term reliability and maintenance cost of marine PV installations. Despite the value of real UAV-acquired field images, several limitations make them unsuitable for direct use as a training dataset. First, the quantity of usable images is limited due to strict access conditions, weather constraints, and limited inspection intervals. Second, the diversity of salt morphology and camera perspectives is insufficient, as environmental parameters such as lighting, angle, and deposition extent are largely uncontrollable. Third, many images contain complex background noise, reflections, and occlusions that hinder precise annotation. These limitations result in datasets that are sparse, imbalanced, and suboptimal for deep learning training, thereby restricting the model's generalization performance. To address these limitations and support the development of a robust salt detection framework, we constructed a customized salt deposition dataset that simulates field conditions while allowing controlled variability. A comparison of on-site datasets and custom datasets showed strong consistency in edge texture and crystallization morphology, as illustrated in Figure 3, the red rectangular box in the figure provides a magnified view of the edge textures and crystalline states of the on-site dataset and the customized dataset.validating the authenticity of the simulation approach. The custom dataset was acquired using a self-designed dataset capture frame, composed of three core modules: a mobile guide rail for positional stability, a sliding device for smooth image sweep across PV panels, and a flexible imaging module capable of mounting different camera types, as shown in Figure 4. This platform allowed image capture under varying conditions of illumination and perspective to replicate real offshore variability. The data acquisition process involved manually simulating salt crystallization on PV module surfaces. A saline solution, prepared to match seawater concentration, was poured evenly over PV surfaces while continuously stirring to ensure consistency. No constraints were imposed on the pouring angle or volume to preserve randomness in deposition patterns. The resulting deposits closely replicated the heterogeneous appearance of actual marine salt crystallization. No restrictions were applied to ambient lighting or camera angles during photography to diversify the dataset further. As a result, the raw dataset contained several hundred high-resolution images, representing a wide range of salt morphologies, deposition densities, and environmental perspectives. This enhanced variability directly contributes to improved robustness in model learning [28]. A series of augmentation techniques was applied to expand the dataset, including scaling, trimming, rotation, and horizontal flipping, effectively increasing data volume and representation diversity. Following augmentation, each image was annotated using LabelMe for annotation, with salt edges manually traced to ensure pixel-level ground truth masks. The final dataset includes 300 annotated images, which were randomly divided into a training set (240 images) and a validation set (60 images) using an 8:2 ratio. An independent test set was also curated for performance verification. Experimental validation showed that the improved model achieved high segmentation accuracy across varying lighting and salt morphologies, confirming the practical applicability and robustness of the constructed dataset in offshore PV scenarios. 2.3. Semantic Thresholding for Low-Level Feature Enhancement 2.3.1. Semantic Ambiguity in Salt and Gridline Features In offshore floating PV systems, visual ambiguity between salt deposition patches and surface gridlines poses a fundamental challenge to accurate image segmentation. As illustrated in Figure 5, these two semantic entities—although physically distinct—often present similar visual characteristics in grayscale representations, particularly in terms of brightness and structure. Gridlines are arranged in a regular, topological pattern aligned with the PV module architecture, forming a lattice-like background feature. In contrast, salt crystallization is highly irregular, randomly distributed across the panel surface, and exhibits fragmented, diffuse morphology. However, due to their co-existence on the module surface, their projections frequently overlap or occlude each other in captured images, leading to semantic entanglement in pixel-level interpretation [29]. Furthermore, salt deposits and gridlines tend to exhibit high grayscale intensities, especially under strong or uneven illumination. This results in overlapping peaks in the global grayscale histogram—forming a bimodal distribution with low inter-class separability. Consequently, traditional thresholding methods struggle to accurately separate the two, often misclassifying gridlines as salt or vice versa, as schematically represented in Figure 5. This inherent semantic ambiguity undermines the reliability of global segmentation algorithms and necessitates the incorporation of region-aware, feature-guided strategies to improve discrimination. To tackle this issue, the subsequent sections introduce a semantic-aware framework that enhances feature separability by leveraging local statistical cues and adaptive thresholding. These methods collectively enable more precise identification of salt regions while suppressing interference from structurally similar background features. 2.3.2. Adaptive Thresholding Strategy for Semantic Feature Extraction To mitigate the semantic entanglement between salt crystallization and PV module gridlines outlined in Section 2.3.1, an adaptive thresholding strategy is adopted to enhance feature separability at the pixel level. Conventional global thresholding algorithms—such as Mean, Otsu, or Sauvola methods—typically operate under the assumption of uniform grayscale distributions and apply a single threshold value across the entire image. While these methods are computationally efficient, their performance is often compromised in heterogeneous scenes, particularly when local illumination varies or when salt deposition partially overlaps with background structures. The image is first decomposed into multiple spatial sub-regions based on its structural layout to address this challenge. Local grayscale distributions are independently analyzed within each sub-region to determine a region-specific binarization threshold. This procedure enables the algorithm to respond adaptively to local intensity contrasts and textural features otherwise lost in global processing. Formally, the input image I is divided into n non-overlapping sub-regions (as shown in Figure 6, n = 9), the area indicated by the red rectangular box 1 corresponds to the clean region of the PV module, while the area indicated by the red rectangular box 2 corresponds to the salt-crystallized region on its surface; both are the primary subjects of this study. This segmentation process is based on the structural characteristics of the image, partitioning the original image into multiple smaller regions. The image information within each region can then be processed more independently, thereby reducing interference between different regions and enhancing the local accuracy of image processing. In the formula, I_s represents the image to be segmented, and I_n denotes the segmented regions. Seg refers to the segmentation algorithm. I_s represents the image to be segmented, and I_n denotes the segmented regions as in Equation (1). For each region I_i , an adaptive threshold B_i is computed using a localized thresholding operator: Through this formula framework, B_n serves not only as a simple adaptive threshold but also encapsulates semantic information, thereby enabling the model to perform more precise pixel-level classification and segmentation. The obtained semantic information is then utilized to process the image. M_n(x, y) represents the segmented semantic information obtained through the thresholding algorithm, and E(x, y) denotes the pixel values corresponding to the grid lines in the image. A value of 1 indicates that the pixel (x, y) has been classified as foreground (salt deposition region), while a value of 0 indicates that it has been classified as background, as in Equation (3). This process preserves the localized semantics of salt deposition while attenuating interference from repetitive background structures such as gridlines or busbars. By incorporating this localized semantic guidance into the segmentation pipeline, a more discriminative and context-sensitive representation of low-level features is achieved, which lays the foundation for the subsequent selection of an optimal thresholding method. 2.3.3. Region-Based Semantic Filtering via Otsu Thresholding To select the most suitable algorithm for regional threshold computation, several classic binarization methods—including Otsu [30], Average, Sauvola [31], and Niblack [32] thresholding—were comparatively evaluated on the constructed salt deposition image dataset [33]. These methods were assessed based on their ability to suppress structural background interference while maintaining high fidelity in representing salt crystallization patterns [34]. Experimental results, shown in Figure 7, indicate that Otsu thresholding consistently achieves superior performance in terms of inter-class separability and segmentation clarity, the red box identifies the noise in the algorithm’s output image that impacts the final model’s output. Otsu’s method determines the threshold by maximizing the between-class variance, which effectively enhances contrast between salt efflorescence and the PV module surface even in cases of grayscale ambiguity. The Otsu algorithm adopts a clustering-based approach to optimize threshold selection by evaluating how different threshold values separate the image into two distinct classes. It calculates the proportion of pixels in each class and the inter-class variance between them. The optimal threshold is determined by maximizing the inter-class variance across the global range of gray levels, based on the total number of image pixels [35]. As illustrated in Figure 8, the histogram demonstrates the concept of maximum inter-class variance in the grayscale image. The blue and green curves represent the fitted probability density functions of the foreground and background, respectively. The optimal threshold (red line) is selected by maximizing the variance difference between the two classes, thereby achieving the best possible separation between foreground and background. In other words, the optimal threshold corresponds to the maximum inter-class variance, indicating the greatest distinction in mean gray levels between the two classes. Specifically, the Otsu algorithm begins by initializing the salt image and defining variables for foreground and background. The means of the foreground and background are then computed. Let w₁ denote the proportion of foreground pixels and w₂ = 1 − w₁ the proportion of background pixels. The inter-class variance, representing the degree of separability between the two classes, is calculated as: where μ₁ and μ₂ are the mean gray levels of the foreground and background, respectively, given by: Here, p(i) denotes the normalized probability of each gray level i, obtained from the histogram h(i) as: where i is the gray level and L is the total number of pixels in the image. Finally, for each image region I_i , Otsu computes the optimal threshold B_i as: To accommodate illumination variation and spatial heterogeneity, Otsu thresholding was applied in a region-wise fashion. Each image was divided into nine sub-regions (i.e., n = 9), and Otsu thresholding was performed individually on each. This region-based application significantly reduced semantic confusion caused by uniform global thresholding, especially in overlapping or gridline-dense areas. The semantic maps generated from region-wise Otsu processing were fused into a composite semantic channel and combined with the original RGB image to form a 4-channel input to the Mask R-CNN model. As shown in Figure 9, this approach improves foreground-background separability, particularly in visually ambiguous regions. The segmentation model exhibits enhanced salt detection performance by leveraging Otsu-derived semantic guidance while minimizing false positives from gridline interference. This semantic-aware preprocessing framework thus provides a robust foundation for feature optimization in complex marine inspection scenarios. It significantly improves the precision and reliability of salt detection on offshore floating PV modules. 2.4. Semantic-Integrated Salt Detection via Mask R-CNN Through the screening of basic models, it is found that two models stand out for application in this scenario: Robust Principal Component Analysis (RPCA) via Low-rank plus Sparse (L+S) Decomposition [36], and Mask R-CNN. From the perspective of application scenarios and core capabilities, RPCA via L+S Decomposition focuses on interference removal and structure extraction at the data level (providing high-quality data for subsequent tasks), while Mask R-CNN focuses on object-level detection and segmentation (enabling end-to-end output of object information); the two play different roles in the technical chain and exhibit complementarity. Since the core of this study is to achieve object-level detection and segmentation of salt deposition on the surface of PV modules, which requires direct acquisition of the spatial location and morphological information of salt deposition, Mask R-CNN is selected as the basic framework for method improvement in this study. In the final stage, the semantic information generated from thresholding is integrated into this enhanced Mask R-CNN model to realize precise instance-level segmentation of salt deposits. He et al [37] proposed Mask R-CNN based on Fast Region-based Convolutional Neural Network (Fast R-CNN) [38]. Mask R-CNN can predict both the class and bounding box of the target, with a more complex model architecture. Mask R-CNN introduces an additional branch that employs a fully convolutional network structure to apply pixel-level segmentation masks to each candidate region. Processing the input feature maps outputs a binary mask that indicates whether each pixel belongs to the corresponding instance. It has been widely applied in fields such as medical diagnosis [39] and agriculture [40], and has gradually evolved into a mature technology. Based on the characteristics of salt deposition on the surfaces of offshore floating PV modules and the practical requirements for salt removal, this study draws upon other object detection frameworks. It employs Mask R-CNN for the image recognition of salt deposition in offshore floating PV systems. The overall architecture can be illustrated as follows: The acquired images of salt deposition on the surfaces of PV modules are first fed into the convolutional neural network to extract high-level features. During this process, the region proposal network generates candidate regions to assist in identifying areas that may contain salt deposition. The mask branch module then processes these candidate regions to generate masks for the salt deposition targets, thereby delineating the specific areas of salt accumulation. The mask enables precise extraction of the salt deposition contours by performing pixel-level segmentation for each salt deposition target. Simultaneously, the box regression module accurately localizes the bounding box of the salt deposition by computing the minimum enclosing rectangle for each target, providing spatial information. The classification module then classifies the detected targets to determine whether they correspond to salt deposition or PV modules. In addition, a sliding window mechanism is employed to generate multiple anchor boxes on the feature map and to evaluate the target probability for each anchor. Finally, after processing through multiple modules, the framework outputs the mask, bounding box, and class information for the salt deposition targets. The network architecture of Mask R-CNN is illustrated in Figure 10. In the aforementioned thresholding process, semantic information is introduced to enhance the effectiveness of image segmentation, ensuring that both grid lines and salt deposition patches are correctly classified according to their semantic characteristics. After completing the threshold-based binarization for each segmented region, the nine processed sub-images are stitched together according to their original spatial positions. The stitched image retains the independently processed results of each segmented part while preserving the overall structural coherence of the image, the green box in the output indicates the salt-dissolving area identified by the model. At this stage, the stitched image is treated as an additional channel and combined with the original Red (R), Green (G), and Blue (B) channels to form a four-dimensional space input for the Mask R-CNN model for object detection and segmentation. Integrating semantic information into the image processing pipeline enhances the model's understanding of image semantics, thereby improving its performance in complex scenarios. Compared with the original unsegmented images, the improved framework significantly enhances the model's capability to handle complex backgrounds. Through the effective utilization of semantic information and the adjustment of the image segmentation strategy, this study not only optimizes the application of the thresholding algorithm in image processing but also improves the accuracy of the Mask R-CNN model in detecting salt deposition on PV modules. This improved approach provides a more precise and efficient solution for image recognition tasks in practical applications, demonstrating strong adaptability and robustness, particularly in the presence of complex structures and noise within the images. Given the irregular and difficult-to-identify boundaries of salt deposition, this study proposes a semantic information-based framework for recognizing salt deposition on the surfaces of offshore floating PV modules. This framework enhances the accuracy of salt deposition detection and segmentation by processing the salt deposition image data through a semantic information-guided thresholding logic. A four-dimensional space input is constructed by integrating the thresholding channel, which is optimized based on semantic information, with the original RGB image. This enables the model to not only learn the color information from the RGB image but also capture the critical textures and structural features highlighted by the thresholding process. In this manner, the model can achieve a comprehensive and in-depth understanding of the image information pertaining to salt deposition on PV module surfaces, allowing it to identify and segment salt deposition regions accurately. The improved network architecture is illustrated in Figure 11, the red box indicates the salt precipitation zone, while the green box indicates the photovoltaic module zone.