Detection of visual faults in photovoltaic modules using a stacking ensemble approach

Abstract

Faults in photovoltaic (PV) modules may occur due to various environmental and physical factors. To prevent faults and minimize investment losses, fault diagnosis is crucial to ensure uninterrupted power production, extended operational lifespan, and a high level of safety in PV modules. Recent advancements in inspection techniques and instrumentation have significantly reduced the cost and time required for inspections. A novel stacking-based ensemble approach was performed in the present study for the accurate classification of PV module visible faults. The present study utilizes AlexNet (a pre-trained network) to extract image features from the aerial images of PV modules with the aid of MATLAB software. Furthermore, J48 algorithm was applied to perform the feature selection task to determine the most relevant features. The features derived as output from the J48 algorithm were passed onto train eight base classifiers namely, Naïve Bayes, logistic regression (LR), J48, random forest (RF), multilayer perceptron (MLP), logistic model tree (LMT), support vector machines (SVM) and k-nearest neighbors (kNN). The best performing five classifiers on the front run with higher classification accuracies were selected to formulate three categories of stacking ensemble groups as follows: (i) three-class ensemble (SVM, kNN, and LMT), (ii) four-class ensemble (SVM, kNN, LMT, and RF), and (iii) five-class ensemble (SVM, kNN, LMT, RF, and MLP). A comparison in the performance of the aforementioned stacked ensembles was evaluated with different meta classifiers. The obtained results infer that the four-class stacking ensemble model (SVM, kNN, LMT, and RF) with RF as the predictor achieved the highest possible classification accuracy of 99.04%.

1. Introduction

Electricity demand across the world has seen a drastic rise in recent times due to scientific innovations, technological advancements and population outbursts. At present, fossil fuels including coal, natural gas and oil have been considered the major contributors to the production of electricity accounting for about 79.7% [1]. Furthermore, widespread fossil fuel usage has accelerated the exhaustion of natural fossil fuel supplies, which might run out in a matter of decades. Furthermore, heavy reliance on fossil fuels has extended their contribution towards greenhouse gas emissions thereby resulting in climatic changes and global warming [2]. Considering the alarming scenarios, the focus of electricity production has shifted from fossil fuels to renewable energy sources to meet the rising energy demands and deliver green energy. Reports reveal that renewable energy sources contribute about 27.3% of the total electricity production across the globe with PV-based power production accounting for about 2.8%. PV-based power production is placed next to wind energy power generation as one of the leading contributors to power generation [3]. PV-based power generation employs PV modules to convert the incident light into electricity. Yearlong availability and wide accessibility of solar energy have attracted numerous capitalists and investors. To support the claim, one can observe that the total global PV installed capacity has gradually improved and can reach around 440 GW (by the end of 2022) [4]. Moreover, the drop in manufacturing cost of PV modules during the last three decades (by 99%) has claimed the interest of capitalists and investors towards PV power generation.

Apart from all the advantages mentioned above, the PV industry faces several challenges that co-exist in recent times such as (i) dynamic operating conditions, (ii) initial installation cost, (iii) fault occurrences, (iv) module reliability, (v) performance degradation and (vi) various other necessary parameters [5]. The continual outdoor operations and exposure to dynamic climatic conditions can induce fault occurrences including, burn marks in PV modules, discoloration, glass breakage, delamination and snail trails. Such faults can affect the power output, life span, reliability and elevate the concern for safe operation. Studies reveal that the annual loss in power output in PV modules reaches up to 18.9% due to the faults incurred [6]. The setbacks have raised a situation that necessitates on-time and accurate fault diagnosis such that the reliability, safety, lifespan and performance of PV modules are conserved. Fault diagnosis works with a prime objective of detecting and classifying faults such that preventive measures for a particular situation can be planned promptly. Overall, fault diagnosis is considered a necessary operation in large PV farms. Conventionally, fault diagnosis in PV farms was executed through visual inspections performed by trained professionals. However, such inspections demand huge manpower, are highly time-consuming and non-feasible over large PV module installations. Currently, technological advancements have delivered numerous non-destructive inspection strategies involving, ultraviolet fluorescence imaging, photoluminescence imaging, electrical measurements, electroluminescence imaging and infrared thermography [7].

The application of unmanned aerial vehicles (UAV) to large-scale PV plant monitoring has received major attention among industrialists, capitalists and investors. Recent developments in UAV technology and owing to the versatile application range of UAV, they have been adopted in several fields like disaster relief, surveillance, cargo carrier, large-scale inspections, photography and search & rescue. The usage of UAVs can help reduce the time consumed for inspections accompanied by minimal human interference and non-destructive inspections. Inspections of PV systems using UAV technologies have been discussed in several literature provided as follows. Thermal cameras installed in UAVs were used to conduct contactless and non-destructive examinations by the authors in Ref. [8]. With the aid of thermal cameras, the identification of faults is confined only towards detecting hotspots due to the poor resolution of thermal cameras and the higher speed of UAVs. Hotspot occurrence in a PV module can be a significance of several fault types like partial shading, corrosion and short circuits. Solder bond failure and micro cracks. Detection of the specific fault type using thermal images can be a challenging task due to the lack of information from the pseudo-color thermal images [9]. The problems faced while using thermal cameras can be eliminated by swapping the thermal imaging cameras with a digital camera of high resolution such that visible faults can be easily identified. Equipping digital cameras in UAVs can help in acquiring true color images from PV modules that can identify faults including burn marks in PV modules, discoloration, glass breakage, delamination and snail trail. Numerous image processing strategies like edge detection [10], image mosaicing [11], aerial triangulation [12] and correlated texture feature extraction [13] were used to identify faults in PV modules. However, the quality of the obtained UAV images has a significant impact on how well the strategy functions. Also, the UAV factors include vibrations transferred due to flight, velocity of the wind, light reflections and haze.

In recent times, numerous fault diagnosis problems have been resolved with the aid of a convolutional neural network (CNN) due to their exceptional performance display for images with low resolution. An array of CNN layers is stacked to create deep learning architectures that can automatically perform feature extraction, selection and classification on input images in machine vision uses [14]. Several works of fault diagnosis in PV modules adopting deep learning and machine learning techniques are presented in Table 1.

Table 1.

Literature work on PV module fault diagnosis using deep learning and machine learning.

Methodology	Technique used	Reference
Deep Learning	CNN with SVM (Digital images)	[14]
	CNN (Electroluminescence images)	[15]
	CNN (Digital images)	[16]
	Deep CNN with VGG16 backbone (Thermal images)	[17]
	Pre-trained AlexNet (PV array faults)	[18]
Machine Learning	Support vector machines	[19,20]
	K nearest neighbour	[6,21]
	Naïve Bayes	[22]
	Random forest	[[23], [24], [25]]

ML algorithms are intelligent and consist of their strengths and weaknesses. To improve the generalization capacity of individual algorithms, researchers have proposed ensemble learning techniques to eliminate the shortcomings of individual techniques and combine the strengths of the individual techniques. The ultimate objective of the ensemble technique is to enhance the accuracy of weak algorithms (classifiers) and to produce a stable model that displays better performance than individual algorithms. Stacking, boosting and bagging have been used as the pre-dominant ensemble techniques over the years. Among the ensemble techniques, stacking has proven to provide the highest generalized classification accuracy. In this technique, the various weak learning algorithms are considered in which the predictions obtained from the initial classification are fed as input to the stacked meta-classifier to derive the final classification. Stacking helps in developing strong classification models with minimal bias to the allied components. To support the effectiveness of ensemble methods various literature are discussed as follows. An ensemble model with a combination of Naïve Bayes, SVM and kNN was proposed by Eskandari et al., to identify line faults in a PV module [26]. A stacking and bagging-based ensemble strategy was used by Justin et al. to find PV defects. The classification accuracy of the stacking-based ensemble was 94%, whereas the accuracy of the bagging method was 79.5% [27]. Dhibi et al. used a hybrid ensemble model of support vector machine (SVM), k nearest neighbour (kNN) and decision trees to categorize string-level PV module errors. Ensemble learning techniques have historically been used to apply numerical data obtained from I–V characteristics observed at PV plants or through other analytical models [28]. The use of ensemble learning approaches with features extracted from image data is still not fully investigated, leaving room for more research. The following observations were identified owing to the above-discussed literature.

• 1.Stacking-based ensemble approaches were not attempted in diagnosing PV module fault scenarios with a combination of deep learning (feature extraction) and machine learning (feature selection and classification) techniques.
• 2.Most of the studies attempted in PV module fault diagnosis utilized infrared and electroluminescence images rather than digital images. The usage of infrared and electroluminescence images was confined only to the presence of hotspots or microcracks. Additionally, such techniques were considered challenging in the detection and identification of multiple fault scenarios.
• 3.Acquisition of image data from PV modules using UAVs is a highly challenging task. Furthermore, the availability of PV module data with fault scenarios is very scarce and not available in public repositories.
• 4.Machine learning and deep learning techniques were applied individually for diagnosing faults in PV modules. However, a combined approach is still in the nascent stages of development. Deep learning techniques have proven to be exceptional feature extractors while machine learning techniques deliver precise and accurate classification results over numerical data.

The aforementioned challenges state that there exists a definite essential to create an innovative and advanced fault diagnosis method. However, the efficiency of different algorithms in classifying instances completely depends upon the individual working principle of the algorithms. To overcome the deficiencies concurred by the ML algorithms during classification, ensemble techniques were proposed to help in delivering enhanced classification performances. The overall workflow of the proposed ensemble technique is presented in Fig. 1.

Fig. 1.

Proposed stacking ensemble method for PV module fault diagnosis.

In the current study, a stacking-based ensemble technique is suggested to diagnose PV module faults. In the proposed work, the top five classifiers (among eight individual classifiers) were selected to form three categories of stacked ensemble groups namely, (i) three class (support vector machine (SVM), k nearest neighbour (kNN) and logistic model tree (LMT)), (ii) four class (SVM, kNN, LMT and random forest (RF)) and (iii) five class (SVM, kNN, LMT, RF and multilayer perceptron (MLP)). The predictions of every stacked group were fed as input to six different meta-classifiers including SVM, kNN, LMT, RF, MLP and J48. Among the different combinations, four class stacked ensembles with RF as meta classifier delivered a classification accuracy of 99.04% outclassing individual classifier performances. The contributions and novelty of this study are summarized below.

• 1.A stacking-based ensemble approach was proposed in the present study to detect and classify faults in PV modules. The approach combines multiple machine-learning algorithms to enhance the accuracy of the created model. The use of such ensemble methods can revolutionize the PV module fault diagnosis field by leveraging the capability of several models to achieve higher classification accuracy.
• 2.Digital images of PV modules were acquired with the aid of a UAV for six different test conditions of PV modules namely, snail trail, delamination, glass breakage, discoloration, good panel and burn marks. The collected images were augmented to artificially expand the image dataset. The process of augmentations helps in enhancing the learning capability of the machine learning model thereby increasing the chances of accurate classification.
• 3.By using a pre-trained AlexNet to extract image features, the indicative of different types of faults were identified with the extracted significant features. This approach represents a novel and innovative way of using machine learning to analyze image data. The obtained features were selected using the J48 decision tree algorithm.
• 4.The individual classification performance of the considered base classifiers was assessed initially while stacking-based ensembles were formulated from the top five performing individual base classifiers. The results derived were analyzed and the best-performing classifier combination was suggested for real-time.

Overall, this study introduces several innovative techniques for fault detection and classification in PV modules, including the use of UAVs for image collection, data augmentation techniques for machine learning and ensemble approaches for fault detection. These novel approaches have the potential to significantly improve the accuracy of fault diagnosis in PV modules, paving the way for more efficient and effective maintenance and repair of these important components.

2. Experimental process

Determining the condition of a PV module as good or faulty is the primary goal of the proposed study. Whenever a PV module is discovered to be in faulty condition, the proposed methodology focuses on determining the type of PV module fault. The experimental setup is described in the current section as, a data acquisition process and a short description of various PV module faults considered in the study.

2.1. Experimental setup

The experimental setup considered in the study is made up of PV modules (with good and fault conditions), a UAV monitoring platform installed with sensors installed on-board, a digital camera with high resolution, a ground control station and multiple processors [29]. Udhaya Semiconductors Ltd produced the PV modules used in the study, and they have been in operation for over 11 years. The complete data acquisition was carried out in a laboratory environment with PV modules placed at different locations and the images were acquired using a DJI Mavic 2 Zoom drone. A drone pilot with the use of a portable remote control operated the UAV to acquire PV module images for various conditions. A wireless transmission medium aided in transferring the images acquired to a data storage system. The stored images were preprocessed prior to being fed as input to pre-trained networks for image feature extraction. The extracted image features were further downsized using a feature selection algorithm (J48). The selected and most significant features were passed onto the ML classifiers to process the classification task. A total of six PV module conditions (5 faulty & 1 good) were utilized during the acquisition of image data. The detailed specifications of PV modules adopted in the study are presented in Table 2.

Table 2.

Detailed specification of PVM.

Parameter	Value
Model Name	USP-36
Type	Monocrystalline
Current (Isc)	2.25 A
Weight	3.5 kg
Dimensions	1011 × 435 × 36 mm
Voltage (Voc)	20.6 V
Maximum Power Point Voltage (Vmpp)	17 V
Maximum Power	36 W
Maximum Power Point Current (Impp)	2.1
Efficiency	9–10%
Number of Cells	36

The drone was operated between a height range of 1 and 5 m above the PV module throughout the data collection process to collect image data. To capture images of PV modules, the drone was operated twice, for a total of around 14 min each time. Burn marks, discoloration, delamination, glass breakage, snail trails, and good were all test conditions for PV modules that were positioned in various locations across the lab for image data collection. In around 2.5 min, 100 images were captured for each test condition for each PV module. Fig. 2 shows an example of UAV-acquired images.

Fig. 2.

PV module sample images acquired for PVM conditions snail trail, burn marks, discoloration and good panel.

2.2. Experimental procedure

The overall experimentation was conducted in five different phases: (i) image data acquisition using UAV, (ii) data augmentation on acquired images, (iii) application of pre-trained AlexNet for feature extraction, (iv) involving J48 decision tree algorithm in the selection of significant or contributing features and (v) classification of features using three, four and five class stacked ensemble models. In the first phase of data acquisition, UAV was used to take pictures of PV module conditions including, glass breakage, good panel, burn marks, delamination, discoloration and snail trails. The UAV controlled through a hand-held remote controller was made to fly over PV modules placed at several locations under laboratory conditions. The remote controller was developed in such a way as to acquire images through wireless modules and store them temporarily that can later be transferred into a storage device through a memory card or external cables. During the acquisition, a total of 600 images (representing 100 images per class) were captured and saved inside individual folders. Deep learning networks are capable of forming unique fault patterns when trained with numerous data consisting of different orientations. Additionally, the biggest challenge imposed while working with deep learning techniques involves the creation or acquisition of datasets. Hence, to elevate the pre-trained network performance and improve the feature learning capability, the second phase of data augmentation is adopted in the study. Data augmentation artificially expands the image dataset through the application of simple image transforms like noise, shift, warp, flip, rotate, zoom and blur. The present study augments the original dataset of 600 images to 3150 images (525 images per class) through the application of image transforms. The details of the image transform applied to the dataset are presented in Table 3. The number of instances created was made uniform to eliminate biasing during classification. The augmented dataset was resized to a size of 227 × 227 pixels prior to feeding into pre-trained AlexNet for feature extraction. The features from the final layer of the pre-trained AlexNet network were extracted and saved into a data file (‘.csv’ file). The extracted features were split into training and testing datasets with a train-test split ratio of 70 % and 30%. Post extraction of features, the most significant features that can positively contribute towards classification were selected using a feature selection algorithm (J48). Finally, the selected features were classified using machine learning classifiers and an ensemble strategy was applied to determine the condition of PV modules.

Table 3.

Basic image transforms applied to produce an augmented PVM dataset.

Transform operation	Flip (Horizontal, Vertical)	Blur	Rotation (Clockwise, Anticlockwise)	Noise	Warp
Value	90°	Gaussian	0°–180°	Random	40

2.3. Visible faults in PVM

The prevalence of faults in PV modules can be created due to the dynamic operating conditions induced by thermal stresses, severe climatic change and environmental uncertainty. Faults in a PV module can significantly affect the operational lifespan, reliability and performance. The most common faults encountered in PV modules have been displayed in Fig. 3 (a) – (e) along with Table 4 providing a brief description of the PV module faults.

Fig. 3.

Various PVM visual faults.

Table 4.

Visual fault occurrence in photovoltaic modules.

S.No	Visual Faults in PVM	Effect on Modules	Reason for the occurrence of a fault
1	Burn marks [30]	Safety hazards and performance degradation	Failure in solder bond, disruption in ribbon and heating at local regions
2	Discoloration [31]	Change in physical color into brown or yellow, loss in output power	Extensive exposure to humidity, heat and UV rays
3	Snail trail [32]	Quicker Degradation	Micro crack stress induction at the corners
4	Glass breakage [33]	Lower Irradiance, Corrosion and Moisture invasion	Stress due to thermal expansion, shocks transmitted during installation and transportation
5	Delamination [34]	Moisture invasion leading to corrosion	Decrease in the adhesive properties between glass, encapsulant and cover

3. Pre-trained AlexNet-based feature extraction

The current section provides a background of the CNN accompanied by the feature extraction process using pre-trained AlexNet and the feature selection process using the J48 decision tree algorithm performed in the study. Additionally, a brief description of the adopted classifiers and the proposed stacking-based ensemble approach is described below.

3.1. Background of convolutional neural networks

Recent years have witnessed the gradual development of deep learning as an influential tool being adopted to solve various computational intelligence and machine vision problems. Deep learning architectures are developed using non-linear-natured convolutional neural networks (CNNs). CNN exhibits high levels of feature extraction capability and expands the compatibility over low-resolution images [35]. Each CNN has three main levels in its architecture (besides a number of special layers and hyperparameters) namely, convolutional, pooling and fully connected layers. The primary function of each layer is described as follows.

• •Convolutional layer – The key feature extracting or learnable layer that consists of numerous learnable filters or kernels that generate an optimal feature map based on the input data and formulate specific patterns for each data fed as input.

• •Pooling layer – The layer is stacked along with the convolutional layer that focuses primarily on reducing the dimensional complexity thereby recognized as a downsampling layer.

• •Fully connected layers – The penultimate layer of a deep learning algorithm that helps in creating vectors of image matrices. Fully connected layers equipped with softmax activation functions perform the classification task for multiclass problems (sigmoid activation for binary class problems).

During the training process, image patterns are continually learned by the convolutional and pooling layers. The feedback concept termed error backpropagation is provided in deep learning models to alter the weights of neurons present in CNN to minimize the error occurrence. Traditionally, CNN models are equipped with provisions that can absorb the most important features that give towards enhancing classification accuracy. Constructing and training a CNN architecture from scratch demands huge volumes of properly labeled data. However, the creation of such huge datasets requires domain knowledge, human intelligence, data analytics and time consuming. Owing to the challenges posted above, numerous researchers have endorsed and verified the usage of pre-trained networks successfully for custom applications. Pre-trained networks display superior feature extraction abilities as they have been trained over large volumes of image data. Several pre-trained network models like VGG16, AlexNet, ResNet50 [36], GoogleNet [37] etc., have been discussed in various kinds of literature and their pre-trained versions are accessible in free repositories for easy access.

3.2. Extraction of features using pre-trained AlexNet

The process of feature extraction helps in reducing the variables to express and understand a relatively large dataset. In the present work, Pre-trained AlexNet was used to extract the features from the obtained PVM pictures. The pre-trained AlexNet model was designed and introduced at the annual ImageNet Large Scale Visual Recognition Challenge (ILSVRC) competition by Alex Krizhevesky et al., to solve image classification tasks. The network set a trend in computer vision applications by outperforming every renowned technique. AlexNet was designed to classify 1000 different object classes by undergoing training with over 1.2 million images of high resolution. Fig. 4 represents the structure of AlexNet involved in the process of feature extraction. The model is constructed with eight deep layers that include one out-put softmax layer, two fully connected and five convolutional layers. PVM aerial image features were extracted using AlexNet and deposited into a data file for further processing into the selection of features and classification of faults. Prior to feeding AlexNet with images for feature extraction, the images must be resized to 227 × 227 pixels since AlexNet is designed to accept images of the aforementioned size. In the first convolutional layer, three foremost operations namely, convolution, pooling (max) and local response normalization are carried out. The convolutional layer consists of 96 learnable filters each of 11 × 11 size that convolve, learn, extract and deliver a simpler image output of pixel size 55 × 55. The derived output of the image is further passed into the subsequent convolutional layer consisting of 256 receptive kernels along with a max pooling layer of 3 × 3 size that delivers an image output of size 27 × 27 pixels. The output image retrieved is further processed into the consecutive third, fourth and fifth convolutional layers containing rectified linear units (ReLU) activation functions to derive an image of size 13 × 13. The processed image is finally pushed into two fully connected layers containing 4096 parameters followed by an output softmax layer that helps in classifying image classes. ReLU is predominantly adopted in deep learning networks due to its competence in solving non-linear tasks. Also, the usage of ReLU activation functions consumes minimal time in comparison with other activation functions. Deep learning models consist of a major drawback in the form of overfitting that results in poor output performance of the model. Thus, fully connected layers are supplied with dropout layers to eradicate the challenge of overfitting. The current study applies a 0.5 ratio dropout layer. The most significant characteristic of CNN is that class discriminative features are learned automatically based on the labels embossed in the datasets that enable classification. Of late, transfer learning has proved to be an efficient technique in extracting image features that facilitate classification on user-defined image datasets with minor variations made in the last few layers of pre-trained models. The present work extracts features from the neurons activated in the final layer (‘fc8’) of pre-trained AlexNet. Specifically, the fc8 layer in AlexNet consists of 1000 neurons that are completely connected to the 4096 neurons in the preceding fc7 layer with the aid of trained weights. During inference, the fc8 layer takes the 4096-dimensional feature vector output by the fc7 layer and computes a 1000-dimensional vector using the learned weights. The fc8 layer in AlexNet maps the high-level features learned by the previous layers. Here, each neuron in fc8 gets input information from the fc7 layer (all neurons) which are weighed by the corresponding weights. In a way, here, a feature fusion takes place and gives better features. The extracted features were stored in a “.csv” data file that contains 3150 rows representing images of every class, 1000 columns representing image features and one column depicting the class names. The detailed specification of AlexNet architecture is presented in Table 5.

Fig. 4.

Extraction of features using pre-trained AlexNet.

Table 5.

The architecture of pre-trained AlexNet.

Layer	Details
Convolution 1	11X11, Max pooling, 96 filters, LRN, ReLU
Convolution 2	3X3, Max pooling, 256 filters, LRN, ReLU,
Convolution 3	384 filters, ReLU
Convolution 4	384 filters, ReLU
Convolution 5	256 filters, ReLU

4. J48 decision tree-based feature selection

The process of selecting the utmost contributing features (towards classification) among numerous extracted features is termed feature selection. The presence of redundant data (features) can have a degrading impact on the classification performance of the model thereby elevating the computational complexity. Thus, feature selection can assist in eliminating non-contributing features such that the individual classifier performance is improved. Several literature studies have adopted the J48 decision tree as a feature selection tool due to its ability to display data effectively. The J48 decision tree displays a graph-like structure that bears a resemblance to a tree with a set of classification rules. Decision trees are made up of parts similar to that of a tree namely, nodes, branches, leaves and a root. Each component in a decision tree represents a significant parameter described as follows: (i) nodes – represent class attributes linked from the root to the leaves through branches, (ii) leaves – denote the class labels, (iii) branches – representation of the rules formed or collective decisions and (iv) root node– represents the most significant node and descends in search of a pure leaf. Highly significant and useful features that are instrumental in improvising the classification performance are estimated using suitable criteria with the aid of decision nodes. The present work adopts the J48 algorithm to identify and choose the prominent features from the mined aerial image features [[38], [39], [40], [41]].

To determine the optimal number of features an experimentation was carried out by altering the minimum number of object criteria in the J48 algorithm. The criterion value displaying minimal time for computation with optimal classification accuracy was selected. Additionally, the features involved for the particular criterion value are selected to perform classification. The observations were tabulated and strategized as a 2D graph with a minimum number of objects on the X-axis against the time taken and classification accuracy on the Y-axis. Fig. 5 represents the plot describing the experimentation carried out during the process of feature selection. From Fig. 5, one can observe that the performance of the decision tree classifier degrades with a gradual rise in the minimum number of objects. Nevertheless, the time required for model building displayed a decreasing trend with an increasing number of objects. Additionally, one can observe an abrupt descent in the time consumed for model building when the minimum number of objects was 70. Such a sudden fall in time denotes the occurrence of optimal value that displayed an 89.24% classification accuracy by consuming 4.78 s to build the model. The identified optimal value corresponds to 70 minimum objects that were filled with 12 contributing features. A pictorial representation of the decision tree representing the selected features is provided in Fig. 6. The decision tree is established based on the depth-first technique which states that the order of significance degrades from highest to lowest. The feature with high significance is denoted as the uppermost node while the feature with poor significance is denoted as the lowermost node. Features 51, 150, 359, 394, 582, 770, 918, 927, 961, 963 and 972 are the selected features to facilitate PVM fault classification.

Fig. 5.

Plot representing the experimentation involved during feature selection with number of objects (X-axis) vs time taken to build model and classification accuracy (Y-axis).

Fig. 6.

Decision tree with selected features (70 min no of objects).

5. Classification methods

The current study adopts the WEKA software tool to perform the feature selection and classification tasks. The software consists of numerous algorithms that can generate base classifiers and ensemble models. WEKA is open-source software that is designed to carry out supervised and unsupervised processes in the form of regression, clustering, classification, data visualization and association. Classification of features is the process carried out after selecting the most significant features from the extracted aerial image features. In the present section, a brief description of the intelligent classifiers (SVM, kNN, LMT, RF and MLP) adopted in the study is described along with the proposed ensemble classification models.

5.1. Support vector machines (SVM)

Support vector machines are supervised machine learning algorithms that can be implemented to solve classification and regression problems. SVM algorithms are linear models that exhibit good performance over linear and non-linear tasks. The basic working principle of SVM is to create a hyperplane or draw a line that separates the dataset into individual classes. If the data points available in the hyperplane are not oriented linearly then SVM transforms the data points to a higher dimensional space through nonlinear transforms. Initially, SVM algorithms were designed as dichotomizers or binary classifiers. The primary aim of the algorithm is to identify the optimal hyperplane by reducing the generalization error and increasing the distance between the hyperplane separation. Practically, the datasets corresponding to a single class cannot be separated in the original space by a straight line. Consequently, the data points present in the original space must be transformed into a new or higher feature space in which a straight hyperplane can be identified to separate the classes. SVM algorithms are generally preferred over other machine learning algorithms due to their good generalization capability, strong theoretical foundation, ability to derive global solutions for classification and inhibition towards dimensional complexity [42].

5.2. K-nearest neighbour (kNN)

Numerous machine-learning processes were employed to solve classification and regression problems. Amidst all the existing procedures, k- Nearest Neighbour has been accepted extensively in pattern recognition due to the lazy working strategy, simple structure and parameter-free nature. Majority voting is the basic working principle of kNN wherein the instances are classified depending upon the neighboring object votes collected from the internal parameter space. The classification of objects is carried out by the classifier based on feature similarity. The parameter-free characteristics of kNN enable the classifier to eliminate assumptions over the disseminated data. Also, kNN models inherit a lazy learning strategy that makes them reluctant to adapt new models or learn new features that assist in generalizing data. Overall, the classification of datasets using kNN involves cluster formation that calculates neighboring object distance to distinguish between image features. Efficient result production, faster implementation on huge datasets and effectiveness over noisy data are some advantages of utilizing a kNN classifier [43].

5.3. Logistic model tree (LMT)

Logistic model tree (LMT) belongs to the tree-based classifiers that work by combining the learning methods of decision tree and logistic regression. LMT works just like a decision tree with minimal modifications. Initially, the data is split based on the information gain obtained using a logistic variant. Further, every node of the tree is equipped with a logistic regression model created with the aid of the LogitBoost algorithm. The pruning of trees is carried out using the classification and regression tree (CART) algorithm. To avoid overfitting in the training data, LMT adopts cross-validation to determine the range of LogitBoost iterations.

5.4. Random forest (RF)

As the name implies, a random forest is a group of decision trees that functions as a supervised learning approach to deal with classification and regression problems. The basic working principle of RF relies upon the information gain received from multiple decision trees. Subsequently, a voting strategy is equipped over the decision trees to forecast the final predictions that deliver the best results. The advantage of using RF dwells in the elimination of model overfitting owing to the average collective decisions performed. RF has outperformed other machine learning classifiers in terms of classification performance because of its ensemble technique, high accuracy, simple adaptation, and adaptable design. The working of RF can be described using the following steps.

• •Initially, the input dataset is utilized to determine the data samples randomly.

• •Consequently, every sample data is denoted in a decision tree format with the results obtained collectively from the decision tree created.

• •Finally, the possible outcomes of a particular problem are predicted using a voting strategy.

5.5. Multilayer perceptron (MLP)

Perceptron was introduced to tackle linearly separable classification problems. However, the application of perceptron cannot be expanded for problems with datasets that are not linearly separable. Multilayer perceptron (MLP) was introduced to counteract and classify datasets that are not linearly separable. In general, the major difference between a perceptron and MLP is the presence of multiple hidden layers between input and output layers in MLP. Additionally, MLPs are generally termed feed-forward neural networks consisting of input, output and hidden layers. The working of MLP is provided as follows: Alike perceptron, the input data is forwarded into an MLP by considering the dot product of input with the hidden layer weights. The obtained dot product value is restrained at the hidden layers that are not pushed forward unlike perceptron.

• •Like perceptron, the input data is forwarded into an MLP by considering the dot product of input with the hidden layer weights. The obtained dot product value is restrained at the hidden layers that are not pushed forward unlike perceptron.

• •The hidden layers in MLPs are applied over activation functions like tanh, sigmoid and rectified linear units to impose non-linearity in the weights calculated.

• •The calculated weights obtained from the activation functions are passed onto the next MLP layer with corresponding weights. The above steps are continued until the final output layer is reached.

• •The backpropagation algorithm is attached to the final output layer (in case of training) to propagate the variance between the input and output such that the weights are adjusted to minimize error. On the other hand, sigmoid or softmax functions are assigned (in case of testing) to deliver the classification results.

5.6. Proposed ensemble classification models

Stacking is a powerful machine-learning technique that has been used successfully in a wide range of applications, including image recognition, natural language processing, and financial forecasting. It is particularly useful when the dataset is complex and contains multiple variables or when multiple models perform well on different parts of the dataset. By combining the strengths of these models, stacking can produce more accurate predictions than any individual model alone. The main idea behind stacking is to use the strengths of multiple models to compensate for their weaknesses. For example, one model may perform well on some parts of the dataset but poorly on others, while another model may have the opposite strengths and weaknesses. By combining the predictions of these models, the overall accuracy can be improved. To implement a stacking-based ensemble approach, the training data is first split into multiple subsets. Each subset is used to train a different machine-learning model. Then, the predictions of these models are combined using a me-ta-model, which learns to weigh the predictions of each model and produce a final output. The meta-model can be trained on a separate validation set to ensure that it does not overfit the training data. The present study involves a stacking-based ensemble approach that consists of two classification stages. The first stage involves several intelligent classifiers like SVM, kNN, LMT, RF, MLP, logistic regression, J48 and Naïve Bayes to categorize the PV module image features and the performance of individual classifiers was assessed. During the second stage, a meta-level classification strategy involving stacking was utilized to improve the classification accuracy by combining the predictions made by individual classifiers. In the current study, three stacking-based ensemble classification models (Fig. 7) were designed that are described as follows.

• •Three class ensemble – Developed by merging three base classifiers namely, SVM, kNN and LMT whose outputs are merged into the LMT meta classifier.

• •Four class ensemble – Developed by merging four base classifiers namely, SVM, kNN, LMT and RF whose outputs are merged into the RF meta classifier.

• •Five class ensemble – Developed by merging five base classifiers namely, SVM, kNN, LMT, RF and MLP whose outputs are merged into the RF meta classifier.

Fig. 7.

Proposed three, four and five class stacking-based ensemble models.

To provide a clear understanding of the stacking procedure, one can consider for instance, the three-class stacking ensemble in which three classifiers namely, SVM, kNN and LMT produce individual outputs P1, P2 and P3 (corresponding outputs represented in green circles). These outputs are amalgamated or ensembled with the help of a meta-classifier to perform stacking operations. Amongst the adopted meta-classifiers, LMT provides the highest classification accuracy for three class ensembles. The above process is carried out for four class and five class ensembles that adopted RF as the meta-classifier to produce the highest classification accuracy.

6. Results and discussions

The prime objective of the present work is to detect and classify visual faults in a PV module using deep learning features and machine learning classifiers. The drone images acquired from PV modules were pre-processed and fed into the pre-trained AlexNet network for image feature extraction. The collected features were stored in a “.csv” data file from which the most significant and contributing features were selected using the J48 decision tree algorithm. The selected features were initially classified with the aid of intelligent base classifiers like SVM, kNN, LMT, RF, MLP, logistic regression, J48 and Naïve Bayes. The effectiveness of the individual classifiers was assessed prior to the formation of ensemble classification models. The top five classifiers were adopted in the formation of three categories of stacked ensemble models. The selected features along with the predicted knowledge were passed into the stacking-based ensemble models for performance evaluation. The selected feature dataset was split into a 70-30 % train-test split ratio for further processing. Also, a ten-fold cross-validation was adopted to determine the classifiers performance. This section discusses the impact of features on classification performance as well as a comparison of basic classifiers and ensemble classification models.

6.1. Effect of features on classification performance

The dataset comprises a uniform dataset comprising of 3150 images representing six test conditions (525 images for each condition). The features of the aerial image dataset were mined from the penultimate layer (‘fc8’) of AlexNet delivering 1000 image features for every image. The literature states that the complete feature set extracted might not significantly contribute toward the effective classification of PV module test conditions. Additionally, the presence of inappropriate features in a dataset can hinder the classification performance thereby consuming more time for training and increasing the complexity during computation. Thus, it is vital to eradicate unwanted image features such that the performance of classifiers is enhanced. The J48 decision tree technique is typically used to remove unimportant characteristics and choose significant ones. The decision tree representing the set of selected features considered in the current study is presented in Fig. 6. The decision tree represents the importance of the features in a descendant order in which the feature of high importance is placed at the top while the features with the least significance are placed at the bottom. Noncontributing features are neglected automatically and will not be displayed during the visualization of the tree. The process of dimensionality reduction using the J48 algorithm is provided as follows. Initially, the root node (highly significant feature) of the decision tree is considered and the classification performance is assessed and recorded. Similarly, in the next step, the root note along with the next significant node are combined and the classification performance of the combination is evaluated and recorded. The method is accomplished continuously for all the available combinations of features in the decision tree and classification accuracy for each combination is recorded. The effect of features on classification performance is depicted in Fig. 8.

Fig. 8.

Effect of features on classification performance.

From Fig. 8, one can observe that there is an increase in the classification accuracy from 53.52% to 84.22% for the initial four feature combinations. Nevertheless, only minimal variation in classification accuracy can be observed for all other feature combinations. Owing to the observations made, one can infer that the maximum accuracy (89.36%) was obtained for 19 selected features. However, a good classification accuracy (87.81%) was obtained with 12 selected features. Additionally, one can observe a sudden fall in the time consumed for model building from Fig. 7 for 70 minimum number of objects (corresponding to 12 features selected). Thus, selecting 12 features is considered an optimal solution. Furthermore, reducing the number of features can help in minimizing the hardware requirements, complexity in model computation, cost and time.

6.2. Performance evaluation of base classifiers

The present study adopts three intelligent classifiers (SVM, kNN, LMT, RF, MLP, logistic regression (LR), J48 and Naïve Bayes (NB)) to perform classification on the features selected (using J48). Snail trail, glass breakage, delamination, discoloration, good panel and burn marks were the PV module conditions considered in the study. The PV module image dataset acquired from UAV was formulated as a uniform dataset consisting of 525 images from each PV module condition accounting for 3150 images in total. The penultimate layers of AlexNet were employed to extract image features that delivered 1000 image features for every image passed and were stored as a.csv data file. Post-feature extraction the most important and contributive J48 decision tree technique was used to choose the features. which selected 12 important features namely, features 51, 150, 359, 394, 582, 685, 770, 918, 927, 961, 963 and 972. The selected features were split into training-test split ratios of 70-30% before feeding them into machine learning classifiers. The training set was accommodated with 2208 images while the test set was filled with 942 images. Additionally, tenfold cross-validation was carried out to determine the effectiveness of the training process. Hyperparameter tuning was carried out to determine the optimal hyperparameters that enhance classifier performance. The individual performance of each classifier was assessed for training, validation and test accuracies. Table 6 depicts the list of hyperparameters adopted for each classifier. The performance of adopted classifiers for training, validation and testing accuracies are illustrated in Table 7. From Table 7, one can infer that SVM achieves a maximum classification accuracy of 98.51% followed by kNN, LMT, RF and MLP with 98.30%, 98.30%, 98.08% and 97.23% respectively. The confusion matrices corresponding to the top five performing classifiers are presented in Fig. 9(a)–9(e).

Table 6.

Optimal hyperparameter selection for base classifiers.

S.No	Base Classifiers	Hyper-parameters (Optimal)
1	SVM	Calibrator – Logistic Regression, Epsilon – 1E-12, Kernel - Poly
2	kNN	kNN = 1, Distance function – Euclidean distance, Search Algorithm – Linear NN search, Distance Weighting – No
3	LMT	Minimum no. of Instances – 400, No. of boosting iterations – −1
4	RF	MinNum – 1.1, Minimum Variance Proportion – 0.001, Seed – 1, K Value – 0,
5	MLP	Hidden layers – a, Momentum – 0.2, Learning rate – 0.3
6	LR	Max iterations – −1, Ridge – 1E-8
7	J48	Minimum no. of objects – 2, Confidence factor – 0.25
8	NB	No hyperparameter tuning required

Table 7.

Performance of intelligent base classifiers.

S.No	Classifier	Training Accuracy (%)	Validation Accuracy (%)	Test Accuracy (%)
1	SVM	99.36	98.50	98.51
2	kNN	100.00	98.68	98.30
3	LMT	100.00	97.37	98.30
4	RF	100.00	98.09	98.08
5	MLP	99.27	97.41	97.23
6	LR	95.74	95.19	95.22
7	J48	98.95	94.61	94.90
8	NB	91.50	91.16	91.93

Fig. 9.

(a) Confusion matrix of SVM classifier

Fig. 9(b) Confusion matrix of kNN classifier

Fig. 9(c) Confusion matrix of LMT classifier

Fig. 9(d) Confusion matrix of RF classifier

Fig. 9(e) Confusion matrix of MLP classifier.

6.3. Performance evaluation of stacked ensemble models

Three types of stacked ensemble classification models namely, three class (SVM, kNN, LMT), four class (SVM, kNN, LMT, RF) and five class (SVM, kNN, LMT, RF, MLP) ensembles were proposed in the present study. Meta classification stacking strategy was applied to the selected features of the PV module image dataset. Several base classifiers like were applied and the performance of the ensemble models was evaluated. Table 8, Table 9, Table 10 present the performance of three-class, four-class and five-class ensemble models for various base classifiers. Enhancement of classification performance was the primary objective considered in the present study. Individual classifiers SVM, kNN, LMT, RF, MLP, LR, J48 and NB achieved good classification performance with SVM being the front-runner with 98.51%. At times, decisions made by individual classifiers can be enhanced by acquiring collective suggestions from other classifiers in the form of an ensemble. Hence, an attempt was made initially to combine multiple classifiers to upgrade the classification performance. An experiment was carried out using three categories of stacked ensembles with different meta-classifiers to determine the best combination that produced enhanced classification. Among various combinations, four class ensembles (SVM, kNN, LMT, RF) with RF as meta classifier combined to achieve a classification accuracy of 99.04%. On the other hand, five class ensembles (SVM, kNN, LMT, RF, MLP) with RF as a classifier also attained similar classification accuracy. Based on the experiments carried out and results obtained, one can suggest four class ensembles over five classes. The reason for selection is that five class ensembles even after the addition of another classifier produced the same results as achieved by four class ensembles. Also, the presence of an additional classifier elevates the computational complexity and dimensions of the proposed stacked ensemble. Hence, a class ensemble (SVM, kNN, LMT, RF) with RF as meta classifier is suggested for real-time PV module visual fault diagnosis.

Table 8.

Performance of three class ensemble models (SVM, kNN, LMT).

S.No	Meta Classifier	Training Accuracy (%)	Validation Accuracy (%)	Test Accuracy (%)
1	SVM	99.81	98.91	98.51
2	KNN	99.77	98.59	98.40
3	LMT	100.00	99.18	98.93
4	RF	99.81	98.77	98.83
5	J48	100.00	98.91	98.72

Table 9.

Performance of four class ensemble model (SVM, kNN, LMT, RF).

S.No	Meta Classifier	Training Accuracy (%)	Validation Accuracy (%)	Test Accuracy (%)
1	SVM	100.00	98.95	98.72
2	KNN	99.95	98.82	98.62
3	LMT	100.00	99.04	98.93
4	RF	100.00	98.91	99.04
5	J48	99.81	98.77	98.72

Table 10.

Performance of five class ensemble model (SVM, kNN, LMT, RF, MLP).

S.No	Meta Classifier	Training Accuracy (%)	Validation Accuracy (%)	Test Accuracy (%)
1	SVM	99.95	98.86	98.72
2	KNN	99.81	98.68	98.51
3	LMT	100.00	98.82	98.72
4	RF	100.00	98.82	99.04
5	J48	99.81	98.41	98.72

From Table 8, Table 9, Table 10, one can observe that four class and five class ensembles with RF as a classifier display uniform performance with a classification performance of 99.04%. Four class ensembles are suggested for fault diagnosis of PV modules owing to the dimensionality issues and computational complexity induced by an additional classifier in five class ensembles. Additionally, according to the findings from Table 9, Table 10, it can be inferred that the validation accuracy of four class ensembles was better than that of five class ensembles resulting in 98.91% and 98.82% respectively. The overall computational time involved in building four-class and five-class ensembles is observed as 33.21 s and 50.12 s respectively. Fig. 10 represents the confusion matrix of four class stacked ensemble with RF as meta classifier. Opting for four class ensembles can help in reducing the training time, computational complexity and hardware requirements.

Fig. 10.

Confusion matrix of four class stacked ensemble with RF meta classifier.

6.4. Performance evaluation of stacked ensemble models with state-of-the-art techniques

To portray the superiority of the proposed stacking-based ensemble models, a performance comparison with various state-of-the-art techniques was presented in Table 11. Based on the comparison in Table 11, one can suggest that stacking-based ensembles provide superior classification accuracy than the listed conventional methods. Additionally, the proposed method can be utilized to diagnose real-time faults in PV modules to detect visible faults.

Table 11.

Performance comparison of the stacked ensembles with state-of-the-art techniques.

Reference	Methodology	Classification Accuracy (%)
[44]	Custom CNN	79.06
[45]	Custom CNN	94.30
[46]	Custom CNN	95.07
[16]	Custom CNN	97.90
[47]	Pretrained CNN + Random Forest	98.25
[21]	Pretrained CNN + k-nearest neighbour	98.95
Proposed	Stacked Ensemble	99.04

7. Conclusion

In this study, several PV faults namely, burn marks, discoloration, delamination, glass breakage and snail trail were detected and classified on the faulty modules along with good panels with the aid of a stacking-based ensemble approach. The images of PV module conditions were acquired using UAV under laboratory conditions. The acquired images were augmented and pre-processed prior to being fed as input to pre-trained AlexNet. Feature extraction was performed using the pre-trained AlexNet with features from the penultimate layer (fc8) extracted and stored into a.csv file. Essential and contributing features were selected among the features extracted using the J48 decision tree algorithm. The selected features were fed into the individual classifiers like SVM, kNN, LMT, RF, MLP, LR, J48 and NB to assess the classification performance. Based on the results obtained the top five performing classifiers were selected. A stacking-based ensemble strategy was formulated using three categories of ensemble classes (three, four and five class ensembles) to enhance the classification accuracy. Various machine learning classifiers were used as meta classifiers (SVM, kNN, J48, RF, LMT) inside the stacking ensemble classifier. The experimentation was conducted to determine the best-performing stacking ensemble category and meta-classifier pair. The obtained results enumerate that four class stacked ensembles (SVM, kNN, LMT, RF) with RF as meta classifier displayed higher performance than three class ensembles with a classification accuracy of 99.04%. On the other hand, five class stacked ensembles with RF as meta-classifiers also returned a similar classification performance of 99.04%. However, considering the dimensionality expansion, computational complexity and time taken to build the model, a four-class stacked ensemble was preferred over five-class ensembles. Thus, adopting stacking-based ensemble techniques has enhanced the performance of base classifiers. As a future scope, various feature extraction strategies can be employed and classified using different ensemble strategies.

Data availability

Data will be made available upon request to the corresponding author.

Funding source

No funding source is involved in the present work.

CRediT authorship contribution statement

Naveen Venkatesh S: Writing – original draft, Visualization, Software, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Divya Sripada: Writing – review & editing, Validation, Supervision, Resources, Investigation, Formal analysis, Data curation, Conceptualization. Sugumaran V: Writing – review & editing, Validation, Supervision, Resources, Project administration, Methodology, Investigation, Funding acquisition, Formal analysis, Conceptualization. Mohammadreza Aghaei: Writing – review & editing, Validation, Supervision, Resources, Methodology, Investigation, Funding acquisition, Conceptualization.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Nomenclature

PV

Photovoltaic

kNN

K nearest neighbour

SVM

Support vector machine

LMT

Logistic model tree

MLP

Multilayer perceptron

RF

Random forest

LR

Logistic regression

GW

Giga watt

UAV

Unmanned aerial vehicle

CNN

Convolutional neural network

A

Ampere

kg

Kilogram

mm

Millimeter

V

Voltage

W

Watt

I_sc

Current

V_oc

Voltage

V_mpp

Maximum power point voltage

I_mpp

Maximum power point current

csv

Comma separated value

ILSVRC

ImageNet Large Scale Visual Recognition Challenge

ReLU

Rectified linear units