Handling consumer vulnerability in e-commerce product images using machine learning

Absract

Need

In recent years, secondhand products have received widespread attention, which has raised interest in them. The susceptibility issues that consumers encounter while buying online products in reference to the display images of the products are also not well researched.

Motivation

Retailers employ clever tactics such as ratings, product reviews, etc., to establish a strong position thereby boosting their sales and profits which may have an indirect impact on the consumer purchase that was not aware of that retailer's behavior. This has led to the novel method that has been suggested in this work to address these issues.

Proposed methodology

In this study, a handling method for reused product images based on user vulnerability in e-commerce websites has been developed. This method is called product image-based vulnerability detection (PIVD). The convolutional neural network is employed in three steps to identify the fraudulent dealer, enabling buyers to purchase goods with greater assurance and fewer damages.

Summary

This work is suggested to boost consumers' confidence in order to address the issues they encounter when buying secondhand goods. Both image processing and machine learning approaches are utilized to find vulnerabilities. On evaluation, the proposed method attains an F1 score of 2.3% higher than CNN for different filter sizes, 4% higher than CNN-LSTM when the learning rate is set to 0.008, and 6% higher than CNN when dropout is 0.5.

Secondhand goods, Machine learning, Convolutional neural network, Product image, Consumer vulnerability.

1. Introduction

Be it irregular, occasional, or irreversible, vulnerability is a problem influencing everyone at any stage as it will become erratic as the time evolves. A consumer who is vulnerable will experience damage as a result of their unique situation, especially when a business is not acting with reasonable standards of care. Vulnerable personal circumstances can also include, but are not limited to, long-term or short-term enduring health problems (physical or mental), emotional trauma or neglect, having a physical impairment, having trouble understanding English or having weak language skills, struggling with addiction, or taking care of someone in multiple situations.

The most common choice for businesses is an e-commerce website created by a technology company because it offers a wide range of online purchasing and selling options. In comparison to a standard website, which is typically used to verify and gather information, e-commerce website platforms enable the consumer to make purchases without visiting a physical store. Shopping on e-commerce websites has several benefits for customers because they offer a big selection of things at fantastic discounts. The consumers' need for the best website production firm for e-commerce will be a crucial step. In e-commerce, the marketing firm acts as the primary customer representative for the business.

Vendors and the sellers on E-commerce websites had to be very careful and pay attention to attract customers for their products displayed online thereby preventing the loss as the disruptive attitudes and the imbalances may impact the business greatly. The new customers will get know about the quality and the reliability of the products by exploring the feedbacks and ratings given by the customers for the products which will, in turn, increase the business sales. The volume and nature of online payment network protection attacks have increased in parallel with the massive expansion in electronic payments.

Common third-party modules used by websites, including shopping cart software, have been identified as vulnerable to these kinds of attacks. These growing web program vulnerabilities, SQL link insertion, and cross-site scripting are used by many attacks. Price manipulation, SQL injection, information disclosure, path disclosure, cross-site scripting, and buffer overflow are some of the various kinds of vulnerabilities. Secondhand product consumption has expanded as a result of online auction and trade platforms. Secondhand product consumption has been performed for decades and is defined as the reusability of old things without losing their principal function.

The possibility of secondhand online trading mainly lies in the possibility of extending the product's life span thereby ignoring extra environmental stresses due to the purchase of newer products. Some European countries had a long history of using second-hand products. The practice of purchasing secondhand or used products from the United Kingdom, for instance, is deeply rooted in the society. Focusing on the current financial crisis, individuals from several nations, including Spain and France, have successfully entered the second-hand market due to various economic causes.

There are conventional indications, such as socioeconomic class, that may discourage some groups from engaging in the purchase of used second-hand goods. However, the current state of the internet and its associated technologies has led to the creation of numerous new electronic gadgets that offer consumers the best options for shopping and making purchases. The market for used goods has been popularized by the use of social networks and smartphones by all socioeconomic groups in society, and as a result, demand for used second-hand products is growing daily. Thus, it is crucial to focus on the variables affecting the purchasing of used goods.

According to a survey (Richard Carufel, 2017), which asked U.S. consumers about their preferences when making purchases online, many respondents preferred the consistency of a company's online promotional images over other factors like social networking, branding, or product specification. Innovative imaging technologies have come into the spotlight as a result of the expanding market need for high-quality images and a variety of product shots. 47 percent of American internet users said that high-quality product photos are important when deciding whether or not to buy a particular brand.

The standards and image of a company are now represented by digital marketing, thus firms are required to provide the highest quality online images in order to move clients through the purchase process. This is because brands will give advertising imagery precedence over social network posts. One third, or 37.9%, of online shoppers in the United States said that outside factors have an influence on their decision. Another third, or 33.7 percent, however, thinks that it might have an impact. As a result, retailers and marketer will concentrate on that as well as enhance their online marketing branding, which has been shown to increase the additional sales.

Additionally, 50.5 percent of online shoppers in the United States like to check the product's photographic images (taken from all angles) before making a purchase. Consumers were used to measure the quantity of online promotional images as well as the accuracy of the images, according to the research. Even the decision of when to tap “purchase” by the customer may be significantly influenced by adding the product or services to a model in the product picture. The study's findings show that brands shouldn't rely solely on real-world photos, and website photography tends to increase the likelihood that customers will shop online.

However, there are no studies that specifically address the risks that customers take when making purchases based on online product photos. Hence based on the Machine learning method, the proposed system is developed to prevent high false-positive rates thereby improving the reliability of the consumers. The suggested system encourages consumers to buy goods or services with greater assurance and without feeling vulnerable.

The objectives of the proposed method are given below:

• •While e-commerce is still expanding, retailer competition for sales and revenue has also been growing significantly. As a result, most retailers today have adopted innovative tactics to take advantage of e-commerce websites and to strengthen their positions. sn addition to code modification, the risky technique also involves click farms, and false customer reviews and ratings. Retailers employ these smart tactics, along with others, to boost their earnings and capture more market share.
• •Consumers on e-commerce websites are extremely vulnerable. A significant portion of consumers are impacted by the character, quality, or risk profile of a product, which has grown increasingly complex. A method based on machine learning and image processing has been presented for accurately identifying product damage in order to get over these drawbacks.

The contributions of this work are:

• •To identify risk factors for consumers when buying things online by looking at the provided product photos.
• •A machine learning-based methodology called the Product image-based vulnerability detection (PIVD) approach is suggested to address reused product image-based consumer vulnerability. In this proposed method, Convolutional Neural Network (CNN) is also utilized.
• •Four alternative methods were applied to the displayed product pictures to determine the efficacy of the proposed (PIVD) method. The dealer's products and their vulnerability are also determined using this way.

2. Literature survey

The previous research on used goods, image analysis, machine learning, and emotion detection approaches is covered in this part.

2.1. Secondhand products

(Kim et al., 2017) discovered the ideal performance level to improve the second-hand product for every inspection and to address the anticipated maintenance costs throughout the product's life cycle from the perspective of the user. They derived an explicit method for calculating the routine maintenance costs incurred over the product's life cycle from the given cost structures. Additionally, they examined the method for determining the ideal stage of development when failure rates followed the Weibull distribution.

On the other hand, a cost model was presented by (N.Darghouth et al., 2015) to identify the reliability enhancement level for second-hand products that are acquired with a free warranty repair (FRW) form. For the purpose of enhancing the dealer's cost reliability, they examined the periodic PM. This model aids in determining whether or not the PM behavior used during the warranty cycle is advantageous from the perspective of the dealers. They consider choosing the best update standard while forgoing routine preventive maintenance (PM) during the warranty period in order to reduce the likelihood of early defects.

(Dr. V. Krishna Mohan et al., 2014) evaluated customer perception with a study of 50 people from various 5 local stores in and near Tochigi Prefecture, Japan. From the study, it was found that the buyers were interested in purchasing the used products, and also, pricing has been identified as fair, as the customers will not experience blotting or social guilt while buying those used products. This paper also explored the cause for consumer loyalty in those shops and noticed that this store shouldn't be wrong for antique merchandise and artifact shops.

Also (Valerie M Thomas et al., 2003), addressed secondhand business development and the competition for new items has been decreasing as there are surplus products available that can be sold on the market. As there is no steady availability of existing used goods, there will be rising competition amongst second-hand buyers to buy fresh items which will also boost the materials cost. Moreover, if second-hand sales reduce the supply of new goods, this will not be generally one-for-one. When the price of recycled items exceeds the purchasing of fresh merchandise, this will be a simple beneficial feature given by recycled merchandise vs. new ones (Hristova, 2019). investigated some of the major trends in the second-hand goods market, along with their causes and impacts over retail in the digital society.

In the presented project, a conventional fuzzy reasoning tool was employed (Ghosh et al., 2021). developed a fuzzy rule-based system for decision-making based on Perceived Environmental Knowledge, Perceived Environmental Attitude, and Green Purchase Behavior among consumers who are related to Eco-friendly products. Green purchase behaviour (GPB) perceived environmental knowledge (PEK), and perceived environmental attitude (PEA) were chosen as fuzzy input variables, each of which has a set of five language factors. Consumers who are “Eco friendly” or “Non-ecofriendly” were divided into two sets in the output.

2.2. Identification of fake reviews

Nave Bayes Classifier, Logistic Regression, and Support Vector Machines were the classifiers utilised in this research by (Kolli et al., 2015) to investigate and determine whether the review is truthful or untrue. To build a model to identify false reviews, they have used linguistic variables including the existence of unigrams, the frequency of unigrams, the presence of bigrams, the frequency of bigrams, and the length of the review. The Yelp Challenge Dataset has been used by this research to locate the phone reviews since it provides both linguistic and behavioural data that can be used to identify fraudulent reviews.

Also (Rodrigo et al., 2019), suggested a system for identifying the false feedback measured in the consumer electronics industry and for classifying the fake reviews in the consumer electronics domain, they have built a dataset for the classification of fake reviews in four various cities centered on scraping methods and defined a feature structure for the detection of fakes. For classifying reviews, they have also established a fake system depending on the suggested structure. They have obtained an F-Score of 82% on the classification function and as per the Friedman test; the Ada Boost classifier was seemed to be effective through statistical means.

Ranking systems that are frequently harmed by profile injection attacks or anomalous scores caused by collective suggestion processes were addressed by (Zhihai Yang et al., 2018). The primary issues attackers constantly running into are introducing malicious profiles that strongly score a particular item or injecting malicious profiles that tend to lower an item's popularity. Due to this, the vulnerable client encounters numerous issues when they believe fake ratings on e-commerce websites. In order to get around the challenging challenges of calculating similarity and extracting features, a method for spotting anomalous ratings or attacks has been created.

In addition (Yuhong Liu et al., 2017), incorporated a quintile regression model for analyzing the important variables in online consumer preferences and exposed the promotional impact on the sales outcomes of the goods. Such effects are measured not only by the ability of the intruder to exploit but also by the unique properties of the desired commodity and the self-exciting force of the sector. Motivated by these findings, a novel iterative rating attack was formulated, and its efficiency is also validated through the experiments. They also researched and listed the economic effect of various influencing factors on product sales/download.

2.3. Image analysis and emotion detection techniques

New neural network models that combine the conventional bag-of-words, word context, and user emotions have been proposed by (Petr Hajek et al., 2020). These models specifically use three sets of features to attempt to understand at the text level: (1) n-grams, (2) phrase embedding, and (3) various lexicon-based emotion signals. The misleading feedback has been divided into four categories using a high-dimensional classification. The proposed approach has been contrasted with the other methods to show the effectiveness of classification.

(Budhi et al., 2021) suggested a novel framework for measuring the ratings of online reviews using machine learning techniques. They found that a number of texts preprocessing methods, including negation word identification, word elongation correction, and part of speech lemmatization paired with Terms Frequency and N-gram words, can improve accuracy. Additionally, they showed how well the general emotion of lexicons like SenticNet 4 and SentiWordNet 3.0 can be leveraged to generate the features.

A novel method for discovering de-blocking that would dynamically acquire object demonstrations from a deep learning system was presented out by (Xianjin Liu et al., 2019). Prior to deblocking, hierarchical characteristics were analysed with a convolutional neural network (CNN) that was administered, and the best features were extracted by CNN by utilising the sliding window.

The main requirement for the development of e-commerce is the achievement of online automatic product categorisation. Jia et al. (2010) developed a quick supervised image classifier based on the class-specific Pyramid Histogram of Words (PHOW) descriptor and Image-to-Class distance (PHOW/I2C) by examining the characteristics of product photos. During the training phase, the local features are heavily sampled and represented as soft-voting PHOW descriptors. Following this, the means and variances of the distribution of each visual word from each labelled class are used to construct the class-specific descriptors.

The growth of used goods decreases the desire for new ones, as shown by the aforementioned works, yet the warranty period is discovered to have an impact on sales and how consumers were behaving while purchasing secondhand items. The market for second-hand products is very strong. Also, it examines the difficulties customers encounter when making purchases based on falsified reviews and how machine learning models were used to identify these reviews.

However, most attempts result in defects and untraceable outcomes (Karode and Werapun, 2021). The aforementioned papers also covered the application of machine learning techniques to sentiment and emotion analysis. In conclusion, none of the aforementioned research projects have concentrated on both image analysis of product photos and emotion recognition in product reviews. Since these works do not compare the same goods with other dealers and don't take into consideration factors like product quality, appearance, wear and tear, or quality, all of these factors were taken into account in the proposed model. These works have not accounted for the factors like quality, appearance, wear and tear of the product, and also, this work doesn't compare the same product with other dealers, and hence, all these factors were accounted for in the proposed model.

3. Methodology

3.1. Factors contributing to consumer vulnerability

The consumers are getting cheated and disappointed just by trusting the product images that are displayed on the E-commerce website and ordering online. Following are the consequences that a customer will face if they order a product just by trusting the image displayed alone.

• 1)Quality and appearance of Product

The quality of the product is of utmost important while purchasing a product from online Ecommerce websites as it is the only thing that helps in retaining the satisfaction and loyalty of the customers. This will greatly reduce the risks associated thereby eliminating the cost of replacing the damaged products. Today, if a consumer is not satisfied with the cost and quality of the product anytime, they are buying from an Ecommerce website, they will definitely move on to some other competitors to buy the same product.

Hence, quality is what that matters everything and it must be the sole commitment made by the sellers to the consumers and those kind of quality products are said to commonly called as the premium products. On the other hand, there is a chance that the product will malfunction after being purchased because the original quality of the product cannot be determined from the product pictures that are posted on e-commerce websites. The product might not fulfill the standard as per the description viewed online. The Product size, colour, and material type are all examples of the Quality indicated here.

It is not possible for the vulnerable consumers to identify the size of the property with the displayed images alone as the appearances and sizes of the image may get varied depending upon the position or the angle of the image that is taken. Also, the product color may also get varied from the original displayed image due to reasons like the lighting conditions at which they are being captured with high resolution. As a result, people rely on the product's external look to judge its quality.

• 2)Physical damage

Additionally, e-commerce companies are being accused of selling defective and damaged goods, which accounts for the fact that more than 20% of delivered goods are returned because of physical defects. Simultaneously, few buyers will purchase things online that they cannot return because they would have done so had the sellers of e-commerce websites offered them a discount or free shipping instead (Saleh, 2018). This is taking place because customers were purchasing the product only on the basis of the product's displayed photos, despite the fact that any physical defects in the product may be hidden or not apparent. There is a possibility of a financial risk involved when the service does not meet the acceptable quality based on the vendor's online records.

• 3)Reimbursed products

It will be challenging to fight the urge to buy a premium quality item at an expensive “throwaway” price because some retailers or dealers give large discounts for reimbursement products. Although e-commerce businesses may have lower administrative costs than conventional retailers, this market difference wouldn't be significant when it comes to marketed items.

For branded or premium goods, there will typically be a standard discount accessible. If the customers are receiving greater discounts than these, the product needs to be taken into account as suspicious product. Reimbursed products cannot be identified solely by looking at the product photos that are provided, and as there are few opportunities to test or physically examine the product before making an online purchase, customers must rely on the website's minimal information of the item.

• 4)Increasing natural quality of the image

In general, the quality of the product picture matters more than the actual quality of the goods because a nice image will influence customers' decisions about whether or not to purchase the product from an e-commerce website. Customers may leave the website and stop buying things if the product's image is of poor quality. As a result, vendors must be careful to provide high-quality resolution pictures that enhance the natural quality of the product image. However, in practice, sellers weren't paying attention to the quality of the product as they advertised or presented.

• 5)Mismatched product from the display

The product should be exactly as agreed upon or ordered on the product image presented, however occasionally customers have reported receiving a different product than what they had seen when completing the order. Customers can get the faulty goods or the wrong color instead of what they requested, which makes them unhappy with the sellers or vendors on e-commerce websites. They begin to doubt online shopping as a result of this.

• 6)Wear of Product

The damage that occurs naturally and eventually as a result of frequent use or ageing has been known as wear and tear of the material. It is impossible to determine how much a product has been used from the images that are provided, making it uncertain as to how long the goods will remain in good condition.

• 7)Hiding sensitive information of the product

However, the dealers were hiding the essential information of the displayed product images from customers. Every image provides crucial information about the goods, and high-quality photographs can enhance customer interest in and trust in the product as well as conversion rates. Consumers may be misled into purchasing a fake product when they purchase goods online because they neglect to verify such details while making purchases or placing orders.

• 8)Additional or Extra things with the product

The goods and other products could be included in the displayed product image. This might trick customers into purchasing things. Therefore, the consumers expecting the product to look exactly as it does on the display may lead to disappointment.

3.2. Handling product images-based vulnerability using 4 ways

An important part of deciding whether the products will get success or not during the test is to classify the defects. There are 4 different ways available for analyzing and finding the factors that pay way to consumer vulnerability and these ways are very useful for the consumers with the vulnerability as it helps them buying the products b checking and comparing it with the others right before ordering that product online. This would ensure the consumer with trust and confidence in buying the products online and the four ways for evaluating the product images are:

• •Identifying incorrectly uploaded dealer/retailer images of damaged goods.
• •Determining how long a product has been in use by comparing the displayed product images with those from other retailers or the related brand company.
• •Reviewing consumer feedback comments after making a purchase.
• •Analyzing past purchases or transactions with the same dealers.

The aforementioned methods can be carried out manually, however manual inspection may frequently result in errors, making it ineffective and challenging to maintain at all times. As a result, a machine learning-based model can be utilized to solve problems more effectively through the process of continuous learning.

• (i)Proposed Technique

Images are the first thing that shoppers look at when browsing a product listing on the web to decide whether or not the product will seem as it is described. While many shoppers won't notice the percentage of faults in the exhibited second-hand products, the product images will assist them analyze the product's consistency and information better than any definition. Therefore, an intelligent damage detection tool that automatically highlights the level of damage in product images while preventing fraud and errors is essential for a quicker evaluation and greater level of customer satisfaction. Such a process is still difficult, nevertheless, for a number of reasons.

The existing software tools (Francois, 2021) available are amazon recognition, Talkwalker, etc. and these tools will not consider the factors like step-by-step analysis, both text, and image-based analysis, and the supervised learning methods thus, resulting in lesser accuracy during image analysis. Hence, a novel technique considering all these factors a “Machine learning-based Product Fault Identification (ML-PFI)” has been proposed here in this work. The images are classified into two types as damaged and not damaged by the detection process executed by the proposed method. Then, by comparing it with the other dealers, the product usage time will be found the proposed method. Then, by using either the consumer reviews or their history, the behavior of dealers can be analyzed as shown in Figure 1.

• [1]Detecting damage in product image based on image analysis

Figure 1.

The workflow of proposed work.

(i) ML-PFI technique-based damage detection

Sending the updated and unaffected pixels to the model actually requires user interaction. As shown in Figure 2, the suggested model employs Convolutional Neural Networks (CNN) (Yan Wang, 2017) to categorize the collected features as well as extract features from dissimilarity images.

Figure 2.

Architecture of ML-PFI technique.

➢ Obtaining Dissimilarity image

At first, the comparison is done between the image 1 which is the real image of the product that is the brand product image and the image 2 which are the images that are being displayed in the E-commerce websites. A vector of the function that is largely focused on the picture of dissimilarity has been used to represent each pixel. These vectors which are referred to as the data will be used to train and test a classifier model to produce a map of binary change.

For each pixel to be classified as modified or unmodified, numerical features must be identified, and the initial step is to assess both the pixel's own grey level and the grey levels of its surrounding neighbours. For a square neighbourhood with a dimension of N x N, each pixel can be represented by a vector of n = N²gray-levels. The basic variance between the two detected grey levels will be further replaced throughout the picture differentiation procedure by a calculation between two gray-level vectors.

Various comparisons and measurements of dissimilarity between two gray-level vectors are proposed, especially for comparing pixels in binocular stereo vision applications. The simple sum of the measure of squared differences is being used as the square of the Euclidean distance and is described using the following Eq. (1):

$$\text{SSD}\left(\text{x},\text{y}\right)={\left|\text{f}1-\text{f}2\right|}^2$$ (1)

In the above equation, f1 and f2 denotes the vectors of the gray level of pixels in the N × N window placed at pixel (x, y) in the first and second data image, correspondingly. Eq. (1) is derived from Yan Wang (2017). Since the two neighborhoods are varying, the SSD calculation of dissimilarity provides the large values and the dissimilarity of the picture will be defined as the matrix of all SSD values which is linearly converted to a grayscale level image. After obtaining dissimilarity images, features are extracted using the CNN technique (Tao Li et al., 2019).

➢ Feature Extraction using CNN

Deep CNN is becoming a well-liked deep learning method. The convolution procedure will look through the entire sample's data using several convolution kernels that reflect the relevant characteristics. This ensures that the CNNs have the necessary tolerance for noise and shifting. Based on the advantages of CNN, product photos are categorized using a deep CNN model.

Figure 3 shows the five layers that have been utilizes in the CNN block. The five layers shown are the input layer, the output layer, the max pooling layer, the fully connected layer, and the convolution layer. By considering the target pixel index as (x, y), the classification model integrates the knowledge coming from the target pixel as well as the pixels that accompany it.

Figure 3.

Layers in CNN.

A multi-layered neural network called CNN has many convolution layers and levels of pooling. The weights of the rectangular patches (filters) that make up the first convolution layer are discovered during the training stages. These rectangular patches (filters) are smaller than the source pictures. The feature maps will be extracted from the input photos using this layer and these filters or kernels, which will be used to extract information from the images at very low rates.

The second layer employed by CNN is the pooling layer, which reduces the spatial dimension of the image at each pooling layer by activating some activation function over a rectangular frame, such as peak or average over a rectangular region. This lowers the number of parameters that must be calculated, which lowers calculations at layers.

When compared with the other traditional neural networks, a CNN model will be found to provide several fully connected layers. Those layers are fully connected to the previous layer activations where the fully connected layers have been occupied by the output layer. In this output layer, the neurons count in the dataset is equivalent to the number of object classes in the dataset. Eqs. (2), (3), (4), (5), (6), (7), (8), (9), (10), (11), (12), (13), (14), (15), (16), and (17) are derived using (Li et al., 2019; Batbaatar et al., 2019; Jaljouli et al., 2018).

➢ Deep CNN (DCNN model)

To identify the pictures that are damaged, the features extraction capability and CNN's classification efficiency has been efficiently integrated by the DCNN model where a fully connected layer will generate the feature vector in which the dimension is proportional to the neurons count in this layer. For the pixel (X, y), X includes all the target pixel data along with various neighbor pixels, and y denotes the ground truth label. Following the convolution layer and then the max-pooling layer, the fully connected layer F4 enters a vector F_x and then this vector will be loaded into the final output layer for measuring the y_o label.

An error value has been determined depending on y_o and y and this is backpropagated over the complete network. Also, by utilizing the gradient descent algorithm, the entire weights will be altered and while training the cube CNN, it is possible to feed all samples to the network and the output may be arranged to its associated output F4 layer and its label y into a training dataset, from which the CNN could be trained effectively. Thus, the cubic samples are used by the CNN model for deriving the high representative classification features.

➢ Training procedure of DCNN model

Under the assumption of a pixel, a feature vector obtained by cube CNN has been employed. For the extraction of feature vector, CNN will be trained using the training data and the Cross-Entropy is employed throughout the cube CNN model as the loss function and it is described in the following Eq. (2):

$$E=-\frac{1}{n}\sum _{i=1}^n\left[y^i\log y_0^i+\left(1-y^i\right)\log \left(1-y_0^i\right)\right]$$ (2)

Where $y_0^i$ represents the output signal of the i^th neuron in the output layer. Also, $y^i$ denotes the corresponding target signal. Following that, component contributions to the final E were propagated backwards, allowing every weight value in the network to be adjusted in accordance with E. The component contribution of each weight “w” on the final E might be seen by looking at the partial derivative of E and the weight value written as w. Over a specific task, multiple training rounds will be given for the cube CNN depending upon the gradient descent and the back propagation and once the training is completed, the cube CNN will feed the entire set of texting dataset into the network.

Then for each observation, the outputs of all the neurons are collected which will be then combined into an array that reflects the features derived from the respective raw sample results. A new sample was produced with its label y. However, a training dataset has been created after passing all training data set samples into the network. For classification of the retrieved features, the CNN classifier is trained using the training dataset and the DCNN model uses deep CNN to extract features from the raw data.

A fully connected layer is utilized to prepare for feature extraction before the classifier uses high representative features to classify each pixel. This classifier will be fed into the trained cube CNN to classify new samples. Further data can be put into the CNN classifier using these derived characteristics. The final CNN output is a binary change map, which shows a class of changed and unaltered pixels in the product image.

(ii) ML-PFI based suspicious product dealer detection

Based on ongoing learning and monitoring by the suggested ML-PFI technique, the proportion of damage in the displayed product images assists in identifying the suspect dealer. This will make it easier to compare the product images displayed with other similar images offered by other vendors. The suggested ML-PFI technique as stated in B (I) section can be applied similarly to accomplish this. The target image in this case will be image 1, and the other dealer's product images will be image 2.

When compared to other product images, this machine learning-based technique offers a percentage of damages in the displayed product images. This will direct customers to buy products with less damage. Here, the model will take into account factors like the Product's cost and the description provided by the dealers before comparing it to other dealers. These parameters will be fed into the model during the training phase.

In addition to the physical damage specified in the product description, damages resulting from wear and tear of the product over its usage period should be taken into consideration. The model can readily find the wear and tear (damages) of the new product within the allotted time with continuous learning. A new product, for instance, will experience minor usage damage over the allotted time.

In addition to nominal losses and damages mentioned in the description, the machine learning-based model will take other damages into account. This allows customers to purchase a product for a reasonable price depending on these damages. Customers can buy this product with trust and confidence if damages and the cost of the product are discovered to be minimal when compared to those of other dealers.

Otherwise, that specific dealer and his products should be noted for monitoring if there is a significant price differential between similar products sold by various dealers. A dealer and the products are identified as suspect or vulnerable if the same dealer is consistently discovered to be malicious during comparison. Next, the customer reviews and the dealer's history (sales made on earlier products) will be examined.

• [2]Detecting dealer's behavior based on text analysis

The reviews have a significant impact on whether a product sells more or less. Sales are increased by positive reviews while decreased by negative ones. The consumer reviews contain information about the dealer's behavior. Therefore, determining whether reviews are genuine or fake and reviewing history is crucial for figuring out how a dealer and their products behave. To find the semantic emotions of the consumer, the convolutional neural network is used here.

Not only are the reviews considered by the proposed model as it also considers the history of the dealer to check whether the product has been sold genuinely or not. From the reviews found, an analysis will be conducted on the emotions expressed by the consumers both the positive and negative reviews using the CNN and long short-term memory. For the same text input, semantic and emotional word embedding are implemented separately in two sub-networks.

CNN will be used to extract the emotion specifics, whilst BiLSTM will gather the contextual data. Then, BiLSTM and CNN are employed to encode semantic as well as emotional signals. Then, using combined semantic and emotion phrase encoding, final representations are integrated to identify more emotion.

Figure 4 shows the framework of the model for text emotion detection which is comprised of two sub networks: one is BiLSTM network for word semantic encoder and the second one is CNN network for emotion encoder. For identifying the emotions from the text, the output from the sub-networks has been utilized. The identical N-word sequence was used to load both sub-networks, and each word will be transformed into a d-dimensional word vector.

Figure 4.

Architecture for text analysis using CNN (Batbaatar et al., 2019).

The embedding layer is an essential part of neural networks as it enables the conversion of each word into a fixed-length vector of defined size. The resultant vector is a dense one having real values instead of just 0's and 1's. The fixed length of word vectors helps us to represent words in a better way along with reduced dimensions and this layer is initialized with random weights and will learn the embedding for all the words in the training dataset. Word2Vec is one of the most popular techniques to learn word embedding from the network (Saxena, 2021). Within that result, the sequence representation is encoded by the word embedding layer as $Z_{emo}$ (emotion word embedding in Eq. (3)) and ${\text{Z}}_{\text{sem}}$ (semantic word embedding in Eq. (4)) matrices as,

$${\text{Z}}_{\text{emo}}=\left[{\text{W}}_1^{\text{emo}}\dots \dots {\text{W}}_{\text{t}}^{\text{emo}}\dots \mathrm{..}{\text{W}}_{\text{N}}^{\text{emo}}\right]\in {\text{R}}^{\text{N}\times \text{d }}$$ (3)

$${\text{Z}}_{\text{sem}}=\left[{\text{W}}_1^{\text{sem}}\dots \dots {\text{W}}_{\text{t}}^{\text{sem}}\dots \mathrm{..}{\text{W}}_{\text{N}}^{\text{sem}}\right]\in {\text{R}}^{\text{N}\times \text{d }}$$ (4)

Whereas ${\text{W}}_{\text{t}}^{\text{e}}$ and ${\text{W}}_{\text{t}}^{\text{s}}$ denotes the emotion and semantic word vectors of word ${\text{W}}_{\text{t}}$ in the sequence correspondingly.

➢CNN network for emotion encoder

For extracting the emotion features from the emotion-based word embedding, CNN has been used in which the features will be extracted through these layers together with the convolution layers in CNN. The structure of CNN is illustrated in Figure 5. The CNN input can be denoted as ${\text{the Z}}_{\text{emo}}$ matrix. Also, emotion word vectors, $w_t^{emo}$ is loaded into CNN. Moreover, the word embedding vectors will be combined as the feature vector v of the sequence as shown in the following Eq. (5):

$$\text{v}={\text{w}}_1^{\text{emo}}\oplus {\text{w}}_{\text{t}}^{\text{emo}}\dots \mathrm{..}{\text{w}}_{\text{N}}^{\text{emo}}$$ (5)

Whereas $\oplus$ represents the concatenation operator of vectors.

Figure 5.

CNN structure (Batbaatar et al., 2019).

A local emotion feature vector xi was constructed by measuring convolution in the first convolution layer using numerous filters with varying window sizes s as well as for each possible word window size. Every convolution operation in a word window ‘s’ forms a new local context feature vector $x_i^s$. For every potential word window in the input sequence, a local feature mapping vector is produced by the convolution filter. Along with this vector, the convolutional process is accompanied which outputs a new vector and it is given in Eq. (6),

$${\text{x}}^{\text{s}}={\text{x}}_1^{\text{s}}\dots ,{\text{x}}_{\text{i }}^{\text{s}}\dots ,{\text{ x}}_{\text{N}-\text{s}+1}^{\text{s}}$$ (6)

Following the convolution operation, the max-pooling operation will be carried out on the new feature vector ${\text{x}}_{\text{i }}^{\text{s}}$ and this vector is produced with the help of the convolution layer. The vector x is mapped by the max-pooling layer to a fixed-length vector using Eq. (7). The number of hidden units in the convolution layer is represented by the vector's length, which is referred to as a hyperparameter and is chosen by the user. By selecting the top number of features by the max-pooling layer, the most important information about the emotion feature will be preserved. Each word's context information and word order were taken into consideration during the pooling process.

$$x_{max}^s=\text{max}\{{\text{x}}_1^{\text{s}}\dots ,{\text{x}}_{\text{i }}^{\text{s}}\dots ,{\text{x}}_{\text{N}-\text{s}+1}^{\text{s}}\}$$ (7)

After the pooling operation, there has been a vector as there are multiple feature maps. All vectors generated by the max-pooling layer are combined to a single feature vector ${\text{h}}_{\text{emo}.}$ using following Eq. (8):

$${\text{h}}_{\text{emo }}=\left[x_{max}^s\right],s\in \left[s_{min},s_{max}\right]$$ (8)

At last, the emotion sequence vector ${\text{h}}_{\text{emo }}$ will be loaded to the hidden layer.

$${\text{g}}_{\text{emo}}=\text{f}\left({\text{w}}_{\text{emo}}{\text{h}}_{\text{emo }}+{\text{b}}_{\text{emo}}\right)$$ (9)

Whereas from Eq. (9), ${\text{g}}_{\text{emo}}$ represents the emotion encoder result and ${\text{w}}_{\text{emo}}$ and $b_{\text{emo}}$ represent the parameters of the non-linear f activation function.

➢BILSTM Network for Semantic Encoder

The bidirectional LSTM is used for extracting the hidden state of every word through outlining the details both in forward and backward directions which helps the model of semantic text information, and it is seen in Figure 6. The LSTM input can be denoted as ${\text{Z}}_{\text{sem}}$ matrix and the Semantic word vectors, $w_t^{sem}$ is loaded into LSTM.

Figure 6.

BiLSTM structure (Batbaatar et al., 2019).

Forward LSTM and backward LSTM were referred to as $\overrightarrow{\text{LSTM}}$ and $\overleftarrow{\text{ LTM}}$. While $\overrightarrow{\text{LSTM}}$ reads words from left to right, and $\overleftarrow{\text{LSTM }}$ reads words in reverse.

$$\overrightarrow{{\text{h}}_{\text{t}}}=\overrightarrow{\text{LSTM }}(w_t^{sem},\overrightarrow{{\text{h}}_{\text{t}-1}}),\text{t}\in \left[1,\text{N}\right]$$ (10)

$$\overleftarrow{{\text{h}}_{\text{t}}}=\overrightarrow{\text{LSTM}}\left(w_t^{sem},\overleftarrow{{\text{h}}_{\text{t}+1}}\right),t\in \left[1,\text{N}\right]$$ (11)

To represent every $w_t^{sem}$, forward hidden state $\overrightarrow{{\text{h}}_{\text{t}}}$ and reverse hidden state $\overleftarrow{{\text{h}}_{\text{t}}}$ are concatenated which are calculated using Eqs. (10) and (11). Then from the final hidden state of the encoded ${\text{h}}_{\text{sem}}$ semantic sequence shown in Eq. (12), the vector has been loaded into the hidden layer.

$${\text{h}}_{\text{sem}}={\text{h}}_{\text{N }}$$ (12)

$$g_{sem}=f\left({\text{w}}_{\text{sem}}{\text{h}}_{\text{sem }}+b_{\text{sem}}\right)$$ (13)

Whereas in Eq. (13), $g_{sem}$ denotes semantic encoder output, ${\text{w}}_{\text{sem}}$ and $b_{\text{sem}}$ represent the activation function (f) parameters.

➢Emotion Recognition

After the creation of emotion encoding ${\text{g}}_{\text{emo}}$ and semantic encoding $g_{sem}$, vectors c are eventually concatenated and loaded into the feed-forward layer using Eq. (14):

$$c=\left[{\text{g}}_{\text{emo}},g_{sem}\right]$$ (14)

$$o=f\left(w_{0}c+b_0\right)$$ (15)

Whereas in equation (15), f denotes the feed-forward layer, $w_0$ represents weight, and $b_0$ represents weight parameters correspondingly. ‘o' denotes the feed-forward layer output. The output from the final step has been taken by Softmax classifier and acts as an input o. For every sequence, the emotion y is predicted by considering the sequence with N-words. The sequence emotion annotations are represented as. $\text{Y }\left(\text{Y}=Y_1,Y_2,\dots .Y_M\right).$

The predicted values $y^{\prime }$ may be computed as using Eqs. (16) and (17):

$$p\left(y|X\right)=softmax\left(w_{p}o+b_p\right)$$ (16)

$$y^{\prime }=\arg maxp\left(y|X\right)$$ (17)

Whereas p denotes the predicted probability of emotion label, $w_p$ and $b_p$ represents the classification layer parameters. The above-mentioned four steps are very useful in detecting issues faced by consumers before purchasing products.

4. Experimental analysis

4.1. Experimental environment

The dataset is taken from the OLX website for analyzing the proposed method with the dataset containing 300. The experiment is performed on Intel Core i7 3.5 GHz processor machine with NVIDIA GPU enabled 4 GB RAM and MATLAB programming platform. The dataset is split into two groups with 80% training data and 20% testing data. For the representation of text in deep learning models, billion 300-dimension word vectors are used. The varying window size is used. The hyper parameters of the model are listed in Table 1.

Table 1.

Hyper parameter setting.

Parameter	CNN	BiLSTM
Learning rate	0.008	0.008
Batch size	128	128
Number of filters	100	100
Filter size	[4,5,6]	[4,5,6]
Dropout	0.5	0.5

4.2. Evaluation of the model with sample images

As mentioned in the above section, the image of the product taken from the specific dealer is analyzed in three steps and these steps are then analyzed with the multiple dealers. From the analysis with multiple dealers, it can be determined whether the product provided by the particular dealer is genuine and safe or not. First, the OLX image of the used product is compared with the reference image, which is the image of a brand-new item.

The percentage of damage was then examined using the proposed methodology and compared with a brand-new product image. The cost and description of the specific dealer product are then taken into account and compared to those of other dealers in the following stage. Finally, after taking into account the history of that particular dealer and any evaluations or ratings offered by customers, the dealer's behavior will be discovered.

• •Detecting/Spotting damages in the displayed product images

A reference image and a second-hand product image provided by the dealer from the OLX website are considered as shown in Plate 1.

Plate 1.

(A) Reference image (B) Phone image in OLX.

To find damages in Plate1 (B), the above RGB images will be converted into grayscale images initially as I = rgb2gray (RGB). This will simplify the transformation of the true-color image RGB into the grayscale image I. The 8 ∗ 3 = 24 bits are required for storing a single colour pixel of an RGB picture (8 bits for each colour component). Storage space will be reduced when an RGB image is converted to a grayscale image since each grayscale pixel requires 8 bits to be stored.

By eliminating the hue and saturation information and keeping the luminance, the RGB2gray function in MATLAB converts RGB images into grayscale images. Plate 2 represents the grayscale image of a brand-new product and phone product in OLX.

Plate 2.

(A) Reference image (B) Phone image in OLX.

Plate 3 shows the binary image of a brand-new product and phone product in OLX. The image in grayscale has been transformed to a threshold-based binary image and by transforming grayscale image into a binary image; image processing will be very simple. During the threshold period, the threshold value will be determined. Then, all gray level values that are less than the set threshold value will be classified as 0 (background as black), and all gray level values that are equal to or greater than the determined threshold value will be classified as 1 (foreground as white).

Plate 3.

(A) Reference image (B) Phone image in OLX.

The binary image of a brand-new product and phone product in OLX is shown in Plate 3.

$$\text{g}1\left(\text{x},\text{y}\right)=\{\begin{array}{l}1,f1\left(\text{x},\text{y}\right)\ge T\hfill \\ 0,otherwise\hfill \end{array}$$

Whereas threshold image pixel at (x, y) is indicated by g1 (x, y). The greyscale image pixel at (x, y) is indicated by f1 (x, y). After acquiring these binary images, the suggested technique makes it simple to identify any differences between them. CNN uses the resulting dissimilarity image to extract features after that. The image was altered in the CNN input layer to produce a feature vector in the CNN output layer.

After extracting the features, the CNN classifier will partition the image's pixels into changed and unchanged category. The final output from the CNN classifier is shown in Plate 4 in which the image will be representing the changed pixels in the product image and this will help to identify the percentage of the damages in the product.

Plate 4.

Final output from CNN classifier.

• Analyzing cost and description of the Product

After identifying the damages in the image, the suggested model would examine the product's price and description, comparing it to the prices of similar products from other dealers to determine whether or not the price is reasonable in the perspective of damage level percentage. If the cost difference between the dealers for the same product is too great, the dealer for that product is observed and watched.

Also, the dealer doesn't specify the damage in the place of the fingerprint sensor side in the description. This suggests that the vendor is attempting to try and hide information regarding sensor damage. Plate 5 depicts the product image's back side, which is covered by a phone case. It gives the Product a negative impression.

Plate 5.

Backside of the product image.

• Analyzing the reviews and history of the Product

Last but not least, by taking into account the information about the customer reviews offered, the ratings or reviews provided by customers were taken into consideration together with the history of that particular dealer. By stating that this Product has 4GB RAM memory in the description, the dealer is attempting to cheat the customers. However, the Moto Turbo (XT 1225) product does not have 4GB of RAM. For this model, there is just 3GB of RAM available. This false information is discovered by the proposed approach from the reviews.

The dealer's past behavior is checked using the proposed model, which allows for an efficient analysis of both the product's damages and the dealer's real behavior in the previous phases. This will help customers buying the safe second-hand products with more confidence and without the help of machine learning technology; this would be more difficult to predict and avoid these steps easily.

As the machine learning methodology is having the ability to process all the difficult and redundant data very easily, this machine learning methodology is utilized in this proposed method to safeguard against all the fraudulent activities. When compared to the manual research, the proposed work shows that machine learning can effectively identify the percentage of faults in used second-hand products.

5. Result and discussion

To evaluate the performance of the proposed model, the following parameters are used and they are:

• ➢Number of filters in CNN
• ➢Impact of Filter Sizes
• ➢Learning rate
• ➢Dropout
• 1)The link between the number of filters in CNN and the F1 score

The effect of using the CNN filters while implementing different types of methods like CNN, CNN with LSTM, and the proposed technique is shown in Table 2 in which the hidden layer size was chosen between the ranges 100–500.

Table 2.

Relationship between several filters in CNN and F1 score.

No. of filters in CNN	F1 SCORE
	CNN	CNN-LSTM	Proposed method
100	73.8	75.5	74.4
200	71	75.7	74.9
300	73.9	74.3	75
400	74	74.8	75
500	73.9	74.5	75.1

The relationship between the number of filters and the F1 score is shown in Figure 7 and from the results, it is found that when the number of filters gets increased, the proposed model will give the best results. When compared with the other models, the hybrid CNN-LSTM will deliver the best results if the numbers of filters are selected within the range of 100–200. In this proposed work, the numbers of filters are set default to 100.

• 2)Impact of filter sizes in CNN

Figure 7.

Comparison between the number of filters in CNN and F1 score.

The effect of using CNN filters sizes in various methods such as CNN, CNN with LSTM, and the proposed technique is shown in Table 3.

Table 3.

Relationship between filter sizes in CNN and F1 score.

Filter size in CNN	F1 SCORE
	CNN	CNN-LSTM	Proposed method
[1,2,3]	71.2	72.8	74.6
[2,3,4]	73.2	74.2	75
[3,4,5]	72.8	73.8	75.1
[4,5,6]	73	74	75.3
[5,6,7]	72.2	72.8	74.8

The proposed method is found to achieve a good F1 score of 75.3% when a filter size is set to 4, 5, and 6 as shown in Figure 8. Also, various size of filters used for testing are [1,2,3] [2,3,4], [3,4,5] [4,5,6] and [5,6,7]. The best result is obtained for a proposed method.

• 3)Learning rate

Figure 8.

Relationship between filter sizes in CNN and F1 score.

The impact of learning rate in different methods like CNN, LSTM with CNN, and the proposed technique is shown in Table 4.

Table 4.

Relationship between learning rate and F1 score.

Learning rate	F1 SCORE
	CNN	LSTM with CNN	Proposed method
0.002	72	71	72
0.004	75	73	74
0.006	68	74	71
0.008	74	71	75
0.01	73	70	70

The learning rate must be significant in order to optimise the offsets and weights. Although the rate of learning appears to be high, the system becomes unstable since it could exceed the maximum. If the learning rate is low, training will take more time and the relationship between learning rate and F1 score is shown in Figure 9.

Figure 9.

Relationship between learning rate and F1 score.

In the proposed model, the high results will be attained for different learning rates. When compared to other models, the CNN model yields the best results while the learning rate drops to 0.01. The learning rate is set between 0.002 and 0.01 to test the learning rate effect. High learning rates can end up causing the model to lack stability. It could take longer to train the model when the learning rate value is low. As a result, in this work, the learning rate is set to 0.008 to achieve the optimal performance of the model.

• 4)Dropout

Table 5 displays how learning rate impacts several methods, including CNN, LSTM, and the proposed method. The dropout method is employed during the training phase to prevent over-fitting issues. The results are outstanding in the proposed work when the dropout value is modified.

Table 5.

Relationship between dropout and F1 score.

Dropout	F1- score
	CNN	CNN with LSTM	Proposed
0	74	72	74
0.2	73	74	75
0.4	71	77	78
0.6	70	71	77
0.8	69	70	74

The comparison of dropout and F1 scores for the proposed and other existing techniques is shown in Figure 10. The models are evaluated using a range of dropouts between 0.0 and 1.0. Every sub-network has two dropouts, one of which will be at the end of the CNN emotion encoder and the other at the conclusion of the BiLSTM semantic encoder. The proposed work performs well when compared to the other models and the dropout value of this work is set to be 0.5.

Figure 10.

Comparison between dropout and F1 score.

5.1. Findings

The findings of our research work are:

• •Consumers of E-commerce websites are vulnerable to fraudulent activities and vulnerabilities. From the proposed method, it is very simple to detect the vulnerabilities faced by consumers in E-commerce websites.
• •By accounting the advantages of image processing method, and machine learning method, it is very simple to handle the vulnerabilities efficiently.
• •The proposed method also protects the consumer by helping them to escape from fraudulent activity.
• •Without using machine learning, it is almost impossible to predict and prevent this at a large scale. Machine learning can only be used by a small number of existing methods to detect the fraudulent activities of the e-commerce websites.
• •The proposed method uses the machine learning methodology as this is the most powerful technique to prevent fraudulent activities with the ability to process tedious and redundant data.
• •The four-level verification is carried out before purchasing the product as it was found that the accuracy is high while detecting the products vulnerabilities.
• •Additionally, it was discovered that the current approach does not apply any processing techniques to identify security vulnerabilities in e-commerce websites.
• •It is discovered that this is the first work to employ image processing for product damage detection.
• •It has been discovered that using this strategy will help consumers purchase secondhand products with assurance, trust, and confidence.
• •Furthermore, by comparing them to other dealers, shoppers can buy secondhand commodities with less damage.

5.2. Implications

The following are the implications of our research work:

• •Despite the fact that e-commerce websites are committed to serving the needs of their users, some of the dealers and sellers there are engaging in fraudulent behaviors by falsifying reviews and original photos of their products.
• •Customers may suffer as a result of this on that specific e-commerce website. Consequently, this may lead customers to lose trust and confidence in that specific e-commerce website, which may result in a negative perception of that e-commerce website.
• •A decline in the number of purchases made by customers resulted from the reputation of the e-commerce website being impacted. Consumers, legitimate dealers, and e-commerce websites will all be negatively impacted by the seller or dealer behaviour on e-commerce platforms, whether directly or indirectly.
• •Customers can therefore obtain appropriate products while handling this consumer susceptibility, avoiding the poor ones. E-commerce websites will reap more benefits and increase their profits as their reputation grows.
• •The problems faced by the consumers will all be prevented using this proposed method as this method is having the potential to detect them earlier.

6. Conclusion

A product image-based vulnerability detection (PIVD) approach has been proposed in this paper to handle the product images that are reused based on the consumer vulnerability. To help consumers in buying the products that are less damaged, the technique named CNN has been utilized ad implemented in three steps to find the fake dealers. CNN is utilized to make the consumers buy products with more confidence. In addition to this to illustrate the proposed model's effectiveness, several extensive experiments have been conducted and analyzed on multiple public datasets. As the feature extraction and classification steps may vary depending upon the product types, a generic architecture will be designed in the future that suits all types of product mages.

Declarations

Author contribution statement

Sarvjeet Kaur Chatrath, Dr. Yogesh Chaba: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Dr. Gurdip Singh Batra: Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Data availability statement

The data that has been used is confidential.

Declaration of interests statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.