Identify doctors’ online performance and prioritize service attributes: A framework with aspect-based sentiment analysis and empirical investigation

Abstract

Objective

This study aims to prioritize service attributes related to doctors’ online performance based on patients’ reviews on online healthcare platforms (OHPs).

Method

We propose a three-stage framework based on uncertainty reduction theory. First, perceived service attributes are extracted from review texts through aspect-based sentiment analysis using deep learning models. Second, the impact of these attributes on customer satisfaction and subsequent consultation demand is prioritized using extreme gradient boosting and an econometric model, respectively. Third, the service strengths and weaknesses of individual doctors are evaluated, to provide recommendations for targeted improvements through importance-performance analysis.

Results

A dataset of 445,435 reviews involving 49,024 doctors from an OHP were analyzed based on our framework. We find that attributes reducing cognitive uncertainty such as attitude are more influential than those addressing behavioral uncertainty like professional skill or fit uncertainty like overall experience.

Conclusion

We built a three-stage framework for mining the perceived service attributes and ranked their priority, which is conducive to doctors’ use of patient feedback on OHPs to adjust their service focus and develop strategies to improve service quality.

Introduction

The relationship between service quality and performance is fundamental in service operations management. ¹ In the medical field, due to information asymmetry, patients often lack the expertise to evaluate providers’ professional knowledge and service performance. ² Therefore, patients are increasingly relying on online reviews and electronic word-of-mouth, utilizing the different types of information to reduce the uncertainty and evaluate the service quality in their decision-making progress. ³ A survey showed that nearly three-quarters (72%) of the respondents searched online reviews before finding a good doctor (https://www.softwareadvice.com/resources/how-patients-use-online-reviews/). Patients tend to search for cues in reviews that signal doctors’ unobservable abilities and service efforts.^4,5 Doctors can also prioritize their own service attributes based on patient reviews to formulate strategies for improved service quality.^6,7

Several studies have demonstrated the impact of online reviews on patients’ choice of doctors, considering factors such as rating, ⁸ number of reviews, ⁹ review length, ¹⁰ and the emotions reflected in the reviews. ¹¹ Additionally, previous research has highlighted the influence of various aspects mentioned in reviews, including concrete abilities such as technical competence^12,13 and physician knowledge, ¹⁴ as well as nontangible abilities such as communication skills^12,13,15,16 and doctors’ attitudes. ¹⁵ Methodologically, qualitative analysis based on manual coding^12,13,15,17 and quantitative analysis involving topic modeling, such as latent Dirichlet allocation (LDA),^16,18 have been commonly used. Other areas such as aspect-based sentiment analysis have also shown promise.^14,19,20

However, existing approaches present two key limitations: (1) most previous studies only considered the influence of online word-of-mouth or overall service quality on user demands, failing to capture the fine-grained emotions of different service attributes; (2) prior research lacks prioritizing online service attributes and providing improvement strategies. After identifying the online service quality, doctors have no idea how to improve their service quality to better meet the expectations of patients. In other fields, research on perceived quality has begun to emphasize attribute prioritization and cause analysis,^21,22 to improve service quality accordingly. ²³ Compared with self-stated importance, statistical and artificial intelligence-based approaches are more reliable for identifying perceived quality.^24,25 In summary, this study aims to fill this gap by addressing two research questions:

• RQ1: How can an attribute prioritization framework, integrating uncertainty classification and aspect-based sentiment analysis from online reviews, address the limitations of analyzing doctors’ general online performance?

• RQ2: How to propose strategies to differentially improve doctors’ online service quality based on the empirical analysis of service attributes?

To answer these questions, we propose a three-stage framework based on online reviews. In the first stage, a deep learning model that combines conditional random fields (CRFs), an attention-based long short-term memory model (ATAE-LSTM), and k-means was developed to identify service attributes. And based on uncertainty reduction theory, ²⁶ different service attributes were classified into the information that reduce cognitive, behavioral and fit uncertainty, separately. In the second stage, patient satisfaction and subsequent online consultations were assessed as two dimensions of doctors’ online performance, measuring the importance of different service attributes using extreme gradient boosting (XGBoost) and a two-way fixed effect model, respectively. We explored the preferences of patients for different types of information that reduces uncertainty in the process of measuring doctors’ online performance. In the third stage, attributes were prioritized using importance-performance analysis (IPA) ²⁷ and combined with the results of the econometric model to provide suggestions to doctors for improved service quality.

To validate this framework, we analyzed 445,435 textual reviews of 1,980,061 consultations involving 49,024 doctors at one of the largest online healthcare platforms (OHPs), covering data from January 2017 to December 2019. Our findings show that attributes related to reducing cognitive uncertainty are more significant than those related to reducing behavioral and fit uncertainty.

This study makes three contributions. First, a novel conceptual framework is proposed for assessing doctors’ online performance and prioritizing service attributes. This framework introduces a service quality sentiment analysis approach that integrates various machine and deep learning methods to extract and measure distinct dimensions of doctor service quality from textual reviews, thereby contributing to recent developments in sentiment analysis of OHPs. Second, the theoretical scope of the uncertainty reduction theory is extended to online healthcare by examining the role of different informational uncertainties in doctors’ online performance. Doctors can systematically categorize service quality attributes according to their specific roles in mitigating distinct types of patient information uncertainty. We find that compared with behavioral and fit uncertainty, patients care more about cognitive uncertainty when considering online healthcare quality. When facing patients with slightly different expectations for specific service attributes, our findings are also applicable, increasing the generalization of this research. Third, our findings offer valuable practical insights, suggesting that doctors should prioritize enhancing service attributes that can reduce cognitive uncertainty, such as attitude and personal integrity, which influence patient satisfaction and choice more strongly than professional ability. In addition, doctors can determine the optimal path for service quality improvement based on their unique service quality analysis results.

Related literature

Perceived quality using aspect-based analysis

In recent years, advanced methodologies have also begun to be applied to review quality analysis in various fields, such as restaurants, movies, and online communities. Many studies have applied comprehensive machine learning and deep learning frameworks,^14,28–30 which are mostly based on a language model with attention mechanism, namely, bidirectional encoder representations from transformers (BERT).^22,31–35

Other studies have employed machine-learning models. Mejia et al. ¹⁹ identified and extracted service dimensions using nonnegative matrix factorization and found that the proposed quality metrics can help predict future restaurant performance. Jiang et al. ³⁶ integrated aspect-based sentiment analysis with dynamic topic modeling, a sentiment lexicon, and the Stanford natural language processing package to derive dimension-specific sentiments and estimate their effects on the movie box office. Liu and Chen ²⁰ built a hierarchical service quality model using LSTM to study the sentiments of different attributes in online reviews.

Table 1 summarizes the online review research of recent years using aspect-based sentiment analysis and related methodologies.

Table 1.

Perceived quality using aspect-based sentiment analysis.

Study	Domain	Methodology
Bai et al. 32	Restaurant	Support vector machine (SVM) model + a semisupervised learning method based on BERT model
Jiang et al. 36	Movie	DTM + a sentiment lexicon + Stanford NLP package
Alamoudi and Alghamdi 28	Restaurant	A new unsupervised deep learning approach based on semantic similarity
Mejia et al. 19	Restaurant	NMF
Zhang et al. 30	Hotel	An unsupervised approach by integrating word embedding, cooccurrence, and dependency parsing
Xu et al. 14	Offline doctors’ services	A machine learning model combining comprehensive text-mining techniques and sentiment analysis
Chang et al. 34	Airline	BERT model
Qiu et al. 31	Answer in online health Q&A community	BK-XGBoost method based on BERT model
Liu and Chen 20	Hotel	A hierarchical deep learning model based on LSTM
Fang et al. 29	Online doctors’ services	A machine learning model combining comprehensive text-mining techniques and sentiment analysis
Nie et al. 37	Hotel	A framework by embedding techniques of text mining and deep learning
Li et al. 38	Restaurant	Machine learning-based conditional survival forest model
Zhao et al. 33	Restaurant	A multitask learning model based on BERT model
Yang et al. 22	Automobile	SVM model + BERT model
Ma et al. 35	Online food delivery	BERT model

Note: NMF: nonnegative matrix factorization; DTM: dynamic topic modeling; NLP: natural language processing; BERT: bidirectional encoder representations from transformers; XGBoost: extreme gradient boosting; LSTM: long short-term memory model.

However, to the best of our knowledge, few studies have focused on aspect-based analyses of online reviews and their impact on doctors’ service performance. Xu et al. ¹⁴ utilized the service quality proxies extracted from textual reviews to forecast doctor demand in offline hospital settings and shed light on aspect-based service quality for offline doctor selection. Fang et al. ²⁹ demonstrated that both technical and functional quality cues within unstructured textual reviews affect online doctor selection; however, they did not specifically address distinct service attributes. In mainstream aspect-based sentiment analysis in the computer science domain, researches often combine LSTM and a hierarchical attention mechanism ³⁹ to focus on the connection between various aspects. Our research also applies this idea to online review analysis of doctors’ service quality. We adopted the model based on CRF and LSTM with a hierarchical attention mechanism, since LSTM has a faster training and inference speed and is more suitable for scenarios with higher real-time requirements, compared with BERT. Besides, CRF can fine-tune the model under a limited sample through structured feature engineering. Because our goal is to provide strategies for improving the service quality of doctors on the OHPs, it is necessary to constantly update the review data. Departing from conventional BERT-based aspect-based sentiment detection approaches, we implemented a decomposition framework that strategically balances computational efficiency with model generalizability. Meanwhile, we added XGBoost subsequently to enhance the interpretability of the model. This framework includes analyzing doctors’ service attributes and how to improve doctor service performance satisfaction such as patient satisfaction and demand, thus bridging a crucial gap in this domain.

Online reviews analysis on OHPs

Existing research has demonstrated the significance of online reviews for patients’ choices of OHPs, as they provide a rich source of data to track doctors’ care quality. ⁴⁰ The literature on online reviews is becoming increasingly comprehensive. Scholars have established that ratings, ⁸ number of reviews, ⁹ and emotional tendencies ¹¹ influence the selection of doctors. Chou et al. ¹⁰ highlighted the significance of review length, readability (measured by Flesch Reading Ease Index), and reviewer rankings as quality indicators.

However, there is a growing body of research on the topic analysis of online reviews, encompassing both qualitative and quantitative approaches. Studies have indicated that both the hard and soft abilities of doctors are essential considerations for patients when making choices.^29,41 For instance, patients pay attention to doctors’ professional qualities, such as technical competence,^12,13 physician knowledge, ¹⁴ and personality traits,^12,42 such as communication skills^12,13,15,16 and attitudes.^15,43 The experience of seeing a doctor is also crucial in offline service reviews,^4,17 including aspects such as convenience, ¹⁶ waiting and service time, ¹⁴ environment,^4,12,14 and cost. ⁴

Table 2 summarizes the aspects of doctors in online reviews that patients focus on, providing details and methodologies on studies involving topic analysis in online reviews.

Table 2.

Factors discovered in previous related works.

Aspects of online review	Study	Detail	Methodology
Rating	Michael Luca and Sonal Vats 8
Reviewers’ ranking	Chou et al. 10
Quantity	Lu and Wu 9
Length	Chou et al. 10
Readability	Chou et al. 10
Emotional tendency	Han et al. 11
Topic	Detz et al. 12	Personality traits, technical competence, communication, access to physician, office staff/environment, and coordination of care.	Manual coding
	Lagu et al. 13	Communication and interpersonal skills, technical skills, facility/office experience, staff characteristics, patient care, feedback about survey.	Manual coding
	Ryskina et al. 15	Perceived attitudes, communication, and clinical expertise.	Manual coding
	Asanad et al. 17	Patient–physician experience, medical and surgical treatment, office staff and analysis of worth.	Grounded theory methodology
	Liu et al. 16	Competence, communication, treatment, and convenience.	LDA
	Shah et al. 4	11 topics such as doctor value, treatment/operational process and so on in patients’ satisfaction reviews; 4 topics such as uncomforting environment and treatment cost in patients’ dissatisfaction reviews	LDA
	Wan et al. 42	14 service attributes, mostly important attributes are trust, phraseology, overall service experience, word of mouth and personality traits.	PageRank algorithm
	Xu et al. 14	Bedside manner, diagnosis accuracy, waiting time, service time, insurance process, physician knowledge, and office environment.	Naive Bayes analyzer
	Fang et al. 29	Technical and functional quality	Machine learning
	Ghosh et al. 44	Hospital administration, treatment experience, responsiveness, and waiting time	LDA
	Aghakhani et al. 18	Review comprehensiveness (i.e. the number of topics covered in a review).	LDA
	Chen and Lee 41	Service-related topics, clinical- and treatment-related topics, general topics	LDA
	Dai et al. 43	Nine topics such as physician ethics, doctor's attitude, and diagnostic result.	LDA

Note: LDA: latent Dirichlet allocation.

The commonly used attribute identification methods in previous researches have some limitations. For example, manual coding methods,^12,13,15 while effective for attribute identification, struggle to scale with large review volumes and lack dynamic prioritization mechanisms; Topic modeling techniques such as LDA^{4,16,18,41,44} often produce overlapping attributes and fail to capture nuanced sentiment variations across specific service dimensions. Our study made full use of the rich information in online reviews to build a framework for online quality identification of doctors and prioritization of service attributes, showcasing the advantages of perceived quality identification based on artificial intelligence and statistical methods.^24,25

Uncertainty reduction theory

Uncertainty reduction theory ²⁶ suggests that individuals employ three general categories of information-seeking strategies, passive, active, and interactive, to reduce uncertainty and increase the other party's predictability. This theory has been adopted in virtual worlds, ⁴⁵ crowdfunding, ⁴⁶ and accommodation platforms, ⁴⁷ to understand consumer behavior. Previous research has shown that uncertainty is negatively related to consumer trust ⁴⁸ and purchase intention, ⁴⁹ perceiving services as riskier than goods. ⁵⁰ Consequently, consumers acquire relevant information to reduce uncertainty, which helps them evaluate the services. ³

In the context of OHPs, online reviews are a vital channel for patients to acquire additional information as an active strategy to reduce uncertainty, considering that information asymmetry exists between patients and doctors. ⁴¹ Reviews are also the most easily observed information, in addition to the passive information received from doctors, such as job titles and hospital ranks on their homepages. Similar to other services, uncertainty can be divided into quality and fit uncertainties. ⁴⁷ Fit uncertainty is unrelated to assessing doctor quality, such as professional skills or morals, and refers to the uncertainties associated with progress communication that best matches patient preferences. Quality uncertainty can be divided into two categories: cognitive and behavioral, ⁵¹ emphasizing that reducing patients’ uncertainty in belief attitude (such as doctor's attitude, character, and other soft powers) and behavioral uncertainty (such as doctor's solution, professional skills, and other hard powers) is equally important. ⁴² Employing uncertainty reduction theory, we consider the impact of different service attributes on service performance (including patient satisfaction and follow-up online demand) from the aspects of cognitive, behavioral, and fit uncertainty.

Methods

In this section, to address our research questions, we propose a three-stage framework for mining perceived service attributes. In Stage 1, perceived service attributes are extracted from review texts. Stages 2A and 2B prioritize the impact of these attributes on customer satisfaction and subsequent consultation demand, respectively. In Stage 3, individual service strengths and weaknesses are evaluated to offer recommendations for improving specific service attributes. The detailed process is illustrated in Figure 1.

Figure 1.

Research framework.

Stage 1: Service attribute identification

The identification process for aspect-based sentiment analysis within patient reviews was performed using a systematic four-step approach, which is represented in Figure 2.

Figure 2.

Service attributes identification model.

First, preliminary cleaning of the data was performed, followed by word segmentation using Jieba (https://github.com/fxsjy/jieba/). To preserve the connection between sentences, the English word placeholder was retained. Additionally, word vectors were calculated using word2vec ⁵² to enhance subsequent analyses. After word segmentation, continuous bag-of-words, which uses context to predict the semantics of intermediate words, was employed to train on the data to obtain a word2vec model.

Second, a CRF model was utilized to extract aspect-based attribute words ⁵³ using selected randomly coded reviews with attribute word labels. Subsequently, the trained model was applied to all the remaining comments for attribute word labeling.

Third, attribute word sentiment was labeled based on ATAE-LSTM. Each attribute word for training data was labeled with a sentiment of 1 (positive) or −1 (negative). This coded dataset played a crucial role in assessing the aspect-based sentiment analysis model and was subsequently applied to the entire dataset to determine the emotional values across all attribute words. The ATAE-LSTM model adds an attention mechanism ⁵⁴ and attribute word-splicing based on the LSTM model. The attention mechanism simulates the data processing method of the human cognitive style. It obtains the mutual importance of two-word segments and assigns different weights to the data, thus helping the model to better recognize and utilize attribute words. When processing a paragraph, multiple attribute words with different emotional polarities may exist. Attribute word splicing was adopted to best utilize different sentiments. Each attribute word in each sentence paragraph was spliced using the original sentence paragraph and input into the model to obtain the sentiment at the aspect level. The structure of ATAE-LSTM is shown in Figure 3.

Figure 3.

Structure of ATAE-LSTM model.

Note: ATAE-LSTM: attention-based long short-term memory model.

First, the word vector of the attribute word and original sentence paragraph were joined individually to obtain a vector that was input into the ATAE-LSTM model to obtain the hidden layer vector H. The hidden layer vector H was then joined with the word vector of the attribute and input into the attention layer to obtain attention vector γ. Finally, the probabilities of different sentiment classifications were obtained using the softmax layer.

Finally, the attribute words were clustered into attribute surfaces based on their word vectors using k-means. This process resulted in the identification of attribute surfaces and the corresponding emotional values in each review.

Stage 2A: Service attribute priority ranking

In Stage 1, various service attributes and their corresponding sentiments are identified in each review. A large volume of review texts on OHPs can lead to cognitive overload. To mitigate this, we ranked the importance of each service attribute in relation to patient satisfaction for each review.

To capture the nonlinear correlation between sentiments of different service attributes and patient satisfaction while filtering out irrelevant factors, we employed XGBoost alongside the permutation importance test. XGBoost, which is an enhancement of the gradient boosting decision-tree (GBDT) algorithm, ⁵⁵ is a powerful predictive tool. As a boosted version of the classification and regression tree method, multiple decision trees are sequentially constructed to minimize prediction error. The tree-based structure allows complex nonlinear relationships to be managed effectively with high interpretability. In terms of interpretability, XGBoost outperforms many deep learning models, such as neural networks. ⁵⁶ The permutation importance test randomly shuffles the values of a variable to break the correlation with the dependent variable. The resulting decrease in prediction performance indicates the significance of the variable in predicting the outcome. By applying this test to all variables, relative importance can be assessed.

Previous research indicates that customers’ self-stated importance is unreliable as they often rate all attributes as highly important. ⁵⁷ Therefore, we relied on statistical and artificial intelligence-based approaches to explore the relationship between attribute performance and overall performance, measured by overall satisfaction.^24,25

Stage 2B: Service attributes affecting demand verification

Beyond patient satisfaction, service performance may also lead to an increase in future consultations. Online reviews are an important channel for analyzing user demand. ⁵⁸ Identifying the service attributes that drive more consultations helps doctors prioritize improvements. To address this, we applied a two-way fixed-effects model to analyze how attribute sentiments in reviews impact each doctor's online consultations. By calculating the average sentiment of each attribute surface from each doctor's monthly reviews, their influence on the volume of online consultations in the subsequent month was assessed.

In this model, the number of consultations serves as the dependent variable, whereas the sentiment values of the various service attributes act as independent variables. Missing values were filled in with zero since the missing values means patients doesn’t have any emotional tendency toward this service attribute. ³⁸ Using monthly observations, the two-way fixed-effects model is structured as follows:

$$Consultatio{n}_{i,t}={\beta }_0+X_{i,t-1}\cdot \mathrm{\Gamma }+Ctr{l}_{i,t-1}+{\mu }_i+{\theta }_t+{\epsilon }_{it},$$ (1)

where Consultation_i,t is the number of consultations for doctor i in period t; X_i,t−1 is the average sentiment value for each corresponding service attribute of doctor i in period t-1; Ctrl_i,t−1 are the control variables, and we control the number of consultations and the number of reviews for doctor i in time period t-1 with the variable Consultation_i,t−1 and Review_i,t−1; μ_i and θ_t capture doctor and time fixed effects; ɛ _it is random noise. We conducted log transformation on count variables and cluster errors at the doctor level for a robust estimation.

Stage 3: Individual strength and weakness representation

Based on the findings from stages 2A and 2B, insights can be acquired into strengths and weaknesses of doctor services, enabling targeted improvements. We adopted IPA ²⁷ with manual adjustments. IPA is a common tool for understanding consumer satisfaction and developing improvement strategies and has been widely used in many fields.^25,35,59,60 Figure 4 shows an example of IPA. The horizontal axis represents performance (perceived quality), and the vertical axis represents importance. The midpoint value was used to define the quadrant divisions, which is the standard approach. The figure was divided into four quadrants: Q1 (high quality, high importance), Q2 (low quality, high importance), Q3 (low quality, low importance), and Q4 (high quality, low importance). IPA helps prioritize quality improvements, with the suggested order being Q2 > Q3 > Q1 > Q4, emphasizing attributes with poor performance but high importance.

Figure 4.

Example of importance-performance analysis.

The data-centered method is the most common way to place the cross-hair for IPA.^61,62 This article utilizes the means to set the cross-hair. The average sentiment for each service attribute for every doctor was calculated, thereby generating an IPA based on the attribute priorities identified in Stage 2A. For the attributes in Q3 or Q4 that significantly affect consultation demand in Stage 2 B, special attention is required to prevent potential patient dissatisfaction from reducing future consultation volumes and associated financial incentives.

Results

Research context and data

To address the research questions proposed in this study, we collected data from https://www.chunyuyisheng.com. This platform is one of the largest online healthcare service providers in China that offers consultation services. Each doctor on the platform has a dedicated homepage showcasing essential details, such as their name, title, affiliated hospital, department, and a comprehensive record of previous consultations and ratings. Patients can choose a doctor, report their chief complaint, upload inspection reports, or initiate consultations via online text or telephone. Doctors can diagnose or offer suggestions based on the information they provide. After consultation, patients need to provide a review with the rating of “Quite satisfied,” “Satisfied,” or “Not satisfied.” Textual reviews are optional.

Our data collection spanned from January 2017 to December 2019 and encompassed a vast dataset of 1,980,061 consultations involving 49,024 doctors. Within this comprehensive dataset, 445,435 consultations (approximately 22.50%) included textual reviews, which were the focus of our research. Specifically, 1,665,209 (i.e. 84.10% of all) had a review of “Quite satisfied,” while 332,566 provided textual reviews (i.e. 19.97% of “Quite satisfied reviews”); 188,259 (i.e. 9.51% of all) had a review of “Satisfied,” while 25,039 provided textual reviews (i.e. 13.30% of “Satisfied” reviews); and 126,593 (i.e. 6.39% of all) had a review of “Not satisfied,” while 87,830 provided textual reviews (i.e. 69.37% of “Not satisfied” reviews). Dissatisfied patients were far more likely to leave written comments than satisfied patients, which aligns with the negative bias of the consumer mentality. ⁶³ This might help us identify more aspects where users are dissatisfied with the service attributes. However, since our current research is to provide improvement strategies and enhance the quality of service based on patients’ perceived quality, which placed greater emphasis on the deficiencies in the service, this will not deviate from the improvement guidelines we propose to doctors.^14,29,42,43

Results of service attributes identification

Figure 5 shows the data used for service attribute identification based on the model in Figure 2. First, to train CRF, 2000 reviews were randomly coded, labeling attribute words and their corresponding emotional values with 1 (positive), or −1 (negative). Finally, 627 reviews were labeled with 798 attribute words. This dataset was used to evaluate the attribute word identification performance with 80% as training data and 20% as testing data. The accuracy, precision, and recall of the CRF model were 99.15%, 86.53%, and 98%, respectively. We then applied the trained model to all the remaining comments for attribute word labeling and identified 156,123 reviews with 1379 unique attribute words among the 445,511 reviews.

Figure 5.

The data used in the service attributes identification model.

Second, to improve the performance of aspect-based sentiment analysis, we adopted a stratified sampling method, selecting 1% of the reviews (i.e. 1561) for attribute word sentiment labeling, whereby the corresponding emotional values for each attribute words were labeled with 1 (positive), and −1 (negative) for 1989 attribute words in 1561 reviews. Subsequently, 10-fold experiments were conducted at the paper level, and statistical tests were run for performance evaluation using the developed ATAE-LSTM model. For hyperparameter tuning, the network was trained by backpropagation in minibatches, and gradient-based optimization was performed using the Adam update rule. The initial learning rate was 0.001, and the number of nodes in each layer was 200 for the hidden layer. The performance of the model was measured using 80% of the training data and 20% of the testing data. Finally, the accuracy, precision, and recall obtained for the ATAE-LSTM model were 94.62%, 97.07%, and 94.46%, respectively, which are better than those of the baseline models. The performances of the baseline models are shown in Table A1, with the evaluation metrics in Appendix A1. The trained model was then applied to all the reviews with attribute words, thereby obtaining 1379 attribute words. After deleting the attribute words that appeared only once, 642 attribute words were retained.

Third, the attribute words were clustered based on their word vectors using the k-means algorithm. During clustering, the parameter K was tested in the range of 2 to 20. According to the elbow method, ⁶⁴ K can be 13 or 15. Comparing the performance of manual clustering based on our research context, K was set to 13. The clustering results are shown in Figure A1 in Appendix A2. For reference, we retained the clustering results for 15 clusters in Table A2 in Appendix A2.

Table 3 displays the 13 clusters with their ID, total number of attribute words, and selected attribute words. A name was assigned to each cluster based on the meaning of the words in each cluster. After clustering, the meanings of the keywords in categories 1 and 2 were found to be very similar; hence, we decided to merge these two categories. Categories 12 and 13 are noisy words that are not related to this research and involve relatively few reviews; therefore, we excluded them from the subsequent empirical analysis. Finally, we obtained 10 categories of service attributes: Explanation, Response, Attitude, Morality, Professional skill, Solution, Suggestion, Communication, Phraseology, and Overall experience.

Table 3.

Cluster and examples of attribute words.

ID	Name	Number of attribute words	Attribute words (for example)	Number of reviews included this attribute
1	Explanation(1)	59	讲(explain),解释(explanation),分析(analyze)	12,395
2	Explanation(2)	72	解答(explanation),讲解(explanation),指导(guidance)	24,952
3	Response	52	回复(response),回答(answer),说(say)	74,215
4	Attitude	58	态度(attitude),服务态度(service attitude)	31,526
5	Morality	54	德艺双馨(both moral and artistic),乐于助人(be ready to help),心地(mind)	1202
6	Professional skill	59	专业(profession),帮助(help),效果(effect),专业知识(professional skills)	5772
7	Solution	66	意见(advice),问题(question),解决问题(solve the problem),方案(solution)	7165
8	Suggestion	45	建议(suggestion),提议(propose)	10,614
9	Communication	29	沟通(communication),交流(communication),交谈(chatting)	1457
10	Phraseology	38	感觉(feeling),语气(tone),声音(voice),语言(language)	460
11	Overall experience	60	答复(reply),服务(service),对待(treat),咨询(consultation)	8312
12	Noisy (1)	45	English words, 呃(modal particle)	296
13	Noisy (2)	4	同理(In a similar way)	16

Among these, Explanation and Response are more closely related to the conversation process in therapy. For example, Explanation describes the patient's perception of the doctor's explanation of the condition and includes a variety of words that express the meaning of “explain,” and other keywords such as “analyze” and “guidance.” Response includes keywords such as “response,” “answer,” and “say.” Attitude describes the doctor's attitude toward the patient during the treatment process, whereas Morality is the patient's perception and description of the doctor's character. Reviews of these attributes can help patients resolve quality uncertainties, primarily cognitive uncertainties, by providing more information about the doctors’ soft abilities. Professional skills, Solution, and Suggestion describe doctors’ abilities and effects in the consultation process according to patients’ disease information, pertaining to information related to doctors’ hard abilities that can reduce behavioral uncertainties. Communication shows patterns of chatting in more detail. Phraseology describes the tone and voice attributes of the doctor in communicating with the patient and includes keywords such as “feeling,” “tone,” “voice,” and “language.” Overall experience refers to the patients’ description of their medical experience. The information provided by these attributes is more about reducing fit uncertainty than quality uncertainty because communication patterns and tone are not clearly “good” or “bad” criteria although patients have personal preferences.

The number of reviews that included each type of service attribute was summarized, finding that patients were most concerned about doctors’ Response and Explanation since online consultation is a problem-oriented help-seeking behavior. In addition, Attitude of doctors was an aspect that patients mentioned frequently in their reviews as more important than Solution, Suggestions, and Professional skill.

Results of service attribute priority ranking

We conducted a 10-fold cross-validation to predict whether a patient is satisfied with the 10 attributes (a rating of “Quite satisfied” or “Satisfied,” means the patients are satisfied with this consultation). Our results with XGBoost show that all 10 service attributes, excluding noise, can predict whether the patient is satisfied, based on permutation importance tests (PI > 0 and p-value < .05). The technical details of XGBoost are shown in Table A3 in Appendix A3. Table 4 reports the attribute importance results, with variables sorted according to permutation importance. We also used the random forest as the robustness test, which is shown in Table A4 in Appendix A3. The results are almost consistent with XGBoost, showing the reliability of our findings.

Table 4.

Important variables for predicting satisfaction.

Information category	Attributes	P.I.	p-value
Cognitive uncertainty	Response	0.3628	.000
Cognitive uncertainty	Attitude	0.0595	.000
Cognitive uncertainty	Explanation	0.0510	.000
Behavioral uncertainty	Professional skill	0.0284	.000
Behavioral uncertainty	Solution	0.0223	.000
Behavioral uncertainty	Suggestion	0.0214	.000
Fit uncertainty	Overall experience	0.0167	.000
Fit uncertainty	Communication	0.0022	.000
Fit uncertainty	Phraseology	0.0016	.000
Cognitive uncertainty	Morality	0.0007	.000

According to the uncertainty reduction theory, we found that except for Morality, all attributes that can help reduce quality uncertainty are more important than attributes related to fit uncertainty. This shows that patients care more about a doctor's ability than their personal preference for OHPs. Notably, cognitive uncertainty reduction is more important than behavioral uncertainty reduction, and doctors’ morality is less important, but this may be because doctors’ morality is more difficult to observe in the context of online healthcare consultations.

Results of service attributes affecting demand verification

Table 5 presents summary statistics of the main variables. Popularity varies from one doctor to another. The consultation volume per month ranged from 0 to 239. On average, the sentiments of Response and Communication are negative, and their quality must be enhanced in subsequent services.

Table 5.

Descriptive statistical analysis results.

Variables	N	Mean	Std Dev	Max	Min
Consultationi,t	1,764,864	1.1219	5.8050	239	0
Reviewi,t	1,764,864	0.2524	1.3783	60	0
Responsei,t	1,764,864	−0.0023	0.1611	1	−1
Attitudei,t	1,764,864	0.0089	0.1153	1	−1
Explanationi,t	1,764,864	0.0106	0.1209	1	−1
ProfessionalSkilli,t	1,764,864	0.0004	0.0557	1	−1
Solutioni,t	1,764,864	0.0001	0.0614	1	−1
Suggestioni,t	1,764,864	0.0026	0.0734	1	−1
OverallExperiencei,t	1,764,864	0.0017	0.0657	1	−1
Communicationi,t	1,764,864	−0.0001	0.0283	1	−1
Phraseologyi,t	1,764,864	0.0000	0.0161	1	−1
Moralityi,t	1,764,864	0.0006	0.0254	1	−1

Table 6 shows how the results of service attributes affect demand. Notably, the sentiment related to reducing cognitive uncertainty, that is Explanation, Response, Attitude, and Morality, has a positive effect on doctor's demand. In the context of seeking assistance on OHPs, patients prioritize receiving detailed explanation and comprehensive replies from doctors. This intuitive finding underscores the correlation between positive comments regarding explanations/responses and increased demand for medical consultations, highlighting the importance of reducing cognitive uncertainty in OHPs.

Table 6.

Results for econometric model.

Information category	Attributes	(1)
Cognitive uncertainty	Responsei,t−1	0.068*** (0.002)
Cognitive uncertainty	Attitudei,t−1	0.029*** (0.008)
Cognitive uncertainty	Explanationi,t−1	0.011 (0.008)
Behavioral uncertainty	ProfessionalSkilli,t−1	0.003 (0.015)
Behavioral uncertainty	Solutioni,t−1	0.017 (0.013)
Behavioral uncertainty	Suggestioni,t−1	−0.018 (0.011)
Fit uncertainty	OverallExperiencei,t−1	0.003 (0.013)
Fit uncertainty	Communicationi,t−1	−0.009 (0.028)
Fit uncertainty	Phraseologyi,t−1	−0.009 (0.049)
Cognitive uncertainty	Moralityi,t−1	0.062** (0.031)
Control variables	Consultationi,t−1	0.614*** (0.003)
	Reviewi,t−1	0.078*** (0.006)
Individual fixed effect		Yes
Time fixed effect		Month
No of observation		1,715,840
No of subject		49,024
Adj. R-square		0.614

Note: Standard errors are indicated in parentheses.

*p < .1, **p < .05, ***p < .01.

Attributes related to reduced behavioral and fit uncertainty did not affect the number of subsequent consultations. This observation underscores the enduring presence of information asymmetry between patients and doctors regarding OHPs. Given the challenge of gauging doctors’ professional competence in this online environment, patients naturally gravitate toward assessing the characteristics and dedication of healthcare providers. This preference for evaluating the softer aspects of doctors’ capabilities suggests an ongoing need for mechanisms to bridge information gaps in OHPs.

In patient satisfaction, the importance of Morality was highlighted. Response and Attitude remained important, retaining the same order. When doctors conduct quality assessment and improvement, they need to comprehensively evaluate the factors influencing these two types of performance, to choose the direction of further improvement.

It is worth noting the evaluation here may be biased for measuring the true online service quality of doctors since dissatisfied patients were far more likely to leave written comments. ⁶³ However, it is unbiased for the online service quality perceived by patients (because patients can only construct their evaluation of the online service quality of doctors from these already published textual comments). Since it is difficult for patients to understand the true online service quality of doctors from other channels, doctors need to emphasize this perceived quality by patients and formulate their own service improvement strategies based on this.

Example of individual advantages and disadvantages representation

Two doctors, A and B, from the same department (Traditional Chinese Medicine Department) were randomly selected from the data, whereby the average scores of each service attribute were entered into the IPA chart as follows:

As shown in Figure 6, the perception of IPA highlights discrepancies in doctors’ online service performance, indicating that each doctor has unique priorities for quality improvement. For example, behavioral uncertainty factors, such as Professional skill and Solution are the attributes to be most urgently improved for Doctor A (Q2), whereas they are less critical for Doctor B (Q1). In contrast, cognitive uncertainty factors such as Response and Explanation are top priorities for Doctor B (Q2) and less critical for Doctor A (Q1). In the case of Doctor A, although morality (Q3) is not the highest priority (compared to Q2), it still requires urgent attention because it influences the number of consultations. Fit uncertainty factors, such as Overall experience also require Doctor B's attention (Q3), reminding him to be mindful of different patient needs and adjust his communication style and pace accordingly.

Figure 6.

Two importance-performance analysis examples.

Discussion

Implication

Here, novel theoretical implications were introduced. First, a framework was developed to identify doctors’ online performance and prioritize service attributes. Previous studies did not propose improvement strategies following quality identification; however, our study offers a complete framework. This framework identifies service attributes that are significantly linked to doctors’ online performance, showing that, in the context of OHPs, cognitive uncertainty information has a stronger impact on performance than behavioral or fit uncertainty. These findings deepen our understanding of the mechanisms that influence OHP performance.

Second, we verified that online reviews serve as effective proxies for service quality in OHPs. While previous studies have often assessed doctors’ online service quality using standardized signals such as professional title, reputation, and ratings,^4,65 exploration of unstructured text data has been limited. This study advances the service quality literature by exploring aspect-based online service attributes, attributing the service attributes identified from the online reviews respectively to the information that reduces cognitive, behavioral, and fit uncertainty. And we found the uncertain information that patients were most concerned about, avoiding the issue of cognitive overload that doctors encounter when analyzing the perceived online service quality from patients. ²² It expands the application of uncertainty reduction theory to online healthcare.

Finally, from the perspective of methodology, a conceptual framework was proposed for service attribute extraction based on aspect-based sentiment analysis. While prior research primarily used topic analysis methods to examine the impact of online reviews on patients’ choice of doctors,^4,16,18,41 studies utilizing aspect-based sentiment analysis have been limited in scope, typically focusing only on technical and functional quality ²⁹ or offline doctor selection. ¹⁴ Our model goes beyond capturing the overall sentiment in online medical reviews by delving into more detailed emotional aspects. Considering that we have 445,435 textual reviews, our capture of patients’ perception of service quality in different attributes is relatively comprehensive. This methodology contributes to the recent trend of using sentiment analysis for OHPs and can be extended to other online review analyses.

This study has important practical implications for doctors, patients, and other healthcare platforms. On the one hand, doctors were informed that patients place a significant emphasis on service attitude and personal morality. These aspects, which can be improved relatively quickly, have a more pronounced impact on patient choice than on personal or professional abilities. Clear communication, quick responsiveness, and friendly attitudes can attract more patients and provide a directional guide for the development of doctors within OHPs. For specific doctors, the IPA results provide actionable guidance to refine their service strategies: (1) They can allocate maximum resources to Q2 attributes (low quality, high importance) identified through quadrant analysis, particularly those showing significant gaps between individual and average practitioner performance. And then pay attention to the service attributes in Q3 (low quality, low importance) to make up for shortcomings in all aspects; (2) They should maintain their quality, which means sustain excellence in Q1 (high quality, high importance) competencies; (3) They need to focus on some service attributes that may affect their consultation demand, even these attributes don’t show high importance based on IPA analysis, such as the morality.

However, considering the backdrop of doctor–patient information asymmetry, we highlight that online reviews in OHPs play a crucial role as a channel for patients to access reliable information about service quality. Patients can refer to these online platforms to make informed choices about trustworthy healthcare providers. Finally, the online platform may enhance its services by offering doctors’ performance analysis reports derived from our framework. As the example given in the example of individual advantages and disadvantages representation section, the platform can issue the IPA analysis diagrams to each doctor. Such reports can help doctors identify their strengths and weaknesses in the disease market, enabling them to improve their service quality to better meet user needs.

Limitation and future work

However, our study has some limitations, offering several avenues for extension. First, we concentrated on evaluating service attributes in terms of their positive and negative values without considering the nuances of emotional word intensity. To delve deeper into this aspect, we propose a more granular exploration of emotional values associated with various characteristics, which allows a more comprehensive investigation of the influence of aspect-based emotional polarity on online consultation. Second, a supervised deep learning model that utilized a manually coded dataset was employed in the service attribute identification stage. In future studies, we recommend exploring unsupervised models that do not rely on subjective human understanding to broaden the scope of our research. Third, due to platform limitations, our dataset included primarily short-text reviews, potentially limiting the availability of comprehensive information and aspects that patients prioritize in their service attribute evaluations. Meanwhile, our research can only capture the service attributes expressed by patients on this platform, but cannot capture other additional information that reduces the uncertainty between doctors and patients. To address these limitations, future research should prioritize the analysis of long-text attributes, and integrate the review information from different OHPs, or supplement qualitative research such as interviews with patients to acquire a more thorough understanding of medical quality evaluations. Furthermore, our study predominantly reflects the unique cultural context of online healthcare quality evaluation in China. Future studies should examine the generalizability of our findings and extend the analysis to different cultural contexts for a more comprehensive assessment.

Conclusion

Although the relationship between service attributes in patients’ online reviews and doctors’ performance has been widely explored using topic modeling^16,18,41 or traditional offline channels, ⁶⁶ our findings contribute to the literature on doctor–patient interaction mechanisms from a text-mining viewpoint through aspect-based sentiment analysis. We built a three-stage framework for mining the perceived service attributes and ranked their priority, which is conducive to doctors’ use of patient feedback on OHPs to adjust their service focus and develop strategies to improve service quality. First, in perceived quality identification, we built an advanced deep learning model to identify aspect-based sentiments from patients’ online reviews and achieved higher performance than traditional deep learning models. Second, we found that information related to reducing cognitive uncertainty is generally more important than reducing behavioral and fit uncertainty in verifying doctors’ performance. Finally, based on IPA, we rank the priority of the attributes of doctors’ service quality. IPA provides each doctor with a strategy to improve the quality of their services. This three-stage framework answer the RQ1, developing an attribute prioritization framework for doctors’ online performance based on OHPs reviews. Furthermore, the framework generates actionable service improvement strategies by synthesizing attribute-specific emotion evaluations across individual practitioners, resolving the RQ2. This approach operationalizes service optimization through a dual focus: (1) prioritizing enhancement of high-impact attributes with suboptimal patient sentiment and (2) strategically addressing discrepancies between patient expectations and provider performance metrics. This mechanistic alignment between diagnostic assessment and targeted intervention establishes a systematic pathway from textual feedback analysis to online healthcare service refinement.