Table 5.

Training results of the seven algorithms with six, five, and three labels for the COVID-19 case study.

Number of labels and multilabel classification technique			F1 score (%)		Recall (%)		Precision (%)		AUCa (%)		Accuracy (%)		Hamming loss (%)
Sixb
	Binary relevance	54.6		52.8		56.6		64.0		15.6		33.3
	Classifier chains	53.9		49.8		58.7		64.2		18.5		32.3
	Label power set	58.6		59.4		57.9		66.5		22.2		31.8
	Adapted algorithm (MLKNNc)	54.5		51.0		58.4		64.4		10.0		32.4
	BERTd	88.2		86.7		89.7		90.3		62.0		8.8
	AraBERTe	82.0		84.4		79.8		86.0		50.5		13.6
	NBSVMf	64.3		51.7		85.0		73.1		20.7		21.7
Fiveg
	Binary relevance	57.0		56.0		58.1		63.1		15.8		35.9
	Classifier chains	56.2		53.0		59.9		63.3		18.3		35.1
	Label power set	60.8		63.4		58.4		65.0		22.0		34.8
	Adapted algorithm (MLKNN)	56.5		54.6		58.7		63.1		10.4		35.7
	BERT	87.3		87.9		86.7		88.9		59.0		10.9
	AraBERT	86.3		92.7		80.7		88.6		53.9		12.1
	NBSVM	55.2		40.6		86.4		67.9		17.9		28.0
Threeh
	Binary relevance	68.5		69.0		68.0		69.2		36.9		30.8
	Classifier chains	69.7		68.1		71.4		71.2		39.9		28.7
	Label power set	70.3		69.0		71.5		71.6		40.1		28.3
	Adapted algorithm (MLKNN)	71.6		70.7		72.6		72.8		41.4		27.1
	BERT	94.8		96.4		93.3		94.9		93.2		5.1
	AraBERT	93.3		94.8		91.9		93.5		85.3		6.5
	NBSVM	70.6		59.6		86.5		75.4		46.5		24.2

^aAUC: area under the receiver operating characteristic curve.

^bThe six labels are “COVID-19 name,” “slang term of COVID-19,” “symptom,” “cause,” “place of the disease spread,” and “infected with category.”

^cMLKNN: multilabel adapted k-nearest neighbors.

^dBERT: bidirectional encoder representations from transformers.

^eAraBERT: transformer-based model for Arabic language understanding.

^fNBSVM: support vector machine with naive Bayes features.

^gThe five labels are “COVID-19 name,” “symptom,” “cause,” “place of the disease spread,” and “infected with category.”

^hThe three labels are “COVID-19 name,” “slang term of COVID-19,” and “infected with.”