Table 1.
Language codes for both FastText-LID and Twitter-LID tools
| Language | FastText | |
|---|---|---|
| Afrikaans | af | – |
| Albanian | sq | – |
| Amharic | am | am |
| Arabic | ar | ar |
| Aragonese | an | – |
| Armenian | hy | hy |
| Assamese | as | – |
| Asturian | ast | – |
| Avaric | av | – |
| Azerbaijani | az | – |
| Bashkir | ba | – |
| Basque | eu | eu |
| Bavarian | bar | – |
| Belarusian | be | – |
| Bengali | bn | bn |
| Bihari | bh | – |
| Bishnupriya | bpy | – |
| Bosnian | bs | bs |
| Breton | br | – |
| Bulgarian | bg | bg |
| Burmese | my | my |
| Catalan | ca | ca |
| Cebuano | ceb | – |
| Cherokee | – | chr |
| Central-Bikol | bcl | – |
| Central-Kurdish | ckb | ckb |
| Chavacano | cbk | – |
| Chechen | ce | – |
| Chinese-Simplified | – | zh-cn |
| Chinese-Traditional | – | zh-tw |
| Chinese | zh | zh |
| Chuvash | cv | – |
| Cornish | kw | – |
| Corsican | co | – |
| Croatian | hr | – |
| Czech | cs | cs |
| Danish | da | da |
| Dimli | diq | – |
| Divehi | dv | dv |
| Dotyali | dty | – |
| Dutch | nl | nl |
| Eastern-Mari | mhr | – |
| Egyptian-Arabic | arz | – |
| Emiliano-Romagnolo | eml | – |
| English | en | en |
| Erzya | myv | – |
| Esperanto | eo | – |
| Estonian | et | et |
| Fiji-Hindi | hif | – |
| Filipino | – | fil |
| Finnish | fi | fi |
| French | fr | fr |
| Frisian | fy | – |
| Gaelic | gd | – |
| Gallegan | gl | – |
| Georgian | ka | ka |
| German | de | de |
| Goan-Konkani | gom | – |
| Greek | el | el |
| Guarani | gn | – |
| Gujarati | gu | gu |
| Haitian | ht | ht |
| Hebrew | he | he |
| Hindi | hi | hi |
| Hungarian | hu | hu |
| Icelandic | is | is |
| Ido | io | – |
| Iloko | ilo | – |
| Indonesian | id | in |
| Inuktitut | – | iu |
| Interlingua | ia | – |
| Interlingue | ie | – |
| Irish | ga | – |
| Italian | it | it |
| Japanese | ja | ja |
| Javanese | jv | – |
| Kalmyk | xal | – |
| Kannada | kn | kn |
| Karachay-Balkar | krc | – |
| Kazakh | kk | – |
| Khmer | km | km |
| Kirghiz | ky | – |
| Komi | kv | – |
| Korean | ko | ko |
| Kurdish | ku | – |
| Lao | lo | lo |
| Latin | la | – |
| Latvian | lv | lv |
| Lezghian | lez | – |
| Limburgan | li | – |
| Lithuanian | lt | lt |
| Lojban | jbo | – |
| Lombard | lmo | – |
| Lower-Sorbian | dsb | – |
| Luxembourgish | lb | – |
| Macedonian | mk | – |
| Maithili | mai | – |
| Malagasy | mg | – |
| Malayalam | ml | ml |
| Malay | ms | msa |
| Maltese | mt | – |
| Manx | gv | – |
| Marathi | mr | mr |
| Mazanderani | mzn | – |
| Minangkabau | min | – |
| Mingrelian | xmf | – |
| Mirandese | mwl | – |
| Mongolian | mn | – |
| Nahuatl | nah | – |
| Neapolitan | nap | – |
| Nepali | ne | ne |
| Newari | new | – |
| Northen-Frisian | frr | – |
| Northern-Luri | lrc | – |
| Norwegian | no | no |
| Nynorsk | nn | – |
| Occitan | oc | – |
| Oriya | or | or |
| Ossetic | os | – |
| Pampanga | pam | – |
| Panjabi | pa | pa |
| Persian | fa | fa |
| Pfaelzisch | pfl | – |
| Piemontese | pms | – |
| Polish | pl | pl |
| Portuguese | pt | pt |
| Pushto | ps | ps |
| Quechua | qu | – |
| Raeto-Romance | rm | – |
| Romanian | ro | ro |
| Russian-Buriat | bxr | – |
| Russian | ru | ru |
| Rusyn | rue | – |
| Sanskrit | sa | – |
| Sardinian | sc | – |
| Saxon | nds | – |
| Scots | sco | – |
| Serbian | sr | sr |
| Serbo-Croatian | sh | – |
| Sicilian | scn | – |
| Sindhi | sd | sd |
| Sinhala | si | si |
| Slovak | sk | – |
| Slovenian | sl | sl |
| Somali | so | – |
| Shona | – | sn |
| South-Azerbaijani | azb | – |
| Spanish | es | es |
| Sundanese | su | – |
| Swahili | sw | – |
| Swedish | sv | sv |
| Tagalog | tl | tl |
| Tajik | tg | – |
| Tamil | ta | ta |
| Tatar | tt | – |
| Telugu | te | te |
| Thai | th | th |
| Tibetan | bo | bo |
| Tosk-Albanian | als | – |
| Turkish | tr | tr |
| Turkmen | tk | – |
| Tuvinian | tyv | – |
| Uighur | ug | ug |
| Ukrainian | uk | uk |
| Upper-Sorbian | hsb | – |
| Urdu | ur | ur |
| Uzbek | uz | – |
| Venetian | vec | – |
| Veps | vep | – |
| Vietnamese | vi | vi |
| Vlaams | vls | – |
| Volapük | vo | – |
| Walloon | wa | – |
| Waray | war | – |
| Welsh | cy | cy |
| Western-Mari | mrj | – |
| Western-Panjabi | pnb | – |
| Wu-Chinese | wuu | – |
| Yakut | sah | – |
| Yiddish | yi | – |
| Yoruba | yo | – |
| Yue-Chinese | yue | – |
| Undefined | und | und |