A comprehensive voice dataset for Hindko digit recognition

Abstract

Hindko is a language primarily spoken in Northwestern areas of Pakistan. Approximately eight million people speak the Hindko language. According to its native speakers, it is 7^th largest language of Pakistan and 2^nd largest language of Khyber Pakhtunkhwa. The Hazara region is the cultural hub of Hindko language. About 80% of the population in districts like Haripur, Abbotabad and Mansehra speak Hindko. The spoken content of Hindko covers a wide range of subjects, including religion, education, poetry, politics, theater, and more. Despite all this, Hindko lacks a voice recognition system that could enhance accessibility, preserve the language, and promote digital inclusion for its speakers. This paper presents a voice recognition dataset that consists of 17,597 voice samples, and is accessible to the public for academic and research purposes. The dataset consists of 20 Hindko digits ranging from 1 to 20 and all the voice samples are taken from the students and staff and faculty of Pak-Austria Fachhochschule Institute of Applied Science and Technology.

Specifications Table

Subject	Computer science, Machine Learning
Specific subject area	Computer Science, Artificial Intelligence, Natural Language Processing, Signal Processing, Machine Learning, Voice Recognition.
Type of data	Audio
Data collection	Dataset is acquired from the recording of the native speakers. These native speaker are the students and staff members of Pak-Austria Fachhochschule institute of Applied Sciences and technology.
Data format	.wav
Data source location	Pak-Austria Fachhochschule, Institute of Applied Sciences and Technology, Pakistan
Data accessibility	https://data.mendeley.com/datasets/yjhz8z7mv5/1
Related research article	None

1. Value of the Data

The Hindko voice dataset is beneficial for the researchers who are working on signal processing, natural language processing and especially those focusing on the speech recognition of Hindko language. The proposed dataset is valuable for all professionals engaged in natural language processing.

The proposed dataset has the followings cultural and linguistic values:

• •Educational Resource:

  The proposed dataset is beneficial for educators and students of Hindko language. It is valuable for developing learning aid, teaching materials, and other educational resources that enhance the understanding and appreciation of the language. The Hindko Voice dataset is also beneficial for creating educational tools for teaching Hindko numerals to learners. It can also be helpful for developing games and applications that aid in language learning.

• •Dialect Comparison:

  Hindko has six dialects that are Peshawari, Kohati, Awankari, Ghebi, Chacchi, and Hazara Hindko. This dataset can help compare how numbers are pronounced across different dialects or languages.

• •Preservation of Language Heritage:

  The Hindko language holds significant historical and cultural values. It is mostly spoken in the Hazara division of Khyber Pakhtunkhwa. The Hindko voice dataset helps preserve the language heritage by recording and documenting how numbers are spoken in Hindko and creates a valuable record of the language phonetic and phonological features. This dataset also aids in cultural preservation by maintaining the spoken form of Hindko, which helps maintain a connection to cultural practices, traditions, and knowledge that are often conveyed through language.

• •Text to Speech Systems:

  The proposed dataset is useful for developing text-to-speech systems. This dataset can help ensure that the system correctly pronounces numbers in Hindko. Accurate pronunciation of numbers is crucial for practical applications like virtual assistants or automated customer service systems.

• •Benchmarking:

  The proposed dataset can serve as a benchmark for evaluating the performance of different voice recognition systems on controlled and straightforward tasks.

2. Background

Hindko is a Pakistani language that is mostly spoken in northwestern areas of Pakistan. There are approximately 8 million native speakers of the Hindko language. According to their native speakers it is the 7^th largest language of Pakistan and the 2^nd largest language of Khyber Pakhtunkhwa [1]. The Hazara division is considered the cultural hub of Hindko language. 80% of the population in Hazara division speak Hindko [[2], [3]]. The spoken content of Hindko covers a wide range of subjects including religion, education, poetry, politics, theater, and many more [4].

Despite its large number of speakers, Hindko does not currently have an automated speech recognition system. The primary purpose of creating this dataset is to create automated speech recognition system for Hindko language. The dataset mainly focuses on 20 voice samples which consist of hinkdo digits from 1 to 20 [5]. As this is the first voice dataset of Hindko language so it is expected to motivate other researchers to explore this area further and create a perfect automated speech recognition system of Hindko.

3. Data Description

The dataset mainly focuses on 20 Hindko digits that are from 1 to 20. Approximately 300 individuals from the Pak-Austria Fachhochschule Institute of Applied Sciences participated in this project. We requested every individual to speak these 20 digits and send the recording via WhatsApp [6]. Each individual was asked to send three recordings in three different accents. We used Audacity audio editing software to refine and prepare the voice dataset. We imported each audio recording into Audacity software one by one and applied various audio modification techniques, such as noise reduction, normalizing volume, equalization, silence removal, pitch and speed adjustment, and format conversion [7]. Since the 20 Hindko digits were in a single raw audio file, we used the selection tool to carefully identify the points at which each digit needed to be split. Using the split function, we then created individual segment for each digit from the raw audio file. Next, we labeled each audio segment and saved it in a separate folder. Since each raw audio file were 20 digits, so 20 folders were created with a label from 1 to 20. Every audio segment is saved to their respective separate folder. The dataset consists of 17,597 samples and is 974MB in size. A bar chart (Fig. 1) shows the number of samples in each class. It can be observed that each class has a balanced number of samples, ranging between 800 to 950. Fig. 2 displays a pie chart showing the percentage of samples in each class. Each class contributes 4.6% to 5.4% of the total samples in the dataset.

Fig. 1.

Number of samples in each class of Hindko voice dataset.

Fig. 2.

Percentage of samples in each class of Hindko voice dataset.

4. Experimental Design, Materials and Methods

This study mainly focuses on the development of a Hindko voice dataset that opens the way for an automated speech recognition system for the Hindko language. To accomplish this task, we mainly focused on the 20 most commonly used Hinko digits, ranging from 1 to 20, and developed the Hindko voice dataset. Our experimental design includes audio sample collection, applying audio modification techniques to refining audio, splitting individual audio digit from raw audio files, labeling each audio segment, and maintaining separate folders for each segment class

The following experimental design were used in dataset creation:

• •Audio Sample Collection
• •Applying Audio Modification Techniques
• •Splitting Individual Audio Digits
• •Labeling Each Audio Segment
• •Maintaining Separate Folder for Each Segment Class.

4.1. Audio sample collection

The dataset consists of 20 digits and 17,597 samples. About 300 individuals participated in this study. All individuals were native speakers of Hindko language. We asked each individual to speak these 20 digits in one recording and send the recording via WhatsApp. Each individual was asked to send three voice recordings using three different accents.

4.2. Applying audio modification techniques

We used Audacity audio-editing software to refine and prepare the voice dataset. We imported each audio file to Audacity software individually and applied various audio modification techniques, including noise reduction, volume normalization, equalization, silence removal, pitch and speed adjustment, and format conversion.

• i)Noise reduction

  To improve the clarity of the audio, we applied noise reduction techniques that involved identifying and reducing background noise or static that might interfere with the quality of the data.

• ii)Volume Normalization:

  To ensure consistency across the dataset, we used normalization to adjust the volume levels of the recording. This normalization process helps in ensure that no single audio recording is disproportionately soft or loud compared to others.

• iii)Equalization:

  To enhance audio clarity, equalization was employed by adjusting the frequency bands. Equalization helps reduce unwanted frequencies that could affect the clarity and quality of the recordings and also emphasizes certain aspects of the voice recordings.

• iv)Silence Removal:

  To make the recordings more fluid we removed unwanted silences and pauses.

• v)Pitch and Speed Adjustment:

  In some cases, we adjusted the pitch and speed of the audio segments to match specific requirements.

4.3. Splitting individual audio digits

Since the 20 Hindko digits were recorded in a single raw audio file, we carefully identified the points where each digit needed to be split using the selection tool. Then, by using split function, we created individual segments for each digit from the raw audio file.

4.4. Labeling of every audio segment

We labeled each audio segment based on the digit it represented, with labels ranging from 1 to 20. For example, audio segments containing the digit '1′ were labeled as ‘1’, audio segments containing the digit ‘2’ were labeled as ‘2’, and so on, up to the digit ‘20’. Each labeled audio segment was then saved in a corresponding folder named after the label. For instance, audio segments labeled with ‘1’ were saved in a folder named ‘1’, those labeled with ‘2’ were saved in a folder named ‘2’, and so on, up to folder ‘20’. This system ensured that each audio segment was organized according to its assigned label for efficient storage and retrieval.

4.5. Maintaining separate folder for every segment class

We maintained a separate folder for each segment class and each audio segment was saved in its respective folder. As there were 20 digits in one raw audio file, 20 folders were created with a label from 1 to 20.

4.6. Data analysis and visualization

We applied audio visualization techniques to understand the patterns of the audio samples and to check the uniqueness of each digit class [8]. For visualization, we used a popular Python package called Librosa, which is used for analyzing and processing audio and music data. The visualization techniques we employed include Waveform and Spectrogram [9]. Each audio sample was represented in its waveform format, as shown in Fig. 3. The waveform shows the amplitude of the audio signal over time, where the x-axis represents time and the y-axis represents amplitude. This visualization provides insight into the structure of the audio signal, allowing us to see the variations in amplitude that characterize different digits. Then, we proceeded to compute the spectrograms for each digit class, as shown in Fig. 4. The spectrogram is a visual representation of the frequency spectrum of the audio signal over time, with the x-axis representing time, the y-axis representing frequency, and the color intensity indicating the strength of a particular frequency at a given time [10]. The spectrogram provides a clearer understanding of the frequency content of the audio, which is crucial for distinguishing between different digit classes based on their unique spectral patterns. From these visualizations, we can conclude that each representation, whether waveform or spectrogram, provides a unique and distinct depiction of the individual digits. This uniqueness and distinctiveness in the representation of audio samples make it easier for any machine learning or deep learning model to effectively learn and differentiate these unique features.

Fig. 3.

Wave form of each class sample.

Fig. 4.

Spectrogram of each class samples.

Limitations

Not applicable.

Ethics Statement

All individuals who contributed to the creation of these voice recognition dataset provided their informed permission. Each person included in this dataset gave their permission by signing a consent form. It is also important to stress that the gathered audio reveal no personally identifiable details about the subjects. The ethical approval was not required for this study.

CRediT Author Statement

Tanveer Ahmed: Conceptualization, Formal analysis, Data curation, Writing – original draft, Visualization, Project administration. Maqbool Khan: Writing – original draft, Writing – review & editing, Project administration. Khalil Khan: Writing – original draft, Writing – review & editing, Project administration. Ikram Syed: Writing – original draft, Writing – review & editing, Project administration. Syed Sajid Ullah: Writing – original draft, Writing – review & editing, Project administration.

Data Availability

• Mendeley DataHindko Voice Dataset (HVD) (Original data).