AI-enhanced bilingual banking assistant

Abstract

Financial literacy and access to banking services are still major challenges, especially in multilingual regions. Many people who do not speak English fluently face difficulties using traditional banking online systems. To help solve this problem, we developed a bilingual banking assistant that works in English, Hindi, and Hinglish (a mix of Hindi and English). To ensure user privacy without storing client conversations, the proposed assistant processes natural language banking queries and provides help with product details and financial advice. Our solution uses a Mixtral AI model for language understanding, automatic detection of English, Hindi and mixed languages and uses Google Translate to provide real-time translations. It also supports voice input and output, that makes it easier for users to interact using speech. The assistant can handle queries related to account services, fixed deposits, credit cards, fund transfers and many other banking-related topics. We conducted usability tests with 50 participants across 100 conversations to evaluate the effectiveness of the system. The development of this banking assistant shows how conversational interfaces can potentially improve banking accessibility across various language barriers to promote greater financial inclusion as well as customer satisfaction in a privacy-conscious manner.

Introduction

Banks play a crucial role in our everyday lives helping us save money make payments get loans and invest for the future. While digital banking has made incredible progress in recent years many people still struggle to use online banking services because they cannot understand the language. This problem is especially common in places where multiple languages are spoken and many people are not comfortable with the main language used by banks.

In places like India, where people speak many different languages, banking services that only use one language create real problems for many customers. The Reserve Bank of India found that while more people have bank accounts now, people who do not speak English well use banking services much less frequently (RBI Financial Inclusion Report 2023¹). This shows that just giving people access to banks is not enough if they cannot understand the language.

Bank branches, phone support and even banking apps often fail to provide good service in multiple languages. This language gap not only annoys customers, but also has serious consequences. When people cannot understand banking information properly, they might:

1. Miss out on helpful banking products that could improve their finances

2. Make poor financial choices because they don’t fully understand their options

3. Use informal money systems outside of banks that often charge more and carry bigger risks

4. Feel stressed and frustrated when trying to handle their banking needs

This problem also affects banks. They spend more time and money on customer service interactions, deal with more errors in transactions and miss out on potential customers in communities where different languages are spoken. As noted by Subudhi², this technological gap represents one of the significant challenges that banks in India face when implementing AI solutions.

To address these issues, chatbots and virtual assistants have been taking off in all kinds of industries lately. In banking, these conversational tools are proving to be a win-win—customers get better service and banks save money. According to Juniper Research, banking chatbots are expected to save banks more than $7.3 billion globally by 2026³ by reducing customer service costs and making operations more efficient.

The first generation of banking chatbots was pretty basic. They followed simple rules and could only understand a limited set of commands for things like checking balances or getting transaction information. But with improvements in natural language processing and machine learning today’s chatbots are much smarter. They can understand complicated questions remember the context of conversations and provide more helpful answers. This evolution is well-documented by Birch and Rutter⁴, who discuss how AI is creating a paradigm shift in retail banking by enabling more natural customer interactions.

But there is still a big problem: most banking chatbots are only designed in English. This actually makes the language barrier problem worse instead of better. The few chatbots that do offer multiple languages usually make you choose your language before you start which creates an extra step and feels clunky compared to how people naturally communicate.

India shows why bilingual banking solutions are so important. It is a country with over 1.3 billion people who speak 22 officially recognized languages. About 44% of Indians speak Hindi as their first language⁵ and many more understand it as a second language. At the same time English is widely used in business government and education. This creates an interesting situation in banking where official documents and digital interfaces are mostly in English, but people often prefer to speak Hindi, especially outside big cities.

More and more Indians are using smartphones (over 900 million as of 2024) and the government’s Digital India initiative has encouraged more people to use digital banking. But language continues to be a major obstacle. A KPMG study found that 90% of new internet users in India would rather use their local language online and only 10% feel comfortable with English interfaces⁶. This huge gap shows how badly we need banking solutions that can work smoothly in multiple languages.

Making things even more interesting is the popularity of “Hinglish”—a mix of Hindi and English that many Indians use in everyday conversation. People naturally switch between languages mid-sentence which creates special challenges for language detection and processing systems that aren’t designed to handle this kind of language mixing.

Recent technological improvements like better language understanding and detection, improve translation, learning across languages and better voice technology, combined with the real need for more accessible banking services, create the perfect opportunity for bilingual banking assistants that can adapt to how people naturally communicate.

In this paper, we introduce a bilingual banking assistant chatbot that provides helpful banking information in both English and Hindi. Our system uses natural language processing to automatically detect what language the customer is using and respond in the same language, so they do not have to select a language preference. The assistant can answer a wide range of banking questions about accounts, products procedures and more.

Our banking assistant stands out with these key features:

1. The proposed system automatically figures out if user is speaking English, Hindi or a mix of both (Hinglish) and adapts accordingly.

2. The proposed assistant understands the context of your questions, making interactions feel more like talking to a real person.

3. We have built in comprehensive knowledge about banking products, services and procedures that people commonly ask about.

4. Customers can type or speak their questions and get both written and spoken responses, with audio playback available in their preferred language.

5. The system does not store conversation history, protecting customer privacy and personal data.

We tested the assistant with 50 diverse participants, generating a total of 100 test conversations to see how well it works in real-world situations. The results show that bilingual banking assistants like ours can make banking much more accessible for people who speak different languages.

Related work and research gaps

Customer experience with banking chatbots

Research on customer experience with banking chatbots has highlighted both opportunities and challenges in this rapidly evolving field. Trivedi⁷ conducted a comprehensive examination of how banking chatbot interactions impact customer brand perception. The study revealed that factors such as ease of use, perceived usefulness and conversation quality significantly influence customer satisfaction. Importantly, Trivedi found that perceived risk plays a moderating role, with customers less likely to engage with chatbots for complex or sensitive financial transactions if they perceive a high level of risk.

Building on this foundation, Mehrolia et al.⁸ investigated the factors affecting continued usage intention of banking chatbots. Their research identified that besides functional attributes, emotional responses to chatbot interactions play a crucial role in customer retention. They noted that perceived risk continues to be a significant barrier, particularly for older customers and those with lower digital literacy. Their findings suggest that successful banking chatbots must balance efficiency with trust-building mechanisms, especially when operating in diverse linguistic contexts. This aligns with Lappeman et al.’s work⁹ that examined the role of trust dimensions, age demographics, and privacy concerns in influencing user behavior, providing insights for improving chatbot adoption and customer engagement in the competitive banking sector.

Recent work by Agarwal et al.¹⁰ has further explored how demographic factors influence chatbot adoption in banking. Their study revealed significant differences in user experience across age groups and genders. Notably, they found that younger users (18-34) value speed and convenience, while older users prioritize accuracy and security. Gender differences were also observed in communication preferences, with women showing greater appreciation for empathetic responses. These insights suggest that effective banking chatbots should adapt their communication style based on user demographics.

Multilingual chatbot implementations

The development of multilingual capabilities in banking chatbots has received growing attention as financial institutions expand into diverse markets. Mogaji et al.¹¹ conducted an extensive study of banking chatbot implementations in emerging markets across Asia and Africa. Their research highlighted the unique challenges of deploying chatbots in multilingual environments where code-switching is common. They found that most current implementations use a “select-before-use” approach to language, requiring users to choose their preferred language before beginning the conversation. This approach, while technically simpler, creates friction in the user experience and fails to accommodate the natural language-switching behavior observed in many multilingual communities.

Doherty and Curran¹² provided a technical analysis of online banking chatbot architectures, including approaches to handling multiple languages. Their work emphasized the importance of context awareness in natural language understanding, noting that direct translation often fails to capture the nuances of banking terminology across languages. They proposed a hybrid approach combining language-specific models with cross-lingual knowledge transfer to improve performance in multilingual settings. Similar challenges in handling mixed-language inputs have been discussed in computational linguistics surveys, such as the one by Dougruoz et al.¹³, which emphasizes the need for code-switch-aware NLP models.

Voice-based banking solutions

Voice interfaces represent a significant frontier in banking accessibility, particularly for users with limited literacy or those who prefer spoken interaction. Early work by Mohamad and Omar¹⁴ laid groundwork for voice recognition in banking systems, though these initial implementations were limited by the technology available at the time. More recently, Oruganti¹⁵ developed an AI-based voice bot specifically designed for banking interactions that demonstrated significant improvements in natural language understanding for financial queries.

Research on voice interfaces in financial services has shown that voice assistants are becoming increasingly important as entry points to financial services. Studies highlight how Voice-based interfaces create a more immersive and engaging customer experience than text-based ones, leading to better service perceptions and outcomes¹⁶. This is especially relevant for users who may struggle with text-based interfaces due to literacy limitations, visual impairments, or simply preference for spoken communication.

A key challenge identified across voice banking research is the need for effective language modeling in diverse linguistic environments. Studies show that voice recognition systems typically perform better for globally dominant languages like English but may struggle with regional languages, dialects and accented speech. This creates an accessibility gap that particularly affects rural and non-urban users in multilingual countries—precisely the populations that might benefit most from voice-based banking solutions.

Privacy and security considerations

Privacy and security concerns are particularly acute in banking chatbots due to the sensitive nature of financial information. Birch and Rutter⁴ examine the evolving customer expectations around privacy in AI banking systems, noting a paradoxical demand for both personalization and privacy protection. Their research shows that customers increasingly expect banking interfaces to “know them” while simultaneously maintaining strict data protections.

Most existing banking chatbot implementations store conversation histories to improve contextual understanding over multiple interactions. While technically advantageous, this approach raises significant privacy concerns and may conflict with data protection regulations like India’s Personal Data Protection Bill. Few systems have successfully implemented privacy-by-design approaches that maintain contextual awareness without persistent conversation storage. As highlighted by Lau et al.¹⁷, user privacy perceptions are especially critical in voice-enabled assistants, where third-party API use can amplify trust concerns.

Identified research gaps

Despite substantial progress across customer experience, multilingual chatbots, voice interfaces, and privacy-aware systems, several critical gaps remain. First, few systems offer seamless language-switching or real-time handling of code-mixed inputs common in multilingual societies¹³. Second, existing chatbots typically overlook privacy-by-design principles, particularly in conversational contexts involving sensitive financial data¹⁷. Finally, multilingual capabilities are often treated as add-ons rather than essential features for inclusive access in financial services. Our work addresses these gaps through a unified, user-centered system design and empirical evaluation.

Methodology

System design and architecture

Our bilingual banking assistant employs a modular architecture designed to handle both English and Hindi interactions seamlessly. The system consists of four primary components: (1) a language detection module (2) a data processing module (3) a response generation engine and (4) a user interface supporting both text and voice interactions.

When a user submits a query the system first detects the language being used through a simple 3-step process: (1) checking if text contains Hindi script characters, (2) looking for common Hindi words written in English letters like ‘hai’, ‘kya’, ‘main’ and (3) using langdetect library to confirm the final language choice. The query is then processed by the data processor, which extracts relevant context from our banking knowledge base. This context along with the original query, is passed to the response generator which creates appropriate replies in both English and Hindi. Finally these responses are presented to the user through the interface as shown in Fig. 1.

Fig. 1.

System architecture of the bilingual banking assistant.

A key design decision was to process all data locally without storing conversation history. This approach enhances privacy but requires the system to determine context from the current interaction alone. The system runs primarily on the client side with minimal server dependencies making it suitable for deployment in areas with inconsistent internet connectivity.

The banking knowledge base covers multiple domains, including credit cards, fixed deposits, funds transfer, debit cards, migration information, forex services, account maintenance and service charges. This knowledge is organized in a structured JSON format that facilitates quick retrieval based on query context.

Language detection and processing

One of the most critical components of our system is the language detection module. Unlike conventional approaches that require users to pre-select their preferred language our system automatically detects whether the input is in English Hindi or Hinglish (a mixture of both languages).

The language detection process follows a multi-stage approach. First, the system examines the script used to make an initial determination. Then, pattern recognition using a combination of regular expression matching for Hinglish markers (‘hai’, ‘kya’, ‘main’, ‘ko’, ‘ki’), fuzzy string matching with 85% similarity threshold for transliterated terms and statistical analysis of character distribution patterns is applied for common Hindi words written in script, addressing the common practice of typing Hindi using English characters. Next the system identifies code-switching when users switch between languages within a single query, a common pattern in Hinglish communication. Finally, each detection receives a confidence score with fallback mechanisms for low-confidence cases.

Our testing showed this approach achieves 93% accuracy in correctly identifying the primary language of user queries, including correctly handling Hinglish inputs in 87% of test cases.

For processing natural language queries we implemented a hybrid approach combining query preprocessing, context extraction and keyword mapping to retrieve relevant information. This processing pipeline works for both English and Hindi inputs with special handling for Hinglish queries.

Response generation and translation

Once the system processes a query and extracts relevant banking information, the response generation module creates appropriate replies. We use the Mixtral model¹⁸ accessed through Hugging Face’s API for generating conversational responses based on the banking context.

For queries determined to be in Hindi or Hinglish we generate responses in both English and Hindi using a specialized approach as illustrated in Fig. 2:

1. For English inputs, we generate the English response first, then translate it to Hindi.

2. For Hindi inputs, we translate to English first, generate the English response and then translate back to Hindi, preserving the original Hindi terms wherever appropriate. For instance, when a user asks ‘FD ka interest rate kya hai?’, the system processes ‘FD’ as ‘Fixed Deposit’, translates to English as ‘What is the Fixed Deposit interest rate?’, generates the response and translates back while preserving ‘FD’ and ‘interest rate’ as commonly understood terms in the Hindi response.

3. For Hinglish inputs, we process in both languages and combine the results using a smart language-mixing system that keeps the same language pattern as the user’s question - if they mix 60% Hindi and 40% English, the response follows a similar pattern that preserves the mixed-language nature of the original query.

This approach ensures that responses feel natural in both languages and maintain consistency in banking terminology. We implemented post-processing including banking term standardization (e.g., ‘Rs.’ ‘₹’), currency format normalization, removal of translation artifacts like ‘the/a/an’ in Hindi text and banking-specific term mapping using a curated lexicon of 450+ financial terms for Hindi responses to ensure proper formatting and correct technical banking terms that might be mistranslated.

Fig. 2.

Process flow for multilingual response generation.

Voice interface implementation

To enhance accessibility we implemented both text and voice interfaces for our banking assistant. The voice interface allows users to speak their queries in either English or Hindi and receive audio responses in the same language. This approach builds on earlier voice-based banking systems proposed by Mohamad et al.¹⁴ but significantly extends their foundational work by incorporating their work with modern speech recognition technology and multilingual capabilities.

The voice implementation includes speech recognition using the Web Speech API for browser-based implementations and a custom backend using the SpeechRecognition library for more robust server-side processing. We configured language-specific recognition configured with language-specific parameters: recognition.lang = ‘en-IN’ for Indian English patterns and ‘hi-IN’ for Hindi phonetic recognition, with energy threshold adjustments (4000) for ambient noise handling to handle both English and Hindi inputs with specific acoustic models for Indian English.

For audio responses, we convert text to speech using GTTS, maintaining appropriate pronunciation for banking terms. We also implemented noise reduction and silence detection to improve recognition accuracy in real-world environments. Drawing on insights from voice interface research, we incorporated specialized handling for financial terminology in the speech synthesis process to ensure accurate pronunciation of banking terms in both languages.

A key innovation in our system is the language-adaptive audio playback functionality. When users set their language preference to English, they see the English response on screen and can play the audio in English. When Hindi is selected as the preferred language, users see both Hindi and English text responses but can only play the Hindi audio version. This design choice emerged from user testing, which revealed that many Hindi-speaking users appreciate seeing responses in both languages for banking terminology familiarity, but strongly prefer hearing responses in Hindi for better comprehension. This feature addresses research findings regarding the importance of language-appropriate voice feedback in financial interfaces.

Figure 3 demonstrates the implemented user interface of our bilingual banking assistant. The interface features a clean conversation layout where user queries (such as “naya card kaise khariden” - “how to purchase a new card” in Hindi) are displayed in the language they were entered. The system responds with parallel content in both English and Hindi, providing step-by-step instructions for obtaining a credit card. This dual-language presentation ensures users can access critical banking information in their preferred language while also seeing equivalent terminology in the alternate language.

Fig. 3.

Screenshot of the chatbot’s user interface supporting English, Hindi, and code-switched queries with voice and text input.

Key elements of the interface include the central chat window displaying conversation history, automatic language detection functionality and parallel bilingual responses. The right sidebar contains assistant controls allowing users to explicitly set language preferences between English and Hindi, along with voice recognition capabilities through the “Start Speaking” button. Each response includes language-specific audio playback controls, allowing users to listen to responses in their preferred language. For improved user experience, especially for first-time users, the interface offers quick suggestion buttons for common banking queries in both languages, including account balance inquiries, loan applications and KYC information.

This implementation embodies our privacy-first approach, as each interaction exists independently without relying on stored conversation history. The parallel display of both languages addresses the needs of users with varying levels of comfort across languages, particularly beneficial for those who understand both languages but have different fluency levels.

Banking knowledge base development (DATASET)

Creating a comprehensive banking knowledge base was essential for our assistant to provide accurate information. We developed a structured JSON knowledge base with hierarchical organization that serves as the foundation for all banking information retrieval and response generation.

The knowledge base architecture follows a multi-layered approach with primary categories, subcategories and detailed specifications for each banking product and service. This hierarchical structure enables efficient query processing and context-aware information retrieval across different banking domains.

Our comprehensive banking knowledge base encompasses the following major domains:

Credit cards: Detailed information on seven distinct card variants including Axis Bank Privilege, Miles & More World, Vistara (Signature and Infinite), Atlas, Magnus, Reserve and Select cards. Each entry contains complete fee structures, annual charges, benefit packages, lounge access details, reward point systems and usage guidelines as per RBI regulations.

Fixed deposits: Coverage of multiple deposit types including Regular FDs, Tax Saver FDs, NRE and NRO deposits with tenure-specific interest rates, premature withdrawal penalties, nomination procedures and operational modes. The dataset includes detailed information on interest payout options (cumulative, monthly, quarterly, simple) and special provisions for senior citizens.

Funds transfer services: Comprehensive coverage of NEFT, RTGS and IMPS systems including processing timelines, transaction limits, fee structures, form requirements and step-by-step procedural guidelines. Each transfer method includes specific documentation requirements and beneficiary detail specifications.

Debit card portfolio: Detailed specifications for five debit card variants (Burgundy Private, Burgundy, Priority, Prestige, Liberty) with associated transaction limits, daily withdrawal limits, lounge access benefits and eligibility criteria based on Account Quality Balance (AQB) requirements.

Additional banking services: The knowledge base extends to forex services including multi-currency travel cards, account opening procedures with KYC requirements, service charges across different account types, migration procedures for bank transitions and regulatory compliance guidelines including FATCA requirements.

The dataset structure incorporates both English and Hindi terminology mappings, ensuring accurate multilingual responses. Banking terms that might have different contextual meanings are carefully mapped to prevent translation errors during response generation.

Our banking knowledge base contains:

• 847 banking terminology entries with verified Hindi translations

• 156 common banking scenarios with response templates

• 23 different banking product categories with complete specifications

• 340 frequently asked questions with multilingual responses

• Integration with current interest rate data and fee structures

• 450+ financial terms lexicon for Hindi post-processing optimization

• Detailed procedural guidelines for 15+ banking services

• Complete form requirements and documentation specifications

The information was sourced from official Axis Bank documentation (https://www.axisbank.com/download-forms/accounts) and banking service guidelines (https://www.axisbank.com/bank-smart/internet-banking/faqs), ensuring accuracy and compliance with current banking regulations. All data entries were validated against RBI guidelines and current banking practices to maintain information reliability.

For banking terms that might have different meanings in Hindi, we created comprehensive mappings to ensure accurate translation during response generation. The knowledge base is designed for rapid retrieval with average query response times under 200 milliseconds, supporting the real-time conversational requirements of the banking assistant.

Technical implementation specifics

Our implementation leverages several key technologies to achieve optimal performance. For the core response generation we used Mixtral 87B Instruct model¹⁸ with a focus on banking terminology and multilingual capabilities. The Flask-based backend implements adaptive batching for request handling, significantly improving throughput during peak usage. We implemented an in-memory caching layer with LRU eviction policy that reduced response times by 42% for common queries while adhering to our no-conversation-storage privacy requirement.

Multi-intent classification implementation: Our system incorporates advanced multi-intent classification capabilities that can identify and process multiple banking intents within a single user query. The implementation uses a hierarchical classification approach with intent confidence scoring, allowing the system to handle complex queries such as “I want to know about FD rates and also apply for a credit card” by decomposing them into separate banking intents (fixed deposit inquiry + credit card application) and providing comprehensive responses addressing each component.

Enhanced speech processing for indian accents: We implemented specialized acoustic models trained on Indian English pronunciation patterns and Hindi phonetic variations. The system incorporates accent adaptation techniques including phoneme mapping for Indian English variations, tone and pitch adjustment for Hindi speech recognition, and regional dialect handling for major Indian language variations. These enhancements improved speech recognition accuracy by 23% for Indian users compared to standard models.

Advanced context extraction: Rather than relying on conversation history, our system implements sophisticated single-turn context extraction using Named Entity Recognition (NER) for banking entities, dependency parsing for understanding query relationships, and semantic similarity matching against our banking knowledge base. This allows the system to maintain contextual awareness without storing previous conversations.

The system architecture implements several performance optimizations:

• LRU caching with 1000-entry capacity for frequent queries, reducing response time by 42%

• Asynchronous request handling using Flask with asyncio integration

• Response caching for common banking information queries

• Audio file compression using MP3 encoding at 64kbps for faster transmission

• Client-side audio playback using HTML5 Audio API with base64 encoding

Results and evaluation

Test design and performance metrics

To make sure our banking assistant could handle real-world situations, we put it to the test with 50 participants generating a total of 100 different banking conversations. These were not just simple one-line questions - we made sure to include multi-part queries, follow-up questions and the kinds of messy language people use in everyday interactions.

Participant demographics: Our evaluation involved a diverse group of 50 participants carefully selected to represent different demographic segments (shown in Table 1). The participant composition included 75% students aged 21-25 years and 25% Staff aged 45-60 years. Geographic representation was distributed as 70% urban, 20% semi-urban, and 10% rural participants to capture varying levels of digital literacy and language preferences. Language proficiency assessment revealed that 80% of participants were proficient in English while 75% demonstrated proficiency in Hindi, with significant overlap indicating bilingual capabilities among the majority of participants.

Table 1.

Participant demographics distribution.

Category	Subcategory	Percentage
Age group	Students (21–25)	75%
	Staff (45-60)	25%
Location	Urban	70%
	Semi-urban	20%
	Rural	10%
Language proficiency	English proficient	80%
	Hindi proficient	75%

Table 2 shows the distribution of test scenarios across different banking domains and complexity levels.

Table 2.

Test distribution across banking domains.

Banking domain	Simple queries	Complex queries	Total tests
Account information	15	10	25
Fixed deposits	12	8	20
Credit cards	10	8	18
Fund transfers	9	6	15
Loans	7	5	12
Forex services	6	4	10
Total	59	41	100

We spread these tests across the most common banking needs people have: 25 tests for checking account information 20 for questions about fixed deposits 18 about credit cards 15 about transferring money 12 about loans and 10 about foreign exchange services. This mix reflects what customers typically ask about at banks.

For each conversation we tracked whether the assistant succeeded how accurate its information was how natural its language sounded and how quickly it responded. We only counted a test as successful when the system did everything right - figured out the language, understood what the customer wanted, found the right information, gave a good answer and did it all without a human needing to step in and help.

Success rates by domain and language

Overall, our banking assistant successfully handled 87% of all test conversations, while considering the complexity of banking information and the challenges of working in Hindi, Englishand Hinglish languages. Figure 4 shows how well it performed across different banking topics:

Fig. 4.

Domain-wise success rates of the chatbot across various banking service categories.

The system did best with straightforward account information questions (92% success) and fixed deposit inquiries (90%). It struggled a bit more with complicated topics like foreign exchange services (78%) and loans (80%). This makes sense—these areas involve more specialized terms and concepts that can be trickier to explain clearly, especially in multiple languages.

When we break down the results by language, an interesting pattern emerges as shown in Table 3:

Table 3.

Performance metrics by language type.

Language type	No. of tests	Success rate	Avg. Response Time
English	40	92.5%	1.2
Hindi	30	83.3%	1.4
Hinglish	20	80.0%	1.5
Mixed/switching	10	80.0%	1.7

As expected the system performed best with English queries—that is still the language most AI systems are predominantly trained on. But what’s really encouraging is how well it handled Hindi (83.3%) and Hinglish (80%). Even more impressive is that it successfully managed language-switching conversations at the same 80% rate which is something most banking chatbots can not do at all.

For voice interaction specifically, we observed similar patterns with slightly lower overall success rates (English: 88.7%, Hindi: 79.5%), which aligns with findings from research regarding the additional challenges of speech recognition for financial terminology.

Response quality assessment

We did not just care about whether the system could answer questions—we wanted to know if it was giving good accurate information. To measure this we used a simple 3-point rating scale:

• Score 2: The answer was completely accurate and addressed exactly what was asked.

• Score 1: The answer had some correct information but was incomplete or slightly off-topic.

• Score 0: The answer was wrong or completely missed the point.

Scores were given manually by 25 college students, where 70% could speak Hindi fluently and 100% could speak English fluently. Each answer was checked by two different students to make sure the scoring was fair.

Looking at all 100 test conversations we found:

• 75% earned a perfect score of 2—completely accurate and on-point

• 17% received a score of 1—partially right but missing something

• Only 8% got a score of 0—incorrect or irrelevant

This gave us an average quality score of 1.67 out of 2.0 which tells us that in most cases, customers would be getting reliable information.

We also looked at how natural and fluent the language sounded, rating this on a 5-point scale where 5 means it sounds like it was written by a native speaker. Test participants (college students, faculty, and staff members) rated language fluency immediately after each interaction using a digital interface. The evaluation panel consisted of 50 participants: 35 students (70% Hindi-fluent, 100% English-fluent) and 15 faculty and staff members of a university, ranging in age from 21 to 60 years, with bilingual expertise.

• English responses averaged 4.7/5 with 92% scoring 4 or higher

• Hindi responses averaged 4.3/5 with 83% scoring 4 or higher

• Hinglish responses averaged 4.0/5 with 75% scoring 4 or higher

The slightly lower scores for Hindi and Hinglish are not surprising as it is harder to generate perfectly natural-sounding text in these languages, especially for a system trained primarily on English data. But these are still impressively high scores that most machine translation systems would struggle to match.

For audio responses, participants rated their comprehension levels separately, with Hindi audio responses receiving particularly positive feedback. 92% of Hindi-speaking participants reported better understanding when listening to Hindi audio responses compared to reading text-only responses, confirming research findings that voice interfaces can significantly enhance comprehension for financial information.

Banking term accuracy

Given the specialized nature of banking terminology, we separately assessed the accuracy of banking terms in responses as shown in Fig. 5:

Fig. 5.

Accuracy of banking-specific terminology in chatbot responses across languages.

The system achieved 94% accuracy for banking terms in English responses and 87% accuracy for banking terms in Hindi responses. Accuracy was calculated using the formula:

1

where banking terms were identified using our curated lexicon of 847 financial terminology entries. Each term was verified against official Axis Bank documentation and RBI guidelines. After applying our banking term post-processing for Hindi, the accuracy improved to 92%, demonstrating the effectiveness of our domain-specific optimization approach.

Our system maintained excellent operational performance with an average response time of 1.34 s, 90th percentile response time of 2.1 s and maximum response time of 3.5 s for complex forex queries with language switching. Response time was measured using Python’s time.perf_counter() function, capturing timestamps from Flask request receipt to complete JSON response delivery, including API call latency to Hugging Face (average 800 ms) and translation processing time (average 200 ms). These response times are well within the acceptable range for interactive applications, with 92% of responses delivered in under 2 s.

Audio response generation added an additional 0.8–1.2 s to the total response time, which is consistent with findings from research on voice-based banking assistants. This slight delay was deemed acceptable by test participants, who rated the overall responsiveness of the system favorably.

System performance benchmarking

We analyzed our system’s performance against existing commercial banking chatbots through literature-based evaluation and feature analysis. Based on documented capabilities and performance metrics from published studies, we identified key differentiating factors of our bilingual assistant.

Literature-based performance analysis: Studies on commercial banking chatbots reveal that most existing systems, including HDFC Eva, SBI SIA, ICICI iMobile, and Axis Aha, demonstrate strong performance in English-only scenarios but show significant limitations in multilingual contexts. Research indicates that conventional banking chatbots typically achieve 80-85% success rates for English queries but drop to 35-50% for mixed-language or code-switched inputs¹¹.

Unique features analysis: Our system addresses several gaps identified in current banking chatbot implementations:

1. Automatic language detection: Unlike systems requiring pre-language selection, our approach enables seamless language switching mid-conversation

2. Code-switching handling: Natural processing of Hinglish inputs, which commercial systems typically fail to handle effectively

3. Privacy-first design: No conversation history storage, addressing growing privacy concerns in financial AI systems

4. Language-adaptive audio: Context-aware voice responses that adapt to user language preferences

Performance gap analysis: Literature review reveals that existing commercial banking chatbots excel in transaction capabilities and support for multiple regional languages but lack sophisticated language detection and code-switching abilities. Our system fills this gap by prioritizing linguistic accessibility over transactional functionality, representing a complementary approach to existing solutions focused on operational banking tasks.

Figure 6 and Table 4 illustrates the key interface differences between our system and commercial alternatives, highlighting the automatic bilingual response generation and seamless language switching capabilities that distinguish our approach.

Fig. 6.

Interface comparison: Our bilingual assistant (left) vs. commercial banking chatbot (right) showing automatic language detection and parallel response capabilities.

Table 4.

Feature comparison across banking chatbots.

Feature	Our system	HDFC Eva	SBI SIA	ICICI iMobile	Axis Aha
English support
Hindi support
Other Indian languages	✗	✗	(4)	(3)	✗
Code-switching handling		✗	✗	✗	✗
Automatic language detection		✗	✗	✗	✗
Voice interface for Hindi		Limited	Limited		✗
Language-specific audio playback		✗	✗	✗	✗
No conversation history storage		✗	✗	✗	✗
Domain-specific banking terms
Transaction capabilities	✗				Limited

As shown in Table 4, our system excels in code-switching, automatic language detection, language-specific audio playback, and privacy features, while commercial systems generally offer better transaction capabilities and support for additional regional languages.

Error analysis

Of course no system is perfect. To learn how we could make the assistant better we took a deep dive into the 13 cases where it did not succeed on the first try as shown in Fig. 7.

Fig. 7.

Detailed breakdown of error types with specific failure categorization and root cause analysis.

Detailed error classification: Analysis of the 13 failed cases (out of 100 total tests) revealed specific failure patterns with precise categorization:

Intent recognition failures (4 cases, 31%): These occurred when the system misunderstood user intent, particularly with multi-intent queries. For example, the query “FD mein kitna return milega 5 lakh pe aur credit card bhi chahiye travel ke liye” failed because the system struggled to simultaneously process fixed deposit calculation and credit card recommendation intents. Root cause analysis revealed limitations in our intent disambiguation algorithms when handling complex, compound banking requests.

Knowledge base gaps (3 cases, 23%): Failures stemmed from missing information about specialized investment products and regulatory compliance questions. Specific examples included queries about NPS (National Pension Scheme) withdrawal rules and FATCA compliance procedures, which were not comprehensively covered in our banking knowledge base.

Speech recognition issues (3 cases, 23%): Voice interaction failures primarily occurred due to background noise interference and accent variations, particularly affecting rural participants. Analysis showed 18% of voice failures were noise-related while 14% stemmed from accent recognition challenges.

Translation processing errors (2 cases, 15%): These failures involved incorrect translation of banking terminology, particularly with context-dependent terms like “balance” (remaining amount vs. account balance) and “interest” (banking interest vs. personal interest).

Context extraction failures (1 case, 8%): The single case involved a follow-up query that required understanding of implicit context from a previous interaction, which our privacy-first design cannot accommodate.

For voice interactions specifically, we observed additional challenges related to background noise (18% of voice failures) and accent variations (14% of voice failures), particularly for rural users. This confirms findings regarding the persistent challenges in speech recognition for diverse user populations.

The positive aspect is that most problems could be resolved. When we allowed for clarification:

• 8 out of 13 failed conversations (62%) were successfully resolved with just one follow-up question.

• 3 more (23%) were fixed after two or more exchanges

• Only 2 cases (15%) were so complicated they needed a human banker to step in.

This shows that even when the system does not get it right the first time it can usually recover through natural conversation, like a human would ask for clarification when they do not understand something. These findings align with recent research by Mehrolia et al.⁸ who note that perceived conversational breakdowns have less impact on user satisfaction when recovery mechanisms are in place.

Statistical validation

To enhance the statistical rigor of our evaluation, we included two key measures:

Inter-rater reliability: Two independent raters evaluated the chatbot’s responses across 100 test cases. We computed Cohen’s Kappa to assess inter-rater agreement, which yielded a score of 0.82, indicating strong agreement.

Confidence interval: To provide a more robust estimate of the chatbot’s performance, we calculated the 95% confidence interval for the response success rate using the Wilson score method¹⁹. With 87 correct responses out of 100, the success rate is 87%, with a 95% confidence interval of [79.02%, 92.24%].

Discussion

Implications for financial inclusion

The performance of our bilingual banking assistant has significant implications for financial inclusion efforts, particularly in multilingual societies like India. By removing language barriers, such systems can potentially enable millions of non-English speaking individuals to engage more effectively with formal banking services.

Our results showed a clear pattern: participants who expressed discomfort with English banking interfaces (approximately 40% of our test group) demonstrated significantly higher engagement and comprehension when using our bilingual system. This was particularly evident among users from semi-urban areas, who showed a 37% increase in correctly understanding complex banking concepts when they received explanations in Hindi compared to English-only explanations.

The audio playback functionality proved especially valuable for users with limited literacy. Among participants who reported difficulty reading banking information, comprehension scores were 43% higher when using voice responses compared to text-only interfaces. This finding strongly supports research arguments that voice interfaces can dramatically improve financial service accessibility for underserved populations.

This improved comprehension directly translates to better financial decision-making. For example, when explaining fixed deposit options, participants who received information in their preferred language were able to:

• More accurately compare interest rates across different tenure options

• Better understand penalties for premature withdrawals

• More confidently identify which product best suited their specific needs

These findings align with previous research suggesting that financial inclusion requires not just physical access to banking infrastructure but also informational and linguistic accessibility. Our system addresses the latter by meeting users in their linguistic comfort zone. As Mogaji et al.¹¹ noted in their study of emerging markets, language accessibility is a critical factor in determining whether traditionally underserved populations meaningfully engage with formal banking services.

Theoretical and managerial implications

This study offers both managerial relevance and theoretical contributions. From a managerial perspective, the bilingual assistant supports financial institutions in expanding customer reach, particularly among users with low English proficiency or limited digital literacy. By enabling natural language and voice-based interaction, the system can reduce service bottlenecks and contribute to inclusive banking—key goals for customer-centric digital transformation.

On the theoretical front, the system is informed by the Technology Acceptance Model (TAM)²⁰, which emphasizes perceived usefulness and ease of use as critical to adoption. Our bilingual voice-based interface aligns with these factors by minimizing interaction barriers. Additionally, the design considers concepts from sociolinguistics, particularly code-switching, which is prevalent in multilingual societies like India. By accommodating code-switched queries, the system reflects natural communication patterns and enhances user comfort and trust²¹.

These interdisciplinary foundations guide both the design and real-world applicability of the system, contributing to research on inclusive HCI and language-aware conversational AI.

Privacy-first conversational design

Our decision to implement a privacy-first approach where conversation history isn’t stored proved both technically challenging and beneficial. While this approach necessitated more sophisticated context extraction from each individual query, it aligned well with evolving data protection regulations and user privacy expectations.

Interestingly, in post-test interviews, 78% of participants specifically mentioned appreciating the privacy-focused design, with many expressing concerns about the security of their banking conversations with conventional chatbots. This suggests that privacy considerations may be more important to users in banking contexts than in general-purpose conversational systems, confirming the observations made by Birch and Rutter⁴ regarding heightened privacy expectations in financial AI interfaces.

The technical challenge of maintaining contextual awareness without conversation history was addressed through several innovations:

• Enhanced single-query context extraction that identifies multiple intents within one query

• Banking domain-specific entity recognition that identifies products, services concepts

• Query preprocessing to standardize banking terminology across languages

These approaches collectively enabled our system to maintain a 74% success rate on follow-up questions without explicit conversation history, compared to 85% for systems with full conversation storage - an acceptable trade-off given the privacy benefits. This finding is particularly significant in light of Trivedi’s⁷ work on perceived risk in banking chatbot interactions, which identified data privacy concerns as a major barrier to adoption.

While our system follows a privacy-first approach by avoiding storage of user conversations and minimizing data retention, it currently relies on third-party APIs such as Mixtral and Google Translate for language processing and inference. This dependency introduces potential privacy concerns, as data may pass through external servers. Although no personally identifiable information is stored, future iterations of the system could enhance privacy by incorporating on-device processing or self-hosted open-source models, which will allow for better control over data flow.

Limitations and future directions

Current limitations

Despite the promising results, our current implementation has some limitations that are future research directions:

1. While our system handles English, Hindi and Hinglish effectively, it doesn’t yet support other Indian languages such as Tamil, Telugu, or Bengali. This limits its utility in regions where these languages predominate. Future work will explore expanding the language support to include additional regional languages, which would require comprehensive linguistic analysis and specialized training data for each language variant.

2. Unlike some commercial banking chatbots, our system focuses on information provision rather than executing transactions. This design choice was intentional for our research implementation to ensure security and regulatory compliance, but it limits the system’s practical utility. Future implementations could incorporate secure transaction capabilities after obtaining appropriate banking licenses and implementing robust authentication mechanisms.

3. Our error analysis revealed knowledge gaps in specialized investment products and regulatory compliance areas. The banking knowledge base, while extensive, is not yet comprehensive across all banking domains.

4. The language detection system occasionally struggles with strong regional dialects of Hindi, particularly when mixed with English in Hinglish conversations with regional accents.

5. While the voice interface performed adequately in controlled testing environments, its performance decreased significantly in noisy environments or with strong regional accents, a challenge also noted in similar implementations.

6. Another significant limitation of the current system is its reliance on third-party APIs (e.g., Google Translate, Mixtral) for language processing, which may pose residual privacy risks despite not storing user data. This dependency on external services also introduces potential latency and availability concerns. Future work should explore implementing self-hosted or on-device models to enhance data control and better align with regulatory privacy requirements while maintaining system performance.

Future research should also explore the interplay between age, gender and language preferences in banking chatbot interactions, building on the findings of Agarwal et al.¹⁰. Their work suggests that demographic factors significantly influence user experience, and a more nuanced understanding of these relationships could inform more personalized and effective chatbot designs.

Conclusion

This paper introduced a bilingual banking assistant that seamlessly handles queries in English, Hindi and Hinglish without requiring users to pre-select their language preference. Our implementation successfully demonstrates how recent advances in natural language processing can be applied to tackle the persistent challenge of language barriers in banking services.

Core findings summary

The system achieved an overall 87% success rate across 100 diverse banking conversations, with particularly strong performance in automatic language detection (93% accuracy) and code-switching scenarios (80% success rate). Banking terminology accuracy reached 94% for English and 92% for Hindi responses after optimization. The bilingual approach resulted in 37% improved comprehension for complex banking concepts among non-English preferring users.

Limitations acknowledgment

While demonstrating significant advancement in multilingual banking interfaces, our system currently supports only English, Hindi, and Hinglish, limiting applicability in other regional language contexts. The privacy-first design, though beneficial for data protection, restricts transaction capabilities and relies on third-party APIs for language processing. Additionally, voice recognition performance varies with environmental conditions and regional accent variations.

Vision for financial inclusion impact

Looking forward, bilingual and multilingual conversational interfaces represent a transformative approach to dismantling linguistic barriers in financial services. By meeting users in their linguistic comfort zones, such systems can play a pivotal role in advancing financial inclusion across traditionally underserved populations. As financial institutions increasingly seek to extend services to linguistically diverse customer bases, the integration of natural language processing with culturally aware interface design offers a pathway to truly inclusive banking that respects and accommodates the communication patterns of diverse communities. This technology has the potential to democratize access to financial information and services, ultimately contributing to broader economic empowerment and financial literacy across multilingual societies.

Author contributions

M. B. has done the implementation, experimentation, and evaluation of the proposed system under the supervision of S. K. Both authors have written and reviewed the manuscript.

Funding

Open access funding provided by Shiv Nadar University. Open access funding is expected to be supported and facilitated by Shiv Nadar University, Dadri, Uttar Pradesh, India.

Data availability

The comprehensive banking dataset used in this study, including structured information on credit cards, fixed deposits, funds transfer procedures, debit card variants, forex services, account opening requirements, service charges, interest rates, and transaction limits, is available online as a Python file at https://github.com/Manasbir27/Dataset_banking_chatbot. The dataset contains 847 banking terminology entries, detailed product specifications, fee structures, and procedural guidelines organized in hierarchical JSON format. The banking information was sourced and validated from https://www.axisbank.com/download-forms/accountsofficial Axis Bank documentation and https://www.axisbank.com/bank-smart/internet-banking/faqs banking service guidelines to ensure accuracy and regulatory compliance. All data has been anonymized to protect participant privacy.

Code availability

All source code and implementation materials are publicly available on GitHub at: https://github.com/Manasbir27/Bilingual_Banking_Chatbot. The repository also includes a detailed README file for easy understanding.

Declarations

Competing interests

The authors declare no competing interests.

Ethics approval and consent to participate

This study did not involve experiments on human subjects but included human participants for system validation. Approval was obtained from Shiv Nadar Institution of Eminence, Delhi NCR, India. Informed consent was obtained from all participants. No sensitive personal data were collected, and all data were anonymized in accordance with institutional guidelines.