Person
Person

Project 3: QA Bot with Multi-turn RAG for e-Commerce

Overview:

Developed an end-to-end Question Answering (QA) bot for the Malay language using a modern Retrieval-Augmented Generation (RAG) architecture. The project demonstrates a clear progression from a foundational V1 prototype to a professional V2 system, which is a bilingual, API-driven solution with an advanced Retrieve-and-Re-rank pipeline and is deployed on Hugging Face Spaces.

Project Description

This project showcases the complete lifecycle of an AI application, from a foundational prototype to a professional, scalable system. It addresses the challenge of building a specialized Question Answering (QA) bot for the Malay language by progressively developing two distinct versions.


V1: The Self-Hosted Prototype

This version served as a technical baseline to prove the core concepts. It was a monolithic Streamlit application where all components were self-hosted and integrated into a single file. The bot was a Malay-only QA system that used a pre-trained BERT model fine-tuned on a custom dataset. The retrieval pipeline utilized a local Sentence Transformer and a FAISS index, with a self-hosted google/mt5-small then google/flan-t5-smallmodel for generating answers.


V2: The Professional Multilingual Assistant (API-Driven)

This version represents a significant re-architecture into a modern, production-ready system. It features a decoupled, bilingual architecture with a secure FastAPI backend that implements an advanced Retrieve-and-Re-rank pipeline. The generative component is powered by the Google Gemini API for enhanced quality, and the knowledge base is managed by Pinecone, a cloud-native vector store. This version also incorporates advanced techniques like Query Transformations and is showcased via a responsive Streamlit chat interface, demonstrating a full progression from a prototype to a scalable, API-driven solution.


Comparison

Category

V1 (Self-Hosted Prototype)

V2 (Professional API)

AI & ML

Python, LangChain, Sentence Transformers (MiniLM-L12-v2)

Python, LangChain, Sentence Transformers (Bi- & Cross-encoder), unstructured

Backend & UI

Streamlit (Monolithic)

FastAPI (Backend) + Streamlit (Frontend)

Vector Store

FAISS (Local File)

Pinecone (Cloud Vector DB)

Generative Model

Google flan-t5-small (Self-Hosted)

Google Gemini gemini-2.5-flash-lite-preview-06-17 (API Integration)

Deployment

Hugging Face Spaces (Standard SDK)

Docker, Hugging Face Spaces (Custom Docker)

Problem Statement

Building a high-quality Question Answering (QA) system for a specialized domain in a low-resource language like Malay poses a significant challenge. An initial prototype (V1) successfully demonstrated the core Retrieval-Augmented Generation (RAG) concept, but was limited by its monolithic architecture and critical LLM performance bottlenecks. Specifically, small, self-hosted models failed at answer synthesis, necessitating a robust fallback mechanism. This highlighted the need for a more scalable and professional solution; where the idea for V2 was coming out.
Car Side View

Methodologies & Project Workflow

Phase 1: V1 Prototype (Self-Hosted)

Objective: To build and deploy a functional, self-hosted RAG QA bot to validate core concepts.


Level 1: Fine-Tuned Model (Local & Colab Execution)
  • Key Methodologies & Steps:

    • Project Setup: Created an isolated virtual environment (p3env) and used Git for version control.

    • Data Preparation: Loaded a custom SQuAD-formatted JSON file, transformed it into a Hugging Face DatasetDict, and split it for training and evaluation.

    • Fine-Tuning: Fine-tuned a multilingual model on a custom Malay dataset using the Hugging Face Trainer API, leveraging Google Colab's GPU for faster processing.


Level 2: RAG Integration
  • Key Methodologies & Steps:

    • Vector Embedding: Used a Sentence Transformer to create embeddings for an e-commerce knowledge base.

    • Vector Store: Stored embeddings in a persistent FAISS index, which was saved to Google Drive.

    • LangChain Implementation: Utilized the LangChain framework to build the end-to-end RAG chain.


Level 3: Demo Deployment
  • Key Methodologies & Steps:

    • Interactive Web App: Created a user-friendly demo with Streamlit.

    • Public Deployment: Deployed the single-file application publicly on Hugging Face Spaces.


Phase 2: V2 Professional Upgrade (API-Driven)

Objective: To re-architect the project into a professional, multilingual, and API-driven system to address the limitations of V1.


Level 4: Project Restructure & Cloud Migration
  • Key Methodologies & Steps:

    • Decoupled Architecture: Implemented a scalable folder structure separating the backend (FastAPI) and frontend (Streamlit).

    • Cloud Vector Store: Migrated the knowledge base to Pinecone, a cloud-native vector store.


Level 5: Advanced Retrieval & Generation
  • Key Methodologies & Steps:

    • Retrieve-and-Re-rank: Implemented a sophisticated pipeline to re-rank retrieved documents, enhancing answer accuracy.

    • Query Transformations: Used an LLM to rephrase user questions to improve retrieval.

    • Generative Model: Integrated the powerful Google Gemini API for high-quality, fluent answer generation.


Level 6: Multilingual API & Final Deployment
  • Key Methodologies & Steps:

    • Bilingual Functionality: Engineered the system to automatically detect and respond in both Malay and English.

    • API Development: Developed a secure FastAPI server to handle all backend logic.

    • Final Deployment: Deployed the complete backend/frontend application to a single Hugging Face Space using a setup.sh script to ensure a resilient two-process application.

Key Results & Model Performance

The results provide a decisive, quantitative validation of the V2 architecture's superiority.

Metric

V1 (Self-Hosted) Score

V2 (API-Driven) Score

Improvement

Answer Relevancy

0.037

0.594

+1517%

Answer Faithfulness

0.467

0.714

+53%

Context Recall

0.100

0.350

+250%

Context Precision

0.333

0.336

+1%

(Note: The answer_relevancy for V1 was near-zero, with many jobs failing and returning NaN. The final average reflects this profound lack of performance.)


Key Findings & Analysis

  • V2 is Over 16x More Helpful: The most critical user-facing metric, Answer Relevancy, shows that the V2 system is over 16 times more effective at providing useful, on-topic answers. This is the ultimate proof of the upgrade's success, transforming the system from a non-functional prototype into a genuinely helpful AI assistant.

  • Generation Quality is the Key Differentiator: The +53% improvement in Answer Faithfulness and the massive leap in Relevancy are directly attributable to V2's use of the powerful Gemini API. It proves that the choice of a capable generative model is paramount for a successful RAG system.

  • Advanced Retrieval Architecture is Validated: The 3.5x improvement in Context Recall demonstrates the power of V2's retrieve-and-re-rank pipeline. It is significantly better at finding *all* the necessary information to construct a comprehensive answer, a task at which V1's simple vector search struggled.

  • V1 System Instability: Evaluated metrics prove that v1 have instability issues as per observed manually/qualitatively, which leads to unrelevant generated output from smaller model used: `"google/flan-t5-small"`.

Key Features & Technical Details

  • Multilingual & Conversational: The final solution is a bilingual assistant that natively supports both English and Malay. It leverages the Google Gemini API to maintain conversation history for natural, multi-turn dialogue.


  • Advanced RAG Pipeline: Implemented a sophisticated "Retrieve-and-Re-rank" strategy, which is a state-of-the-art technique for improving retrieval accuracy in RAG systems. It uses a fast bi-encoder for efficient retrieval from Pinecone, followed by a Cross-Encoder for high-precision re-ranking of results before generation.


  • Decoupled & Scalable Architecture: The solution was re-architected from a monolithic application into a decoupled system with a robust FastAPI backend that handles all AI logic and a responsive Streamlit frontend for the user interface.


  • Powerful Generative AI: The generative component is powered by the Google Gemini API (gemini-2.5-flash-lite), ensuring fluent, accurate, and synthesized answers.


  • Transparent & Debuggable: A key feature is a "Developer Mode" in the UI that reveals the exact retrieval sources used to generate an answer, showcasing the transparency and reliability of the RAG pipeline.


  • Cloud-Native Vector Store: The project demonstrates proficiency with production-level services by migrating the knowledge base from a local FAISS index to a cloud-native vector store, Pinecone.


  • Robust Docker Deployment: The entire two-process application is containerized with Docker and deployed to a single Hugging Face Space, ensuring a consistent runtime environment for both the backend and frontend services.


  • LLM-Powered Query Transformations: Implemented an advanced RAG technique using an LLM to rephrase a user's question, which further improves the quality of the initial retrieval step.

Technical Stack

  • Generative AI: Google Gemini API (gemini-2.5-flash-lite), Hugging Face Models

  • Vectorization & Retrieval: Sentence-Transformers (paraphrase-multilingual-mpnet-base-v2), FAISS, Pinecone (Vector Store), LangChain

  • Backend & Frontend: FastAPI, Streamlit

  • Containerization & Deployment: Docker, Hugging Face Spaces

  • Frameworks & Libraries: Python, Pandas, Scikit-learn, Pytest, LangChain

  • Version Control: Git, GitHub

Challenges & Solutions

Challenge: LLM Performance Bottlenecks with Self-Hosted Models

  • The Problem: The initial V1 prototype utilized small, self-hosted LLMs that consistently failed at answer synthesis, resulting in fragmented or low-quality responses.

  • The Solution: A robust fallback mechanism was implemented in V1 to handle these failures. For V2, a strategic decision was made to switch to a powerful, API-based LLM, the Google Gemini API, which immediately resolved the performance bottlenecks and enabled the bot to generate high-quality, fluent answers.


Challenge: Architecture Migration for Scalability

  • The Problem: The V1 prototype was a single-file, monolithic application that was difficult to manage, debug, and scale, and was not representative of a production-ready system.

  • The Solution: The project was completely refactored into a professional, decoupled V2 microservice architecture. A dedicated FastAPI server was created as the backend to handle all AI logic, with a separate Streamlit application serving as the frontend client. This separation of concerns enables scalability and maintainability.


Challenge: Advanced Retrieval Pipeline for Factual Accuracy

  • The Problem: A basic RAG pipeline can retrieve irrelevant information, leading to the LLM generating incorrect or poorly contextualized answers.

  • The Solution: An advanced "Retrieve-and-Re-rank" pipeline was implemented. This process uses a fast bi-encoder (paraphrase-multilingual-mpnet-base-v2) for efficient initial retrieval and a more precise Cross-Encoder for re-ranking, ensuring only the most relevant documents are passed to the LLM.


Challenge: Deployment Complexity on Hugging Face Spaces

  • The Problem: Deploying a multi-process application (FastAPI backend and Streamlit frontend) on a single Hugging Face Space is not a standard workflow and presents significant configuration challenges.

  • The Solution: A custom Docker container and a setup.sh script were created to orchestrate the two processes. This involved managing the startup sequence and port forwarding, demonstrating a critical skill for deploying non-trivial, multi-service applications in a containerized environment.


Challenge: Real-World Git & Environment Management

  • The Problem: As the project evolved, it was necessary to navigate real-world Git issues (e.g., merging "unrelated histories") and resolve persistent Python library conflicts between different components (e.g., Streamlit and PyTorch).

  • The Solution: These issues were methodically resolved, demonstrating a mastery of Git for managing a clean repository and Python virtual environments (venv, requirements.txt) for ensuring a consistent, conflict-free development environment.

Car Rear Zoom View
Car Front View
Car Front Zoom View
Car Side View

Future Enhancements & Improvement

  • Continuous Integration & Deployment (CI/CD): Implement a full CI/CD pipeline using GitHub Actions to automate the testing and deployment of the backend and frontend. This would ensure that code changes are validated and deployed seamlessly and reliably.


  • Production Monitoring: Integrate monitoring tools such as Prometheus and Grafana (as explored in Project 2) to track API latency, request rates, and other key metrics for the FastAPI backend. This would provide real-time visibility into the application's performance in a production environment.


  • Automated Model Evaluation: Develop a system to periodically evaluate the performance of the RAG pipeline. This could involve using a benchmark dataset to automatically track changes in retrieval accuracy and answer quality over time.


  • Advanced Retrieval Strategies: Experiment with and implement more advanced retrieval techniques. This could include using Hybrid Search (combining keyword and vector search) or a multi-query approach to further enhance the relevance of retrieved documents.


  • Database Integration: Enhance the application by adding a database (Amazon RDS or PostgreSQL) to store conversation history and user feedback. This would enable the collection of data for continuous model improvement and a more personalized user experience.


  • Advanced UI/UX: Refine the Streamlit frontend with a more sophisticated user interface, including features like chat bubbles, user authentication, and a dedicated admin dashboard for monitoring.


  • Cloud-Native Deployment: Migrate the entire two-process application from a single Hugging Face Space to a more robust, cloud-native deployment. This could involve deploying the FastAPI backend to a serverless platform like AWS Lambda and API Gateway (as demonstrated in Project 1), and the Streamlit frontend to a dedicated hosting service, showcasing a more scalable architecture.

More Works

(GQ® — 02)

©2024

More Works

(GQ® — 02)

©2024

Person
Person

Project 3: QA Bot with Multi-turn RAG for e-Commerce

Overview:

Developed an end-to-end Question Answering (QA) bot for the Malay language using a modern Retrieval-Augmented Generation (RAG) architecture. The project demonstrates a clear progression from a foundational V1 prototype to a professional V2 system, which is a bilingual, API-driven solution with an advanced Retrieve-and-Re-rank pipeline and is deployed on Hugging Face Spaces.

Project Description

This project showcases the complete lifecycle of an AI application, from a foundational prototype to a professional, scalable system. It addresses the challenge of building a specialized Question Answering (QA) bot for the Malay language by progressively developing two distinct versions.


V1: The Self-Hosted Prototype

This version served as a technical baseline to prove the core concepts. It was a monolithic Streamlit application where all components were self-hosted and integrated into a single file. The bot was a Malay-only QA system that used a pre-trained BERT model fine-tuned on a custom dataset. The retrieval pipeline utilized a local Sentence Transformer and a FAISS index, with a self-hosted google/mt5-small then google/flan-t5-smallmodel for generating answers.


V2: The Professional Multilingual Assistant (API-Driven)

This version represents a significant re-architecture into a modern, production-ready system. It features a decoupled, bilingual architecture with a secure FastAPI backend that implements an advanced Retrieve-and-Re-rank pipeline. The generative component is powered by the Google Gemini API for enhanced quality, and the knowledge base is managed by Pinecone, a cloud-native vector store. This version also incorporates advanced techniques like Query Transformations and is showcased via a responsive Streamlit chat interface, demonstrating a full progression from a prototype to a scalable, API-driven solution.


Comparison

Category

V1 (Self-Hosted Prototype)

V2 (Professional API)

AI & ML

Python, LangChain, Sentence Transformers (MiniLM-L12-v2)

Python, LangChain, Sentence Transformers (Bi- & Cross-encoder), unstructured

Backend & UI

Streamlit (Monolithic)

FastAPI (Backend) + Streamlit (Frontend)

Vector Store

FAISS (Local File)

Pinecone (Cloud Vector DB)

Generative Model

Google flan-t5-small (Self-Hosted)

Google Gemini gemini-2.5-flash-lite-preview-06-17 (API Integration)

Deployment

Hugging Face Spaces (Standard SDK)

Docker, Hugging Face Spaces (Custom Docker)

Problem Statement

Building a high-quality Question Answering (QA) system for a specialized domain in a low-resource language like Malay poses a significant challenge. An initial prototype (V1) successfully demonstrated the core Retrieval-Augmented Generation (RAG) concept, but was limited by its monolithic architecture and critical LLM performance bottlenecks. Specifically, small, self-hosted models failed at answer synthesis, necessitating a robust fallback mechanism. This highlighted the need for a more scalable and professional solution; where the idea for V2 was coming out.
Car Side View

Methodologies & Project Workflow

Phase 1: V1 Prototype (Self-Hosted)

Objective: To build and deploy a functional, self-hosted RAG QA bot to validate core concepts.


Level 1: Fine-Tuned Model (Local & Colab Execution)
  • Key Methodologies & Steps:

    • Project Setup: Created an isolated virtual environment (p3env) and used Git for version control.

    • Data Preparation: Loaded a custom SQuAD-formatted JSON file, transformed it into a Hugging Face DatasetDict, and split it for training and evaluation.

    • Fine-Tuning: Fine-tuned a multilingual model on a custom Malay dataset using the Hugging Face Trainer API, leveraging Google Colab's GPU for faster processing.


Level 2: RAG Integration
  • Key Methodologies & Steps:

    • Vector Embedding: Used a Sentence Transformer to create embeddings for an e-commerce knowledge base.

    • Vector Store: Stored embeddings in a persistent FAISS index, which was saved to Google Drive.

    • LangChain Implementation: Utilized the LangChain framework to build the end-to-end RAG chain.


Level 3: Demo Deployment
  • Key Methodologies & Steps:

    • Interactive Web App: Created a user-friendly demo with Streamlit.

    • Public Deployment: Deployed the single-file application publicly on Hugging Face Spaces.


Phase 2: V2 Professional Upgrade (API-Driven)

Objective: To re-architect the project into a professional, multilingual, and API-driven system to address the limitations of V1.


Level 4: Project Restructure & Cloud Migration
  • Key Methodologies & Steps:

    • Decoupled Architecture: Implemented a scalable folder structure separating the backend (FastAPI) and frontend (Streamlit).

    • Cloud Vector Store: Migrated the knowledge base to Pinecone, a cloud-native vector store.


Level 5: Advanced Retrieval & Generation
  • Key Methodologies & Steps:

    • Retrieve-and-Re-rank: Implemented a sophisticated pipeline to re-rank retrieved documents, enhancing answer accuracy.

    • Query Transformations: Used an LLM to rephrase user questions to improve retrieval.

    • Generative Model: Integrated the powerful Google Gemini API for high-quality, fluent answer generation.


Level 6: Multilingual API & Final Deployment
  • Key Methodologies & Steps:

    • Bilingual Functionality: Engineered the system to automatically detect and respond in both Malay and English.

    • API Development: Developed a secure FastAPI server to handle all backend logic.

    • Final Deployment: Deployed the complete backend/frontend application to a single Hugging Face Space using a setup.sh script to ensure a resilient two-process application.

Key Results & Model Performance

The results provide a decisive, quantitative validation of the V2 architecture's superiority.

Metric

V1 (Self-Hosted) Score

V2 (API-Driven) Score

Improvement

Answer Relevancy

0.037

0.594

+1517%

Answer Faithfulness

0.467

0.714

+53%

Context Recall

0.100

0.350

+250%

Context Precision

0.333

0.336

+1%

(Note: The answer_relevancy for V1 was near-zero, with many jobs failing and returning NaN. The final average reflects this profound lack of performance.)


Key Findings & Analysis

  • V2 is Over 16x More Helpful: The most critical user-facing metric, Answer Relevancy, shows that the V2 system is over 16 times more effective at providing useful, on-topic answers. This is the ultimate proof of the upgrade's success, transforming the system from a non-functional prototype into a genuinely helpful AI assistant.

  • Generation Quality is the Key Differentiator: The +53% improvement in Answer Faithfulness and the massive leap in Relevancy are directly attributable to V2's use of the powerful Gemini API. It proves that the choice of a capable generative model is paramount for a successful RAG system.

  • Advanced Retrieval Architecture is Validated: The 3.5x improvement in Context Recall demonstrates the power of V2's retrieve-and-re-rank pipeline. It is significantly better at finding *all* the necessary information to construct a comprehensive answer, a task at which V1's simple vector search struggled.

  • V1 System Instability: Evaluated metrics prove that v1 have instability issues as per observed manually/qualitatively, which leads to unrelevant generated output from smaller model used: `"google/flan-t5-small"`.

Key Features & Technical Details

  • Multilingual & Conversational: The final solution is a bilingual assistant that natively supports both English and Malay. It leverages the Google Gemini API to maintain conversation history for natural, multi-turn dialogue.


  • Advanced RAG Pipeline: Implemented a sophisticated "Retrieve-and-Re-rank" strategy, which is a state-of-the-art technique for improving retrieval accuracy in RAG systems. It uses a fast bi-encoder for efficient retrieval from Pinecone, followed by a Cross-Encoder for high-precision re-ranking of results before generation.


  • Decoupled & Scalable Architecture: The solution was re-architected from a monolithic application into a decoupled system with a robust FastAPI backend that handles all AI logic and a responsive Streamlit frontend for the user interface.


  • Powerful Generative AI: The generative component is powered by the Google Gemini API (gemini-2.5-flash-lite), ensuring fluent, accurate, and synthesized answers.


  • Transparent & Debuggable: A key feature is a "Developer Mode" in the UI that reveals the exact retrieval sources used to generate an answer, showcasing the transparency and reliability of the RAG pipeline.


  • Cloud-Native Vector Store: The project demonstrates proficiency with production-level services by migrating the knowledge base from a local FAISS index to a cloud-native vector store, Pinecone.


  • Robust Docker Deployment: The entire two-process application is containerized with Docker and deployed to a single Hugging Face Space, ensuring a consistent runtime environment for both the backend and frontend services.


  • LLM-Powered Query Transformations: Implemented an advanced RAG technique using an LLM to rephrase a user's question, which further improves the quality of the initial retrieval step.

Technical Stack

  • Generative AI: Google Gemini API (gemini-2.5-flash-lite), Hugging Face Models

  • Vectorization & Retrieval: Sentence-Transformers (paraphrase-multilingual-mpnet-base-v2), FAISS, Pinecone (Vector Store), LangChain

  • Backend & Frontend: FastAPI, Streamlit

  • Containerization & Deployment: Docker, Hugging Face Spaces

  • Frameworks & Libraries: Python, Pandas, Scikit-learn, Pytest, LangChain

  • Version Control: Git, GitHub

Challenges & Solutions

Challenge: LLM Performance Bottlenecks with Self-Hosted Models

  • The Problem: The initial V1 prototype utilized small, self-hosted LLMs that consistently failed at answer synthesis, resulting in fragmented or low-quality responses.

  • The Solution: A robust fallback mechanism was implemented in V1 to handle these failures. For V2, a strategic decision was made to switch to a powerful, API-based LLM, the Google Gemini API, which immediately resolved the performance bottlenecks and enabled the bot to generate high-quality, fluent answers.


Challenge: Architecture Migration for Scalability

  • The Problem: The V1 prototype was a single-file, monolithic application that was difficult to manage, debug, and scale, and was not representative of a production-ready system.

  • The Solution: The project was completely refactored into a professional, decoupled V2 microservice architecture. A dedicated FastAPI server was created as the backend to handle all AI logic, with a separate Streamlit application serving as the frontend client. This separation of concerns enables scalability and maintainability.


Challenge: Advanced Retrieval Pipeline for Factual Accuracy

  • The Problem: A basic RAG pipeline can retrieve irrelevant information, leading to the LLM generating incorrect or poorly contextualized answers.

  • The Solution: An advanced "Retrieve-and-Re-rank" pipeline was implemented. This process uses a fast bi-encoder (paraphrase-multilingual-mpnet-base-v2) for efficient initial retrieval and a more precise Cross-Encoder for re-ranking, ensuring only the most relevant documents are passed to the LLM.


Challenge: Deployment Complexity on Hugging Face Spaces

  • The Problem: Deploying a multi-process application (FastAPI backend and Streamlit frontend) on a single Hugging Face Space is not a standard workflow and presents significant configuration challenges.

  • The Solution: A custom Docker container and a setup.sh script were created to orchestrate the two processes. This involved managing the startup sequence and port forwarding, demonstrating a critical skill for deploying non-trivial, multi-service applications in a containerized environment.


Challenge: Real-World Git & Environment Management

  • The Problem: As the project evolved, it was necessary to navigate real-world Git issues (e.g., merging "unrelated histories") and resolve persistent Python library conflicts between different components (e.g., Streamlit and PyTorch).

  • The Solution: These issues were methodically resolved, demonstrating a mastery of Git for managing a clean repository and Python virtual environments (venv, requirements.txt) for ensuring a consistent, conflict-free development environment.

Car Rear Zoom View
Car Front View
Car Front Zoom View
Car Side View

Future Enhancements & Improvement

  • Continuous Integration & Deployment (CI/CD): Implement a full CI/CD pipeline using GitHub Actions to automate the testing and deployment of the backend and frontend. This would ensure that code changes are validated and deployed seamlessly and reliably.


  • Production Monitoring: Integrate monitoring tools such as Prometheus and Grafana (as explored in Project 2) to track API latency, request rates, and other key metrics for the FastAPI backend. This would provide real-time visibility into the application's performance in a production environment.


  • Automated Model Evaluation: Develop a system to periodically evaluate the performance of the RAG pipeline. This could involve using a benchmark dataset to automatically track changes in retrieval accuracy and answer quality over time.


  • Advanced Retrieval Strategies: Experiment with and implement more advanced retrieval techniques. This could include using Hybrid Search (combining keyword and vector search) or a multi-query approach to further enhance the relevance of retrieved documents.


  • Database Integration: Enhance the application by adding a database (Amazon RDS or PostgreSQL) to store conversation history and user feedback. This would enable the collection of data for continuous model improvement and a more personalized user experience.


  • Advanced UI/UX: Refine the Streamlit frontend with a more sophisticated user interface, including features like chat bubbles, user authentication, and a dedicated admin dashboard for monitoring.


  • Cloud-Native Deployment: Migrate the entire two-process application from a single Hugging Face Space to a more robust, cloud-native deployment. This could involve deploying the FastAPI backend to a serverless platform like AWS Lambda and API Gateway (as demonstrated in Project 1), and the Streamlit frontend to a dedicated hosting service, showcasing a more scalable architecture.

More Works

(GQ® — 02)

©2024

Person
Person

Project 3: QA Bot with Multi-turn RAG for e-Commerce

Overview:

Developed an end-to-end Question Answering (QA) bot for the Malay language using a modern Retrieval-Augmented Generation (RAG) architecture. The project demonstrates a clear progression from a foundational V1 prototype to a professional V2 system, which is a bilingual, API-driven solution with an advanced Retrieve-and-Re-rank pipeline and is deployed on Hugging Face Spaces.

Project Description

This project showcases the complete lifecycle of an AI application, from a foundational prototype to a professional, scalable system. It addresses the challenge of building a specialized Question Answering (QA) bot for the Malay language by progressively developing two distinct versions.


V1: The Self-Hosted Prototype

This version served as a technical baseline to prove the core concepts. It was a monolithic Streamlit application where all components were self-hosted and integrated into a single file. The bot was a Malay-only QA system that used a pre-trained BERT model fine-tuned on a custom dataset. The retrieval pipeline utilized a local Sentence Transformer and a FAISS index, with a self-hosted google/mt5-small then google/flan-t5-smallmodel for generating answers.


V2: The Professional Multilingual Assistant (API-Driven)

This version represents a significant re-architecture into a modern, production-ready system. It features a decoupled, bilingual architecture with a secure FastAPI backend that implements an advanced Retrieve-and-Re-rank pipeline. The generative component is powered by the Google Gemini API for enhanced quality, and the knowledge base is managed by Pinecone, a cloud-native vector store. This version also incorporates advanced techniques like Query Transformations and is showcased via a responsive Streamlit chat interface, demonstrating a full progression from a prototype to a scalable, API-driven solution.


Comparison

Category

V1 (Self-Hosted Prototype)

V2 (Professional API)

AI & ML

Python, LangChain, Sentence Transformers (MiniLM-L12-v2)

Python, LangChain, Sentence Transformers (Bi- & Cross-encoder), unstructured

Backend & UI

Streamlit (Monolithic)

FastAPI (Backend) + Streamlit (Frontend)

Vector Store

FAISS (Local File)

Pinecone (Cloud Vector DB)

Generative Model

Google flan-t5-small (Self-Hosted)

Google Gemini gemini-2.5-flash-lite-preview-06-17 (API Integration)

Deployment

Hugging Face Spaces (Standard SDK)

Docker, Hugging Face Spaces (Custom Docker)

Problem Statement

Building a high-quality Question Answering (QA) system for a specialized domain in a low-resource language like Malay poses a significant challenge. An initial prototype (V1) successfully demonstrated the core Retrieval-Augmented Generation (RAG) concept, but was limited by its monolithic architecture and critical LLM performance bottlenecks. Specifically, small, self-hosted models failed at answer synthesis, necessitating a robust fallback mechanism. This highlighted the need for a more scalable and professional solution; where the idea for V2 was coming out.
Car Side View

Methodologies & Project Workflow

Phase 1: V1 Prototype (Self-Hosted)

Objective: To build and deploy a functional, self-hosted RAG QA bot to validate core concepts.


Level 1: Fine-Tuned Model (Local & Colab Execution)
  • Key Methodologies & Steps:

    • Project Setup: Created an isolated virtual environment (p3env) and used Git for version control.

    • Data Preparation: Loaded a custom SQuAD-formatted JSON file, transformed it into a Hugging Face DatasetDict, and split it for training and evaluation.

    • Fine-Tuning: Fine-tuned a multilingual model on a custom Malay dataset using the Hugging Face Trainer API, leveraging Google Colab's GPU for faster processing.


Level 2: RAG Integration
  • Key Methodologies & Steps:

    • Vector Embedding: Used a Sentence Transformer to create embeddings for an e-commerce knowledge base.

    • Vector Store: Stored embeddings in a persistent FAISS index, which was saved to Google Drive.

    • LangChain Implementation: Utilized the LangChain framework to build the end-to-end RAG chain.


Level 3: Demo Deployment
  • Key Methodologies & Steps:

    • Interactive Web App: Created a user-friendly demo with Streamlit.

    • Public Deployment: Deployed the single-file application publicly on Hugging Face Spaces.


Phase 2: V2 Professional Upgrade (API-Driven)

Objective: To re-architect the project into a professional, multilingual, and API-driven system to address the limitations of V1.


Level 4: Project Restructure & Cloud Migration
  • Key Methodologies & Steps:

    • Decoupled Architecture: Implemented a scalable folder structure separating the backend (FastAPI) and frontend (Streamlit).

    • Cloud Vector Store: Migrated the knowledge base to Pinecone, a cloud-native vector store.


Level 5: Advanced Retrieval & Generation
  • Key Methodologies & Steps:

    • Retrieve-and-Re-rank: Implemented a sophisticated pipeline to re-rank retrieved documents, enhancing answer accuracy.

    • Query Transformations: Used an LLM to rephrase user questions to improve retrieval.

    • Generative Model: Integrated the powerful Google Gemini API for high-quality, fluent answer generation.


Level 6: Multilingual API & Final Deployment
  • Key Methodologies & Steps:

    • Bilingual Functionality: Engineered the system to automatically detect and respond in both Malay and English.

    • API Development: Developed a secure FastAPI server to handle all backend logic.

    • Final Deployment: Deployed the complete backend/frontend application to a single Hugging Face Space using a setup.sh script to ensure a resilient two-process application.

Key Results & Model Performance

The results provide a decisive, quantitative validation of the V2 architecture's superiority.

Metric

V1 (Self-Hosted) Score

V2 (API-Driven) Score

Improvement

Answer Relevancy

0.037

0.594

+1517%

Answer Faithfulness

0.467

0.714

+53%

Context Recall

0.100

0.350

+250%

Context Precision

0.333

0.336

+1%

(Note: The answer_relevancy for V1 was near-zero, with many jobs failing and returning NaN. The final average reflects this profound lack of performance.)


Key Findings & Analysis

  • V2 is Over 16x More Helpful: The most critical user-facing metric, Answer Relevancy, shows that the V2 system is over 16 times more effective at providing useful, on-topic answers. This is the ultimate proof of the upgrade's success, transforming the system from a non-functional prototype into a genuinely helpful AI assistant.

  • Generation Quality is the Key Differentiator: The +53% improvement in Answer Faithfulness and the massive leap in Relevancy are directly attributable to V2's use of the powerful Gemini API. It proves that the choice of a capable generative model is paramount for a successful RAG system.

  • Advanced Retrieval Architecture is Validated: The 3.5x improvement in Context Recall demonstrates the power of V2's retrieve-and-re-rank pipeline. It is significantly better at finding *all* the necessary information to construct a comprehensive answer, a task at which V1's simple vector search struggled.

  • V1 System Instability: Evaluated metrics prove that v1 have instability issues as per observed manually/qualitatively, which leads to unrelevant generated output from smaller model used: `"google/flan-t5-small"`.

Key Features & Technical Details

  • Multilingual & Conversational: The final solution is a bilingual assistant that natively supports both English and Malay. It leverages the Google Gemini API to maintain conversation history for natural, multi-turn dialogue.


  • Advanced RAG Pipeline: Implemented a sophisticated "Retrieve-and-Re-rank" strategy, which is a state-of-the-art technique for improving retrieval accuracy in RAG systems. It uses a fast bi-encoder for efficient retrieval from Pinecone, followed by a Cross-Encoder for high-precision re-ranking of results before generation.


  • Decoupled & Scalable Architecture: The solution was re-architected from a monolithic application into a decoupled system with a robust FastAPI backend that handles all AI logic and a responsive Streamlit frontend for the user interface.


  • Powerful Generative AI: The generative component is powered by the Google Gemini API (gemini-2.5-flash-lite), ensuring fluent, accurate, and synthesized answers.


  • Transparent & Debuggable: A key feature is a "Developer Mode" in the UI that reveals the exact retrieval sources used to generate an answer, showcasing the transparency and reliability of the RAG pipeline.


  • Cloud-Native Vector Store: The project demonstrates proficiency with production-level services by migrating the knowledge base from a local FAISS index to a cloud-native vector store, Pinecone.


  • Robust Docker Deployment: The entire two-process application is containerized with Docker and deployed to a single Hugging Face Space, ensuring a consistent runtime environment for both the backend and frontend services.


  • LLM-Powered Query Transformations: Implemented an advanced RAG technique using an LLM to rephrase a user's question, which further improves the quality of the initial retrieval step.

Technical Stack

  • Generative AI: Google Gemini API (gemini-2.5-flash-lite), Hugging Face Models

  • Vectorization & Retrieval: Sentence-Transformers (paraphrase-multilingual-mpnet-base-v2), FAISS, Pinecone (Vector Store), LangChain

  • Backend & Frontend: FastAPI, Streamlit

  • Containerization & Deployment: Docker, Hugging Face Spaces

  • Frameworks & Libraries: Python, Pandas, Scikit-learn, Pytest, LangChain

  • Version Control: Git, GitHub

Challenges & Solutions

Challenge: LLM Performance Bottlenecks with Self-Hosted Models

  • The Problem: The initial V1 prototype utilized small, self-hosted LLMs that consistently failed at answer synthesis, resulting in fragmented or low-quality responses.

  • The Solution: A robust fallback mechanism was implemented in V1 to handle these failures. For V2, a strategic decision was made to switch to a powerful, API-based LLM, the Google Gemini API, which immediately resolved the performance bottlenecks and enabled the bot to generate high-quality, fluent answers.


Challenge: Architecture Migration for Scalability

  • The Problem: The V1 prototype was a single-file, monolithic application that was difficult to manage, debug, and scale, and was not representative of a production-ready system.

  • The Solution: The project was completely refactored into a professional, decoupled V2 microservice architecture. A dedicated FastAPI server was created as the backend to handle all AI logic, with a separate Streamlit application serving as the frontend client. This separation of concerns enables scalability and maintainability.


Challenge: Advanced Retrieval Pipeline for Factual Accuracy

  • The Problem: A basic RAG pipeline can retrieve irrelevant information, leading to the LLM generating incorrect or poorly contextualized answers.

  • The Solution: An advanced "Retrieve-and-Re-rank" pipeline was implemented. This process uses a fast bi-encoder (paraphrase-multilingual-mpnet-base-v2) for efficient initial retrieval and a more precise Cross-Encoder for re-ranking, ensuring only the most relevant documents are passed to the LLM.


Challenge: Deployment Complexity on Hugging Face Spaces

  • The Problem: Deploying a multi-process application (FastAPI backend and Streamlit frontend) on a single Hugging Face Space is not a standard workflow and presents significant configuration challenges.

  • The Solution: A custom Docker container and a setup.sh script were created to orchestrate the two processes. This involved managing the startup sequence and port forwarding, demonstrating a critical skill for deploying non-trivial, multi-service applications in a containerized environment.


Challenge: Real-World Git & Environment Management

  • The Problem: As the project evolved, it was necessary to navigate real-world Git issues (e.g., merging "unrelated histories") and resolve persistent Python library conflicts between different components (e.g., Streamlit and PyTorch).

  • The Solution: These issues were methodically resolved, demonstrating a mastery of Git for managing a clean repository and Python virtual environments (venv, requirements.txt) for ensuring a consistent, conflict-free development environment.

Car Rear Zoom View
Car Front View
Car Front Zoom View
Car Side View

Future Enhancements & Improvement

  • Continuous Integration & Deployment (CI/CD): Implement a full CI/CD pipeline using GitHub Actions to automate the testing and deployment of the backend and frontend. This would ensure that code changes are validated and deployed seamlessly and reliably.


  • Production Monitoring: Integrate monitoring tools such as Prometheus and Grafana (as explored in Project 2) to track API latency, request rates, and other key metrics for the FastAPI backend. This would provide real-time visibility into the application's performance in a production environment.


  • Automated Model Evaluation: Develop a system to periodically evaluate the performance of the RAG pipeline. This could involve using a benchmark dataset to automatically track changes in retrieval accuracy and answer quality over time.


  • Advanced Retrieval Strategies: Experiment with and implement more advanced retrieval techniques. This could include using Hybrid Search (combining keyword and vector search) or a multi-query approach to further enhance the relevance of retrieved documents.


  • Database Integration: Enhance the application by adding a database (Amazon RDS or PostgreSQL) to store conversation history and user feedback. This would enable the collection of data for continuous model improvement and a more personalized user experience.


  • Advanced UI/UX: Refine the Streamlit frontend with a more sophisticated user interface, including features like chat bubbles, user authentication, and a dedicated admin dashboard for monitoring.


  • Cloud-Native Deployment: Migrate the entire two-process application from a single Hugging Face Space to a more robust, cloud-native deployment. This could involve deploying the FastAPI backend to a serverless platform like AWS Lambda and API Gateway (as demonstrated in Project 1), and the Streamlit frontend to a dedicated hosting service, showcasing a more scalable architecture.

More Works

©2024

Create a free website with Framer, the website builder loved by startups, designers and agencies.