forked from AI_team/Philosophy-RAG-demo
143 lines
5.6 KiB
Markdown
143 lines
5.6 KiB
Markdown
# generic-RAG-demo
|
|
|
|
A generic Retrieval Augmented Generation (RAG) demo from Sogeti Netherlands built in Python. This project demonstrates how to integrate and run different backends, from cloud providers to local models, to parse and process your PDFs, web data, or other text sources.
|
|
|
|
## Table of Contents
|
|
|
|
- [generic-RAG-demo](#generic-rag-demo)
|
|
- [Table of Contents](#table-of-contents)
|
|
- [Features](#features)
|
|
- [Getting started](#getting-started)
|
|
- [Project Environment Setup](#project-environment-setup)
|
|
- [Installation of system dependencies](#installation-of-system-dependencies)
|
|
- [Unstructered PDF loader (optional)](#unstructered-pdf-loader-optional)
|
|
- [Local LLM (optional)](#local-llm-optional)
|
|
- [Running generic RAG demo](#running-generic-rag-demo)
|
|
- [config.yaml file](#configyaml-file)
|
|
- [.env file](#env-file)
|
|
- [Chainlit starters](#chainlit-starters)
|
|
- [Dev details](#dev-details)
|
|
- [Linting](#linting)
|
|
|
|
## Features
|
|
|
|
- **Multi-backend Support:** Easily switch between cloud-based and local LLMs.
|
|
- **Flexible Data Input:** Supports both PDFs and web data ingestion.
|
|
- **Configurable Workflows:** Customize settings via a central `config.yaml` file.
|
|
|
|
## Getting started
|
|
|
|
### Project Environment Setup
|
|
|
|
This project leverages a modern packaging method defined in `pyproject.toml`. After cloning the repository, you can install the project along with its dependencies. You have two options:
|
|
1. Using uv
|
|
|
|
If you're using uv, simply run:
|
|
```bash
|
|
uv install
|
|
```
|
|
|
|
2. Using a Python Virtual Environment
|
|
|
|
Alternatively, set up a virtual environment and install the project:
|
|
```bash
|
|
python -m venv .venv # Create a new virtual environment named ".venv"
|
|
source .venv/bin/activate # Activate the virtual environment (use ".venv\Scripts\activate" on Windows)
|
|
pip install . # Install the project and its dependencies
|
|
```
|
|
|
|
### Installation of system dependencies
|
|
|
|
Some optional features require additional system applications to be installed.
|
|
|
|
#### Unstructered PDF loader (optional)
|
|
|
|
If you would like to run the application using the unstructered PDF loader (`pdf.unstructured` setting) you need to install two system dependencies.
|
|
|
|
- [poppler-utils](https://launchpad.net/ubuntu/jammy/amd64/poppler-utils)
|
|
- [tesseract-ocr](https://github.com/tesseract-ocr/tesseract?tab=readme-ov-file#installing-tesseract)
|
|
|
|
```bash
|
|
sudo apt install poppler-utils tesseract-ocr
|
|
```
|
|
|
|
> For more information please refer to the [langchain docs.](https://python.langchain.com/docs/integrations/providers/unstructured/)
|
|
|
|
#### Local LLM (optional)
|
|
|
|
If you would like to run the application using a local LLM backend (`local` settings), you need to install Ollama.
|
|
|
|
```bash
|
|
curl -fsSL https://ollama.com/install.sh | sh # install Ollama
|
|
ollama pull llama3.1:8b # fetch and download as model
|
|
```
|
|
|
|
Include the downloaded model in the `config.yaml` file:
|
|
|
|
```yaml
|
|
local:
|
|
chat_model: "llama3.1:8b"
|
|
emb_model: "llama3.1:8b"
|
|
```
|
|
|
|
>For more information on installing Ollama, please refer to the Langchain Local LLM documentation, specifically the [Quickstart section](https://python.langchain.com/docs/how_to/local_llms/#quickstart).
|
|
|
|
### Running generic RAG demo
|
|
|
|
Please mind due to use of `argparse` the generic RAG demo can not be launched the way `chainlit` documentation recommends.
|
|
|
|
```bash
|
|
chainlit run generic_rag/app.py # will not work
|
|
```
|
|
|
|
Instead, the app can be launched and debugged the usual way.
|
|
|
|
```bash
|
|
python generic_rag/app.py -p data # will work and parsers all pdf files in ./data
|
|
python generic_rag/app.py --help # will work and prints command line options
|
|
```
|
|
|
|
Please configure your `config.yaml` and `.env` file with your cloud provider (backend) of choice. See the sections below for more details.
|
|
|
|
### config.yaml file
|
|
|
|
A config.yaml file is required to specify your API endpoints and local backends. Use the provided `config.yaml.example` as a starting point. Update the file according to your backend settings and project requirements.
|
|
|
|
Key configuration points include:
|
|
- Chat Backend: Choose among azure, openai, google_vertex, aws, or local.
|
|
- Embedding Backend: Configure the embedding models similarly.
|
|
- Data Processing Settings: Define PDF and web data sources, chunk sizes, and overlap.
|
|
- Vector Database: Customize the path and reset behavior.
|
|
|
|
For more information on configuring Langchain endpoints and models, please see:
|
|
|
|
- [langchain cloud chat model doc](https://python.langchain.com/docs/integrations/chat/)
|
|
- [langchain local chat model doc](https://python.langchain.com/docs/how_to/local_llms/)
|
|
- [langchain cloud/local emb model doc](https://python.langchain.com/docs/integrations/text_embedding/)
|
|
|
|
> for local models we currently use Ollama
|
|
|
|
### .env file
|
|
|
|
Set the API keys for your chosen cloud provider (backend). This ensures that your application can authenticate and interact with the services.
|
|
|
|
```text
|
|
AZURE_OPENAI_API_KEY=your_azure_api_key
|
|
OPENAI_API_KEY=your_openai_api_key
|
|
```
|
|
|
|
### Chainlit starters
|
|
|
|
Chainlit suggestions (starters) can be set with the `CHAINLIT_STARTERS` environment variable.
|
|
The variable should be a JSON array of objects with `label` and `message` properties.
|
|
An example is as followed.
|
|
|
|
```text
|
|
CHAINLIT_STARTERS=[{"label":"Label 1","message":"Message one."},{"label":"Label 2","message":"Message two."},{"label":"Label 3","message":"Message three."}]
|
|
```
|
|
|
|
## Dev details
|
|
|
|
### Linting
|
|
|
|
Currently [Ruff](https://github.com/astral-sh/ruff) is used as Python linter. It is included in the [pyproject.toml](pyproject.toml) as `dev` dependency if your IDE needs that. However, for VS Code a [Ruff extension](https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff) exists. |