forked from AI_team/Philosophy-RAG-demo
113 lines
4.0 KiB
Markdown
113 lines
4.0 KiB
Markdown
# generic-RAG-demo
|
|
|
|
A Sogeti Nederland generic RAG demo
|
|
|
|
## Getting started
|
|
|
|
### Installation of system dependencies
|
|
|
|
#### Unstructered PDF loader (optional)
|
|
|
|
If you would like to run the application with the unstructered PDF loader, the application requires system dependencies.
|
|
The two currently used:
|
|
|
|
- [poppler-utils](https://launchpad.net/ubuntu/jammy/amd64/poppler-utils)
|
|
- [tesseract-ocr](https://github.com/tesseract-ocr/tesseract?tab=readme-ov-file#installing-tesseract)
|
|
|
|
```bash
|
|
sudo apt install poppler-utils tesseract-ocr
|
|
```
|
|
|
|
and run the generic RAG demo with the `--unstructured-pdf` flag.
|
|
|
|
> For more information please refer to the [langchain docs.](https://python.langchain.com/docs/integrations/providers/unstructured/)
|
|
|
|
#### Local LLM (optional)
|
|
|
|
The application supports running a local LLM, using Ollama.
|
|
|
|
To install Ollama, please run following commands
|
|
|
|
```bash
|
|
curl -fsSL https://ollama.com/install.sh | sh # install Ollama
|
|
ollama pull llama3.1:8b # fetch and dowload specific model
|
|
```
|
|
|
|
Include the model in the `.env` file:
|
|
|
|
```text
|
|
LOCAL_CHAT_MODEL="llama3.1:8b"
|
|
LOCAL_EMB_MODEL="llama3.1:8b"
|
|
```
|
|
|
|
And run the generic RAG demo with the `-b local` flag.
|
|
|
|
>For more information on installing Ollama, please refer to the Langchain Local LLM documentation, specifically the [Quickstart section](https://python.langchain.com/docs/how_to/local_llms/#quickstart).
|
|
|
|
### Running generic RAG demo
|
|
|
|
Please mind due to use of `argparse` the generic RAG demo can not be launched the way `chainlit` documentation recommends.
|
|
|
|
```bash
|
|
chainlit run generic_rag/app.py # will not work
|
|
```
|
|
|
|
Instead, the app can be launched and debugged the usual way.
|
|
|
|
```bash
|
|
python generic_rag/app.py -p data # will work and parsers all pdf files in ./data
|
|
python generic_rag/app.py --help # will work and prints command line options
|
|
```
|
|
|
|
Please configure your `.env` file with your cloud provider (backend) of choice and set the `--backend` flag accordingly.
|
|
|
|
### .env file
|
|
|
|
A .env file needs to be populated to configure API end-points or local back-ends using environment variables.
|
|
Currently all required environment variables are defined in code at [backend/models.py](generic_rag/backend/models.py)
|
|
with the exception of the API key variables itself.
|
|
More information about configuring API endpoints for langchain can be found at the following locations.
|
|
|
|
- [langchain cloud chat model doc](https://python.langchain.com/docs/integrations/chat/)
|
|
- [langchain local chat model doc](https://python.langchain.com/docs/how_to/local_llms/)
|
|
- [langchain cloud/local emb model doc](https://python.langchain.com/docs/integrations/text_embedding/)
|
|
|
|
> for local models we currently use Ollama
|
|
|
|
An `.env` example is as followed.
|
|
|
|
```text
|
|
# only one backend (azure, google, local, etc) is required. Please addjust the --backend flag accordingly
|
|
|
|
AZURE_OPENAI_API_KEY="<secret_key>"
|
|
AZURE_LLM_ENDPOINT="https://<project_hub>.openai.azure.com"
|
|
AZURE_LLM_DEPLOYMENT_NAME="gpt-4"
|
|
AZURE_LLM_API_VERSION="2025-01-01-preview"
|
|
AZURE_EMB_ENDPOINT="https://<project_hub>.openai.azure.com"
|
|
AZURE_EMB_DEPLOYMENT_NAME="text-embedding-3-large"
|
|
AZURE_EMB_API_VERSION="2023-05-15"
|
|
|
|
LOCAL_CHAT_MODEL="llama3.1:8b"
|
|
LOCAL_EMB_MODEL="llama3.1:8b"
|
|
|
|
# google vertex AI does not use API keys but a seperate authentication method
|
|
GOOGLE_GENAI_CHAT_MODEL="gemini-2.0-flash"
|
|
GOOGLE_GENAI_EMB_MODEL="models/text-embedding-004"
|
|
```
|
|
|
|
### Chainlit starters
|
|
|
|
Chainlit suggestions (starters) can be set with the `CHAINLIT_STARTERS` environment variable.
|
|
The variable should be a JSON array of objects with `label` and `message` properties.
|
|
An example is as followed.
|
|
|
|
```text
|
|
CHAINLIT_STARTERS=[{"label":"Label 1","message":"Message one."},{"label":"Label 2","message":"Message two."},{"label":"Label 3","message":"Message three."}]
|
|
```
|
|
|
|
## Dev details
|
|
|
|
### Linting
|
|
|
|
Currently [Ruff](https://github.com/astral-sh/ruff) is used as Python linter. It is included in the [pyproject.toml](pyproject.toml) as `dev` dependency if your IDE needs that. However, for VS Code a [Ruff extension](https://marketplace.visualstudio.com/items?itemName=charliermarsh.ruff) excists.
|