michel/Philosophy-RAG-demo
1
0
Fork 0
forked from AI_team/Philosophy-RAG-demo
Code Pull Requests Activity

Add filetype key for web chunk

Browse Source
This commit is contained in:
Nielson Janné 2025-03-15 15:35:09 +01:00
parent 865c2c3b8b
commit 37aa171924
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
generic_rag/parsers/parser.py
View File

@ -52,6 +52,7 @@ def add_urls(vector_store: Chroma, urls: list[str], chunk_size: int) -> None:
for split in splits: for split in splits:
split.metadata["source"] = url split.metadata["source"] = url
split.metadata["filetype"] = "web"
all_splits.extend(splits) all_splits.extend(splits)