RAG system

2 items

ARTICLEDEV.to AI·21d ago

Building an Evaluation Harness for Financial RAG: What I Learned About LLM-as-Judge Calibration

The author built a RAG system for financial Q&A using SEC filings and the FinanceBench benchmark. They uncovered a significant discrepancy between LLM-as-judge evaluations and actual performance, leading to lessons on calibrating LLMs for assessment.

Financial AI Benchmarking GPT-4o-mini RAG system

ARTICLEDEV.to AI·18d ago

Gemma 4 가 갑자기 답을 못 했다 — 외부 협업이 24시간 만에 root cause 찾아낸 이야기

PROJECT JAMES, an Ollama-based Graph-RAG system using Gemma 4, encountered an issue where the model abruptly stopped responding in specific stages. After investigating multiple hypotheses and external collaboration, the root cause was identified as a max_tokens cap truncating the model's output.

Ollama Gemma 4 RAG system Debugging