LLM-as-judge evaluator for scoring AI agent trajectories on task completion, tool correctness, and efficiency
"Benchmarking LLMs on structured argument extraction from dense, low-resource halakhic texts using an LLM-as-a-Judge pipeline."
JurisRAG is a citation-enforced legal RAG system delivering grounded answers from Indian legal documents using hybrid BM25 + vector retrieval, Azure AI Search, and Azure OpenAI. Evaluated for correctness, groundedness, completeness, citation quality, and hallucination. Also Uses LLM-as-a-Judge setup for generation evaluation.
DODI: a deterministic Terms of Service deception scorer with an MCP server. A 2015-2024 study of 10 platforms with ToS;DR and LLM-judge validation, plus v2, which weighs each contract against what the store promised.
Multi-agent routing system built on Amazon Bedrock AgentCore and LangChain, utilizing deterministic prompt engineering to handle bug reports and FAQs.
Independent RAG evaluation framework: a fictional knowledge base stress-tested for hallucination, scored by a blinded, cross-family LLM judge validated on a held-out set.
A modular framework for evaluating Retrieval-Augmented Generation (RAG) systems with novel pedagogical metrics (PES) for E-Learning environments. Master's Thesis @ Politecnico di Torino.
A tool that scores an LLM's response to a prompt against a five-part rubric, using Claude as the judge
Microservices infrastructure for web parsing evaluation using Llama 3.2 as an LLM-as-a-Judge and traditional and advanced metrics. Computer Engineering Thesis, Sapienza University of Rome.
Lightweight Rust engine for model-vs-model and adversarial LLM evaluation.
To associate your repository with the llmasajudge topic, visit your repo's landing page and select "manage topics."