Movatterモバイル変換

Skip to content

#

llm-as-evaluator

Here are 14 public repositories matching this topic...

Language:All

Filter by language

All14 Python11 C#1 Jupyter Notebook1

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

prometheus-eval /prometheus-eval

Evaluate your LLM's response with Prometheus and GPT4 💯

python evaluation gpt4 llm llmops vllm litellm llm-as-a-judge llm-as-evaluator

UpdatedApr 25, 2025
Python

Pacific-AI-Corp /langtest

Deliver safe & effective language models

nlp artificial-intelligence benchmarks benchmark-framework model-assessment ai-safety mlops responsible-ai ml-safety trustworthy-ai ethics-in-ai ml-testing large-language-models llm ai-testing llm-test llm-evaluation-toolkit llm-as-evaluator llm-testing

UpdatedOct 25, 2025
Python

IAAR-Shanghai /xFinder

[ICLR 2025] xFinder: Large Language Models as Automated Evaluators for Reliable Evaluation

benchmark regex reliability evaluation dataset gpt phi large-language-models llm cc-by-nc-nd-4 open-compass chatglm qwen lm-evaluation llm-as-a-judge llm-as-evaluator xfinder reliable-evaluation key-answer-extraction judge-model

UpdatedFeb 26, 2025
Python

KID-22 /LLM-IR-Bias-Fairness-Survey

This is the repo for the survey of Bias and Fairness in IR with LLMs.

information-retrieval recommender-systems bias ir fairness large-language-models llm chatgpt llm4rec llm4rs llm-as-a-judge llm-as-evaluator llm4ir

UpdatedSep 4, 2025

zhaochen0110 /Timo

Code and data for "Timo: Towards Better Temporal Reasoning for Language Models" (COLM 2024)

temporal-reasoning sota-model llms rlhf rlaif llm-as-a-judge llm-as-evaluator self-critic-framework colm2024

UpdatedOct 23, 2024
Python

minnesotanlp /cobbler

Code and data for Koo et al's ACL 2024 paper "Benchmarking Cognitive Biases in Large Language Models as Evaluators"

nlp evaluation bias bias-detection llm llms llm-evaluation llms-benchmarking llm-as-judge llm-as-a-judge llm-as-evaluator

UpdatedFeb 16, 2024
Jupyter Notebook

HillPhelmuth /LlmAsJudgeEvalPlugins

LLM-as-judge evals as Semantic Kernel Plugins

semantickernel llm-evaluation llm-as-a-judge llm-as-evaluator

UpdatedAug 22, 2025
C#

djokester /groqeval

Use groq for evaluations

groq llm generative-ai mixtral llm-as-a-judge llm-as-evaluator llama3

UpdatedJul 11, 2024
Python

Kakz /prometheus-llm

PrometheusLLM is a unique transformer architecture inspired by dignity and recursion. This project aims to explore new frontiers in AI research and welcomes contributions from the community. 🐙🌟

deep-learning mcp evaluation pipelines tracing language-model self-organization cognitive-architecture hermeneutics philosophy-of-mind gpt4 llm llmops ollama litellm llm-as-evaluator autopoietic-systems prompt-logging

UpdatedNov 12, 2025
Python

anaishowland /llm-judge-psai

Evaluation system for computer-use agents that uses LLMs to assess agent performance on web browsing and interaction tasks. This judge system reads screenshots, agent trajectories, and final results to provide detailed scoring and feedback.

computer-vision evaluation evaluation-metrics performance-testing evaluation-framework web-agent llm-as-judge llm-as-a-judge llm-as-evaluator computer-use

UpdatedOct 28, 2025
Python

trustyai-explainability /vllm_judge

A tiny, lightweight library for LLM-as-a-Judge evaluations on vLLM-hosted models.

evaluation-metrics llmops llm-evaluation llm-as-a-judge llm-as-evaluator

UpdatedSep 29, 2025
Python

Non-NeutralZero /LLM-EvalSys

Automated evaluation of llm generated responses on aws

aws llmops llm-as-a-judge llm-as-evaluator

UpdatedApr 11, 2025
Python

rafaelsandroni /antibodies

Antibodies for LLMs hallucinations (grouping LLM as a judge, NLI, reward models)

python nli hallucinations llms hallucination-detection llm-as-a-judge llm-as-evaluator

UpdatedJun 13, 2024
Python

LaurentVeyssier /Agentic-workflow-for-project-management

project from UDACITY second module 'agentic workflow' as part of Agentic AI Nanodegree

python agentic llm-as-evaluator agentic-workflow

UpdatedAug 16, 2025
Python

Improve this page

Add a description, image, and links to thellm-as-evaluator topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with thellm-as-evaluator topic, visit your repo's landing page and select "manage topics."

[8]ページ先頭

©2009-2025 Movatter.jp