Claw-Eval is an evaluation harness for evaluating LLM as agents. All tasks verified by humans.

Minimal agent runtime built with DSPy modules and a thin Python loop. Includes CLI, FastAPI server, and eval harness with OpenAI/Ollama support.

Set up and improve harness engineering (AGENTS.md, docs/, lint rules, eval systems, project-level prompt engineering) for AI-agent-friendly codebases. Triggers on: new/empty project setup for AI agents, AGENTS.md or CLAUDE.md creation, harness engineering questions, making agents work better on a codebase.

An open-source evaluation framework specifically for agentic systems — not just LLM outputs, but full agent behavior.

The open-source MultiAgentOps evaluation and verification harness for any industry business workflow.

Linux sandboxing for AI runtimes, code agents, tool executors, and eval harnesses - in one Python file.

Evaluate AI agents with Unix-style pipeline commands. Schema-driven adapters for any CLI agent, trajectory capture, pass@k metrics, and multi-run comparison.

Harness engineering for coding agents: smaller root routers, hard verification gates, migration governance, and repo-specific evals.

PQC Standards Navigator — Agentic RAG over NIST PDFs with citations + eval harness.

Agent Evaluation Harness — write repeatable, measurable evals for AI agents. Python + TypeScript.

Agentic Development Harness - TypeScript. Quality harnesses for AI-integrated web apps: digital twins, prompt evals, code quality gates, accessibility checks.

Adversarial eval harness for any LLM agent pipeline — Claude, OpenAI, or your own. CLI + REST API + MCP server for Cursor/Antigravity.

An agent-native harness engineering starter with structured docs, layered architecture, executable guardrails, evals, and CI.

This should be the first complete agent in a harness w/ MCPs, RAG, Session File Memory, Skills, Usage Meter, Evals etc. FIRST AGENT: GEMINI ARCHITECT

End-to-end refund agent: deterministic policy engine, RAG with FAISS + Pinecone, NVIDIA Nemotron LLM explanations, LangSmith tracing, eval harnesses, and MCP tool interface. Hands-on architecture exploration using AI-assisted development.

A tiny production-ready agent that fetches web pages, extracts and cleans content, and normalizes it to a simple schema. Ships with a FastAPI endpoint, a CLI, a Docker image, and a toy eval harness.

AI agent utility: agent-eval-harness

Agent ecosystem scaffold + eval harness v0

Cross-provider dual-judge scoring across 6 dimensions that predict whether developers will delegate to a coding agent.

No description available

Lightweight evaluation harness for LLM-based agents — scoring, trajectory analysis, and regression gating with pytest.

Framework to benchmark and evaluate multi-agent system performance, accuracy, and efficiency

No description available

No description available

No description available

No description available

Standardized evaluation harness for agentic systems that must be verifiable

Vue 3 eval fixture for Defender agent harness

React 18 eval fixture for Defender agent harness

No description available