The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

From the team behind Gatsby, Mastra is a framework for building AI-powered applications and agents with a modern TypeScript stack.

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

A powerful tool for creating datasets for LLM fine-tuning 、RAG and Eval

AI Observability & Evaluation

Modeling, training, eval, and inference code for OLMo

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.

No description available

AI observability platform for production LLM and agent systems.

Our library for RL environments + evals

The code samples for Learning React by Alex Banks and Eve Porcello, published by O'Reilly Media

Evaluation and Tracking for LLM Experiments and AI Agents

Laminar - open-source observability platform purpose-built for AI agents. YC S24.

A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

Free Images for EVE-NG and GNS3 containing routers, switches,Firewalls and other appliances, including Cisco, Fortigate, Palo Alto, Sophos and more. Master the art of networking and improve your skills!, our repository provides a one-stop solution for a comprehensive hands-on experience.

Interactive Go interpreter and debugger with REPL, Eval, generics and Lisp-like macros

Official github repo for C-Eval, a Chinese evaluation suite for foundation models [NeurIPS 2023]

Python fitting assistant, cross-platform fitting tool for EVE Online

Code to pretrain, fine-tune, and evaluate DreamZero and run sim & real-world evals

OpenSource Production ready Customer service with built in Evals and monitoring

Evaluate your LLM-powered apps with TypeScript

Harbor is a framework for running agent evaluations and creating and using RL environments.

Read Eval Print Loop for Java

OpenAI Frontier Evals

🥤 RAGLite is a Python toolkit for Retrieval-Augmented Generation (RAG) with DuckDB or PostgreSQL

A read-eval-print-loop for php

Skills for AI Evals to compliment the course: AI Evals For Engineers & PMs

The open source post-building layer for agents. Our environment data and evals power agent post-training (RL, SFT) and monitoring.