The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.

ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.

Evaluation and Tracking for LLM Experiments and AI Agents

A database of SDKs, frameworks, libraries, and tools for creating, monitoring, debugging and deploying autonomous AI agents

The missing DevOps layer for coding agents. Flow, feedback, and memory that compounds between sessions.

Transparent telemetry collector for agents

AI agent to automatically check grammar and spelling on documentation files

CrewAI agents that gather and analyze YouTube comments to generate insights to inform better content creation.

Geniusrise: Framework for building geniuses

A collection of cookbooks to help developers get started quickly with the Firecrawl API.

Learn AI, ML, and NLP with interactive Jupyter Notebook tutorials.

`oly` turns long-running and interactive CLI workflows into persistent, supervised sessions for humans and AI agents. Close the terminal, keep the process alive, get notified when input is needed, and jump back in from anywhere.

A radically simple, reliable, and high performance template to enable you to quickly get set up building multi-agent applications

CrewNews is an AI news generator that delivers an unbiased version of the news for a given topic, using Streamlit for the GUI, Llama 3.1 as the LLM (inferenced via the AIML API), CrewAI for building AI agents, AgentOps for testing AI agents, and Exa & Firecrawl as tools

AgentOps: Production-Grade Agentic AI on Kubernetes

Notion-powered control plane for OpenClaw AI agents — monitor, dispatch, tune, backup, and clone your agent fleet

GPT4 for windows, macos and ubuntu

This smart assistant uses advanced Multi-Agent AI systems, leveraging over 10+ agents working asynchronously to perform 20+ tasks. From disease research to personalized diet and exercise plans, as well as doctor consultations and patient journeys, it brings all essential healthcare services into a single platform.

Operational patterns for AI agents and infrastructure from solo to enterprise scale. Applies DevOps + SRE principles to the intersection: infrastructure FOR AI + AI FOR infrastructure. Alpha framework extracting meta-patterns from mission critical environments.

Agent Watch is an AgentOps monitoring library designed for Crew AI applications.

CrewAI AgentOps: Monitor your AI Agents

The open-source MultiAgentOps evaluation and verification harness for any industry business workflow.

agentops mcp

Declarative YAML-based AgentOps framework for building composable AI agents

No description available

This is a smart procurement agents that can help you to find the best product for your company. and generate a report for you.

Demonstrate LLMOps in TypeScript

The system of record for AI agent behavior. AgentOps Replay is an open-source, production-grade observability and audit system for AI agents. Unlike traditional monitoring tools, AgentOps Replay provides cryptographically verifiable, immutable event logs designed for incident investigation, compliance, and post-mortems.