Agent Evals by Galileo
Bring agent evaluations, observability, and synthetic test set generation directly into your IDE for free with Galileo's new MCP server
Ads
Overview
> Learn how to integrate Galileo's Model Context Protocol (MCP) server with AI-enabled IDEs like Cursor and VS Code The Galileo Model Context Protocol (MCP) server enables seamless integration between AI-powered IDEs, such as Cursor, or VS Code with GitHub Copilot, and Galileo's evaluation and observability platform. With MCP, you can access Galileo's capabilities directly from your development environment, including: Creating and managing datasets Running experiments Setting up prompt templates Getting signals on Log streams Integrating Galileo with your code
Installation
Installation instructions are not available yet.
Configuration
{
"mcpServers": {
"galileo_mcp_server": {
"url": "https://api.galileo.ai/mcp/http/mcp",
"headers": {
"Galileo-API-Key": "YOUR-API-KEY",
"Accept": "text/event-stream"
}
}Capabilities
- Create synthetic datasets
Use `Create Datasets` to generate test data with prompt injections, off-topic, or toxic queries.
- Check dataset status
Use `Check Dataset Status` to track progress and preview the first 10 rows of a dataset.
- Create prompt templates
Use `Create Prompt Templates` to build reusable templates with model config and temperature.
- Set up experiments
Use `Set Up Experiments` for step-by-step guidance on running Galileo experiments with metrics.
- Get signals on Log streams
Use `Get Signals` to analyze production log streams and get fix recommendations.
- Integrate with OpenAI or LangChain
Get step-by-step guides for adding Galileo observability and tracing to your apps.
Tags
- testing-qa
- observability-incident
- ai-agent
- development
Related MCP servers
Context7 MCP
OfficialOfficial Context7 MCP server that brings up-to-date, version-specific library documentation and code examples into AI coding prompts.
Chrome DevTools MCP
OfficialOfficial Chrome DevTools MCP server for controlling and inspecting a live Chrome browser from coding agents such as Gemini, Claude, Cursor, and Copilot.
Mastra/mcp
OfficialClient implementation for Mastra, providing seamless integration with MCP-compatible AI models and tools.
GitHub MCP Server
OfficialConnect your AI assistant to GitHub repositories, issues, pull requests, and search.