Remote MCPs What we learned from shipping — John Welsh, Anthropic
AI Summary
This video is a talk by John, a technical staff member at Anthropic with 20 years experience in building large scale systems, about implementing MCP (Model Client Protocol) clients and its usage at scale within a large organization. John explains the rapid growth in AI tool integrations and the associated chaos caused by duplicated functionality and inconsistent interfaces across services. They describe how Anthropic standardized internally on MCP to unify tool calling, authentication, and resource management, improving engineering efficiency and reducing integration complexity.
John discusses the two components of MCP: the JSON RPC message specification and the global transport standard for session management. He emphasizes that standardizing on MCP internally allows engineers to work on interesting problems rather than plumbing and integration details.
He describes the implementation of an MCP gateway at Anthropic, a shared infrastructure component providing a centralized point of entry for MCP connections. The gateway handles URL-based routing, credential management, rate limiting, and observability, simplifying the developer experience. The gateway returns an MCP SDK client session that works for internal and external integrations.
John also covers the transport layer details, mentioning various options like websockets and gRPC, and highlights the internal authentication model centralized at the gateway. This design allows batch jobs and internal services to use credentials seamlessly without repeated authentication.
Additionally, the talk covers security and policy enforcement benefits enabled by the MCP gateway, including auditability, content classification, and banning malicious servers. John adds that MCP support simplifies adding new integrations and future proofs the infrastructure by adopting an industry-wide standard embraced by major AI labs.
Key takeaways include the simplicity of piping JSON streams with MCP, the value of standardizing on a single protocol for tool integration, the importance of building “pits of success” to encourage best practices, and centralizing shared problems like authentication and connectivity to free up engineers to focus on valuable business problems.
The video concludes with John thanking the audience and encouraging the adoption of MCP as a robust solution for managing tool calling at scale in AI-powered organizations.