OppieAI MCP Tool Filter

A Precision-driven Tool Recommendation (PTR) system for filtering MCP (Model Context Protocol) tools based on conversation context. Fetch only relevant tool for the ongoing conversation and save cost while increasing the precision of your LLM Response.

Developed by OppieAI

🎥 Explainer Video

Watch the full explanation of how ToolsFilter works and its impact on LLM performance

Table of Contents

• 🎥 Explainer Video
• Why?
  • The Tool Overload Problem
  • Research Evidence
  • Visual Evidence
• Solution
  • Core Capabilities
  • Infrastructure & Performance
  • Real-World Impact
• Architecture
  • Search Pipeline Architecture
  • Search Strategies
• Performance
  • Latest Evaluation Results
  • LTR Model Performance
  • Optimization Roadmap
• Quick Start
  • Prerequisites
  • Installation
  • Running the Services
  • API Documentation
• Usage Example
• API Endpoints
  • Main Endpoints
  • Response Format
• Development
  • Running Tests & Evaluation
  • Latest Evaluation Reports
  • Code Quality
  • Performance Testing
• Configuration
  • Vector Store Collections
  • Automatic Fallback Mechanism
• Documentation
• References
• License

Why?

The Tool Overload Problem

Modern LLMs with access to large tool suites face a critical performance degradation issue: the more tools available, the lower the accuracy becomes. This phenomenon is well-documented in research and practical implementations:

📊 Research Evidence

Recent studies using MCPGauge evaluated six commercial LLMs with 30 MCP tool suites and revealed alarming findings:

• 9.5% accuracy drop on average when LLMs have automated access to MCP tools
• 3.25× to 236.5× increase in input token volume, creating massive computational overhead
• "Non-trivial friction" between retrieved context and the model's internal reasoning
• Models struggle with instruction compliance when too many tools are available

🎥 Visual Evidence

This accuracy degradation with increased tool count is demonstrated in this analysis video, showing how model performance deteriorates as more tools are introduced.

Solution

Core Capabilities

• 🚀 Multi-Stage Search Pipeline: Semantic + BM25 + Cross-Encoder + LTR ranking
• 🎯 High-Performance Results: Perfect P@1 and MRR across all search strategies
• 🧠 Learning-to-Rank: XGBoost model with 46+ engineered features (NDCG@10: 0.975)
• 🔧 OpenAI Function Calling Compatible: Flat tool structure following OpenAI specification