An MCP-based Chatbot

(English | 中文 | 日本語)

Introduction

👉 Human: Give AI a camera vs AI: Instantly finds out the owner hasn't washed hair for three days【bilibili】

👉 Handcraft your AI girlfriend, beginner's guide【bilibili】

As a voice interaction entry, the XiaoZhi AI chatbot leverages the AI capabilities of large models like Qwen / DeepSeek, and achieves multi-terminal control via the MCP protocol.

Recent Updates

• The project now requires ESP-IDF v6.0.1 or later. ESP-IDF v6.1 is the recommended SDK. ESP-IDF 5.x is no longer supported. The current matrix contains 171 variants; ESP32-S31 builds require IDF 6.1 or later.
• MQTT and BluFi cryptographic code has migrated to PSA Crypto. IDF 6 component splits and third-party dependency compatibility have also been addressed.
• Audio pipeline concurrency, MQTT/UDP packet validation, and release-matrix selection have been hardened.
• ESP32-P4 Rev1 and Rev3 are both supported on IDF 6 with ESP-SR 2.4.7.

Features Implemented

• Wi-Fi, wired Ethernet, USB RNDIS, and ML307/EC801E or NT26 Cat.1 4G networking; supported boards can switch between Wi-Fi and 4G
• Offline voice wake-up with ESP-SR, including customizable wake words
• Two communication transports: WebSocket and MQTT + UDP
• Opus audio streaming with conventional streaming ASR + LLM + TTS pipelines and Realtime end-to-end voice models; AEC-capable hardware supports realtime full-duplex interaction
• Speaker recognition, identifies the current speaker 3D Speaker
• OLED / LCD displays with emoji and rich expression support, plus camera vision input on supported boards
• Battery display and power management
• 39 interface languages, with localized voice prompts where available and English fallback
• ESP32, ESP32-C3, ESP32-C5, ESP32-C6, ESP32-S3, and ESP32-P4 chip platforms
• Wi-Fi provisioning through hotspot or BluFi
• Device-side MCP for device control (Speaker, LED, Servo, GPIO, etc.)
• Cloud-side MCP to extend large model capabilities (smart home control, PC desktop operation, knowledge search, email, etc.)
• Customizable wake words, fonts, emojis, and chat backgrounds with online web-based editing (Custom Assets Generator)

Hardware

Breadboard DIY Practice

See the Feishu document tutorial:

👉 "XiaoZhi AI Chatbot Encyclopedia"

Breadboard demo:

Supports 138 Board Directories and 171 Release Variants (Partial List)

• LiChuang ESP32-S3 Development Board
• Espressif ESP32-S3-BOX-3
• M5Stack CoreS3
• M5Stack AtomS3R + Echo Base
• Magic Button 2.4
• Waveshare ESP32-S3-Touch-AMOLED-1.8
• LILYGO T-Circle-S3
• XiaGe Mini C3
• CuiCan AI Pendant
• WMnologo-Xingzhi-1.54TFT
• SenseCAP Watcher
• ESP-HI Low Cost Robot Dog

Software

Firmware Flashing

For beginners, it is recommended to use the firmware that can be flashed without setting up a development environment.