Problem: The concept needed a handheld device that could hold an AI chat conversation on a touchscreen, connect over WebSocket, and switch between AI backends without new firmware. No device on the market did all three.

Approach: We architected the system around a CYD ESP32 display with an LVGL touchscreen interface and a Python asyncio WebSocket server. The agent runtime is pluggable: Direct LLM, the Hermes agent bridge, or a custom tool-based agent.

Technical details:

• CYD ESP32-2432S028R display with the LVGL UI framework
• Python asyncio WebSocket server with connection management
• Pluggable agent runtime: Direct LLM, Hermes bridge, tool-based agents
• Streaming token display with card rendering
• Conversation history and context held in state
• WiFi with automatic reconnection

Outcome: The concept went from whiteboard to working prototype with a documented build specification. A new AI backend is a new module on the server, with no firmware change.

Stack: Python asyncio, ESP32-2432S028R, LVGL, Arduino/PlatformIO, WebSocket

Sandboxed plugin ran for projects/esp32-ai-terminal and read “ESP32 AI Terminal” through the mediated API.