Skip to main content

Week 15 - Interface and Application Programming

This week's work focused on embedded graphical user interfaces and voice-interactive applications. The hardware platforms included the ESP32-DEV 30-pin development board and the ESP32-S3 development board. On the software side, three development environments were employed—MicroBlocks, Arduino IDE, and ESP-IDF—enabling a complete development and verification pipeline from TFT screen driving to voice dialogue functionality.

Individual Assignments​

  1. Initial Attempt: AI-Assisted Code Generation and Screen Driver Validation With the assistance of an AI programming companion, we rapidly generated and iteratively tested the low-level display firmware for a 1.8-inch SPI-based TFT screen. Basic graphics rendering and text output were successfully implemented on the ESP32-DEV 30-pin board. The hardware connections and SPI communication timing were validated, laying a solid foundation for subsequent firmware encapsulation and extension.

  2. MicroBlocks Firmware Development: TFT Driver Encapsulation and Display Verification Within the MicroBlocks graphical programming environment, we developed and encapsulated a dedicated TFT display driver firmware module. A suite of test cases—including geometric shape drawing, text rendering, and color filling—was designed and executed. The firmware ran stably on the target board, and the display output was visually verified. This effort produced a reusable screen driver component for future interactive applications within the MicroBlocks ecosystem.

  3. XiaoZhi AI Voice Dialogue: Command Recognition and Interactive Implementation On the ESP32-S3 platform, we integrated the XiaoZhi AI voice dialogue feature using the ESP-IDF framework. The implementation covered real-time local voice capture, cloud-based or on-device speech recognition, and action feedback. A complete closed-loop pipeline—voice input → intent recognition → action execution—was established. This preliminary work demonstrated the feasibility of intelligent voice interaction on low-power edge devices.

Group Assignment: Toolchain Comparison and Selection Analysis​

To evaluate the suitability of different development approaches for TFT driving and AI voice applications, our group conducted a systematic comparison of the following three toolchains:

Arduino IDE​

Offers a low entry barrier and a rich ecosystem of community-contributed libraries. It is well-suited for rapid functional validation of basic board features—such as GPIO control, SPI communication, and simple sensor reading. However, it shows limitations in multitasking scheduling, project structure management, and in-depth debugging capabilities.

MicroBlocks (VS Code + PlatformIO)​

Combines the intuitiveness of graphical programming with the engineering flexibility of VS Code, making it ideal for educational scenarios and rapid prototyping. PlatformIO's dependency management and build system effectively enhance team collaboration efficiency. Nevertheless, it underperforms in scenarios requiring low-level register manipulation or stringent real-time responsiveness compared to native SDKs.

ESP-IDF​

Provides the most comprehensive feature set, including a rich component library and native FreeRTOS support. It is the preferred choice for complex projects such as the XiaoZhi AI voice dialogue system. The build system is flexible, and debugging tools are robust. However, the learning curve is steeper, and it demands a higher level of embedded systems expertise from developers.

Summary:​

In conclusion, Arduino IDE is best suited for functional verification, MicroBlocks excels in rapid prototyping and educational demonstrations, while ESP-IDF remains the go-to solution for production-grade products and high-performance applications. In this project, the TFT display module was primarily developed and verified using MicroBlocks, whereas the voice dialogue subsystem was deeply customized with ESP-IDF, thereby leveraging the distinct strengths of each toolchain.