Skip to main content

2. Interface and Application Programming

2.1 AI-Assisted OLED Display​

With the assistance of an AI programming companion, I rapidly generated and iteratively tested the low-level display firmware for a 0.96-inch OLED display screen.

Basic graphics rendering and text output were successfully implemented on the ESP32-DEV 30-pin board.

The hardware connections and SPI communication timing were validated, laying a solid foundation for subsequent firmware encapsulation and extension.

2.2 Use OLED Display with MicroBlocks​

I used the PCB board I fabricated for my assignment a few weeks ago, together with an ESP32 development board.

I flashed it with MicroBlocks firmware, and successfully drove a 0.96-inch OLED display.

This module supports essential graphical operations, including displaying text (e.g., "Hello"), drawing lines, filling rectangles, and rendering circles on the screen.

Here is the blocks code:

Here is the source code

module main
author unknown
version 1 0 
description ''

script 177 139 {
whenStarted
OLEDInit_I2C 'OLED_0.96in' '3C' 0 false
OLEDclear
OLEDshowGDBuffer
repeat 2 {
  sendBroadcast 'dis1'
  waitMillis 1000
  sendBroadcast 'dis2'
  waitMillis 1000
  sendBroadcast 'dis3'
  waitMillis 1000
  sendBroadcast 'dis4'
  waitMillis 1000
}
}

script 767 214 {
whenBroadcastReceived 'dis1'
OLEDclear
OLEDwrite 'Hello!' 0 0 false
}

script 460 276 {
whenBroadcastReceived 'dis2'
OLEDclear
OLEDfillRect 20 20 30 20 false
}

script 695 421 {
whenBroadcastReceived 'dis3'
OLEDclear
OLEDdrawLine 0 0 127 63 false
}

script 484 497 {
whenBroadcastReceived 'dis4'
OLEDclear
OLEDdrawCircle 64 32 10 false
}

Here is the vedio of OLED display:

2.3 XiaoZhi AI Voice Dialogue: Command Recognition and Interactive Implementation​

Hardware Setup​

I am using an ESP32-S3 development board, an INMP441 microphone, a MAX98357 amplifier module, a speaker, and a 0.96 inch OLED screen.

On the ESP32-S3 platform, I integrated the XiaoZhi AI voice dialogue feature using the ESP-IDF framework.

Software Implementation​

I use the the XiaoZhi AI voice opensource project: https://github.com/78/xiaozhi-esp32

The implementation covered real-time local voice capture, cloud-based or on-device speech recognition, and action feedback. A complete closed-loop pipeline—voice input → intent recognition → action execution—was established. This preliminary work demonstrated the feasibility of intelligent voice interaction on low-power edge devices.

I also extended the XiaoZhi AI voice dialogue system to run on an IoT device, enabling voice-controlled interaction with the AI Horse Move system via the MQTT protocol.

To implement this, I used the ESP-IDF framework to modify the xiaozhi-esp32 project, adapting it for seamless integration with my hardware setup.

As part of the integration, I added a new IoT component called HorseCopilot, which acts as the bridge between the voice interface and the movement control logic.

I refer to this setup as the HorseCopilot running on my xiaozhi wifi board.

2.4 MQTT message receive​

I used an ESP32-S3 and an ESP32 to verify MQTT functionality: one board listens for the voice command "Forward" and publishes it via MQTT, while the other subscribes to the topic and moves a toy horse forward accordingly.

I also validated message reception, confirmed firmware stability on the target hardware, and visually checked the display output.

This work resulted in a reusable screen driver component for future MicroBlocks-based interactive applications.

Here is the blocks code:

Here is the vedio of MQTT:

2.5 source file​