Week 11 — Networking and Communications

Overview

This week was about communication between different nodes. In previous weeks, most of my input and output tests happened on one board. This time, I separated the system into two microcontroller nodes and used MQTT to connect them wirelessly.

The XIAO ESP32S3 Sense was used as the input node. It used the camera and a local fruit recognition model to detect whether there was an apple in front of it. When the model detected an apple, the S3 published the message apple through MQTT.

The XIAO ESP32-C3 was used as the output node. It subscribed to the same MQTT topic. When the C3 received the message apple, it turned on the Grove RGB LED for five seconds.

Assignment

Group Assignment

The group assignment was to send a message between two projects. I tested MQTT communication with MQTT Explorer and used it to send messages to the XIAO ESP32-C3 receiver node.

Individual Assignment

The individual assignment was to design, build, and connect wired or wireless node(s) with network or bus addresses and local input and/or output device(s). I built a wireless two-node system: the S3 worked as the input node, and the C3 worked as the output node.

System Design

I used MQTT as the communication method. MQTT uses a publish and subscribe structure. The sender publishes a message to a topic, and the receiver subscribes to that topic. The two boards do not need to be directly connected by wires.

XIAO ESP32S3 Sense camera sees an apple
        ↓
Local fruit recognition model detects apple
        ↓
S3 publishes MQTT message: apple
        ↓
MQTT broker receives the message
        ↓
XIAO ESP32-C3 subscribes to the same topic
        ↓
C3 receives apple
        ↓
Grove RGB LED turns red for five seconds

Step 1: Installing MQTT Explorer

At the beginning, I needed a tool to observe and test MQTT messages before connecting the two boards together. I used MQTT Explorer because it can show MQTT topics clearly and can also manually publish test messages.

MQTT Explorer can be downloaded from its official website: https://mqtt-explorer.com/ . At first, I tried to download the app from the website, but macOS blocked the downloaded application. To avoid spending too much time on this installation problem, I downloaded MQTT Explorer directly from the App Store.

MQTT Explorer Download and Broker Setting

For the MQTT server, I used the HiveMQ public MQTT broker. The broker address was broker.hivemq.com. HiveMQ provides a public MQTT broker for testing MQTT clients and messages: HiveMQ Public MQTT Broker .

In the MQTT Explorer connection window, I filled in the following information. I used the normal MQTT protocol without TLS encryption, so the port was 1883. I did not need a username or password for this public broker.

Setting	Value	Explanation
Name	Week11 Apple MQTT	This is only the local name of my connection in MQTT Explorer.
Protocol	mqtt://	I used the normal MQTT protocol.
Host	broker.hivemq.com	This is the public MQTT broker server used in my test.
Port	1883	Port 1883 is the standard MQTT port without TLS.
Username	Empty	No username was required for this public test broker.
Password	Empty	No password was required for this public test broker.
Validate certificate	Off	I did not use TLS, so certificate validation was not needed.
Encryption / TLS	Off	I used mqtt:// and port 1883.

Step 2: Manual MQTT Test with the C3

Before adding the S3 camera node, I first tested whether the C3 could receive MQTT messages and control the RGB LED. In MQTT Explorer, I manually published two messages: apple and no_apple.

In this early test, apple was used to turn on the RGB LED, and no_apple was used to turn it off. This helped me confirm that the C3, the MQTT topic, and the RGB LED wiring were working correctly.

Manual Publish Setting in MQTT Explorer

After connecting to the broker, I used the Publish panel on the right side of MQTT Explorer to send test messages. This allowed me to test the C3 receiver before adding the S3 camera node.

Publish setting	Value
Topic	litchee/garlen/week11/apple
Payload type	raw
Payload for LED on	apple
Payload for early LED off test	no_apple
QoS	0
Retain	Unchecked

In the final version, I only kept the apple message. The C3 turned on the RGB LED for five seconds after receiving apple, and then the LED turned off automatically.

Step 3: Building the C3 RGB LED Output Node

The output node was built with a XIAO ESP32-C3 and a Grove RGB LED module. The C3 subscribed to the MQTT topic and controlled the RGB LED according to the received message.

Grove RGB LED red wire     → XIAO ESP32-C3 3V3
Grove RGB LED black wire   → XIAO ESP32-C3 GND
Grove RGB LED yellow wire  → XIAO ESP32-C3 D3
Grove RGB LED white wire   → XIAO ESP32-C3 D2

#include <WiFi.h>
#include <PubSubClient.h>
#include <ChainableLED.h>

const char* ssid = "LitcheeLab";
const char* password = "********";

const char* mqtt_server = "broker.hivemq.com";
const int mqtt_port = 1883;
const char* topic = "litchee/garlen/week11/apple";

WiFiClient espClient;
PubSubClient client(espClient);

const int DATA_PIN = D3;
const int CLOCK_PIN = D2;

ChainableLED leds(CLOCK_PIN, DATA_PIN, 1);

void rgbOff() {
  leds.setColorRGB(0, 0, 0, 0);
  Serial.println("RGB LED OFF.");
}

void rgbRedForFiveSeconds() {
  Serial.println("Apple detected. RGB LED will stay ON for 5 seconds.");

  for (int i = 0; i < 50; i++) {
    leds.setColorRGB(0, 255, 0, 0);
    delay(100);
  }

  rgbOff();
  Serial.println("5 seconds finished.");
}

Step 4: Timed MQTT Publishing Test with the S3

After the C3 receiver worked, I tested whether another board could publish messages to the same topic. At this stage, the S3 did not use the camera yet. I programmed the S3 to publish apple every ten seconds. When the C3 received the message, the RGB LED turned on for five seconds.

This was only a communication test. It proved that the S3 could publish MQTT messages and that the C3 could respond to messages from another board.

Step 5: Trying SenseCraft AI

After the basic MQTT communication worked, I started to test apple recognition on the XIAO ESP32S3 Sense. My first method was SenseCraft AI. I connected the S3 to the SenseCraft workspace and loaded an object recognition model.

The SenseCraft preview could recognize the apple successfully. The preview showed the apple label with a confidence value.

However, when I tried to use the MQTT output from SenseCraft, the response was slower than I expected. It also used its own topic structure, such as sscma/v0/#, which made the communication less direct for my final test.

Step 6: Switching to a Local Fruit Recognition Library

To make the response faster and easier to control, I switched to a local fruit recognition Arduino library. This library can run directly on the XIAO ESP32S3 Sense and can classify apples, bananas, and grapes.

Finding the Library

I downloaded the library from the Seeed Studio Edge Impulse tutorial: Seeed Studio Wiki — Edge Impulse for XIAO ESP32S3 Sense . On this page, I found the section called Fruit identification (apples, bananas, grapes) Arduino Library.

xiao-esp32s3-fruits-classify_inferencing.zip

Sketch
→ Include Library
→ Add .ZIP Library
→ xiao-esp32s3-fruits-classify_inferencing.zip

../files/week_11/xiao-esp32s3-fruits-classify_inferencing.zip

#include <xiao-esp32s3-fruits-classify_inferencing.h>

Step 7: Final Local Model and MQTT Integration

In the final version, the S3 ran the fruit recognition model locally. The board only published apple when the model detected apple or apples with a confidence value higher than 0.70.

This final code did not send apple automatically every ten seconds. The delay in the code only prevents repeated messages when the apple stays in front of the camera.

#include <xiao-esp32s3-fruits-classify_inferencing.h>

#include <WiFi.h>
#include <PubSubClient.h>
#include "esp_camera.h"
#include "img_converters.h"
#include "edge-impulse-sdk/dsp/image/image.hpp"

const char* ssid = "LitcheeLab";
const char* password = "********";

const char* mqtt_server = "broker.hivemq.com";
const int mqtt_port = 1883;
const char* topic = "litchee/garlen/week11/apple";

WiFiClient espClient;
PubSubClient client(espClient);

void publishApple() {
  if (!client.connected()) {
    reconnect_mqtt();
  }

  client.loop();
  client.publish(topic, "apple");

  Serial.print("Published to topic: ");
  Serial.println(topic);
  Serial.println("Message: apple");
}

bool appleDetectedNow = false;

for (size_t ix = 0; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) {
  String label = result.classification[ix].label;
  float confidence = result.classification[ix].value;

  Serial.print(label);
  Serial.print(": ");
  Serial.println(confidence, 4);

  label.toLowerCase();

  if ((label == "apple" || label == "apples") && confidence > 0.70) {
    appleDetectedNow = true;
  }
}

if (appleDetectedNow) {
  Serial.println("Apple detected by local AI model.");

  if (millis() - lastAppleSendTime > resendDelay) {
    publishApple();
    lastAppleSendTime = millis();
  }
}

Problems and Solutions

What I Learned

This week helped me understand how to test a networked system step by step. Instead of trying to build the final system immediately, I first tested the MQTT tool, then the C3 receiver, then S3 publishing, then image recognition, and finally the complete integration.

I learned that MQTT is useful because the sender and receiver do not need to be directly connected. They only need to use the same broker and topic. This makes the system flexible and easier to debug.

I also learned that running camera recognition on the XIAO ESP32S3 Sense requires careful memory management. Enabling PSRAM was important for making the camera and local model work together.

Reflection

This week connected several previous topics together: input devices, output devices, embedded programming, and networking. The S3 acted as a smart input node, while the C3 acted as a physical output node.

The most important part was not only sending an MQTT message, but deciding when the message should be sent. The final system only sent apple after the local model detected an apple, instead of sending messages at a fixed time interval.

For my final project, this workflow is very useful. My emotional companion robot may need to recognize visual information and trigger a physical response, such as light, movement, or sound. This week's test gives me a basic structure for that kind of interaction.