Input Devices

Extra Code

Once I had the physical board in my hands, I connected it to the Arduino IDE software and uploaded a simple code to make sure the LED was working correctly.

Adding the Microcontroller to Arduino

1. Download the "Additional Boards Manager" for my ESP32

The following video shows how it was done:

2. Select my ESP32 S3

3. Code to test the LED

/*
  Blink for ESP32-S3
  Correctly mapped: Physical Pin D6 -> GPIO 43
*/

// Define the correct pin according to your hardware
const int ledPin = 43;

void setup() {
  // Configure GPIO 43 as output
  pinMode(ledPin, OUTPUT);

  Serial.begin(115200);
  Serial.println("ESP32-S3: Starting blink on GPIO 43 (Pin D6)");
}

void loop() {
  digitalWrite(ledPin, HIGH);   // Turn LED on
  Serial.println("State: ON");
  delay(1000);

  digitalWrite(ledPin, LOW);    // Turn LED off
  Serial.println("State: OFF");
  delay(1000);
}

Sense Craft

The program I used to load the AI model onto the camera is a web platform called Sense Craft. The steps are:

Jumpers Connection

Sense Craft AI & Model Operation

To learn a bit more about this model, I did some research and found the following.

1. Architecture and Algorithm: MobileNetV2

The SenseCraft person detection model is based on the MobileNetV2 algorithm. This is a deep neural network architecture specifically designed by Google for mobile devices and resource-constrained hardware (such as the XIAO ESP32S3).

2. Model Format and Quantization (INT8 and TF Lite)

For the model to "fit" on the ESP32, two key technologies are used:

• TF Lite (TensorFlow Lite): This is the optimized format for running AI models on embedded systems.
• INT8 Precision (Quantization): This is a very important technical detail. It means the model's weights were converted from complex decimal numbers (FP32) to 8-bit integers. This makes the model much lighter and faster, ideal for the ESP32S3 chip architecture.

3. Classification

The model analyzes the image and classifies the scene into one of two categories (Classes): "Person" or "Not a person". This model determines presence or absence based on the features learned during its training.

4. Workflow in the XIAO ESP32S3 Sense

Logical and Electrical Connection (Signal Level)

1. The Transformation of Data into Voltage

The camera acts as an Input Sensor and the GPIO as an Output. The technical process is as follows:

• Digital Input (Camera): The camera sends a data matrix (pixels) to the processor through an internal data bus. The signal here is complex and high-speed.
• Processing (Inference): The ESP32 interprets that data. If the model detects a person, the software changes the state of an internal register from 0 (False) to 1 (True).
• Electrical Output (GPIO): That logical 1 is physically translated to the selected GPIO pin (for example, the LED pin or GPIO 1/D0).
  • High State: The pin delivers a voltage of 3.3V.
  • Low State: The pin drops to 0V (GND).

2. Explanation of GPIO Behavior

• When you configure "Active state: LED on" in the Sense Craft interface, you are instructing the microcontroller to saturate the internal transistor channel of the GPIO to allow current to flow.
• This is the "signal-level connection": it is the transition from a software analysis (person presence) to a measurable electrical pulse.

3. Synchronization and Latency

There is a small delay called inference latency.

• The camera captures.
• The processor calculates (this takes milliseconds).
• The GPIO changes state.

Code Analysis

In the bottom-right corner, you can see the code the program is currently running.

To understand what is happening inside my device's "brain", I analyzed the performance data it outputs while running. It is like watching the processor's "heartbeat" as it tries to recognize me.

This is how you can see the code running.

Results

Reflections

Files I used this week