Grok helps coding
Dear evaluator. I included the following text for transparency reasons only. It is almost a 100% copy of my interaction with outside knowledge/ GenAI. You are of course completely free to read it but it does not reflect me. I merely used it's output to get more knowledgeable, get inspired, ask more questions, ... If all went well I showed later on what I learned from it and/or used it.
I need a Arduino program in C for a XIAO RP2040. It needs to monitor UART data coming in that follows a JSOn structure. The output comes from a Grove Vision AI v2 module from Seeedstudio. The JSON looks like "perf:{"preprocess":1,"inference":21,"postprocess":0}
boxes:[[145,132,97,138,86,0]]"
You asked specifically for a bare-metal Arduino (C++) sketch for the XIAO RP2040 that monitors the UART and parses the JSON-style structure you showed:
I’ll give you exactly that — no external libraries except the very lightweight ArduinoJson (which is standard in Fab Academy projects anyway). I’ll also include plenty of explanatory comments, debugging tips, and hardware-connection guidance so you can get it running quickly and then adapt it for your final project.
1. Hardware Connection Tips (quick checklist)
- UART pins on XIAO RP2040: Use
Serial1→ TX = D6 (GPIO0), RX = D7 (GPIO1). - Connect to Grove Vision AI V2: Vision TX → XIAO D7 (RX), Vision RX → XIAO D6 (TX), GND → GND, 3.3V/5V → VCC (the module is 3.3 V logic but tolerant).
- Baud rate: The module defaults to 921600 for high-speed inference results — you must use hardware UART (Serial1).
- Power the module from the XIAO’s 3.3 V rail (it draws < 300 mA peak).
2. Required Library
Install ArduinoJson (by Benoit Blanchon) via the Library Manager (Sketch → Include Library → Manage Libraries). Version 7.x works perfectly.
3. Complete Sketch
/*
* XIAO RP2040 + Grove Vision AI V2 UART JSON Monitor
* Fab Academy 2026 - custom parsing example
* Reads the exact format you showed: perf:{...} boxes:[[...]]
* Prints nicely formatted results to Serial (USB) for debugging.
*/
#include <Arduino.h> // For Serial and Serial1 on RP2040
// We use ArduinoJson to turn the semi-JSON string into a real object
#include <ArduinoJson.h>
// Buffer size – the Vision AI output is small, but we give headroom
const size_t BUFFER_SIZE = 512;
char buffer[BUFFER_SIZE];
void setup() {
// USB Serial for debugging / monitoring on your computer
Serial.begin(115200);
while (!Serial) delay(10); // Wait for USB serial (optional on RP2040)
Serial.println(F("=== XIAO RP2040 + Grove Vision AI V2 UART Monitor ==="));
Serial.println(F("Baud 921600 | Parsing perf + boxes JSON-like data"));
// Hardware UART to the Vision AI module
Serial1.begin(921600); // Critical: 921600 baud
Serial.println(F("Serial1 started at 921600 baud – waiting for data..."));
}
void loop() {
// Read any available data from the Vision AI (it sends one block per inference)
if (Serial1.available()) {
// Read until newline (the module usually ends messages with \n)
String raw = Serial1.readStringUntil('\n');
if (raw.length() == 0) return;
// Debug: show exactly what arrived (very useful the first few runs)
Serial.print(F("RAW: "));
Serial.println(raw);
// The string is NOT valid JSON, so we surgically turn it into one:
// Input → perf:{...} boxes:[[...]]
// Output → {"perf":{...},"boxes":[[...]]}
int perfStart = raw.indexOf("perf:");
int boxesStart = raw.indexOf(" boxes:");
if (perfStart == -1 || boxesStart == -1) {
Serial.println(F("⚠️ Unrecognised format – skipping"));
return;
}
// Extract the two parts
String perfStr = raw.substring(perfStart + 5, boxesStart); // e.g. {"preprocess":1,...}
String boxesStr = raw.substring(boxesStart + 7); // e.g. [[145,132,...]]
// Build valid JSON string
String jsonStr = "{\"perf\":" + perfStr + ",\"boxes\":" + boxesStr + "}";
// Parse with ArduinoJson
JsonDocument doc;
DeserializationError error = deserializeJson(doc, jsonStr);
if (error) {
Serial.print(F("JSON parse failed: "));
Serial.println(error.c_str());
return;
}
// === Extract and display data (easy to adapt) ===
JsonObject perf = doc["perf"];
int preprocess = perf["preprocess"];
int inference = perf["inference"];
int postprocess = perf["postprocess"];
Serial.println(F("--- INFERENCE RESULT ---"));
Serial.printf("Performance → Pre: %d ms | Inference: %d ms | Post: %d ms\n",
preprocess, inference, postprocess);
// Boxes can contain zero or many detections
JsonArray boxes = doc["boxes"];
Serial.printf("Detections found: %d\n", boxes.size());
for (JsonArray box : boxes) {
int x = box[0];
int y = box[1];
int w = box[2];
int h = box[3];
int score = box[4];
int target = box[5]; // class ID (depends on your model)
Serial.printf(" Box → x:%d y:%d w:%d h:%d score:%d%% target:%d\n",
x, y, w, h, score, target);
}
Serial.println(F("------------------------\n"));
}
}
4. How to Use & Debug Tips (from one teacher to another)
- Upload the sketch, open the Serial Monitor at 115200 baud.
- Run an inference on the Vision AI (trigger it however your model is set up — usually it auto-runs or you send an AT command).
- You should immediately see the RAW line followed by nicely parsed output.
- Common gotchas & fixes:
- No data? Double-check wiring (TX↔RX crossed) and that the module is powered and running a model.
- Garbage characters? Confirm baud rate is exactly 921600 on both sides.
- Message split across reads? Increase the read timeout or accumulate in a larger buffer until you see both “perf:” and “boxes:”.
- Want to act on the data? Just add your own
if (score > 70 && target == 0)logic right after parsing — perfect for controlling motors, LEDs, etc. - Performance note: At 921600 baud the RP2040 handles this with CPU time to spare, so you can easily add I²C displays, servo control, or Wi-Fi in the same sketch.
5. Optional but recommended next step
If you later decide you want even less boilerplate, the official Seeed_Arduino_SSCMA library does exactly this parsing internally (and works over UART or I²C). It’s great for rapid prototyping, but the code above gives you full visibility into the data flow — which is what you asked for.
Upload it, run a few tests, and let me know what you see in the Serial Monitor. If the exact output format from your module varies slightly (different firmware version, etc.), paste a fresh RAW line here and I’ll tweak the parser instantly. You’ve got this — this is exactly the kind of solid hardware-software bridge that makes Fab Academy projects shine!
Follow-up question
Thx. But I've an addiotion to this. the json does only have to be interpreted. So I get JSON and based on that I want to make a decision. The boxes point out were an object is detected and which how much probability. SO boxes:[[145,132,97,138,86,0]] means in the image at (x,y) 145,132; (width, height) (97,138) with a probability of 86, object belongs to class 0. It is also possible that boxes[] exists meaning no box was detected. It is also possible that there will be multiple boxes at the same time (an array of boxes). I want to know if the box is more to the left, to the right or at the center.
Key clarifications based on the Grove Vision AI V2 documentation (and your example format):
- Each box is [center_x, center_y, width, height, score, class].
- Coordinates are in pixels relative to the camera frame. The module most commonly outputs a 480×480 resolution (this is what you’ll see in almost all SenseCraft/SSCMA examples). I’ve made it a configurable constant so you can change it in one place if your model uses 320×320, 240×240, etc.
- Decision logic (left / center / right):
- Compute the horizontal position using the center_x value (already provided by the module — no need to calculate center manually).
- Divide the image width into thirds:
- center_x < width/3 → LEFT
- center_x > 2*width/3 → RIGHT
- otherwise → CENTER
- Handles zero boxes (no detection), one box, or many boxes (the module can return up to ~10 depending on the model).
- Score is in percent (0–100), so I added a simple threshold (you can tweak it) to ignore weak detections.
- I added a clear “DECISION” section in the output so you can immediately see what the code “thinks” — perfect for debugging or feeding into actuators (servo, LED, motor, etc.).
Here’s the updated complete sketch. Just replace your previous one (everything else stays the same: wiring, baud rate, ArduinoJson library).
/*
* XIAO RP2040 + Grove Vision AI V2 UART JSON Interpreter
* Fab Academy 2026 - position-based decision example
* Parses perf + boxes, classifies each object as LEFT / CENTER / RIGHT
*/
#include <Arduino.h>
#include <ArduinoJson.h>
const size_t BUFFER_SIZE = 512;
char buffer[BUFFER_SIZE];
// ================== CONFIGURATION ==================
const int IMAGE_WIDTH = 480; // Change only if your model uses different resolution
const int CONFIDENCE_MIN = 60; // Ignore detections below this % (adjust as needed)
// ==================================================
void setup() {
Serial.begin(115200);
while (!Serial) delay(10);
Serial.println(F("=== XIAO RP2040 + Grove Vision AI V2 Decision Maker ==="));
Serial.printf("Image width: %d px | Min confidence: %d%%\n", IMAGE_WIDTH, CONFIDENCE_MIN);
Serial1.begin(921600);
Serial.println(F("Serial1 ready at 921600 baud – waiting for inference data..."));
}
void loop() {
if (Serial1.available()) {
String raw = Serial1.readStringUntil('\n');
if (raw.length() == 0) return;
Serial.print(F("RAW: "));
Serial.println(raw);
// Convert the semi-JSON to valid JSON (same as before)
int perfStart = raw.indexOf("perf:");
int boxesStart = raw.indexOf(" boxes:");
if (perfStart == -1 || boxesStart == -1) {
Serial.println(F("⚠️ Unknown format"));
return;
}
String perfStr = raw.substring(perfStart + 5, boxesStart);
String boxesStr = raw.substring(boxesStart + 7);
String jsonStr = "{\"perf\":" + perfStr + ",\"boxes\":" + boxesStr + "}";
JsonDocument doc;
DeserializationError error = deserializeJson(doc, jsonStr);
if (error) {
Serial.print(F("JSON parse error: "));
Serial.println(error.c_str());
return;
}
// --- Performance (for your info) ---
JsonObject perf = doc["perf"];
Serial.printf("Perf → Pre:%dms | Inf:%dms | Post:%dms\n",
(int)perf["preprocess"], (int)perf["inference"], (int)perf["postprocess"]);
// --- Boxes & Decision Logic ---
JsonArray boxes = doc["boxes"];
Serial.printf("Detections: %d\n", boxes.size());
if (boxes.size() == 0) {
Serial.println(F("DECISION: NO OBJECT DETECTED"));
// <-- Insert your action here (e.g. stop motor, turn off LED)
Serial.println(F("------------------------\n"));
return;
}
for (JsonArray box : boxes) {
int center_x = box[0]; // already the horizontal center!
int center_y = box[1];
int w = box[2];
int h = box[3];
int score = box[4];
int target = box[5];
if (score < CONFIDENCE_MIN) {
Serial.printf(" Weak box (score %d%%) ignored\n", score);
continue;
}
// Position decision
String position;
if (center_x < IMAGE_WIDTH / 3) {
position = "LEFT";
} else if (center_x > 2 * IMAGE_WIDTH / 3) {
position = "RIGHT";
} else {
position = "CENTER";
}
Serial.printf(" Box [class %d] score:%d%% pos:(%d,%d) size:%dx%d → %s\n",
target, score, center_x, center_y, w, h, position.c_str());
// Example decision you can expand:
Serial.printf(" DECISION: Object is on the %s → (add your action here)\n", position.c_str());
// e.g. if (position == "LEFT") { digitalWrite(LED_LEFT, HIGH); }
}
Serial.println(F("------------------------\n"));
}
}
Quick tips from one teacher to another
- Test first: Upload, open Serial Monitor at 115200. Point the camera at something your model recognises. You should now see clear “DECISION” lines for every object.
- Multiple objects: The loop handles them one by one. If you want a single overall decision (e.g. “most confident object is on the LEFT”), just add a variable that tracks the highest-score box.
- Change resolution: If your model reports a different frame size (you’ll see it in the raw data sometimes), just edit
IMAGE_WIDTH = 320;(or whatever) and re-upload. - Add real actions: The comment
// <-- Insert your action hereis where you hook up motors, servos, LEDs, Wi-Fi messages, etc. With your software-engineering background this will feel very natural. - Performance: Still lightning fast on the RP2040 — plenty of headroom for extra code.
Run it, point the camera around, and paste a couple of RAW lines + the new output here if you want me to refine the decision logic further (e.g. add vertical position, size-based distance estimate, multi-object priority, etc.). This is solid, production-ready scaffolding for whatever final project you’re building.
You’re making great progress — keep going!
Your lecturer