Skip to content

16. System Integration

Weekly Assignment:

Design and document the system integration for your final project.

Design Overview

The pen lift mechanism is based off How To Mechantronics' pen plotter.

Input: Audio Capture + Transcription (Whisper)

The user speaks a command such as “Draw a flower” into the onboard microphones on the reSpeaker 2-Mics Pi HAT. The reSpeaker captures audio and sends it to the Raspberry Pi. The Raspberry Pi runs Whisper (or some other similar system) to transcribe the audio into text. The transcription is saved or passed directly to the next step.

Processing: Text-to-G-code using AI

The ChatGPT API takes the transcribed prompt with a structured prompt like:

“You are a plotter that outputs simple G-code drawings. Given a prompt, generate G-code to draw it on a 3-axis pen plotter.”

ChatGPT responds with plain G-code text. The result is saved to a .gcode file.

Output: G-code sending to GRBL

This step will involve pySerial or Universal G-code Sender (UGS), probably pySerial. There will be GRBL firmware running on the CNC shield. The Raspberry Pi opens the USB serial port to communicate with GRBL. A script reads and sends the G-code to GRBL using pySerial. GRBL interprets motion commands and moves the stepper motors accordingly.

Interface: Touchscreen Control

Using the 7” Raspberry Pi Touchscreen, there will be an interface (Python UI) that shows the transcribed prompt. I would be able to re-record, retry, or start the plot. Potentially, it would be able to visualize the pending G-code file before sending. Ideally, it could shows plot status (e.g., "Drawing...", "Idle"). At the very least, I need to be able to double check whether the text I want to send through is actually what I want to send to the pen plotter.

Subsystem Interfaces

Below is a flow chart for reference. This was made using Lucid Charts.

Input: Microphone → Speech Recognition (Whisper)

ReSpeaker 2-Mic Pi HAT connects through the GPIO pins on the Raspberry Pi. Power is supplied through the Raspberry Pi GPIO header. It takes in raw audio.

Processing: Whisper → GPT → G-code Generator

Whisper → GPT/LLM: Could be done using OpenAI's ChatGPT API but it would require internet access, or I could use a local model.

GPT → G-code Generator: Uses python script to process and save or stream the G-code.

Output: G-code → GRBL Plotter

Connection between USB Serial, Pi and a GRBL board: Pi USB port to an Arduino Uno with a GRBL-compatible CNC shield. 12V power supply is connected to the CNC shield to power the motors. The Arduino Uno is powered through USB from the Raspberry Pi. The RPi will communicate with GRBL controller through serial (UART) commmunication. It follows that the library is pySerial in Python.

Interface: Touchscreen UI

Connected by DSI ribbon cable and 5V DC power.

Integration

Touchscreen - Raspberry Pi 4

When I purchased the 7 Inch HDMI Touchscreen LCD Display it came with instructions on how to connect it to a Raspberry Pi. Even so, because it was my first time working with a Raspberry Pi, I was still confused. When I tried searching for tutorials online, a lot of them already assumed I had some base level of knowledge I didn't.

Figuring out where each of the three connective parts went was not necessarily the hardest part, but installing the driver was. I ended up following this tutorial to power it on even though it was different than what I had.

After that, I just inserted the MicroSD chip with the driver and powered it on and I got to the landing page.

ReSpeaker 2-Mics Pi HAT - Raspberry Pi 4

The Mic HAT is installed on GPIO headers of the Raspberry Pi. I ended up following this tutorial for the microphone.

I was originally worried that this way of connecting the microphone would block the rest of the GPIO pins I would need for other aspects of my project, but I learned stacking headers would be a solution to that.

With it attached, I was able to get the Raspberry Pi to recognize my microphone by accessing the Command Prompt window on the touchscreen.

CNC Shield - Pen Plotter

GRBL-compatible CNC shield connections with X, Y, and Z axis stepper motors, servo motor, and X, Y, and Z limit switches.

Pen Plotter Lift Mechanism

Below is the 3D parts for the lift mechanism printed out and assembled. The mechansim was inspired by How To Mechantronics' pen plotter.

Methods of Packaging

For my Raspberry Pi 4, I will use a case like this one.

As for cable management, when I get to wiring all of my motors and the Raspberry Pi, I will use a braided cable sleeve, or something like it to make sure they are neatly secured and do not fall all over the place. The motors will be mounted on the pen plotter with custom 3D-printed parts, the microphone attaches to the Raspberry Pi on its GPIO pins, the touchscreen has its own aforementioned stand that could be attached to the pen plotter, and the pen is attached to its lifting mechanism also via 3D-printed parts. Since most of the parts of my pen plotter will be 3D-printed, save for the parts that I buy, I will make sure the parts are of high quality. And since my cable managing will be custom designed, I will make sure ports are accessible in said designs through cutouts.

Testing

When I further advance in getting seperate parts completed, I will test each module individually, then test them together. So for input I would make sure Whisper's transcription is accurate. Then for processing, I'd ensure the GPT-generated G-code is correct and simulate it before running (if I get there). Then make sure the RPi communicates with GRBL properly and USB serial comms are stable. Then make sure my output is outputting and I can repeatedly count on accurate pen plotter movements.

Failure Modes

Audio: There could be misheard commands or poor mic sensitivity that prevents it from properly inputting raw audio from the get-go.

Text Parsing: The prompt could be too vague and the AI might misunderstanding and not generate the proper G-code.

Plotter: There might be missed steps in the g-code or missed steps on the plotter's end. The pen lift servo could have issues and that might lead to the pen not lifting and trailing on the paper. I was originally worried about shaking, so I did not go with a T-shaped pen plotter design and instead went for a CoreXY design so it would be more stable.

Overview

This week was all about bringing the individual components of my final project together into a unified, functional system. This week made me think deeply about how each component depends on the others. Importantly, it forced me to get a deeper grasp on exactly how my machine needs to operate to fulfill the vision I had when I first started this project. It also made me really realize how much work I have cut out for me and how tightly I should stick to the Gantt Chart I made two weeks ago.

Last update: May 14, 2025