Week 15: System Integration

This week, the Task was to continue working on the final project, including programming, design, electronics, and assembly. In particular, it should be considered how to integrate all components of the system in a nice way.

This Week’s Tasks

• Design and document the system integration for your final project

This Week’s Lecture

This week’s lecture covered what could go wrong when designing a product from an engineer’s perspective and how to circumvent that. Topics covered were usability, design for machining, documentation tools accompanying the design process (OSH and a KiCad BOM extension). We were advised to design nice packaging for our product (considering surface finish, look & feel, shape, …) and to consider the routing and mounting of wires. Basic quality assurance methods were highlighted (shaking, burn-in, power cycling, etc.). as an example assignment, I looked at the one by Edwin Dertien. More notes on this can be found in my lecture notes.

Programming

In the input-devices week (9), I set up a gyroscope for reading pitch and roll of the glove.

In the networking-and-communication week (11), I set up the ESP32S3 to send MIDI signals to the laptop via USB-MIDI. Together with my fellow student Niclas, I set up two ESP32S3 boards to communicate with each other. Both can be seen in the two videos below.

At that point I was only using the ESP-IDF for development, because I did not find a USB(-MIDI) library that worked with the ESP32S3 available for the Arduino environment. The only ESP-IDF library available for the BNO085 board only implemented an SPI interface for communication with the sensor board. The developers argued that the I2C communication at the side of the Adafruit board behaves faulty sometimes, so they did not even implement it. However, when I started integrating the gyroscope with the MIDI communication, the SPI communication with the sensor board sometimes did not work.

The measured data are packed into the following struct.

1typedef struct __attribute__((packed)) {
2    uint8_t pitch;
3    uint8_t roll;
4} MIDIData_t;

The struct is packed. That means that data are written directly after another in the memory and there is no padding added. It is explained in this forum post as well. I did this to have complete control over how the structure is assembled on bit-level. The reason for that is that Niclas and I had issues with correctly casting received packets. I do not know if the bit-level position of the values in the struct were playing a role or if the problem was something else, but it worked. Sending the values from the remote to the dongle can be seen below.

I then programmed the dongle to process the pitch as usual CC messages, but the roll as NOTE_ON/NOTE_OFF messages. Furthermore, the remote only sends MIDI data for the gyroscope’s pitch if the value is larger than 20. The reason is how MIDI controllers are mapped to software parameters. It is done by telling the software to listen to any MIDI input and then only change the one knob to be mapped to a parameter. To be able to either send pitch CC messages or send NOTE_ON/OFF messages, the user needs to be able to send only one of type of message at a time. With the current implementation the user just needs to hold the controller so the front faces down to deactivate sending pitch values. This, however, is not optimal, since it is not possible for the user to send values below 20. One could, however adapt the range of the CC value so that the current value 20 would correspond to a new value of 1. The code can be seen below.

 1void midi_send_data(MIDIData_t data)
 2{
 3    // Use roll as a switch.
 4    // ..when the gyroscope is turned downwards, a note-on message is sent. When it is turned upwards, a note-off message is sent.
 5    const int roll_threshold = 63;
 6    // ..introduce hysteresis to prevent oscillations of states.
 7    const int roll_thresh_upper = roll_threshold + 10;
 8    const int roll_thresh_lower = roll_threshold - 10;
 9    if (data.roll > roll_thresh_upper && !midi_roll_last_msg_was_note_on){
10        uint8_t note_on_roll[3] = {
11            NOTE_ON | CHANNEL,
12            MIDI_ROLL_NOTE_NUM,
13            120, // Velocity. 
14        }; 
15        // ..ensure, NOTE_ON and NOTE_OFF can only occur alternating.
16        midi_roll_last_msg_was_note_on = true;
17        #ifdef USB
18        tud_midi_stream_write(CABLE_NUM, note_on_roll, 3);
19        #endif
20        ESP_LOGE(TAG, "NOTE_ON");
21    } else if (data.roll < roll_thresh_lower && midi_roll_last_msg_was_note_on) {
22        uint8_t note_off_roll[3] = {
23            NOTE_OFF | CHANNEL,
24            MIDI_ROLL_NOTE_NUM,
25            120, // Velocity. 
26        }; 
27        midi_roll_last_msg_was_note_on = false;
28        #ifdef USB
29        tud_midi_stream_write(CABLE_NUM, note_off_roll, 3);
30        #endif
31        ESP_LOGE(TAG, "NOTE_OFF");
32    }
33
34    // Use pitch as knob.
35    // ..only activate if certain threshold is exceeded.
36    uint8_t pitch_inv = 127 - data.pitch;
37    const int pitch_thresh = 20; 
38    if (pitch_inv > pitch_thresh){ 
39        uint8_t cc_msg_pitch[3] = { 
40            CONTROL_CHANGE | CHANNEL,
41            MIDI_PITCH_CONTROL_NUM, // Index. Also known as "control number". CC 16-19 indicates a "generic control".
42            pitch_inv, // Data. Also known as "control value". Encodes the "position" of the "knob".
43        };
44        #ifdef USB
45        tud_midi_stream_write(CABLE_NUM, cc_msg_pitch, 3); 
46        #endif
47    }
48}

The below video shows the controller operating the cutoff-value of a low-pass filter which again is applied to a distorted sine oscillator.

Electronics

In the electronics design week (6) I learned how to use KiCad for designing PCBs. In the electronics production week (8) I learned how to fabricate a PCB using a CNC mill for isolation milling. The pictures below show the development process of the electronics of the project.

What I Learned

Packaging

Housing

3D and 2D design was taught in multiple weeks. Most about CAD I learned during the computer-controlled cutting week (3), the computer-controlled machining week (7), and mechanical & machine design week (12). As a CAD software I used FreeCAD. With FreeCAD I designed the housings for the PCBs, as well as the wristbands to mount the remote onto the forearm. Below the different features of the housing is explained. The design is parametric, but rather to document specific dimensions than to make it adjustable. Some parameters can be adjusted, be careful though. There are two FreeCAD projects, one for the dongle and one for the remote. Each project consists of multiple files. In each project, the file called d.FCStd contains a spreadsheet with the corresponding parameters. There, most dimensions are expressed as a sum of an actual dimension and a tolerance associated with it. This way, the dimensions and tolerances can be understood.

Wristband

To mount the PCB onto the forearm, two wristbands were fabricated. These were then threaded through the pockets in the bottom part of the housing.

What I learned

Assembly

Bonus: Surface finish

This section is included, because it is in general interesting for system integration, but it was not necessary for this project.

If it is not enough to just have a print, but to have a very smooth surface, there are different methods to achieve that. One is to use this product by Smooth-On. Another method is to use filler material (called Bondo or Spachtelmasse, in particular Feinspachtel). The procedure is as follows

• Put on safety equipment as described on the product. In my case this was ear, eye and respiratory protection as well as gloves.
• Add a little bit of hardener (included in the product) to the filler material and mix it well.
• Put a good amount onto the print so that every surface is covered. Do not put too much, as it would need to be grind away later.
• Let everything dry, roughly 24 hours.
• Grind everything to have a smooth surface wearing gloves to not put any residual fat or dirt onto the object (also wearing protection as above).
  • first with roughness 120 (not too rough, otherwise this could add gaps to the surface).
  • then with even finer roughness (like 500)
• Then, spray the part.
  • Be careful! Have the nozzle 15 to 20 cm away from piece to not put too much coating in one place.
  • Wait 15 minutes for one layer to dry.
  • Add multiple layers while grinding between adding them. Again, be careful.

Digitial Files

The digital files for this assignment are the ones provided on the final project page.

Use of Language Models

During writing this report, I did not use any language models.