17. Mechanical Design

We are living in a social distancing society under the coronavirus situation.

However, to make the most out of it, 7 students in FabLab Kamakura and FabLab Kannai came together (online of course) to make something great that operates on their own, but even greater when being connected over the network.

Introducing MODS - Machine Orchestra in a Distant Society.

It is an orchestra with each music machine playing on their own, being connected through the network using MQTT.

Network for communication

We discussed about the way of collaboration in options like:

• Option1(Photo transistor sensing): Stream score image streamed using Zoom, individuals read the information using photo transistor to sense the change og light.

• Option2(MQTT): Publish musical note and subscribe it at a distanced location, then make sounds

Option1: Photo transistor sensing - “Haru” Hideo Oguri (Kamakura)

As one of the options to share the musical information and collaborate with several musical machines, I tested the Photo Transistor to read the streamed score image by video conference system ( Zoom, etc. ).

Firstly, I tried PhotoResistor, although it worked fine, I moved to PhotoTransistor because of its fast response.

Also, I tried the sensor below, but it didn’t work on the LED screen on PC, because it was Infrared sensor.

PhotoReflector ( LBR-127HLD, Cut-off Wavelength = 840nm )

The sensors I used were ,,,,,

PhotoTransistor ( NJL7502L, visible light sensor ; Peak Wavelength = 560nm )

I made sensor box which has 4 sensors in it, and attached it on the LED of my MacBook Pro.

( in the test below, only one set of sensor/solenoid was used. )

Test example 1

I programed ESP32 to hit the drum solenoid 5 times/second ( on-time ; 20 msec) when the light sensor detected the black colored area. By scrolling the Black and White spreadsheet, I checked the response of the system.

The sensor position was set to detect the wide black colored band, solenoid was controlled almost fine.

( Although, the scrolling speed was not controlled precisely, the time period of back colored area is about 0.7 second. )

Test example 2

This time, the sensor position was set to detect the narrow black colored band, the number of solenoid ON was not stable ( twice, once, zero ). I think this was caused by the timing of the solenoid control ( programing ).

As a conclusion, this system “Photo Transistor to read the streamed score image by video conference system ( Zoom, etc. )” will work fine, if we can prepare the “Photo Transistor sensor boxes” for each machine and the “Music score image”. But considering the time limitation and to explore the networking technology ( MQTT ), I put this plan on hold.

Option2: MQTT - Tatsuro Homma (Kannai)

Because of time constraint in mechanical designs and machine building week, we decided to use option2(MQTT) because it allows everyone’s machine with ESP32 to receive musical message only by connecting to Wifi network(rather than option1 using photo resistor and tuning the input value)

Also, I checked some communication protocols of services to validate pros/cons to network multiple devices that coverw two-way communication.

	MQTT	HTTP	IFTTT
Overview	Message Queuing Telemetry Transport. publish-subscribe network protocol that transports messages between devices	Hyper Text Transfer Protocl. Communication between web clients and servers	If This Then That a web-based service that creates chains of simple conditional statements
Network entities	a message broker and a number of clients	a web server and a number of clients	web services and web services
Need server?	No(configure MQTT broker)	Yes	No(configure IFTTT web service)
Protocol	Publish/Subscribe (Publisher pushes message to subscriber)	Request/Response (Client requests content to web server. Web socket can cover 2-way communication )	IFTTT itself works on HTTP. One service trigers the other service(ex. when a sensor get values and it to Twitter over IFTTT)
Performance	light	heavier than MQTT(packet is bigger than MQTT)	depends on the web service

From points of easiness of implementation(serverless communication between machines) and performance(quickness of publish/subscribe sequence and small packet), I chose MQTT as a communication protocol.

Building system and experiment in networking week

I did experiment in networking week and did some experiment with Oguri-san and Kimura-san. In detail:

• Guideline and template program using MQTT Pubsub client library on ESP32 (Kannai group page in week of networking)

• MQTT Broker Configuration on AWS IoT (my networking week page)

• Expreiment result(my networking week page)

Consideration about delay between messages in MQTT

We found some issues of this system as a platform to play music in remote location. Though MQTT’s performance is excellent as a communication protocol, it has limitation as “a rhythm/message sequencer” (of course this protocol is not for music:).

• I intended to publish one message as one beat. Of course the time span between messages is affected by the internet performance. It might be good to implement streaming with buffering total or chunk of music information. Another considerable idea is sharing clock time by absolute time in multiple client and sync music if the big size of data is allowed in network and client environment.
• Performance is also affected by home Wifi connection from ESP32.

After some trial experiment with Oguri-san to check rhythm in subscriber end, we found that publish and subscribe one message every 2 seconds is durable span for sequential music. That’s why, the publisher push message every 2 seconds in this system.

Music publisher

As ESP32 cannot do multi-task at the same time, I needed to separate publisher and subscribers in different boards. For visualization and ease of operation, I tried to setup Node-RED on Raspberry Pi as a publisher of musical note.

• Raspberry Pi 3 model B

At first, I connected to MQTT broker on AWS IoT to MQTT broker from RaspPi via Wifi network in my home. After checking the performance, I moved the connection by LAN cable for stability of messaging.

• Node-RED flow for publishing musical note

At function node of “make note”, musical note is made like bellow.

var interval = msg.delay;
var seq = msg.seq;

function makeNotes(t1, t2, t3, t4, t5, t6){
    msg.payload =  {
        "seq":seq,
        "count": msg.count,
        "t1":t1,
        "t2":t2,
        "t3":t3,
        "t4":t4,
        "t5":t5,
        "t6":t6,
        "interval": interval
    }
}

// For limiting note number in range of Kimura-san's recorder.
function checkNote(note){
    if (note > 72) {
        note = note - 12;
    }
    if (note < 60) {
        note = note + 12;
    }
    return note;
}

function melodySequence(seq, t1s) {
    var beat = seq % 8;
    var t1;

    switch(beat){
    case 1:
        //t1 = checkNote();
        makeNotes(t1s, 1, t1s, 1, 0, 1); // ド ソ C
        break;
    case 2:
        t1 = checkNote(t1s + 7);
        makeNotes(t1, 2, t1s, 0, 1, 1); // ソ ミ C
        break;
    case 3:
        t1 = checkNote(t1s + 4);
        makeNotes(t1, 3, t1s, 1, 0, 0); // ミ ド C
        break;
    case 4:
        t1 = checkNote(t1s + 11);
        makeNotes(t1, 4, t1s, 0, 1, 1); // シ ソ C
        break;
    case 5:
        t1 = checkNote(t1s + 7);
        makeNotes(t1, 5, t1s, 1, 0, 0); // ソ ド C
        break;
    case 6:
        t1 = checkNote(t1s + 9);
        makeNotes(t1, 6, t1s, 0, 1, 1); // ラ ミ  C
        break;
    case 7:
        t1 = checkNote(t1s + 7);
        makeNotes(t1, 7, t1s, 1, 0, 1); // ソ ド C
        break;
    default:
        t1 = checkNote(t1s + 4);
        makeNotes(t1, 8, t1s, 0, 1, 0); // ミ ド C
        break;
    }
}

var beat = seq % 8;
switch(1 <= beat && beat <= 4){
    case true:
        melodySequence(seq, 60); // C
        break;
    case false:
        melodySequence(seq, 65);  // F
        break;
}

return msg;

Node-RED flow on Raspberry Pi(link to individual reposictory at gitlab) Node-RED_flow_publish-music-notes_20200603_presented.json

Instrument machines

Elmo the Drunken Conductor - “Yume” Menghe Xu (Kamakura)

When we shot the group video, it was a prototype where Elmo was attached to LEGO and servo motor (and several alcohol bottles) to conduct.

Afterwards, I made the machine using laser cut acrylic.

Design files

• involute gear .svg file

• arm support .dxf file

• leg support .dxf file

• frame .dxf file

• bottom .dxf file

• head holder .dxf file

• Fusion 360 design .f3d file

Slide guitar machine - Tatsuro Homma (Kannai)

Guitar played manipulating a stepping motor and 3 solenoids.

Machine design(link to Autodesk A360)

Software on ESP32(link to individual reposictory at gitlab) mqtt_aws_music_mods_sample_v1_01.ino

Hue - Hiroaki Kimura (Kamakura)

Shinobue (Japanese flute) being controlled using 8 solenoids with the ESP32 to play a melody.

A compressor for a construction site nailer was used as the breather.

Revolving glasses percussion - Yuki Oka (Kamakura)

The machine hits the glass using cam mechanism which is controled by DC motor. The table in the middle can revolve by servo motor and gears. So it can change its sound by switching the target glass.

Tin Tin Arm - Toshiki Tsuchiyama (Kamakura)

A robot arm created by cutting acrylics, moved using servo motor made to hit glasses filled with water.

Hitting Can Machine - Kazuki Yanome (Kamakura)

A geared motor swings the nut tied by a wire, and makes a sound by hitting the nut to a fish can.
The length of the wire is arranged to be able to hit a fish can when the motor is close to a can.

Solenoid Drum Machine - “Haru” Hideo Oguri (Kamakura)

Solenoids beating the miso packages to create a cool rhythm.

Future Development opportunities

• Internet connection over Wifi network occurs delay to communicate MQTT message. It will be good to connect machines by LAN cable. The other idea is separating quick rhythm sequence device from MQTT subscriber and mixing music by the othe way wheareas the most big delay would happen by Zoom meeting system. We could find the other system for real time music session.

• For future opportunity to continue performance, as “team MODS”, we applied to DIY MUSIC on DESKTOP on 25th July 2020 that is spinoff event organized by Maker Faire Tokyo/Kyoto . This is online event that shares the music performance by Zoom video conference from staying at home. If that application is accepted, students who can join would perform in real time together or showing recorded video to audiences in Youtube Live.