14. Interface and Application Programming¶
Weekly Digest (Thu. May 2 to Wed. May 8)¶
| Date | Time | Place | Activities |
|---|---|---|---|
| Fri. May 3 | 10:00-14:00 21:00-24:00 |
Home | Review of the Week14 Global Session |
| Sat. May 4 | 10:00-19:00 | FabLab Kannai | Local Session on Week14 |
| Mon. May 6 | 9:00-15:00 16:30-24:00 |
Home (Nagaoka) | Documentation of Group Assignment for Week14 Documentation of Individual Assignment for Week14 |
| Tue. May 7 | 13:00-18:00 | Office (NID) | Exploration of p5*js |
| 19:30-24:00 | Home (Nagaoka) | Exploration of MIT App Inventor and Documentation of Individual Assignment for Week14 | |
| Wed. May 8 | 13:00-18:00 | Office (NID) | Exploration of p5*js, rbCanvas/p5 and BLE connection with Xiao board |
| 18:30-20:30 | Cafe | Redesign of the UI, Documentation of Final Project Page |
Individual Assignments Overview¶
Here is the list of individual assignments given by the instructors. As for the Group Assignment, please see the Group Assignment page on the FabLab Kannai website:
Write an application that interfaces a user with an input &/or output device that you made
Design of the Final Project Mobile App with MIT App Inventor¶
As is described in my Final Project page, setting up the Bluetooth connection with the device and monitoring it with the mobile app are one of the key components of my project. I really wanted to spend much time considering the user interface of the mobile app and for the sake of time saving, I decided to work on the app development with MIT App Inventor.
1. Comparison with Other Mobile App IDEs¶
I first thought of the other few options, including Android Studio, MacroDroid and “Node-RED”. The reason for choosing the App Inventor is:
-
Android Studio: I tried to learn app development by myself a few years ago with this IDE with a study guide available in Japanese language. I found that it’s a good IDE to finetune the app development to the professional level. But I was discouraged to see its response different from what I had expected. I couldn’t follow the same steps described in the study guide. Also, I had to install the IDE and it was huge.
-
MacroDroid: As part of preparation for my Final Project, I studied the way to develop the mobile app and found that MacroDroid had already been offering the development environment for the users to develop the customized solutions. I installed the app into my mobile phone via Google Play and tried to develop one app that if you come into the 500 meter radius of the home wifi, it automatically turns on the wifi connection to that particular wifi or vice versa. We could develop a very simple trigger-action logic and make and deploy the app with your mobile phone. But at first sight it seemed that the top page is not required. Although we could switch the language setting to English and other languages, the basic development environment is in Japanese.
- Node-RED: Node-RED was recommended by our local instructor. Because I had a past experience in flow-based programming using the MESH IoT Blocks, I was also tempted to learn more about the flow-based programming envirnment like Node-RED. However, my time management alerted me that I should get the result as soon as possible leaving the exploration on Node-RED aside for now.
2. Tutorial¶
To start with, I learned how to develop the mobile app following the first tutorial available on the Get Started site. I just followed the instruction. My first experience with the App Inventor tutorial was described in the Group Assignment page.
3. Design Policy¶
This is the sketch of my idea for the mobile app. It consists of two pages:
-
TOP Page: For setting up the Bluetooth Low Energy (BLE) connection. Also, once it loses the BLE connection with the device, it starts siren and moves to the second page;
-
Second Page: For turning off the siren and moving back to the TOP page.
I considered installing the program to resume the BLE connection automatically. But what the users want first after they lose the BLE connection will be the warning that they should take a look around at what’s happening to their valuables. So I thought I should go without reconnecting to BLE.
4. Page Design¶
For the app page design, I used the past design data I uploaded on the Week1 individual assignment page: Hokusai/Seigaiha/Siren. For siren sound, I downloaded from charge- and loyalty-free BGM Dova Syndrome website.
5. Source Code Available¶
Fortunately I could find the instruction manual “MIT App Inventor - IoT BLE Basic Connection” in the MIT App Inventor page. Basically I followed this instruction and made the basic code blocks.
Please note that I have added a few more blocks to the basic code blocks on BLE connection.
- TOP Page: I added one block that rings a siren and move to the Second page once it loses BLE connection with the device.
- Second page: I added a block that enables the user to stop the siren and move back to the TOP page for reconnection.
6. Test Connection via AI Companion¶
Once both the page design and coding were finished, I proceeded to the test connection with the Raspberry Pi Pico W program for BLE connection I tried in the 3. Alternative Scenario with Raspberry Pi Pico W in Week13.
Then back in the App Inventor, I opened the AI Companion by clicking “Connection” tab. When I clicked “AI Companion”, the “Connect to Companion” dialogue window was popped up.
Then I took my mobile phone and opened the “MIT AI2 Companion” app. (Please note that the app had already been installed.) I chose to scan the QR code shown in the AI COmpanion dialogue. Once the scanning process was done, my mobile opened the TOP page of the app developed.
When I clicked the “Scan” button, then I could receive the list of the Bluetooth devices. Once I got the list of the devices, I clicked the “Stop Scan” button. Then I tapped “rpy-uart” and clicked the “Connect” button. My mobile phone and the device were paired up.
Once my mobile phone loses connection with the device, it started sending out a siren sound and moved to the Second page.
Lastly, after moving to the Second page, I clicked “Stop Siren Now” button. Then the page automatically moved back to the TOP page.
However, during the first test connection, I faced a few problems that need to be rectified.
-
Length of Siren: I found the siren sound was too short for the users to notice. Although I chose the siren sound which could last more than 2 minutes, it went off in just 2 seconds. Also, with the maximum volume, the sound was still a bit small.
-
Failure in the Second Connection and Later: Another issue is that it failed to reconnect to the device again and again. I tried to follow the same steps from scanning to connecting. But every time I clicked the “Connect” button, I saw the status label “Disconnect” before it was actually connected even just once. It looked like the program proceeded to “Disconnect” before they were connected. I spent a few hours considering the alternative blocks to ensure the reconnection but so far I haven’t found the way out. (Note (Wed. May 8): When I somehow rectified the problem in uploading the BLE setup on my Xiao board and did the second connection test, using the Xiao ESP32C3 board, this second connection failure didn’t take place.)
-
Design Failure on the TOP Page: I even tried to build Android App (APK file) to see if it would go better. I exported the app data and installed it in my mobile. Once I tapped the icon, I saw the TOP page missing the Screen Title “Passport Alert App” on top of the TOP page. Another issue was that the siren sound was even smaller when they lost connection. It was almost silent.
-
Problem in Uploading to Xiao Board: For this connection test, I used the RPi PicoW board and my sample code for BLE connection I prepared in Week13. Before I decided to do it with PicoW, I once tried to flash codes to Xiao ESP32C3 board many times only to end up with “Status 2” error again and again. So far I haven’t found the solution on this, too. (Note (Wed. May 8): This problem seemed to occur when I failed to reboot the board. Somehow, I found the way out with Xiao ESP32C3 and did the second BLE connection test. Although the first problem about the siren was still existing, the second issue didn’t occur with the Xiao board.)
7. Final App Design for Final Project (Updates on June 22, 2024)¶
For the final app design that a user could be interfaced with the output device I made, please read 6. Bluetooth Connection in the Final Project page.
The above four issues were somehow addressed. You can see how the whole system works by the following hero shot.
I would also share the App design data both in terms of apk file and aia file. Apk file is for installing the app on your mobile phone. However, in order to customize the App program according to the type of your mobile phone and the MAC address of your board for the Alarm Device, you have to select your phone size and input the MAC address manually. This is one of the unfriendliness of this app for the users.
This issue will be addressed in a month or two. But for the time being, expecting that the users will type the MAC address manually on the MIT App Inventor IDE, I would like to post aia file so that they could import this program and remix or improve it on their laptop.