Objectives

Individual assignment: write an application that interfaces a user with an input &/or output device that you made

Group assignment: compare as many tool options as possible

14. Interface and application programming

UI and its types

A User Interface (UI) is the medium through which a user interacts with a device, software application, or system. It includes all the controls and visual elements that allow users to:

Input commands
Manipulate or retrieve data
Receive feedback

The goal of UI design is to make these interactions intuitive, efficient, and visually pleasing ultimately enhancing the overall User Experience (UX).

Graphical User Interface (GUI): Uses visual components like windows, icons, buttons, and menus. Users interact via mouse, keyboard, touchscreen, etc.

Command-Line Interface (CLI): A text-based interface where users type commands. Efficient for experienced users, especially in system administration and development.

Touchscreen Interface: Found in phones, tablets, kiosks, and more. Enables intuitive interaction through gestures like tapping, swiping, and pinching.

Voice User Interface (VUI) Allows users to interact using spoken commands. Examples: Siri, Alexa, Google Assistant.

Virtual and Augmented Reality Interfaces (VR/AR)

VR: Immerses users in a fully digital environment.
AR: Overlays digital content onto the real world. Interaction happens through headsets, hand-tracking, or controllers.

UI for interactive website

As a part of explaining interface, I tried the codes provided by the instructor.

1.Button

      <!DOCTYPE html>
      <html lang="en">
         
      <head>
      <meta charset="utf-8">
      <meta name="viewport" content="width=device-width, initial-scale=1.0">
      <title>Button</title>
      </head>
      <body>
          button style="margin: auto;" id="press">Click me/button>
          <p id="para">
          <script>
              const button = document.getElementById('press');
              const para = document.getElementById('para');
              button.addEventListener('click' , function(e){
                  para.textContent = para.textContent + 'Clicked';
                  document.getElementById('para').style.color = 'red'
                  button.style.backgroundColor = button.style.backgroundColor === 'red' ? 'gray' : 'red';
              
      
              });

2.Realtimeinput

        <input type="text" id="myInput" name="myInput">
        <div id="output"></div>
        
        <script>
            const inputElement = document.getElementById('myInput');
            const outputElement = document.getElementById('output');
        
            inputElement.addEventListener('input', function(event) {
                const inputValue = event.target.value;
                outputElement.textContent = 'My name is ' + inputValue;
            
            //const input = (e.target.Value).split("");
            if (inputValue.length>8){
              outputElement.style.color = 'red'
            }else if(inputValue.length>4){
              outputElement.style.color = 'blue';
            }else{
              outputElement.style.color = 'green'
        
              }
            });
        
        
            
            </script>

KODULAR

Kodular is a free, drag-and-drop platform that enables users to create Android applications without writing traditional code. It features a visual interface where users can design apps by assembling components like buttons, sensors, and layouts, and it employs a block-based programming system based on Google’s Blockly, similar to MIT App Inventor and Scratch. This approach allows beginners and non-programmers to build functional Android apps using visual logic blocks instead of text-based code. While Kodular itself operates through these visual blocks, the apps created with it are ultimately compiled into Java code for Android. To facilitate real-time testing and debugging, Kodular provides a companion app called Kodular Companion. This app allows developers to instantly preview and interact with their projects on an Android device without the need to compile and install the APK each time. By scanning a QR code or entering a code generated in the Kodular creator interface, users can connect their device to the project and see live updates as they modify their app, making the development process more efficient and user-friendly.

To use kodular, I signed in to kodular. I didn't had an idea about how to use Kodular. So I referred Fabacadamey 2024 studentAnsu Thomas's documentation.

After creating project a new window opens. Kodular Creator has two main views:

Designer: Drag and drop components (buttons, labels, sensors, etc.).
Blocks: Create logic using visual blocks (like Scratch or Blockly).

I wanted to add a background image for my app. So i downloaded an, image . From the image icon The image is uploded. The changes were made in the background image option.

I dragged the horizontal arrangement from the layout. From properties panel I aligned it in the center. Then I draged button From user interface, and placed it inside the horizontal arrangement.I named button 1 as "Turn on" and button 2 as "turn off". I also draged Web from connectivity. Components like Buttons, Labels, Text Boxes etc..can be dragged from the Palette to the Viewer. Non-visible components like Bluetooth, Sensors, or Notifiers are avilable in the bottom of the palette. We can customize properties (text, color, size, etc.) using the Properties panel.

Then I switched to blocks tab. The block tab consists of event blocks, in the left panel. Drag event blocks and add logic.I copied the IP address Of XIAO board from the code and pasted it in the block.

I needed to upload the code in xiao Esp32 C6. I used the pcb from electronics production week. To get the code , I used chatgpt. Prompt and result included in the bottom part.

The code is given below

#include <WiFi.h>
#include <WebServer.h>

#define LED_PIN 18  // Define the GPIO pin connected to the LED

// Replace with your actual Wi-Fi network credentials
const char* ssid = "yourSSID";
const char* password = "Yourpassword";

// Create a web server object that listens on port 80
WebServer server(80);

// Handle the root page by sending an HTML page with ON/OFF buttons
void handleRoot() {
  String html = "<html><head><title>ESP32-C6 LED Control</title></head><body>";
  html += "<h1>ESP32-C6 LED Control";
  html += "<p><a href=\"/on\"><button>Turn ON</button></a></p>";
  html += "<p><a href=\"/off\"><button>Turn OFF</button></a></p>";
  html += "</body>";
  server.send(200, "text/html", html); // Send the HTML page to the client
}

// Turn the LED ON and redirect back to the root page
void handleOn() {
  digitalWrite(LED_PIN, HIGH);
  server.sendHeader("Location", "/");
  server.send(303);
}

// Turn the LED OFF and redirect back to the root page
void handleOff() {
  digitalWrite(LED_PIN, LOW);
  server.sendHeader("Location", "/");
  server.send(303);
}

void setup() {
  Serial.begin(115200);               // Start serial communication
  pinMode(LED_PIN, OUTPUT);           // Set the LED pin as output
  digitalWrite(LED_PIN, LOW);         // Initialize LED as OFF

  WiFi.begin(ssid, password);         // Connect to Wi-Fi
  Serial.print("Connecting to Wi-Fi");
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  Serial.println("\nConnected! IP address: ");
  Serial.println(WiFi.localIP());     // Print local IP address

  // Define routes for the server
  server.on("/", handleRoot);
  server.on("/on", handleOn);
  server.on("/off", handleOff);
  
  server.begin();                     // Start the web server
  Serial.println("HTTP server started");
}

void loop() {
  server.handleClient();              // Handle incoming client requests
}

I uploaded the code in XIAO ESP32 C6

Next we need to test the app. I downloaded kodular companion app in my phone. Then I clicked the test option. I got a QR code. I scanned the code using mobile app.

The result is shown below

Changing the colour of neopixel using kodular

The pcb board from communication week was used here.

I used chatgpt to get the code and logic. the prompt and result included in the bottom part. The initial setup were done as above.

The code I uploaded on esp32 c6 is given below. In place for ssid and password you need to upload the credinials of your wifi.

#include <WiFi.h>                    // Include Wi-Fi library for ESP32
#include <WebServer.h>              // Include WebServer library to create a web server
#include  <Adafruit_NeoPixel.h>      // Include Adafruit NeoPixel library

#define NEOPIXEL_PIN 18             // ✅ Using GPIO 18 for NeoPixel
#define NUMPIXELS 1                 // Number of NeoPixels (in this case, just 1)

const char* ssid = "SSID";          // 🔁 Replace with your Wi-Fi SSID
const char* password = "PASSWORD";  // 🔁 Replace with your Wi-Fi password

Adafruit_NeoPixel pixels(NUMPIXELS, NEOPIXEL_PIN, NEO_GRB + NEO_KHZ800);  // Initialize NeoPixel strip
WebServer server(80);              // Create a web server on port 80

// Function to set NeoPixel color
void setColor(uint8_t r, uint8_t g, uint8_t b) {
  pixels.setPixelColor(0, pixels.Color(r, g, b));  // Set color of pixel 0
  pixels.show();                                   // Update the NeoPixel to show new color
}

// HTTP handler to receive RGB values from URL parameters
void handleColor() {
  if (server.hasArg("r") && server.hasArg("g") && server.hasArg("b")) {  // Check if RGB parameters are present
    int r = server.arg("r").toInt();  // Convert red value from string to integer
    int g = server.arg("g").toInt();  // Convert green value
    int b = server.arg("b").toInt();  // Convert blue value
    setColor(r, g, b);                // Set NeoPixel to received RGB values
    server.send(200, "text/plain", "Color changed to R:" + String(r) + " G:" + String(g) + " B:" + String(b));  // Send response
  } else {
    server.send(400, "text/plain", "Missing RGB parameters");  // Send error if any parameter is missing
  }
}

void setup() {
  Serial.begin(115200);        // Start serial communication for debugging
  pixels.begin();              // Initialize NeoPixel library
  pixels.show();               // Turn off all pixels at startup

  WiFi.begin(ssid, password);  // Start Wi-Fi connection
  Serial.print("Connecting to Wi-Fi");
  while (WiFi.status() != WL_CONNECTED) {  // Wait for Wi-Fi to connect
    delay(500);
    Serial.print(".");
  }
  Serial.println("\nConnected! IP address:");
  Serial.println(WiFi.localIP());  // Print local IP address to access server

  server.on("/color", handleColor);  // Define server route for /color
  server.begin();                    // Start the server
  Serial.println("Web server started");
}

void loop() {
  server.handleClient();  // Continuously handle incoming client requests
}

I used vertical arrangement and added buttons.I dragged web from palette. Colour palette was not avilable. So I manually added buttons for each colour.

I tested it in the app.

Creating a gradient in neopixel

As there wasn't any colour palette available, I had to create one. I searched in youtube and find a tutorial. I followed it to connect the blocks. The designer and blocks view are given below.

initialize global color to ""

Declares a global variable color with no value at the start. It will be assigned a real color value (like "255,0,0") later during app interaction.

call Canvas1.GetPixelColor

This function reads the color of the pixel at the location where the user is dragging.

x = get current X → the X-coordinate of the touch location

y = get current Y → the Y-coordinate of the touch location

set Label1.BackgroundColor to <the color>

This sets the background color of Label1 to match the pixel color from the canvas where the user is dragging.

call Ball1.MoveTo get current X, get current Y

This block moves the ball (Ball1) to the current touch position on the canvas.

get current X / get current Y

: These give the current touch position (coordinates) on the Canvas.

call Canvas1.GetPixelColor

: This block fetches the color at a specific pixel on the Canvas based on the coordinates you give it.

set global color to ...

: This saves that fetched color into a global variable named color

set Web1.URL to

– You’re setting the URL that the Web component will call

join

– Combines pieces of text into one complete string (used to form a valid URL).

url/setcolor?r=

– The base URL path to your web server's color endpoint (e.g.,

http://192.168.1.100/setcolor

), followed by the start of a query string for red

(r=)

is a string like "255,100,50" (from

Canvas.GetPixelColor

split color

breaks it into a list

["255", "100", "50"]

select list item list

with index 1 gets the red component ("255").

Same logic applies for

&g=

and

&b=

with index 2 and 3, for green and blue values.

The tutorial didin't showed how to connect it to a board. So I used chatgpt to get the code. The promt and responses are provided in the bottom part of the page. The code is given below.

#include <WiFi.h>                // Include Wi-Fi library for ESP32
#include <WebServer.h>          // Include WebServer library to create HTTP server
#include <Adafruit_NeoPixel.h>  // Include NeoPixel library for controlling RGB LEDs

#define LED_PIN    18              // Define the GPIO pin connected to the NeoPixel
#define NUM_LEDS   1               // Number of NeoPixel LEDs used

const char* ssid = "SSID";         // Replace with your Wi-Fi SSID
const char* password = "PASSWORD"; // Replace with your Wi-Fi password

Adafruit_NeoPixel pixels(NUM_LEDS, LED_PIN, NEO_GRB + NEO_KHZ800); // Initialize NeoPixel
WebServer server(80);             // Create a web server on port 80

// Root handler: returns a basic text response
void handleRoot() {
  server.send(200, "text/plain", "ESP32 NeoPixel Color Controller"); // Basic root message
}

// Color control handler: expects RGB values as query parameters
void handleSetColor() {
  if (server.hasArg("r") && server.hasArg("g") && server.hasArg("b")) { // Check for RGB arguments
    int r = server.arg("r").toInt();  // Convert 'r' parameter to integer
    int g = server.arg("g").toInt();  // Convert 'g' parameter to integer
    int b = server.arg("b").toInt();  // Convert 'b' parameter to integer

    pixels.setPixelColor(0, pixels.Color(r, g, b)); // Set NeoPixel color
    pixels.show(); // Apply the color to the LED

    server.send(200, "text/plain", "Color set!"); // Send confirmation
  } else {
    server.send(400, "text/plain", "Missing RGB arguments"); // Send error if parameters are missing
  }
}

void setup() {
  Serial.begin(115200);       // Start serial communication
  pixels.begin();             // Initialize NeoPixel library
  pixels.show();              // Turn off all pixels initially

  WiFi.begin(ssid, password); // Start Wi-Fi connection
  Serial.print("Connecting to Wi-Fi");

  while (WiFi.status() != WL_CONNECTED) { // Wait until connected
    delay(500);
    Serial.print(".");
  }

  Serial.println("\nConnected to WiFi.");         // Confirm connection
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());                 // Display IP for access

  server.on("/", handleRoot);                     // Define root route
  server.on("/setcolor", handleSetColor);         // Define /setcolor route
  server.begin();                                 // Start the server
  Serial.println("HTTP server started");          // Confirm server is running
}

void loop() {
  server.handleClient();  // Continuously handle incoming client requests
}

To test the code I opened the Kodular companion app.

The result is shown below.

Group assignment

This week, we explored a variety of tools for creating computer interfaces to better understand how they can be used in our final projects. We worked with Kodular, a visual app builder for Android, to create mobile interfaces without coding. Using PyGame, we developed a simple snake game controlled via a custom macropad. With Processing, we built a visual servo controller using a slider. We also created web-based control interfaces using HTML, CSS, and JavaScript, enabling real-time communication with microcontrollers like the ESP32-S3 over WebSerial for tasks like servo motor control and robot navigation. Lastly, we looked at an advanced Flutter app that interfaces with the Spotify Web API, using an ESP32 to bridge mobile and hardware interaction via HTTP. These tools highlighted different ways to create interactive experiences across mobile, desktop, and web platforms.

Group assignment page

Files

The chatgpt prompt and responses are given below

ChatGPT Prompt 1: write the complete updated code

Prompt 1: write the complete updated code

To control your ESP32-C6 LED over Wi-Fi using Kodular, you can set up a simple web server on the ESP32-C6, and use Kodular's Web component to interact with that server. Below is a complete guide for creating the code for ESP32-C6 Wi-Fi control and the Kodular app setup to control the LED.

✅ Step 1: ESP32-C6 Code (Wi-Fi Web Server)

The ESP32-C6 will create a simple web server that listens for requests to turn the LED on or off. Here’s the Arduino code for it:

#include <WiFi.h>
#include <WebServer.h>

#define LED_PIN 8  // Define your LED pin

// Replace with your network credentials
const char* ssid = "Your_SSID";
const char* password = "Your_PASSWORD";

WebServer server(80);

void handleRoot() {
String html = "<html><head><title>ESP32-C6 LED Control</title></head><body>";
html += "<h1>ESP32-C6 LED Control</h1>";
html += "<p><a href=\"/on\"><button>Turn ON</button></a></p>";
html += "<p><a href=\"/off\"><button>Turn OFF</button></a></p>";
html += "</body></html>";
server.send(200, "text/html", html);
}

void handleOn() {
digitalWrite(LED_PIN, HIGH);
server.sendHeader("Location", "/");
server.send(303);
}

void handleOff() {
digitalWrite(LED_PIN, LOW);
server.sendHeader("Location", "/");
server.send(303);
}

void setup() {
Serial.begin(115200);
pinMode(LED_PIN, OUTPUT);
digitalWrite(LED_PIN, LOW);

WiFi.begin(ssid, password);
Serial.print("Connecting to Wi-Fi");
while (WiFi.status() != WL_CONNECTED) {
  delay(500);
  Serial.print(".");
}

Serial.println("\nConnected! IP address: ");
Serial.println(WiFi.localIP());

server.on("/", handleRoot);
server.on("/on", handleOn);
server.on("/off", handleOff);
server.begin();
Serial.println("HTTP server started");
}

void loop() {
server.handleClient();
}

🔧 Step 2: Kodular Setup

To control the ESP32-C6 from Kodular, you’ll need the Web component to send HTTP requests to the ESP32 server.

Add Components in Kodular:

Button1: Text = "Turn ON"
Button2: Text = "Turn OFF"
Web1: (This will send HTTP requests to the ESP32-C6)

Block Setup:

Here are the blocks to send HTTP requests from the Kodular app to the ESP32 web server:

📡 Button1 - Turn ON (HTTP Request for /on):

when Button1.Click
do
  set Web1.Url to "http://[ESP32_IP]/on"
  call Web1.Get

📡 Button2 - Turn OFF (HTTP Request for /off):

when Button2.Click
do
  set Web1.Url to "http://[ESP32_IP]/off"
  call Web1.Get

Replace [ESP32_IP] with the IP address printed in the Serial Monitor after the ESP32-C6 connects to your Wi-Fi network.

Response Handling:

You can use the Web1.GotText event to handle the response or display any confirmation, but for basic ON/OFF control, you may not need it.

⚙️ Testing the Setup

Upload the ESP32 Code: Upload the Arduino code to your ESP32-C6.
Connect to Wi-Fi: Open the Serial Monitor to get the ESP32's IP address.
Run Kodular App:
- Set the Web1.Url to the ESP32-C6's IP (e.g., http://192.168.x.x/on).
- Tap the "Turn ON" or "Turn OFF" buttons.
Control the LED: The LED should toggle based on the buttons you press in the Kodular app.

📝 Optional: Make ESP32 a Wi-Fi Access Point (No Router Needed)

If you don’t want to rely on a Wi-Fi router, you can set the ESP32-C6 as an Access Point. Modify the setup() function as follows:

WiFi.softAP("ESP32-AP", "123456789");  // Create Wi-Fi AP with SSID and password

This will let your device connect directly to the ESP32-C6.

📱 Test with Kodular

Once everything is set up:

Open the Kodular app on your phone (using Kodular Companion or a compiled APK).
Use the buttons to toggle the LED ON and OFF through the Wi-Fi network.

Let me know if you need further clarification, or if you'd like help with testing it in the Kodular environment!

ChatGPT Prompt 1: ✅ Final ESP32-C6 Code for NeoPixel on GPIO 18

Prompt 1: ✅ Final ESP32-C6 Code for NeoPixel on GPIO 18

Here's the final ESP32-C6 code for controlling a NeoPixel on GPIO 18:

#include <WiFi.h>
#include <WebServer.h>
#include <Adafruit_NeoPixel.h>

#define NEOPIXEL_PIN 18         // ✅ Using GPIO 18 for NeoPixel
#define NUMPIXELS 1             // Number of NeoPixels

const char* ssid = "Your_SSID";         // 🔁 Replace with your Wi-Fi SSID
const char* password = "Your_PASSWORD"; // 🔁 Replace with your Wi-Fi password

Adafruit_NeoPixel pixels(NUMPIXELS, NEOPIXEL_PIN, NEO_GRB + NEO_KHZ800);
WebServer server(80);

// Function to set NeoPixel color
void setColor(uint8_t r, uint8_t g, uint8_t b) {
pixels.setPixelColor(0, pixels.Color(r, g, b));
pixels.show();
}

// HTTP handler to receive RGB values
void handleColor() {
if (server.hasArg("r") && server.hasArg("g") && server.hasArg("b")) {
  int r = server.arg("r").toInt();
  int g = server.arg("g").toInt();
  int b = server.arg("b").toInt();
  setColor(r, g, b);
  server.send(200, "text/plain", "Color changed to R:" + String(r) + " G:" + String(g) + " B:" + String(b));
} else {
  server.send(400, "text/plain", "Missing RGB parameters");
}
}

void setup() {
Serial.begin(115200);
pixels.begin();
pixels.show(); // Turn off all pixels at startup

WiFi.begin(ssid, password);
Serial.print("Connecting to Wi-Fi");
while (WiFi.status() != WL_CONNECTED) {
  delay(500);
  Serial.print(".");
}
Serial.println("\nConnected! IP address:");
Serial.println(WiFi.localIP());

server.on("/color", handleColor);
server.begin();
Serial.println("Web server started");
}

void loop() {
server.handleClient();
}

🧱 Kodular Note
In your Kodular project, ensure that:

You've updated Web1.Url to match your ESP32’s IP, e.g.:

http://192.168.1.115/color?r=255&g=100&b=50

Everything else remains the same — you just needed to use pin 18, which this version does.

Would you like a visual block diagram, .aia file, or a Kodular blocks screenshot?

ChatGPT Prompt 3: there is no colour picker in the panel

Prompt 3: there is no colour picker in the panel

You're right — Kodular removed the classic Color_Picker component from the UI palette in recent updates. But don’t worry — you can still build a color picker using buttons, color blocks, or an extension.

✅ 3 Alternatives to Add Color Picker in Kodular:

Use Basic Buttons with Color Blocks
Use regular Button components and assign them background colors manually.
How:
- Add 4–10 buttons (e.g., red, green, blue, yellow, etc.)
- Set their background colors in the Designer > Properties panel.
- When clicked, each button sends its own RGB values.
Example Blocks:
```
when ButtonRed.Click
do
 call Web1.Get with URL "http://192.168.X.X/color?r=255&g=0&b=0"
```
Use the Dynamic Components Extension + Color List
You can dynamically generate a palette of colors using a grid of buttons or canvases.
Let me know if you want to do this — I can show you how to use the Dynamic Components extension.
Use a Custom Color Picker Extension
Use an external .aix extension to add a full color picker.
Example Extensions:
- ColorTools by Sunny Gupta (AI2 Community)
- Taifun's Color Picker
You can import these in Kodular:
1. Go to Assets > Import Extension (.aix)
2. Upload the .aix file.
3. Drag the new component onto your project.

Would you like:

A prebuilt .aia project with buttons as a color palette?
Or help installing a color picker extension?

ChatGPT Prompt 4: these extensions not working

Prompt 4: these extensions not working

Got it — since the extensions aren't working and Kodular no longer provides a native Color Picker, the most reliable way is to manually create a color palette using buttons.

This method is 100% stable, requires no extensions, and works well with NeoPixel control.

✅ Create a Color Palette Using Buttons (No Extensions)

🎨 Step 1: Designer Setup

Add a HorizontalArrangement or TableArrangement to organize color buttons.
Add several Button components inside — for example:
- ButtonRed
- ButtonGreen
- ButtonBlue
- ButtonYellow
- ButtonWhite
- ButtonOff
For each button:
- Set the Background Color (e.g., red, green, etc.)
- Set Text to the color name (optional)
- Width: 50px or % to make it tile nicely

🧱 Step 2: Add Web Component

Drag in a Web1 component from Connectivity.
Set Web1.Url in the blocks dynamically (don’t set it in the designer).

🧩 Step 3: Blocks

Example for each color:

when ButtonRed.Click
do
  set Web1.Url to "http://192.168.1.115/color?r=255&g=0&b=0"
  call Web1.Get

when ButtonGreen.Click
do
  set Web1.Url to "http://192.168.1.115/color?r=0&g=255&b=0"
  call Web1.Get

when ButtonBlue.Click
do
  set Web1.Url to "http://192.168.1.115/color?r=0&g=0&b=255"
  call Web1.Get

when
::contentReference[oaicite:4]{index=4}



      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      
      

      

    
      Students agreement
        Copyright 2025 Ancy Raju - Creative Commons Attribution Non Commercial 
        Source code hosted at gitlab.fabcloud.org