week 14. Interface and application programming

This week's brief:

Individual assignment:

Write an application that interfaces a user with an input &/or output device that you made

Group assignment:
Compare as many tool options as possible

GROUP ASSIGNMENT
You can view our group assignment here!

INDIVIDUAL ASSIGNMENT

JavaScript

This week we were introduced to javascript functionalities. JavaScript is used to program the behavior of web pages. We add or modify HTML content to create user friendly and interactive websites.

As a stepping stone into this language, we first tried out very simple codes and exercise on how to get real-time previews when a button is clicked or when we type something into a textbox. We manipulated the codes to make the interfacing interactive in different ways.

Our Instructor, Midlaj, taught us how to use JavaScript to make a Webserial . WebSerial is a remote terminal which is accessible via browser to log, monitor or debug your firmware remotely.

I coded the board i made during the electronics design week, to make the LED turn on when "1" is pressed and turn off when "0" is entered. T

Then in an html page we made use of a script that was written to make an interface to connect the serial port and to send the values to the board.

<script>
let port;
let writer;
let reader;
let toggle;

async function connectToSerial() {
    port = await navigator.serial.requestPort();
    await port.open({ baudRate: 9600 });
    
    writer = port.writable.getWriter();
    reader = port.readable.getReader();

    const decoder = new TextDecoder();

    while(true) {
        const { value, done } = await reader.read();
        if (done) break;

        const string = decoder.decode(value);
        document.getElementById('outputMsg').textContent += string;
    }
}

async function sendMessage() {
    const input = document.getElementById('input').value;
    if (!writer) return;

    const encoder = new TextEncoder();
    await writer.write(encoder.encode(input + '\n'));
    document.getElementById('input').value = ""
}

async function toggleLED() {
  if (!writer) return;

  const dataToSend = toggle ? '1' : '0'
  const encoder = new TextEncoder();
  writer.write(encoder.encode(dataToSend)).then(() => {
    if (dataToSend === '0') {
      document.getElementById('toggle').innerHTML = ``;
    } else {
      document.getElementById('toggle').innerHTML = ``;
    }
    toggle = !toggle;
  });
}

document.getElementById('serial').addEventListener('click', connectToSerial);
document.getElementById('upload').addEventListener('click', sendMessage);
document.getElementById('toggle').addEventListener('click', toggleLED);
<script>

An async function enables asynchronous,non-blocking operations, allowing the program to remain responsive while waiting for tasks to complete.

An await operator is used in async functions to pause execution until a completion of task occurs. This makes asynchronous code look and behave more like synchronous code.

A websocket is hosted where you can connect your device and send a 1 or 0 to turn on/ off the LED on the board

Making our own Interface

This week i wanted to explore Pygame for making a game interface with the macropad i made during Output week. As you can see there is an underlying game theme going on throughout the projects i have been doing. I wanted to get used to figuring out the logic required for programming and how the concept of interface works so that i can implement it into my final project . I thought , why not make a simple game and learn and see how much i would have to prep for the making the game for my final project . I already had an idea to somehow program the macropad to making it into a standalone gaming console. So basically this week opened up that oppurtunity to integrate the assignment and doing somethinh fun!. So the goal for me this week was to make a snake game and control the movements using my macropad through serial communication. Whenever the game starts/stops ,snake eats the food or hits a wall the macropad should play a melody to denote the action . Additionally the addressable LEDs should light up when you lose the game. The main challenge is to make the macropad interact with the game on the screen through serial ports and vice versa.

Pygame

Pygame is a cross-platform set of Python modules designed for writing video games. It includes computer graphics and sound libraries designed to be used with the Python programming language.

Pygame requires Python, you can download it from python.org.

To install pygame i used the the pip tool,which is what python uses to install packages.

  python3 -m pip install -U pygame --user

PySerial

Serial communication allows data exchange between a computer and peripheral devices such as microcontrollers, sensors, and other hardware modules. One of the most popular libraries for serial communication in Python is PySerial.It is used to read and write serial data to Arduino or any other Microcontroller.

To install pyserial type :

  pip install pyserial

To check whether the serial communication is working i got this sample code to test. i used VScode to write out the python code first to send data over serial

  # Importing Libraries 
import serial 
import time 
arduino = serial.Serial(port='COM4', baudrate=115200, timeout=.1) 
def write_read(x): 
	   arduino.write(bytes(x, 'utf-8')) 
	   time.sleep(0.05) 
	   data = arduino.readline() 
	   return data 
while True: 
	   num = input("Enter a number: ") # Taking input from user 
	   value = write_read(num) 
	   print(value) # printing the value

Next we have to write a code in Arduino IDE to initiate a connection to python. Make sure you give the correct communication port.

        int x; 
        void setup() { 
          Serial.begin(115200); 
          Serial.setTimeout(1); 
        } 
        void loop() { 
          while (!Serial.available()); 
          x = Serial.readString().toInt(); 
          Serial.print(x + 1); 
        }

Understanding Pygame

It was interesting to learn the coding and programming but since this was a new language it took a while to get the hang of it and understand the commands. So to get started with the language and functions i followed a tutorial to make a basic code that :

makes a display box
makes an object(rectangle)
moves the object with keyboard keys.
moves the object using macropad.

In the tutorial, they only showed how to interact with the screen using the keys on the computer keyboard. So i had to try to make the code to make a serial connection from my PCB to the computer.

I had to upload a code into the microcontroller to send a serial to the python code. This data should be sent according to the switch that is being pressed.That is . If switch 1 is pressed then a serial.write needs to be performed where the string "UP" will be sent to python. .I made the following code after understanding the serial read and write commands.

        //Code for sending data to python when a switch1 input is registered

#define switch1 26
#define switch2 28


int a = 0;
int b = 0;


bool SerUpdate;



void setup() {

  Serial.begin(9600);
  pinMode(switch1, INPUT);
  pinMode(switch2, INPUT);
  SerUpdate = false;
}

void loop() {
  a = digitalRead(switch1);
  b = digitalRead(switch2);
  if (a == LOW && !SerUpdate) {
    Serial.print("a");
    SerUpdate = true;

  } else if (b == LOW && !SerUpdate) {
    Serial.print("D");
    SerUpdate = true;
    delay(300);
  } else {

    SerUpdate = false;
  }
}

The following code was written to test for two switches to write UP and DOWN to pyserial. Similarly all four switches were written.

I modified the python code to read the serial data and move the rectangle positions. .I ran the code and it was reading the switch input data correctly

At first the reading rate was very slow, the rectangle was moving very slowly. after going through the codes we figured out that it was probably due baud rate and increased the rate from 9600 to 115200 to 200000.

There was still a lag while reading serial. We found that it was because the readline command only closed after taking all the inputs given by the user instead of giving it on one by one. So instead of giving a string ling i gave a single character to be read and compared. This worked out correctly and the flow of interface was smooth.

          #code for shifting positon of rectangle 
          #Code from Coding With Russ modified by namita 

          import pygame
          import sys
          import serial
          
          
          # to read serial data
          ser = serial.Serial('COM20', 9800, timeout=1)
          ser.readline()
          
          
          pygame.init()
          # for the display window
          SCREEN_H = 600
          SCREEN_W = 800
          screen = pygame.display.set_mode((SCREEN_W,SCREEN_H))
          pygame.display.set_caption("My Pygame Window")
          
          player = pygame.Rect((300,250,50,50))
          
          run = True
          while run:
              for event in pygame.event.get():
                  if event.type == pygame.QUIT:
                       run = False
                   
              pygame.draw.rect(screen,(128, 0, 128),player)
          
           # to receive
          
              input= ser.read() 
              swinput =str(input,'UTF-8') 
              # variable to store the incoming serial data
              print(swinput)
          
          
          # to send
          
          
          
          #  comparing the strings ,if they match the position shifts 
              if(swinput == "U"):
                  player.move_ip(0,-30)
          
              elif(swinput == "D"):
                  player.move_ip(-30,0) 
                 
              elif(swinput == "J"):
                  player.move_ip(30,0) 
                     
              elif(swinput == "A"):
                  player.move_ip(0,30) 
                 
          
              pygame.display.update()
              
          
          pygame.quit()
          sys.exit()

It turned out to be a plotter now xD.

SNAKE GAME

After playing around with the example code and understanding the serial port communication and pyserial, I moved on to create the main part of the assignment - To make the Snake Game

I need to understand the logic of the game first. I found a code to make a basic snake game work from Edureka . But then again, like the previous example code the keys used to move were from the computer keyboard. So i applied the same logic ,to write serial data from arduino to python when switch is pressed.

For Mapping the keys

I Wrote a code for the switch input provided by the user to be sent as a character to the python code.

I dissected the Python code and the rewrote the code to make the snake move using the macropad


#snake game 1st attempt to control usoing macropad

import pygame
import sys
import serial


ser = serial.Serial('COM20', 9800, timeout=1)
ser.readline()

pygame.init()

SCREEN_H = 600
SCREEN_W = 800
screen = pygame.display.set_mode((SCREEN_W,SCREEN_H))
pygame.display.set_caption("My Pygame Window")

player = pygame.Rect((300,250,50,50))

run = True
while run:
    for event in pygame.event.get():
        if event.type == pygame.QUIT:
             run = False
         
    pygame.draw.rect(screen,(128, 0, 128),player)

    datatosend = 

    input= ser.readline()
    swinput =str(input,'UTF-8')
    print(swinput)

    if(swinput == "UP"):
        player.move_ip(0,-30)

    elif(swinput == "DOWN"):
        player.move_ip(0,30) 
 

    pygame.display.update()

pygame.quit()
sys.exit()


      //arduino ide code for up and down controllers 

      #define switch1 26
      #define switch2 18
      #define switch3 28
      #define switch4 14
      
      int a = 0;
      int b = 0;
      int c = 0;
      int d = 0;
      
      bool SerUpdate;
      
      
      
      void setup() {
      
        Serial.begin(9600);
        pinMode(switch1, INPUT);
        pinMode(switch2, INPUT);
        pinMode(switch3, INPUT);
        pinMode(switch4, INPUT);
       SerUpdate = false;
      
      }
      
      void loop() {
      a = digitalRead(switch1);
      b = digitalRead(switch2);
      c = digitalRead(switch3);
      d = digitalRead(switch4);
      
      
      
      
      
      if (a == LOW && !SerUpdate){
        Serial.print("UP");
        SerUpdate = true;
        delay(300);
        
      }else if (b == LOW && !SerUpdate) {
        Serial.print("JUMP");
        SerUpdate = true;
      delay(300);
      
      }else if (c == LOW && !SerUpdate) {
        Serial.print("DOWN");
        SerUpdate = true;
       delay(300);
      
      }else if (d == LOW && !SerUpdate) {
        Serial.print("ATTACK");
        SerUpdate = true;
        delay(300);
      
      
      }else
      
      
         SerUpdate = false;
      
      
      
        // delay(100);
      
      
      }

swinput is the string in which the value of 'input' from the serial code is being read , given through the ide code.
ser.write(b'0)' when the if condition is satisfied the python code sends '0' to the board which will execute the funtion of turning on the LEDs
ser.write(b'1)' when the if condition is satisfied the python code sends '0' to the board which will excute the funtion of turning off the LEDs

Here there was an issue of the the snake moving very slowly at first. I tried moving around the serial commands within the loops but it did not work properly. So i seeked the help of our instructors Saheen and Midhlaj to figure out where the loop was facing an issue. After many iterations it was found that there were issues with the indentations and where the commands were sitting within the loop.

Next i added on to the ide code to make the melody tones play when the game starts and a game over tune. I also added sound effects when a button is pressed to move the snake and another beep when the snake eats the food.

Then i added the serial write commands into the python code where the actions should be performed.

Another issue that had to be resolved was restarting the gameloop when the user presses the key "c" on the keyboard and quit the game when "Q" is pressed.

The Modified final Codes(as of now)

Python Code

  #snake game by edureka , modified by Namita
# to be controlled using an external macropad

  import pygame
  import time
  import random
  import serial
  
  
  # declared a null variable
  ser = ""  
  
  pygame.init()
   
  white = (255, 255, 255)
  yellow = (255, 255, 102)
  black = (211, 211, 211)
  red = (213, 50, 80)
  green = (0, 255, 0)
  blue = (21, 27, 84)
   
  dis_width = 600
  dis_height = 400
   
  dis = pygame.display.set_mode((dis_width, dis_height))
  pygame.display.set_caption('Snake Game by Edureka modified by Namita')
   
  #caption for display window
  
  clock = pygame.time.Clock()
   
  snake_block = 10
  snake_speed = 15
  #speed of snake 

  font_style = pygame.font.SysFont("lemonmilk", 25)
  score_font = pygame.font.SysFont("montserrat", 25) 
  #fonts for displaying 
  
   
  def Your_score(score):
      value = score_font.render("Your Score: " + str(score), True, white)
      dis.blit(value, [10, 0])
   
   
   
  def our_snake(snake_block, snake_list):
      # dis.blit(img, (snake_list[-1][0], snake_list[-1][1]))
      for x in snake_list:
          pygame.draw.rect(dis, black, [x[0], x[1], snake_block, snake_block])
   
   
  def message(msg, color):
      mesg = font_style.render(msg, True, color)
      dis.blit(mesg, [dis_width /6, dis_height / 3])
  
          
  def serialInit():
      global ser 
      ser = serial.Serial('COM20', 2000000  , timeout=1)
          # ser.read()
  def gameOver():
       ser.write(b'0')
  def gameRestart():
       ser.write(b'1')
  
  
  def gameLoop():
  #for checking is serial port is open/close
      serialInit()
      if ser.is_open:
          print("Serial port is open.")
      else:
          print("Serial port is closed.")
      ser.write(b'1')
      game_over = False
      game_close = False
   
  
      #position coordinates

      x1 = dis_width / 2
      y1 = dis_height / 2
   
      x1_change = 0
      y1_change = 0
   
      snake_List = []
      Length_of_snake = 1
   
      foodx = round(random.randrange(0, dis_width - snake_block) / 10.0) * 10.0
      foody = round(random.randrange(0, dis_height - snake_block) / 10.0) * 10.0
   
      while not game_over:
          
      
      #to quit or restart game
          while game_close == True:
              dis.fill(blue)
              message("You Lost! Press C-Play Again or Q-Quit", red)
              Your_score(Length_of_snake - 1)
              
              if ser.is_open:
                   ser.write(b'0')
                   ser.close()
              pygame.display.update()
  
              for event in pygame.event.get():
                  if event.type == pygame.KEYDOWN:
                      if event.key == pygame.K_q:
                          game_over = True
                          game_close = False
                          
                      if event.key == pygame.K_c:
                          message("Loading...", white)
                          gameLoop()
                         
          if ser.in_waiting > 0:
              input= ser.read()
              swinput =str(input,'UTF-8')
              print(swinput)
  
              if swinput == "D":
                  x1_change = -snake_block
                  y1_change = 0
              elif swinput == "J":
                      x1_change = snake_block
                      y1_change = 0
              elif swinput == "U":
                      y1_change = -snake_block
                      x1_change = 0
                      
              elif swinput == "A":
                      y1_change = snake_block
                      x1_change = 0
   #interfacing with macropad       
  
          
                          
          if x1 >= dis_width or x1 < 0 or y1 >= dis_height or y1 < 0:
              game_close = True
          x1 += x1_change
          y1 += y1_change
          dis.fill(blue)
          pygame.draw.rect(dis, green, [foodx, foody, snake_block, snake_block])
          snake_Head = []
          snake_Head.append(x1)
          snake_Head.append(y1)
          snake_List.append(snake_Head)
          if len(snake_List) > Length_of_snake:
              del snake_List[0]
      #If snake head touches the tail
          for x in snake_List[:-1]:
              if x == snake_Head:
                  game_close = True
   
          our_snake(snake_block, snake_List)
          Your_score(Length_of_snake - 1)
   
          pygame.display.update()
      #when snake encounters food 
          if x1 == foodx and y1 == foody:
              foodx = round(random.randrange(0, dis_width - snake_block) / 10.0) * 10.0
              foody = round(random.randrange(0, dis_height - snake_block) / 10.0) * 10.0
              Length_of_snake += 1
              ser.write(b'2')
   
          clock.tick(snake_speed)
   
      pygame.quit()
  
      quit()
  
  
   
  gameLoop()

Arduino IDE Code

  
// Code for pygame snake

#define switch1 26
#define switch2 18
#define switch3 28
#define switch4 14
//defining all the switches

#define ledPin 12 // neopixel pin

#include  // addressable LED library
#ifdef __AVR__
#include   // Required for 16 MHz Adafruit Trinket
#endif
#include  // library for LCD


// How many NeoPixels are attached to the Arduino?
#define NUMPIXELS 4
Adafruit_NeoPixel pixels(NUMPIXELS,ledPin, NEO_GRB + NEO_KHZ800);

#define DELAYVAL 500
// Adafruit_NeoPixel strip(PIXEL_COUNT, PIXEL_PIN, NEO_GRB + NEO_KHZ800);


int a = 0;
int b = 0;
int c = 0;
int d = 0;

bool SerUpdate;  //for debouncing

int incoming; 

const int buzzerPin = 22;

const int rs = 6, en = 7, d4 = 13, d5 = 11, d6 = 9, d7 = 8;
LiquidCrystal lcd(rs, en, d4, d5, d6, d7);

void setup() {

  Serial.begin(2000000);
  pinMode(switch1, INPUT);
  pinMode(switch2, INPUT);
  pinMode(switch3, INPUT);
  pinMode(switch4, INPUT);
  SerUpdate = false;
  pixels.begin();  // INITIALIZE NeoPixel strip object 
  lcd.begin(2, 16);
  // Print a message to the LCD.
  lcd.print("START GAME");

}

void loop() {
  a = digitalRead(switch1);
  b = digitalRead(switch2);
  c = digitalRead(switch3);
  d = digitalRead(switch4);

  incoming = Serial.read(); // variable storing serial data sent from python code

  if (incoming > -1) {  
    Serial.println(incoming);
  }

  if (incoming == '0') { // if serial read as 0 , game over ( turns on led and plays the game over tune)
 
    gameover_seq();
    gamefail();
    lcd.print("GAME OVER");
    lcd.clear();
    delay(100);
    clearPixel();
  }

  if (incoming == '1') { // if serail is read as 1 , restarts the game(turns LED off and plays the startup  tune)

    clearPixel();
    startup();
    start_seq();
    // lcd.clear();
    // lcd.print("GAME OVER");

  }
  if (incoming == '2') { //serial when the snake eats food

    tone(buzzerPin, 3500, 10);
  }

// checking if switch is pressed
  if (a == LOW && !SerUpdate) { 
    Serial.print("U");
    tone(buzzerPin, 2000, 20);
    SerUpdate = true;
    delay(300);

  } else if (b == LOW && !SerUpdate) {
    Serial.print("J");
    tone(buzzerPin, 2000, 20);
    SerUpdate = true;
    delay(300);

  } else if (c == LOW && !SerUpdate) {
    Serial.print("D");
    tone(buzzerPin, 2000, 20);
    SerUpdate = true;
    delay(300);

  } else if (d == LOW && !SerUpdate) {
    Serial.print("A");
    tone(buzzerPin, 2000, 20);
    SerUpdate = true;
    delay(300);


  } else


    SerUpdate = false;
}


// led sequence when game starts
void start_seq() {


  for (int i = 0; i < NUMPIXELS; i++) {

    pixels.setPixelColor(i, pixels.Color(0, 255, 0));
 
    pixels.show();
    // delay(300);
    clearPixel();
    // delay(DELAYVAL);
  }
}
// led sequence when game is over
void gameover_seq() {
  pixels.clear();
  for (int i = 0; i < NUMPIXELS; i++) {

    pixels.setPixelColor(i, pixels.Color(255, 0, 0));

    pixels.show();
  }
}

//clears led  colours
void clearPixel() {
  pixels.clear();
  pixels.show();
}


// buzzer melody sequence when game starts
void startup() {
  tone(buzzerPin, 262, 500);  // C4
  delay(200);

  tone(buzzerPin, 294, 500);  // D4
  delay(200);

  tone(buzzerPin, 330, 500);  // E4
  delay(200);

  tone(buzzerPin, 349, 500);  // F4
  delay(200);

  noTone(buzzerPin);
  delay(500);
}

// buzzer melody sequence when game is over
void gamefail() {
  tone(buzzerPin, 500, 200);
  delay(200);
  tone(buzzerPin, 500, 200);
  delay(200);
  tone(buzzerPin, 500, 200);
  delay(200);
  tone(buzzerPin, 800, 150);
  delay(150);
  tone(buzzerPin, 500, 500);
  delay(500);
  tone(buzzerPin, 600, 1000);

  delay(10000);
}

Finally the game was done and it was working quite well! I am surprised i was able to do learn and execute this in such a short span of time being completely new to the language and program (Of course with the help of our instructors whenever i was stuck with the programming).

The Game

The Game Window

Playing the Game

Making custom Keycap

Thhe keycaps that i was using above were taken from my instructors keyboard, So i made some custom 3d printed keycaps to indicate the arrowkeys on my macropad and spare his keys. The arrows were engraved on to the keycaps using xtools F1 Ultra.