Project Development

Real life gaming machine in 3rd person’s view

download files

what will it do?

Making a head tracking back-sided mounted cameras. and virtual a reality headset.

we only see the world in first person perspective, since our eyes have never left our face. I wanted to make gaming machine controlling myself in third person perspective.Using Augmented reality, I can display personal information like Heart rate or speed. And can make a simple game using this informations and computer vision.It could make me feel like control myself in game like GTA or Call of Duty.

Plans

1. Head tracking Pan & Tilt module

Components :
MPU6050, Atmega168, ATtiny44, wires, resistors, capacitors

Programming :
Arduino IDE using MPU6050 library

Process :
Sensor module(MPU6050) - I2C communication with Atmega get power from MCU module
MCU modules(Atmega168) - get sensor data from sensor and send command to servo module
Servo module(Attiny44) - control the two servos with receiving data from serial bus

2. Display module

**There are no fab-parts in this module..**
Components : 7'' TFT LCD, HDMI cable, Raspberry pi, Pi-camera module

Process :
Application programming
openCV + pygame - I'm going to split video to each eyes using openCV library.
I can make simple games with pygame library. I'm thinking about some games below.
1. Being a spy - Player has to avoid to meet people. If camera detect any face, game is over.
2. Dancing game - There is a game someone made using openCV and pygame libraries called 'danceCV'

3. Back Mount

Materials :
12mm Plywood for rigs and hinges, 1.5mm plywood for composites, M6 bolts and nuts, Thick rubber band, Fastener, Iso pink for mold, resin and hardener for composites

parts :
Raspberry pi, Fabduino(Atmega 168)

Equipments :
Shopbot CNC router

Process :
Compression composites - layers of 1.5mm plywood
CNC - hinges and rigs with 8mm plywood
Assemble : press-fit between hinges and back mount, bolts and nuts between rigs and hinges

4. Pan Tilt Camera Mount

Materials :
ABS filaments,

Parts :
M6 bolts and nuts, M2 bolts and nuts, Two 9g Mini servos, Servo module(ATtiny 44), Lipo battery

Equipments :
3D printer

Process :
3D printing
Assemble : Press-fit between 3d printing parts, M2 bolts and nuts between servos and mount.

5. Head Mount

**I'm going to modify the google cardboard 7'' lcd**

Materials :
Cardboard, 2 lenses, thick rubber band, double-sided tape

Equipments :
Laser cutter

Update#1

In my Input device assignment, I failed to make MPU6050 I2C communication board with 1/64 diameter endmill. The endmill was too big for the tiny component. So we ordered 0.01 mill. I’m waiting for arriving. Before it's arrived, I did quick test with GY-521 breakout board, arduino Uno and mini servos.

Arduino Code : MPU6050test.ino
Library : Jeff Rowberg has made an I2C lib: http://www.i2cdevlib.com/devices/mpu6050
Reference : http://playground.arduino.cc/Main/MPU-6050
Wire connection : SCL - A5, SDA - A4, INT - D2

It works very well. Next time, I'll try with my own boards.

Update#2

I tried to solder sensor board using a Heat gun with my instructor's help , and I Failed. I designed for MPU6050 I2C communication board. but It is too tiny for cut by 1/64 inch endmill. I trusted my hand and cutter.

designed the board	challenged with hand and knife
Aweful soldered and It doesn't work

Update#3

I design the pan&tilt camera mount using Autodesk Inventor. I wanted to make it to press-fit contruction. I 3d printed it many times for finding the nice tolerances for press-fit. You can download the every design files at here.

<design>

Change dimensions for tolarences	Press-fit construction design
Make fillets for strength	Assembly test

<3D printing>

I 3d printed it with stl files which is exported from Autodesk Inventor. It's press-fit construction without using bolts and nuts, so It's very important that makes right tolerances. I changed value (layer heights, dimensions of the design files) and remake it again and again. I finally found perfect dimensions and setting for the 3d printing. It takes about 3~4 hours for 5 files.

Download STL files

setting : layer height / 0.2mm, infill / 20% (infill / 100% for ServoStand files), print speed / 70%.

tolerance failed it was easy to break	3d print with ABS filament (Ultimaker)
needed parts for assembly	Final assemble

<assembly video>
I made a video to show how to assemble it and it's processes.

Update#4

Raspberry pi setup

It was almost my first raspberry pi project. so I struggled even when I start linux. I have a Raspberry pi Model B. And I followed this tutorial.
1. downloaded OS image file (raspbian) from the page
2. put SDcard in raspberry slot, connect mouse, keyboard, HDMI Cable, micro USB cable.
3. the Power LED is turned on but the lcd didn't get any signal
4. I solved this in config.txt (link). opened the file and changed to hdmi_force_hotplug=1 . then It worked.

Display(LCD) setup

I have a 7'' tft lcd module (CLAA070NC0BCT : datasheet). It is interfacing with LVDS, so I needed to HDMI to LVDS converter like this. but I decided to use DVI to LVDS converter since I could easliy get from a local shop. It was around 30 US dollars. Then, I also needs HDMI to DVI cable.

Pi camera setup

Why Pi-camera module?
I chose The Pi camera module rather than USB webcam. I found the reason at this forum "it's 'run' by the GPU and can dump full frames into RAM at 15 frames a second this is 7.5MB/frame, 15fps = 112.5 Mega BYTES per second .. or you can have full HD resolution 30fps H264 encoded (by the GPU) along with some simultaneous still photos (Google MMAL) all at virtually zero CPU loading"

Get started
Very well documentation on this page
1. The flex cable inserts into the connector situated between the Ethernet and HDMI ports
2. Boot up the Pi and log in
3. sudo apt-get update (It must be online, I failed at first time because I didn't check the wifi connection)
4. sudo apt-get upgrade
5. sudo raspi-config, Navigate to “camera” and select “enable”.
6. Select “Finish” and reboot.

Using MMAL library and raspivid/raspistill source code
I tried to follow this tutorial. However, I failed on step:3 when I compiled it. It seems that everyone get hard times at same reason. I struggled with it and spent a lot of time because of my lack of knowledge about linux and e. I decided to look for another mothod.

I solved with v4 l2 driver. I followed the instruction on this page (korean)

1. enable the camera
pi@rasplay ~ $ sudo raspi-config

2. Activate the v4l2 driver

pi@openmake ~ $ sudo modprobe bcm2835-v4l2

pi@openmake ~ $ ls /dev/video0 
/dev/video0

3. Get registration key

pi@rasplay ~ $ wget http://www.linux-projects.org/listing/uv4l_repo/lrkey.asc && sudo apt-key add ./lrkey.asc    

4. Install the package

pi@rasplay ~ $ sudo apt-get install uv4l uv4l-raspicam

5. test

pi@rasplay ~ $ uv4l --driver raspicam --auto-video_nr --width 640 --height 480 --encoding jpeg --frame-time 0
     

Update#5

I redesigned the Fabduino board. because I'm going to use 12V power source for HDMI to DVI convertor, I will drop the source down to 5V for my fabduino.

UPDATE#5

I designed Fabduino board for sensor Input pinout and Servo pinout. For enough power to servo motor, I used 12V battery connector and Regulator LM2940. sensor Pin out is connected with MPU6050 sensor, and It will do I2C communication.

Eagle files

Trace	Outline and drilling
fabricated with Roland SRM-20	Soldered

After soldered the board, I bootload arduino on it and uploaded arduino code what I used in Update #1. It works well.

Failure and solving

I used 20MHz resonator at first, It doesn't work, not bootloaded at all. I changed it to 8MHz resonator, then It works well. It was a clock problem.

UPDATE#6

ADDING STABILIZING CODE ON IT. It doesn't use sensor data directly, It makes sum of the data and use it's average data when moving servo motor. It works. It reduced servo's vibration, but Not perfectly. I'm going to try with more values.

arduino code

//the Changed parts (only loop function)

void loop() 
       {
 mpu.getMotion6(&ax, &ay, &az, &gx, &gy, &gz);
 
 TotalY=TotalY-valY[i];
 TotalX=TotalX-valX[i];
 valY[i] = map(ay, -17000, 17000, 10, 170);
 valX[i] = map(ax, -17000, 17000, 10, 170);
 TotalY=TotalY+valY[i];
 TotalX=TotalX+valX[i];
 i++;
 if(i>=10)
 {
 i=0;
 PromedioY = TotalY/10;
 myservoY.write(PromedioY);
 Serial.println(PromedioY);
 PromedioX = TotalX/10;
 myservoX.write(PromedioX);
 Serial.println(PromedioX);
 }
 }

UPDATE#7

I made a Head mount display modified Google Cardboard. The Google cardboard has open-source design file but It is only for small smartphone like 5'', I'm going to use 7 inch lcd, so I modified it. Also I don't want to make magnetic button, so I deleted the designs for button.

Check dimensions	Design in Illustrator
Needed parts and equipments, laser cut	LCD cable
Lcd Cable pin out	Assemble with Velcro
lenses assembled and lcd test!

UPDATE#8

UPDATE#9

I didn't know that MPU6050 only can sensing acceleration acurately. When I want to detect z axis rotation, another sensor like compass sensor is needed. For this project, I decided to just using yaw and pitch motion, using just x accel, y accel. But I added joystick module for adding more interfaces. If push button is clicked, the mode is changing. There are two mode. one is automatically head motion mode, and the other is manual mode using joystick.