3D Table:Concept

Multimodal interaction with phisical and virtual world. 3D technology since its first appearance on the market has been able to amaze and attract attention in various fields , from leisure , research and industry . Just like in the movies , futuristic virtual reality is mixed with the real world scenarios opening ever wider interconnection man - machine . Kinect , Leap Motion , Oculus Rift are making this a reality.

Haptic Interfaces. In contexts in which the only vision of what is happening is not enough for the user in order to ensure proper control (surgical robotics , space , remote handling ) , are useful and necessary haptic interfaces , or devices that allow you to manipulate an object , real or virtual , and receive tactile sensations in response (feedback).

The final project is related to a wider idea we want to develop in our R&D laboratory. The idea was born remembering old games , especially the Pin Art , the desk -toy patented Ward Fleming . It is a box containing a matrix crowded of tacks that are free to slide in an independent manner such as to create a three-dimensional surface . From this idea sprang the aim of studying the possible uses of interfaces that can change appearance and shape dynamically as the pixels of a screen.

Starting from a single pixel I‘ll make an architecture composed by an environment for the tracking of objects or gestures through a kinect , mounted on a pedestal , reproducible through simulation environment based on web3d and finally through a pin table consisting of a series of actuators ( servo motors and RGB LED ) and drive all by means of a control logic.

Why a 3D Table?

Create new ways of Human-Machine Interaction,

in order to improve User Experience, Usability and Accessibility

As just wrote a wider idea by starting in small

And ….solid thinking…making (almost) anything and putting together

1. I can interact with my 3D Object in Holographic Pyramid, for example  a City 3D Model. I could rotate the object, highlight a part, explore its composition seeing details that in standard ways I would never have noticed
2. I can highlight a part of object and send it to my SMART Table.
3. SMART Table is composed by:
   • 3D Shape Display where the 3D City  Model is rendered  and related  3D thematic layers can be displayed
   • Two interactive displays built by  a  Interactive Display Builder (IDB): the first one shows the 3D object or its element and users could draw annotations or select details,while the second one shows a virtual keyboard for insert textual annotations by users.
4. A user could watch the 3D Model on the Dynamic Shape Display and see users annotations thanks to augmented reality (ex. by Google Glasses).

The experience could continue by sending the data of an element of 3D model  to a 3D printer…..

3D Table: Applications and Implications

3D Table...in summary

What will it do?

Start in small. A platform of motorized blocks that can render 3D physical content, so that users can interact with digital information in a tangible way. . Using a Deph sensor (Kinect or Xtion Pro Live) to detect depth and rgb maps, a microcontroller system , receives this data and pilots servomotors that raise small parallelepipeds (depth data) and leds (rgb data) reproducing a 2.5 (surface) representation of perceived image frames. An alternative is sending data from an application interface that simulates the reproduction of the surface.

Who's done what beforehand?

The project is similar to the activities carried out in recent years by the Tangible Media Group at MIT Media Lab . The institute presented inFORM a " Dynamic Shape Display" , a motorized platform blocks that can render 3D content into physical , so that users can interact with digital information in a tangible way. In inFORM the images are produced by means of a projector, we want reproduce image map by means of rgb leg of each pin.

What materials and components will be required?

The platform will be composed of:

• Table structure: wood planes, plastic pin, steel wires, servo motors, metal skeleton...
• Electronic equipment: At beginning the idea was to use arduino, finally I will use the new fabkit -sashakit- designed by my colleague Daniele. Leds and other electronics components to control servos. A deph sensor, maybe a Asus Xtion Pro Live.
• Software components for image acquisition, elaboration and for controlling servos

Where will they come from?

Table structure components will come from local providers

How much will it cost?

The system is composed of a set of servors+rgb leds so is modular and extensible. The cost will depend of the matrix dimensions plus a fixed cost for single equipment (for example deph sensor). At beginning in my R&D lab the idea was to make a table with a matrix of 64 pixels/motors. For the final project I scheduled to test a minimun set, that is a line composed by 8 pin/motors. In summary:

• Table structure:metal frames, plywood sheets, nuts and bolts and plexiglass sheet: 60 euros
• Pin system: servo + rgb led+ wires+square aluminum tube+ plexiglass tube and bar +armonic wire:16 euros per pin
• Electronic equipment: at least 4 sashakit (servo, red, green, blue) + other electronics accessories (TLC5940 board: minimum for 16 pins) +deph sensor = 180 euros

What parts and systems will be made?

Most of the structural and electronic components will be made by myself

What processes will be used?

During my Fab Academy assignment I try to think about my final project. In particular I will use the acquired knowledge in the following processes:

• Electronic Production: PCB laser engraving, Soldering components
• Laser cutting
• 3D Printing
• Embedded Programming
• Interface and Application Programming

What tasks need to be completed?

The main tasks:

• 1.Pin movement system
• 2.RGB LEd system
• 2.Depth and image acquisition
• 3.Information transmission
• 4.Servo and Led Controlling

What questions need to be answered?

• 1.Translation of deph and image information in a pin movement
• 2. best approach to command servos and leds

What is the schedule?

Cause the final goal is hard to complete by the final presentation deadline I decided to proceed with a cyclic process in order to achieve incremental results.

How will it be evaluated?

A Fist evaluation step will be controlling pin system by sending data to control boards and move servos and control leds. Once reached this results the next goal is to implement the entire process from 3d acquisition to 3d table reproduction.

3D Table: Intellectual Property and Income

As just described my final project is a small part of a bigger project that involves several ideas about sensor networks, human interaction, virtual 3D imersion and smart cities. Once finisched the 3D table will be used by other collegues in order to integrate other R&D results mainly related to software systems. As regards dissemination activities in the last month I presented the idea to visiting students and presented to a my project kick-off in order to exploit the system as possible media to reproduce 3D data. 

Another exploitation may be, as discussed with my collegue Daniele Ingrassia, to reproduce surface data belonging from sensors mounted on his AAVOID drone system (thermal sensors, deph sensor etc..)

About the licence, I specified a Creative Commons CCA license for all my Fab Academy work and software source code and projects are licensed under the terms of the GNU Lesser General Public License v2.1. For my final project presentation only table boards software is released, cause the other software is under development in other r&d projects.

3D Table: Project Development

The first step was to think about a single pixel in order to define a first version of the table and its structure.

On the top we have a transparent pin moved by a servo motor connected by an armonic steel wire. on the bottom of the pin a rgb led is connected

My final project as described is and will focused mainly on table structure and driving starting from a deph and colour information from a sensor.The following video shows by means of a web application written with web3d libraries in order to simulate the table ( in simulation the pins and motors are many more: are free ;-)

Starting from this early prototype I think about the overall structure, and taking into account the total transmitted torque a metal structure solution has been adopted.

Download the eagle file of table layers

Each table layer contains a component of system table layer ( electronics->servo motors-> transmission->pin-push-> pins and leds. The introduction of a pin-push facilitates the stability of the edge transparent pin during its linear movement.

#define FASTADC 1
#ifndef cbi
#define cbi(sfr, bit) (_SFR_BYTE(sfr) &= ~_BV(bit))
#endif
#ifndef sbi
#define sbi(sfr, bit) (_SFR_BYTE(sfr) |= _BV(bit))
#endif

#include 
#include "Leds.h"
#include "Utils.h"
#include "Tlc5940.h"
#include "tlc_servos.h"


	
#define VREF     5.0  // analog reference
#define SERVO    64   // led number
#define DELTA    61   //  TLC in ms

Leds morse(7);
Utils util;

boolean update            = false;
boolean contact           = false;

char buffer[5] ;
int  p             = 0;
int  delayTime     = 2;
int  debug         = 1; // debug = 0 disabled - debug = 1 enabled
long oldTime       = 0;

void setup()
{ 
  tlc_initServos(); 

  Serial.begin(9600);
  morse.led();
 
  establishContact();
}

void establishContact() {
  while (contact==false) {
    Serial.println('S');
    delay(200);
    
    if (Serial.read()=='K'){//K
      contact=true; 
      morse.off(); // offLed();
      if(debug!=0) Serial.println('K');
     
    }      
  }
}
 
void loop()
{
  if (Serial.available() >0)
  {
    
    morse.led();
    
    char inByte = Serial.read();
    
    if (inByte =='#') // # PP RR servo number and value 
    {
     
      while (p<4)
      {
        buffer[p] = Serial.read();
        Serial.println(buffer[p]);
        p++;
      }
      byte pp    = util.hex2Decimal(buffer[1])+util.hex2Decimal(buffer[0])*16;
      byte angle = util.hex2Decimal(buffer[3])+util.hex2Decimal(buffer[2])*16;
      if(debug!=0){
        Serial.print("# ");
        Serial.print(pp);
        Serial.print(" ");
        Serial.println(angle);
      }
      if (angle>4 && angle<176){ 
        handleServo(pp, angle);
        update=true;
      }
    }

    else if (inByte =='T') // T XX delay time
    {
      while (p<2)
      {
        buffer[p] = Serial.read();
        p++;        
      }
      delayTime = util.hex2Decimal(buffer[1])+util.hex2Decimal(buffer[0])*16;
      if(debug!=0){
        Serial.print("T ");
        Serial.println(delayTime);
      }
    }
    else if (inByte =='S') // S reset servo position
    {
      for(int i=0; iDELTA){
    Tlc.update(); 
    update=false;
    oldTime=millis();
  }
  delay(delayTime); 
}

/****************************************************** TLC SERVOS */
void handleServo(int pin, int angle)
{
  if(debug!=0){
        Serial.println("moving servo ");
        Serial.print(pin);
        Serial.print(" ");
        Serial.print(angle);
      } 
  tlc_setServo(pin+0, angle);
}

#define FASTADC 1
#ifndef cbi
#define cbi(sfr, bit) (_SFR_BYTE(sfr) &= ~_BV(bit))
#endif
#ifndef sbi

#define sbi(sfr, bit) (_SFR_BYTE(sfr) |= _BV(bit))
#endif

#include 
#include "Leds.h"
#include "Utils.h"
#include "Tlc5940.h"

	
#define VREF     5.0  // analog reference
#define LED      64   // led number
#define MAX      2     
#define DELTA    21   

Leds morse(7);
Utils util;

boolean update            = false;
boolean contact           = false;

byte buffer[5] ;
int  p             = 0;
int  delayTime     = 2;
int  lum           = MAX;
int  debug         = 1; // debug = 0 disabled - debug = 1 enabled
long oldTime       = 0;

void setup()
{ 
  Tlc.init();

  Serial.begin(115200);
  morse.led();
  
  establishContact();
}

void establishContact() {
  while (contact==false) {
    Serial.println('R');
    delay(200);
    
    if (Serial.read()=='K'){//K
      contact=true; 
      morse.off(); // offLed();
      if(debug!=0) Serial.println('K');
     
    }      
  }
}
 
void loop()
{
  if (Serial.available() >0)
  {
    
    morse.led();
    
    byte inByte = Serial.read();
    
    if (inByte == '#') // # PP RR led number and value 
    {
      while (p<4)
      {
        buffer[p] = Serial.read();
        p++;
      }
      byte pp    = util.hex2Decimal(buffer[1])+util.hex2Decimal(buffer[0])*16;
      byte led   = util.hex2Decimal(buffer[3])+util.hex2Decimal(buffer[2])*16;
      if(debug!=0){
        Serial.print("# ");
        Serial.print(pp);
        Serial.print(" ");
        Serial.println(led);
      }
        
      handleLed(pp, led);
      update=true;
    }

    else if (inByte == 'L') // L XX intensity
    {
      while (p<2)
      {
        buffer[p] = Serial.read();
        p++;        
      }
      lum = util.hex2Decimal(buffer[1])+util.hex2Decimal(buffer[0])*16;
      if(debug!=0){
        Serial.print("L ");
        Serial.println(lum);
      }
    }

    else if (inByte == 'T') // T XX delay time
    {
      while (p<2)
      {
        buffer[p] = Serial.read();
        p++;        
      }
      delayTime = util.hex2Decimal(buffer[1])+util.hex2Decimal(buffer[0])*16;
      if(debug!=0){
        Serial.print("T ");
        Serial.println(delayTime);
      }
    }
    else if (inByte == 'S') // switch off all leds
    {
      Tlc.clear(); Tlc.update();
      if(debug!=0){
        Serial.println("S ");
      }
    } 
    else if (inByte =='X') // X XX active (1) - no active (0) debug 
    {
      while (p<2)
      {
        buffer[p] = Serial.read();
        p++;        
      }
      debug = util.hex2Decimal(buffer[1])+util.hex2Decimal(buffer[0])*16;
      if(debug!=0){
        Serial.print("X ");
        Serial.println(debug);
      }
    } 
    morse.off(); 
  }
  p=0;

  if(update & millis()-oldTime>DELTA){
    Tlc.update(); 
    update=false;
    oldTime=millis();
  }
  delay(delayTime); 
}

/****************************************************** TLC LED RGB */
void handleLed(int pin, int led)
{
  Tlc.set(pin+0, led*lum);
}

A first try has been performed by adopting a dual layer disposal of motors in order to reduce the distance of neighbour motors respect to the vertical axis of the pi. Obviously the dimension of the motors and their arms are greater than the dimension of each pixel/pin. This first try was useful to test a first release of motor and led drive.

but this solution provide complexity and space reduction on the composition of the entire motor system and transmission (wires are too near). So the approach is to return on a single linear disposal of motors with the support of a rubber tube inside which steel wire flows.

After this test in the second phase I enhanced the table by adding other components, the final part of pin system:

Download box rhino model

And new arm for servors. I modelled and print the arm according to the specification that length of plexiglass is 7 cm and the max excursion has to be of 6cm.

Download the arm rhino model

3D Table: Project Presentation