Final Project – Stage Five (The Finale)

It’s time to hook up the whole system. The three of us met one final time to set up the flute (our lovely assistant, Box Cat) with grapes and the program Rebecca created to make this project interactive. While the Fruit Hero program was being finished, Paul and I worked on getting the Arduino and breadboard into the box and ready for play. We ran into some interesting issues here, and we really had no idea why. We put the wires for the grapes up through the top of the box, and put the boards inside the box. We plugged a 9V battery into the board to see if we could make the system wireless, but then the program wasn’t working. Touching the wires produced no sound. We thought the wiring was wrong, so we adjusted some things and tried again. Still no sound, or sometimes some notes would work and some wouldn’t. We then plugged the board back into the computer, and it worked fine. We scratched our heads in awe, and then went to Google for answers.


Turns out, the problem is in the way the system is grounded. In this forum, a user explains that the system works fine if you touch the battery when touching the sensors, basically allowing your body and floor to be the grounding for the system. So when plugging it in to the computer, the computer acts as the grounding for the system. Pretty neat. Ultimately we went with the plug, since we need the Arduino to be talking and listening to the computer anyway for interaction.

Once Rebecca finished finalizing the Fruit Hero program, we hooked the two up together and played our new game.










Some little details came in later, like adding in the “Call Me Maybe” music sheet and a color change for correct and incorrect notes. We also added in fireworks effects and celebration text to make the experience more dynamic. Since Paul is our musical prodigy, we let him demonstrate our project to the class, and they loved it.  Here is the link to the video:

Final Project – Complete. Thus, we end our time in Physical Computing.


Final Project – Stage Four

Last time, we got the serial connection working between the Arduino and Processing on the computer.

Now, it’s time to get into the technical details of the Grape Cat Hero game also dubbed as Fruit Hero.

This was my first time messing around with Processing.  It was pretty easy to figure it out, but the documentation needs to be updated for a couple of things.

1.  fill(r, g, b) requires ints for the parameters r, g, and b not floats as the documentations says.

Ex: fill(255, 0, 0) produces a red color.

2.  if(myPort.available() > 0)
serialPortValue = myPort.readStringUntil(‘\n’);

if(serialPortValue != null) //You need this check, because Processing will get the buffer that contains the information

//you sent over from Arduino very fast and sometimes it won’t have the next line character at the end because of this.

readString() has the same problem except it has chunks of the data instead of receiving a null value.  The null value

comes from not finding the next line character in readBytesUntil functionality of readStringUntil method.


//Do useful things in here with information received from Arduino


3.  Text can be wrapped around in a text box by adding a new line character tag for each line.

Other than that, it was pretty straightforward as with any other higher level programming language.

The Grape Cat Hero program works by getting the serial port value from the Arduino, which is the note played and checks to see if the note played matches the current note to be played.  If it is correct, then the current note along with the lyric line it matches up with will change its text color to green; otherwise, it will change its text color to red.

The lyrics and notes are read from text files and the lyrics are space separated while the notes are comma separated.  This allows for lyrics to have commas if need be and notes to have spaces if need be.  Since the lyrics and notes are read from text files, this program can be easily extensible by adding more lyric and notes text files with each set representing a level in the game.  To add text files in Processing, you need to do Sketch->Add File and add the text files you want the game to load.  It will place these text files in a data folder that Processing automatically creates for you in the destination folder that you have your main Processing file in.  You can also do the same step with adding pictures for Processing to load, but it is unnecessary for our game.

The lyrics and notes are essentially combined into one text with a next line tag in between the note and its corresponding word from the lyrics.  They are represented by the Note class which has the note information and the note with lyric information combined along with functionality to render the note with the  lyric word and to change the color the note and its corresponding lyric word text.   The Note class also stores the line and row (section) number information to render the notes and lyrics in a column fashion for the song music sheet.  After it renders a set of 11 lines, then it moves on to the next column of notes and lyrics to render.

After finish playing the song, the user is presented with a rating ranging from Uber Awesome to Don’t Call Me Maybe based on the number of notes the user got correct out of the number of notes in the song.  The user also gets a showing of fireworks created by a particle system with random colored particles.

To reset the game, the user can hit the spacebar and play the song again.

Here is a link to all of the files we created and used for the game:


Final Project – Stage Three

Now we got the grapes playing notes, it’s time to send the note String data over to the computer.

Sending String data from the Arduino to the computer is very easy.

First you need to place this line of code in your setup method:

//Initialize serial communication at 9600 baud rate

Then to send String data from the Arduino to Processing, you need to use this line of code:


delay(10);//delay by 10 miliseconds so we don’t send the String data super crazy fast to Processing

For the purpose of our project, we placed it in the loop method, since we are listening for whenever a grape is touched to produce a note.

On the Processing side, we first need to import a library.  You can place this line of code before your setup method in Processing.

import processing.serial.*; //For Arduino to send over the String data to Processing

Then in your setup method, you need to place these lines of code:

//Get the port in the serial list
String portName = Serial.list()[5];// 5 – port number
myPort = new Serial(this, portName, 9600);

For my MacBook Pro laptop, port 5 is the one I am using on the Arduino side.  Your port will be different based on what computer you are using.  You can use this line to see what ports are available on your computer:

println(Serial.list()[5]); //5 is the port number.  To check, put in 0, 1, 2, etc. until you find the String printed out by this function to match the Serial port description for your Arduino Serial Port.  Mine is  /dev/tty.usbmodemfa131.  You can check what Serial Port you have by clicking on Tools -> Serial Port and see which one is checked.

Finally to get the String data value from the Arduino, you need these lines of code in your draw method:

//If we get data from serial port,
//then read it and store it in a variable
if(myPort.available() > 0)
serialPortValue = myPort.readStringUntil(‘\n’);

I used this tutorial to get this part working:

Next step:  Creating the “Call Me Maybe” music sheet with Processing and changing text color based on whether the person got the note right or wrong.

Final Project – Stage Two

It’s time to make this project interesting: we’re going to test out communication between an Arduino board and a piece of fruit. We had one lecture class on this process, so I talked to the teacher about how to do this. She drew a simple diagram, and then directed me to look at the CapSense tutorial.

The capacitiveSensor library turns two or more Arduino pins into a capacitive sensor, which can sense the electrical capacitance of the human body. All the sensor setup requires is a medium to high value resistor and a piece of wire and a small (to large) piece of aluminum foil on the end. At its most sensitive, the sensor will start to sense a hand or body inches away from the sensor.

We actually got this part working fairly quickly, testing with just one grape, then four, then all nine. This was rather quick. The next part can’t be too bad, right?


Next: Polishing, and the Fruit Hero program.





Reblogged from Amanda’s post:

Final Project – Stage One

“Create a coherent interactive experience that involves the Arduino communicating with another device (computer, 2nd Arduino, MIDI instrument, etc.).
Interactive = listening, thinking, and speaking from both parties. Whether it involves one cycle or many, the exchange should be engaging.”

Such are the specifications for our final project in Physical Computing. Paul, Rebecca, and I teamed up once again to create another interesting and engaging piece. We decided to make Fruit Hero: an interactive music experience with a  fruity twist. We’re going to make a musical instrument using grapes as the keys, and a program that shows the player musical notes to match.


The materials:

  • One Arduino Uno
  • Nine Grapes
  • One Speaker
  • Nine 1.5 Mohm Resistors (These are huge!)

Before getting our hands on grapes, we wanted to prototype with buttons to make sure our notes were going to be played correctly. Easy enough, we’ve worked with buttons before. Except that we all forgot how to use buttons on a board. So that took some getting used to, as well as some issues with the organization of button prongs. But eventually we got four buttons onto the board. But then the speaker didn’t work for some reason. After some tweaking, we got this small version working.

Then we wired another breadboard with four more buttons. We ran into some issues with this. Only one side was working. We rewired a few times to no avail. We couldn’t figure out why only some of the buttons were working, and the other weren’t. So we disconnected one and it was working again, but of course with only four buttons. But what was the issue? Turns out, the small ground wire we were using to connect the board together was just a bit too short, so the other breadboard wasn’t being grounded. Oops. We switched out the wires and all nine buttons (we added in two modifier buttons for changes in octaves) were working splendidly.

Next step: Grapes.

Reblogged from Amanda’s post:

Games Brainstorm Day: Cabbageball

cabbage5    +    plastic_baseball_bats    +   short-51-w  +  Lab-Chair

Cabbageball is an adaptation of baseball with a cabbage instead of a baseball and people, wearing t-shirts and spandex shorts, on lab chairs. None of the people are allowed to lift themselves off the chairs and their butts must be in constant contact with the chairs.  They will propel themselves around by rolling their chairs with their feet.  If a person’s butt is not in contact with the chair, they are out until the next round.  A runner can get out by getting tagged with the cabbage if his or her chair is not touching the base.  The person up to bat  will use a plush toy bat to hit the cabbage with.  If the cabbage is destroyed in the game, it is replaced with a new cabbage.

Project Audio


Paul, Rebecca, and I teamed up once again to work on a project, deemed Project Audio. This was a smaller scale project for practice in preparation for the big final project. For this one, we needed to incorporate a speaker and a microphone. I was gone for a lot of the week in which this project started, so Paul and Rebecca came up with the idea to make something that detects temperature, and when you speak to it, it will talk back and light up either green (room temperature), red (warm), or blue (cold).



Paul and Rebecca worked on the project for most of the time without me; I came in handy moreso at the end. They put together the board and wired everything up. When I came in, they were using a capacitor instead of a thermistor (some of them do look very much alike). We fixed this issue, but we were struggling to get the temperature reading just right. We tried changing resistors, thermistors, code…but with no difference in results. Finally, we went to the professor for help. She showed us a diagram of how to wire a thermistor–turns out, we were wiring it wrong. After fixing this, the thermistor was working splendidly.

Now we just needed to fix the microphone. Paul and Rebecca had tested some mic code earlier, so we altered that for the rest of the project. The mic was tricky about detecting sound. We never did get it to work perfectly, but it works well enough. After adjusting the mic and the range for the temperature, our project was done. When we made noise or tapped the mic, the speaker would make a sound depending on the temperature; the higher the temperature, the higher the pitch. Originally we wanted the speaker to output something like a .wav with custom commentary, but we didn’t get that implemented in time. Regardless, it’ still cool.

Project Audio – Complete.

Link to video:






And here’s the code!


/* Lozano's code adapted by W. M. Harris 4.5.14
*Code for the electret microphone
*Based on Knock and Calibrate from Arduino examples
*by David Mellis,Tom Igoe and David Cuartielles
*More info:
*Jose Lozano
#include "pitches.h"

// these constants won’t change:
const int microphonePin = 2; // the amplifier output is connected to analog pin 2
const int triColorBlueLed = 12;
const int triColorGreenLed = 10;
const int triColorRedLed = 8;
const int thermisterPin = A5;
const int speakerPin = 3;

const int hotTemperature = 60;
const int coldTemperature = 50;

// these variables will change:
int sensorReading = 0; // variable to store the value read from the sensor pin
int sensorMax = 0;
int sensorMin = 1023;
int threshold = 0;
int difference;


int melodyCold[] =
int melodyHot[] =
int melodyMeh[] =

// note durations: 4 = quarter note, 8 = eighth note, etc.:
int noteDurations[] =
//4, 8, 8, 4, 4, 4, 4, 4
1//4, 4, 4, 3, 4, 2, 4, 8, 8, 2, 8, 8, 2, 4, 8, 8, 4, 4, 4, 2

void setup() {
pinMode(triColorRedLed, OUTPUT);
pinMode(triColorGreenLed, OUTPUT);
pinMode(triColorBlueLed, OUTPUT);
Serial.begin(9600); // use the serial port
while (millis() < 3000) {
threshold = analogRead(microphonePin);

// record the maximum sensor value
if (threshold > sensorMax) {
sensorMax = threshold;



// signal the end of the calibration period
threshold = sensorMax;


void loop() {
int number = isThereNoise();
if(number == 1)

int isThereNoise()
// read the sensor and store it in the variable sensorReading:
sensorReading = analogRead(microphonePin);
sensorMax = max(sensorMax, sensorReading);

difference = 0;
// if the sensor reading is greater than the threshold:
if ((sensorReading >= threshold)) {
difference = sensorReading-threshold;
Serial.print(difference); //Will send only positive and absolute values of waveform

//make LED brightness proportional to range of sensor values
//Still send only positive vals
difference = max(0, map(sensorReading, threshold, sensorMax, 0, 255));
//Serial.print(” “);

if(difference > 0)
return 1;
return 0;

void figureOutTheTemperature()

float thermisterReading = readingFromThermister(analogRead(thermisterPin));

if(thermisterReading >= hotTemperature)
digitalWrite(triColorRedLed, HIGH);
//analogWrite(triColorRedLed, thermisterReading);

digitalWrite(triColorRedLed, LOW);
else if(thermisterReading <= coldTemperature)
digitalWrite(triColorBlueLed, HIGH);
//analogWrite(triColorBlueLed, thermisterReading);

digitalWrite(triColorBlueLed, LOW);
digitalWrite(triColorGreenLed, HIGH);
//analogWrite(triColorGreenLed, thermisterReading);

digitalWrite(triColorGreenLed, LOW);

// Serial.println(analogRead(thermisterPin));
// Serial.println(thermisterReading);
//delay(4); // Better for Processing showing data

float readingFromThermister(int RawADC)
float Temp;
// See for explanation of formula
Temp = log(((10240000/RawADC) – 10000));
Temp = 1 / (0.001129148 + (0.000234125 * Temp) + (0.0000000876741 * Temp * Temp * Temp));
Temp = Temp – 238.00;//273.15; // Convert Kelvin to Celcius

Serial.print(“Celcius: “); Serial.print(Temp);
Temp *= 1.8;
Temp += 32;
Serial.print(” Fahrenheit: “); Serial.println( Temp);
return Temp;

void playSoundEffect(int temp)
// iterate over the notes of the melody:
for (int thisNote = 0; thisNote < /*20*/1; thisNote++)
// to calculate the note duration, take one second
// divided by the note type.
//e.g. quarter note = 1000 / 4, eighth note = 1000/8, etc.
int noteDuration = 1000/noteDurations[thisNote];
if (temp == 1) //hot
tone(speakerPin, melodyHot[thisNote],noteDuration);
else if (temp == 2) //meh
tone(speakerPin, melodyMeh[thisNote],noteDuration);
else if (temp == 3) //cold
tone(speakerPin, melodyCold[thisNote],noteDuration);

// to distinguish the notes, set a minimum time between them.
// the note’s duration + 30% seems to work well:
int pauseBetweenNotes = noteDuration * 1.30;
// stop the tone playing:

/* Processing code
* from
* code –
*Based on Tickle example.
* The word “tickle” jitters when serial data arrives from serial port
import processing.serial.*;
String portname = “COM1”;
Serial port;
PFont font;
int micro = 0;
float x = (500); // X-coordinate of text
float y = (350); // Y-coordinate of text

void setup()

size(1200, 700);
font = createFont(“Courier”, 24);
// #1 in Serial.list() usually FTDI adaptor
portname = Serial.list()[2];
port = new Serial(this, portname, 9600);

void draw()
rect(0, 0, width, height);

if (port.available() > 0) {
micro =;

x = x + random(-12, 12) + random(-2,2)*micro/2 ;
y = y + random(-12, 12) + random(-2,2)*micro/2;

text(“Sing Me”, constrain(x,0,width-30), constrain(y,0,height));
if (x>width || y > height || y < 0 || x < 0 ){
x = (500); // X-coordinate of text
y = (350); // Y-coordinate of text
void mousePressed() {

x = (500); // X-coordinate of text
y = (350); // Y-coordinate of text

Reblogged from Amanda Imperials website:

Project Runway – Day Three

Before we can get to sewing, we tried adding two more phototransistors to the project; having only one makes for only precise shooting. However, we had lots of issues connecting everything properly, and were receiving no readings from any of the phototransistors. We spent at least two hours just trying to get this working again. Eventually we gave up trying to figure out the issue and just pulled everything out and rewired everything up again. Finally, it worked.

Link to video:


Paul’s working hard.



The main component was done. The next step was getting this bad boy onto a shirt. We considered actually sewing our circuit together, but thought it may be easier to stick the transistors through the shirt. This actually took us forever to do. Maybe next time we’ll learn how to use conductive thread. But overall, the project was done. So we added a little touch of the 80s to make it a little more fun. When shot, the speaker plays a little tune similar to a certain Bon Jovi song. Take a guess.



Project Runway – Completed.




Shot through the heart.


Fritzing! Pretty simple.

Here’s the code!

by Tom Igoe
This example code is in the public domain.

Modified for this project by
Rebecca Vessal
Paul Saidi
Amanda Imperial
// Source:
//define pins
#define irLedPin 4
#define longDistanceRange 50
#define shortDistanceRange 300
#define phototransistorPin A0
#define phototransistorPin2 A2
#define phototransistorPin4 A4
#include “pitches.h”

// notes in the melody:
int melody[] =

// note durations: 4 = quarter note, 8 = eighth note, etc.:
int noteDurations[] =
//4, 8, 8, 4, 4, 4, 4, 4
4, 4, 4, 3, 4, 2, 4, 8, 8, 2, 8, 8, 2, 4, 8, 8, 4, 4, 4, 2

int irRead(int readPin, int triggerPin); //function prototype

void setup()

// wait for the long string to be sent

void playHitSoundEffect()
// iterate over the notes of the melody:
for (int thisNote = 0; thisNote < 20; thisNote++) { // to calculate the note duration, take one second // divided by the note type. //e.g. quarter note = 1000 / 4, eighth note = 1000/8, etc. int noteDuration = 1000/noteDurations[thisNote]; tone(8, melody[thisNote],noteDuration); // to distinguish the notes, set a minimum time between them. // the note’s duration + 30% seems to work well: int pauseBetweenNotes = noteDuration * 1.30; delay(pauseBetweenNotes); // stop the tone playing: noTone(8); } } void loop() { // read and display all phototransistors // value on serial monitor Serial.print(” 1st phototransistor reading = “); int phototransistorReading = analogRead(phototransistorPin); Serial.println(phototransistorReading); Serial.print(” 2nd phototransistor reading = “); int phototransistorReading2 = analogRead(phototransistorPin2); Serial.println(phototransistorReading2); Serial.print(” 3rd phototransistor reading = “); int phototransistorReading4 = analogRead(phototransistorPin4); Serial.println(phototransistorReading4); //If the phototransistors return a higher value than the value from room light //from the laser pointer, then play a sound if(phototransistorReading >= longDistanceRange || phototransistorReading >= shortDistanceRange)
if(phototransistorReading2 >= longDistanceRange || phototransistorReading2 >= shortDistanceRange)
if(phototransistorReading4 >= longDistanceRange || phototransistorReading4 >= shortDistanceRange)

// a short delay

* This function can be used with a panasonic pna4602m ir sensor
* it returns a zero if something is detected by the sensor, and a 1 otherwise
* The function bit bangs a 38.5khZ waveform to an IR led connected to the
* triggerPin for 1 millisecond, and then reads the IR sensor pin to see if
* the reflected IR has been detected

int irRead(int readPin, int triggerPin)
int halfPeriod = 13; //one period at 38.5khZ is aproximately 26 microseconds
int cycles = 38; //26 microseconds * 38 is more or less 1 millisecond
int i;
for (i=0; i <=cycles; i++)
digitalWrite(triggerPin, HIGH);
digitalWrite(triggerPin, LOW);
delayMicroseconds(halfPeriod - 1); // - 1 to make up for digitalWrite overhead
return digitalRead(readPin);

Reblogged from Amanda Imperial’s website:

Project Runway – Day Two

Alright! So today was mostly experimental. We’ve never really worked with phototransistors, so this was a big step for us. We might have killed one in the process (we forgot to put the resistor back in). We were working off of the Lazer Tag link from the first post, and were having trouble getting anything to work. But thankfully, after some shifting around, changing of resistors, and a brand new phototransistor, we got some consistently fair readings.

She's such a good model.

Don't worry, the LED is grounded.

After this feat, we stuck an infrared LED where the LED was and tried to see if our phototransistor could read the light at all. With some new code, it worked like a charm. But it’s extremely precise. We may decide to take a different route with the infrared, but overall made good progress today. We did also add some code to test our speaker component, but no tones came through. We hope it’s just a broken speaker.

Until next time.

Are prototype is almost done.

UPDATE: The speaker was indeed broken. We plugged in a new speaker, and it worked just fine. We also worked a bit more today (but not enough to deserve its own post), and got two Arduino Uno boards to point at each other. However, an LED without a lens to focus it is not the best way of doing a range-based system, so the professor suggested a laser pointer. We went ahead with her suggestion, and we decided we liked this way better. After tweaking some of the original tone code, the laser pointer light, when it hits the transistor, sets off the speaker. Our first steps into the final stages have begun.

Next time: Sewing.



The phototransistor has been pulled away from the board.



Reblogged from Amanda’s website:

Project Runway – Day One

And so begins project number two: Project Runway. Our goal is to make something wearable and interactive. For this challenge, we’ve agreed to take on a project similar to lazer tag. We found a tutorial that will probably help us accomplish this, as we have yet to work with emitters and light sensors. Also, we have an additional partner this time. So we are a team of three.



Our materials (for now) include:

  • Photodiode
  • Light Emitter
  • Speaker
  • Button
  • A few resistors
  • Capacitor
  • Motor *

* Tertiary; these may be used later.

Here are some of the resources we found:

Lazer Tag

Arduino – Lazer Tag

Arduino – Simple IR

Robotix – Simple IR

To be continued when we make more progress!

We got a box!


Reblogged from Amanda’s website: