Category: Physical Computing

You can see the full post with more details here: http://backend.alvarolacouture.com/tangible-dynamics/

This project was the final project for my Introduction to Physical Computing class at NYU. It was co-created with Nicole Ginelli.The idea was the combination of our combined interests in interactive 3D animations, real-time user control, and a desire to put into use several approaches we learned during the semester,

We used a Kinect 2 for depth, luminance, and color tracking. Using this combined approach, we were able to control the tracking in a way where only the spheres would be tracked, and people’s extremities would not. This video feed was managed and programmed using Touchdesigner, which we used to get X and Y coordinates for each of the tracked objects. We then used the OSC communication protocol to send these values to Unity3D, where we assigned each set of coordinates to control objects inside the particle systems we created with the game engine. Simultaneously, we used SPOUT to send the game engine’s camera view as a texture over to Touchdesigner, where we adjusted colors, contrast, and brightness before video mapping the image on top of the table using a short-throw projector. Finally, we also had an Arduino NANO 33 IOT inside each of the spheres you can observe on the table. Each Arduino controlled an independent neo-pixel LED strip. We used UDP communication to control the color and luminance of each of the neo-pixel strips according to the particle simulation we decided to show. In total, we had four different particle simulations. Each one had with own properties, LED color combination, and audio-reactive composition.

Final Project Ideas

Sketches

Previous
Next

Ryoichi Kurokawa

http://www.ryoichikurokawa.com/

Ad/ab Atom

Previous
Next

oscillating continuum

Gabriel Pulecio

http://www.gabrielpulecio.com/

Fractoid 2.1

Previous
Next

Bradley G Munkowitz “Gmunk”

https://gmunk.com

Box

Midterm Part 2

Installation: Everything else

Last week’s post detailed how we started the construction for our spooky Halloween installation. This week we proceeded to install the motors, test the programming, and finish our project.  We had two HS-311 motors, which we tested loosely against the cloth and it didn’t seem to have much strength. We ordered two stronger ones (ref. DS3218) in case we needed more torque. According to the Amazon reviews, these were much stronger and resilient, being used primarily for RC cars.

Materials

Previous
Next

Process

Now came the fun part: How would we attach the motors with the props on to the frame? It was impossible to test the motors without having them installed: we needed to test how much force they could apply against our leather canvas. We tried several positions for the head and the hands, until we found one we were satisfied with.

Previous
Next

To print or not to print…

After several tests and placing blocks of wood against the frame and attaching the motors in a variety of different ways (glue gun, tape, zip ties, you name it), we came upon a problem. David Rios made us realize that unless the motors were firmly installed against the frame they wouldn’t we able to produce the full force we needed. 

After a while of scratching our heads and looking into different DIY ways of setting the motors correctly, Jake Sherwood suggested we should look for a 3D printed solution. Blessed be Jake Sherwood. I immediately came across a free 3D model I could print right away. (model .stl file here). Even better, it worked with both of my servos without any modification.

It took us about 7 hours to print three servo brackets in a Ultimaker 2+ available on the ITP floor. After printing there was residue, so we had to sand them down and use a motor tool to open the holes where we would tie the servo motors.

3D printed brackets for motors

Previous
Next

Installing the elements into the frame

Motors and props

With the brackets printed, we could now attach them to our frame. Since we needed the hands and skull to be more centered, we used pieces of scrap wood to locate the motor brackets closer to the center of the canvas.

Previous
Next

Once we were pleased with everything we proceeded with hammering and gluing down the parts, taking into account the distance to the arduino, the ultrasound sensor, and other elements.

Previous
Next

The sensor

Installing this sensor was the most time-consuming element of the whole project. This sensor brought us a lot of trouble, it’s distance detection was difficult to regulate. After listening to several people, it seems as if this low-cost sensor is not precise nor easy to work with.

In any case, we soldered some cables on to each pin to be able to extend it’s reach. Now we had to choose where to install it so it would achieve it’s purpose: detecting to people so we could scare the shit out of them. I’ll talk about the programming ahead.

 

For the sensor to work correctly it needs to have both beacons uncovered. In other words, there was no way we could get away with hooking it up to the backframe. If we installed the sensor on the top pf the frame we would miss shorter people, or we would have to locate the frame closer to the ground. Neither idea was appealing.

 

We decided to install our spooky frame on a table , with out sensor installed on the front underside side of the table, with a black cloth covering the wiring below the tabletop. This allowed the sensor  to detect objects directly in front of it, about 4 ft. off the ground, making our skull lash out only when someone is in passing right in front of the table.

Perfect.

Previous
Next

Coding the terror

Circuit

The schematic for this project is simple. On one side we have The Arduino connected through USB to a computer. It’s digital pins 11 and 12 are for the trig and echo pins of the ultrasound sensor. It’s digital pins 2,3 and 4 are for controlling each of the servos.

 

The Arduino, sensor, and motors all share a common ground. The servos however are connected to an independent 5V power source.

Code

Previous
Next

As I said previously, we assigned the variables for each of the used arduino pins. We also determined which motor would control which prop. We made several tests with a potentiometer to manually detect which was the initial and final position for each motor in each prop. We found that a 60 degree change between each position achieved the desired effect. In the initial position, you would not be able to see anything pressing against the canvas. In the final position, the props would be totally pressed against the canvas.

While fiddling around with the sensor we found a couple of functions which were useful to debug the distance in centimeters and inches (functions found here). 

We established several states  determined by the distance detected by the sensor. If the sensor detected anything closer than 25 inches, it would activate state 1. If the detection occured between 25 in and 50 in, state 2 would activate. State 3 would activate if anything was detected farther than that. The closer you get, the faster the motors would move the props against the canvas.

 

The Result

https://youtu.be/DL77Z4BPpWE

Thanks for reading!

 

Midterm Part 1

The Idea

So, it was time for the midterms.

I was partnered with Sydney Meyers to come up with ideas to make for the midterm. We went through several fun things to tackle and found several of them were mentioned by Tom Igoe in his  post Physical Computing’s Greatest Hits (and misses).

Our focus was definitely to go for a scary vibe, we wanted people to get frightened by our installation. Unfortunately I can’t show many doodles we had as ideas for the project, since I lost my notebook. However, here are a few of the sketches we went through before we arrived at our main idea. 

 

Creepy sketches

Previous
Next

We thought about a Chucky-like doll which followed you with it’s head (first sketch). It would have several ultrasound sensors around it, and when one was activated it head would turn towards that position. Though the idea seemed cool, we thought probably the doll head’s movement wouldn’t be as fluid since it would depend on the amount of sensors around the doll (sketch #2). We also though we could use the ml5 library with Posenet to detect body movement, but we quickly threw away the idea since it would require a camera and would be harder to control if several people were to get close to it.

It quickly became clear how we should control the detection, how to make the “user” maneuver through space how wanted them to. Right away we thought about how instead of making something detect a person passing by, we might as well make the user go straight to the action point. Reflecting on how cool a cardboard box filled with satanic puppies might look (as usual), we decided to take a more human-focused approach and simulate dead babies instead. Puppy skulls aren’t that easy to get a hand on.

Anyways, we let the idea settle for a while. And then, out of nowhere, came Keanu Reeves.

Final Sketch

https://www.youtube.com/watch?v=7tYBZD3raOQ

We came up with something similar to this scene in the Constantine movie, made in 2005. I seemed like a ghoul, or a demon stretching a skin like material until it almosts bursts. We thought about how it would look great on a picture frame and with a leather-like material similar to skin.

Looking for the stuff

So, we were set. Time to go get the stuff to make it happen. We would need an old frame, some leather, and a backside to mount the electronics. We thought going through thrift shops and antiquity stores would be our best choice. We tried Mother of Junk, which seemed like it had a great collection of items (The first to picture ahead show it’s massive assortment). It did have variety, but the employees were trying to rip us off, so we decided to leave to Brooklyn Vintiques. 

Here we found our $17 frame, and in a thrift store nearby we found a leathery track suit that worked perfectly (picture ahead). We decided to purchase the necessary wood for the backframe, since we needed it to be in perfect state.

Previous
Next

Will it look good?

To test how it would look we cut the suit we bought and adjusted it temporarily behind the windowless frame. We did several tests to see how it would stretch and how much cloth we would have to cut.

Installation: The frame

Materials

Previous
Next

Process

1. We removed the glass from the frame and clamped our cloth around it to measure it’s stretch potential and which was the best placement.
2. According to our previous measurements we cut the cloth according to the frame size and how tense we wanted it to be.
3. We used a staple gun to attach the cloth to the frame, making sure each part would be firmly attached.

Previous
Next

4. We proceeded to cut the wood which would be attached to the frame according to the previous measurements. We decided having a 3 inch depth would give us enough space to mount the motors and Arduino behind the frame. The added frame would be solid wood as well, as to give support to the whole system.

5. We left the backframe with wood glue for a whole day before we proceeded with the next part. To ensure it’s firmness we used clamps.

Previous
Next

Thanks for reading! Heads up for part 2!

 

Human Chasing Camera 1.0

The Idea

During last week I developed a circuit which involved connecting a servo motor to an Arduino and control it using a mouse. This week’s project consisted of developing this idea further on. In addition to this, I wanted to used the ml5.js machine learning library made for P5.js  to control something physical in the world, in this case the servo motor.

It is common knowledge that machine learning algorithms like the one I have used for making these projects are being used today in surveillance systems around the world. Among these sites there are varying degrees of privacy disrupting mechanisms which correspond to geopolitical forces and control mechanisms.

To know how to prepare against these types of surveillance systems effectively it is necessary to learn how it is done.

The Video

The Process

The system consists using a an image recognition software which tracks your body position and uses this data as a way to control the movement of a motor with the camera attached on top, Effectively giving the sense that it is chasing you.

The HS -311 servo motor has enough torque to move the webcam easily, it works perfectly for my purposes.

A more detailed description of how the connection was made between the Arduino and p5 can be found in my last blog post. 

ML5 and Posenet

The ml5 library has an interesting set of functions derived from Tensorflow.js and PoseNet, a convolutional neural network that detects human poses in real time. 

When we use ml5 through a p5 sketch to track someone’s body position, we get a series of X and Y coordinates which correspond to different parts of the body being tracked in real-time. In this project, the camera is only used to track one point: the nose. The reason for this is that we need to grab a point which always corresponds to a front facing user. Also it  seems as if the person tracked and the camera are in a conversation, looking face to “face”.

The Code

Arduino Code Repository : https://github.com/lacouture100/Intro-to-Physical-Computation-ITP/tree/master/Week_5/arduino

P5 Code Repository : https://github.com/lacouture100/Intro-to-Physical-Computation-ITP/tree/master/Week_5/p5

Thanks for reading! 

 

Mouse motor control

The Idea

During this week we developed a circuit which involved connecting a servo motor to an Arduino Nano 33 iot. This week’s project consisted of developing something which I had started working on last week,  to control something physical with some king of haptic or visual interaction through human input.

The Video

The Process

The system consists of using a mouse to control the position of a Servo motor (ref. HS-311). . For dealing with the mouse interaction, I used p5 to send the mouse’s position to Arduino, which translated this value into an angle the servo motor could move. I used serial communication to create a bridge between both software and send the values of the mouse’s position in the Y axis to the Arduino.

The connection between the servo motor and the Arduino was straightforward. The servo motor three female headers, one for power, one for GND and another for communicating with the arduino.

 

I wanted to make some kind of visual feedback on the screen, as well as to debug my serial connection. So I made a bullseye that tracks mouse movement, maps the value between 0 – 179, and sends the mapped value to the Arduino as a serial message. Since this had to be updated in real time, and since p5 and Arduino both communicate in an asynchronous manner at a 9600 baudrate, I would send this updated value every loop.

 

As seen on the image above, I am sending a value and receiving the angle in which the motor has rotated the object attached to it.  

The Code

Arduino Code Repository : https://github.com/lacouture100/Intro-to-Physical-Computation-ITP/tree/master/Week_3/Arduino%20Code

P5 Code Repository :  https://github.com/lacouture100/Intro-to-Physical-Computation-ITP/tree/master/Week_4/p5_code

Thanks for reading! 

 

RGY Arduino digital input and output

During this week we developed a series of circuits which involved connecting buttons and LEDs to an Arduino Nano 33 iot. This week’s project consisted of developing something which would allow me to control something physical and on the screen which related to each other.

The Idea

The device consists of three colored LED lights and three buttons connected to an Arduino. Each button is colored red, blue or yellow. Each corresponds to a light with the same color.

When you press a button, the Arduino receives a digital signal and turns on the light according to color. Additionally, the Arduino communicates with P5 to print the chosen color on the screen.

The Result

The Video

Code Repositories

Arduino Code Repository:  https://github.com/lacouture100/Intro-to-Physical-Computation-ITP/tree/master/Week_3/Arduino%20Code

P5 Code repository:  https://github.com/lacouture100/Intro-to-Physical-Computation-ITP/tree/master/Week_3/P5%20Code

New York City’s MTA device analysis

Assumptions on how its being used, describe the context in which its being used.

The MTA subway device (I will refer to it as MTAD from now on) is present in every subway station in New York City, made for the single use of allowing people to pay for their subway fare.

How they use it, what they do differently, difficulties, easy parts.

The main interface is the touch screen incorporated in the MTAD. Through the touch screen users have several options that include purchasing a new metro card, or filling up one the user already has. In case the user does not have a card, the interface provides options which include different amounts for the user to purchase: a single-ride option, a single-day use card, a monthly-use card, or a custom amount to fill up the card with.

The device has labeled sections (in English) across it’s frontal panel indicating where to insert coins, bills, credit or ATM cards.  It also includes a change and receipt tray, as well as a slot for inserting your metro card or receiving a single-ride card.  For a user who speaks English I would consider the interface to be pretty straightforward.

As I was in the Halsey Street Station in Bushwick, I noticed most people used the recharge option for a card they already possessed. I would bet this is because most people that use this specific station live around the area. I believe in Manhattan I probably would have seen more people buying a single-day use or single ride kind of option due to the amount of turists.

What takes the longest, what takes the least amount of time.

When people use credit or ATM cards they seem to take more because of the time required to confirm the transaction. Inserting a PIN and typing in your Zip code takes time.

How long the whole transaction takes.

In general, people spend less than a minute handling the device, unless they use a credit or debit card. This usually adds up to another minute to the whole process.

Reflecting on Norman’s book “The Design of Everyday Things””we can analyze the MTAD from a System Image perspective, the conceptual model we use to handle the machine from the moment we glance at it is similar to the one we handle when we use an ATM. The device itself is designed with a very similar form and use in mind: You use money (cash or cards) to retrieve an amount of credit for your Metrocard.

Thanks for reading! 

Wooden object frame with a backlight

Previous
Next

The Idea

During this week we developed a series of circuits which involved connected LEDs in series and parallel. This week’s project consisted of developing a Wooden Succulent container with a light switch.

The idea for the project came up when i saw a dollar shop walking through Brooklyn. The small picture frame caught my eye. Great, a victim for the greater good.  I  considered the possibility of creating a switch activated picture frame with some kind of extra dimensionality, as to illuminate an object inside.

The Process

In order to make the device with an independent power supply and to also fit any type of circuit inside of the frame, an expansion would be necessary.

In order to make the expansion, the dimensions and requirements of the circuit were imperative. I concluded two leds, a 220k resistor, a 9V battery and a switch were required.

 

I decided on making the connections between the leds in series directly to a 220k resistor. So, the positive end of the battery would go to a swtich, which would connect to one side of the resistor. 

The other side of the resistor would connect to LED #1 positive end, and LED #1 negative end would connect to LED #2 positive end, connecting it’s negative leg towards ground.

Finally, I drilled a hole into the backside of the  picture frame, attached the expansion module, and inserted the circuit into the new device. As a detail I inserted a Succulent plant which made it look cool.

The Result

Thanks for reading!

Wearable Prototyping

Previous
Next

The Idea

As part of the discussion of what is Physical Computing, we developed a prototype for a universal translator device. The device is meant to be used in situations where the user requires an immediate translation of the conversation at hand. In other terms, it allows someone to have a fluid conversation with another person without any delays in translation. The device consists of two objects:

 

1. A digital watch which serves as a control interface for the device. Thought primarily as an app for smartwatch devices, the interface allows control over translation and language settings, as well as activation functions, volume control, and full visual functionality to control the auricular component. It also allows the transmission of the user’s translated voice to the receptor, allowing each to see a text translation for the conversation in the app screen.

2. An auricular component which allows a direct transmission relayed by the smartwatch device. It allows the user to hear what is being said in their own language, without any delay. This allows a fluid reception of the conversation at hand and in real-time. The auricular includes directional and omnidirectional microphones, in case the user needs to use the device in a crowd and requires multiple translations at the same time.