Categories
Real Time Social Spaces

First Dive into ThreeJS

Creating an environment

Our task this week was to start from Aidan Nelson’s template, which includes the files for a threeJS template that “allows multiplayer three.js scenes with integrated audio/video capabilities. It uses a node server running socket.io to provide multiplayer functionality as well as WebRTC signaling.” After you download and run the npm run start command inside the root folder, a browser window should open and ask for permission to access your browser camera and mic. After you comply you should see your head(fortunately) popping on the surface of a Cube-like character.

It is pretty much as simple and effective as possible to get you up and running with multiplayer capabilities (after you’ve hosted it online), with the main components being scene.js and environment.js. What I did was instantiate a bunch of the spheres that came with the template (5000), and then set up six lights to rotate around the center. I also changed my main shape to the sphere as well. Below you can see some of the tests I made, as well as the GitHub repository, and a couple of questions that still remain.


Questions:

  • How to import and use Three.js objects such as water and sky?
    • I downloaded Water.js and set it inside my js/libs folder, and I made another one inside my assets/objects folder to try it out there too. I then tried importing it in my environment.js and scene.js, but failed to import the object with import { Water } from ./assets/objects/Water.js due to the following error " Cannot use an import statement outside of a module."
  • While testing three.js by itself, and also checking the network tab in chrome dev tools, I didn’t get to see where most of the loading delay for the frontend was happening. Suppose I would want to make a loading bar, what data would I measure? How can I establish a percentage of how much has loaded? Or does it all just kind of have to be faked with asynchronously loading animations/loading bars?

Categories
Real Time Social Spaces

RTSS_P5LiveMedia

Yell to be seen

This quick sketch came as a solution for something I noticed during several zoom meetings I’ve had: some people like to talk too much. As a solution, I came up with a sketch in which your camera feed appears invisible until you start speaking. The more you talk, the more others can see you and the higher up you appear on the screen. It’s meant to be used for a group of people who wish to have a balanced conversation in terms of participation.

Categories
Computational Approaches to Narrative

CAtN Final

Introduction

Two weeks ago, I came up with an idea that functioned more as a momentary placebo amidst finals and all the underlying chaos. I wanted accomplish some kind of way to give a more “magic realism” feel to a particular series of news reports (I initially thought of NYTimes). However, after bouncing around the idea in my head several times I decided it was not a something I was willing to test and possibly fail about due to time. I wanted to make an experiment that led me to something fun, and even an idea that might lead me somewhere else.

In any case, I decided to maintain the Magical Realism element. After taking a look at @MagicRealismBot in Twitter, as well as reading this interview with their creators Ali and Chris Rodley, the idea became clear. I recommend reading the interview, it is entertaining to see the mindset and the process behind such an effective and endearing tool.

I thought about some comments made in class about using GPT-2 to help with thesis process (Neta and Nicole?). In any case, I thought it would be fun to give some of the articles I usually read in one of my favorite New Media online platform, Creative Applications Network, a bit of a twist. Specifically, the way in which the website displays the entries for each article made it easier. I’ll explain in more detail below.

Process

Creative Applications Network

You can follow along this notebook I made in Google Collab. I decided to scrape all of the blog entries from the blog. By blog entry I mean the short pieces of text you see below. Every blog article had one. After scraping the 147 entries, I made a CSV file out of them so I could take a better look.

I noticed most of the entries (or at least the ones I was interested in) followed a similar pattern. “X project” is a “Y adjective to make it sound innovative” “Z New Media name” “Whatever it did”. I made a small diagram of it below.

Right away I thought this type of one-line format (after cleaning up the data a bit) would mix perfectly with Magic Realism Bot’s posts. (weird saying it that way) To clarify, the cleanup involved removing the names of the work, and of the artists. So I took all of those and formatted them into a format acceptable by spacy and the Markov chain algorithm. I wanted to make a script for it. This involved making a script that received a CSV file with one column of text and a name and outputs a new file with a ‘name’, ‘index’, ‘total’, and ‘text ‘ as column headers. All of this is in the notebook. I then proceeded to scrape tweets from the MagicRealismBot on Twitter.

Magic Realism Bot

I will not go into much detail about how I scraped it, since it is on the notebook. However, I just want to make clear I obtained about 3200 tweets, which I took a screenshot of below.

Spacy, Tracery and the fun part

After having both of my sources, I joined them into one CSV file. I processed the file using Spacy and Allison’s guide for a corpus-driven narrative generation. The whole thing was about 4800 entries.

I also used a copy of Allison’s notebook. With Allison’s notebook, I was able to use spacy. Spacy gave me my entities, as well as actions, verbs, objects, etc… which I used as an input for Tracery.

I made a couple of tests to see how it would go. And I think it went pretty well.

AND MY FAVORITE (which sadly I got too excited to remember to even screenshot):

“An interactive installation that facilitates collaboration between a human and a puddle of alcohol.”

Other remarkable examples:

A bisexual fisherman falls in love with the use of CCTV.

A professor reads a poem about an arduino that can destroy metaphysics.

A new life as a performance.

A theologian discovers that Wikipedia does not exist.

The dancer’s body is extended and manipulated as a tool to quantify the world.

An interactive installation and performance inspired by light rays traveling in a latent space of situations.

A project explores possible alternatives of how we experience the materiality of nature through the mediums of fiction.

Exploring behavior-based design systems that are self-aware, mobile, and self-structure / assemble.

To compensate for the lack of material I had from the Blog, I had to duplicate those entries several times, as well as some of the examples shown before. By the end it came out to a 3200(MRbot) vs. 1600 entries.(creativeappsnet + handpicked generated).

Output?

So I came around this talk by Kate Compton (creator of Tracery) who gave the most enlightening talk about procedural generation, and it occurred to me I did not want to leave the output of this experiment only in text.

So through this VR experiment I wanted to include several elements I became captivated by during the semesterโ€ฆ.such as the sense of waiting present in Epitaph, or the idea of spatial(enviromental) storytelling in Bitsy. Also, I felt some of these ideas were brilliant, and on a computer screen I seemed to get distracted with my other 127 tabs.

So ideally, you would be able to walk to other beams of light which generate text with other parameter values. ( If I was to use GPT-2 for example, each beam on a row would contain +0.1 in the temperature parameter). You would be able to pick up some of these and keep them.

EDIT: Worked!

Categories
Time

HORIZONTE – Time Final

My project is called Horizonte, a horizon clock that adjusts it’s color and light patterns in real-time according to its location and the current time of day. The clock itself is a device that provides a WiFi access point for remote configuration, and an autonomous network-based update feature, which allows the user to customize the clock’s parameters. I wanted to stray away from the sense of being precise, a rather lazy, and non-invasive way of letting you know what part of the day it is. The clock gradually shifts from one stage of the day to the next, taking around 5- 20 minutes to shift depending on the event, location, and date.

The project uses an API that is based on the SuncalcJS library to acquire the time for each particular ‘sun event’ in the day, such as Dawn, Sunset, etc… For this particular project, at this stage at least, I am dealing with events that happen between Dawn and Dusk. Why? These are the first and last stages of sunlight, and if you are a fan of ” From Dusk till Dawn”, those are vampire-safe times.

Hardware Used

Power requirements

  • I used a 5V, 10A power supply for both the Arduino and the lights. Same as this one.

Software

  • Arduino IDE
  • NodeJS
  • Suncalc
  • Blender
  • Illustrator
  • Unreal Engine 4 (for visualizations)

Initial Proposal

Initial mockup idea

User Interaction

UI Re-design

FULL PROCESS

I mainly focused on gradients, however I also tried different approaches, as you can also see below.

Testing different lighting properties with the strips.

I quickly found myself struggling to acquire the control I needed, and after a week or two of trying to make my own gradients based off of Adafruit’s limited examples, I was running out of time to get the other things going. After looking for a while I ran into FastLED, a Neopixel control library developed for Arduino which boasts to be better than the original library. Their examples were very varied, complete, and allowed me to find a couple of useful scripts which I took apart to make my own. Specifically, I heavily relied on the ColorPalette example and the nblendPaletteTowardPalette function found in the docs. The code implemented in the clock goes through nine different gradients, transitioning in and out of them during the day.

The main function takes the following parameters:

  • A current time measure in order to determine which sun event should be activated
  • The start time for the current day loop. It resets at midnight.
  • each of the sun events in millis().
void ChangePalettePeriodically_bottom(unsigned long startMillis,
                                      unsigned long currentMillis,
                                      long interval1,
                                      long interval2,
                                      long interval3,
                                      long interval4,
                                      long interval5,
                                      long interval6,
                                      long interval7,
                                      long interval8,
                                      long interval9
                                     ) {
Gradient using FastLED
Seeing the individual sections on the two strips

In terms of wiring and connections, it was pretty straightforward. I used an Arduino MKR 1010 since it had an esp32. From previous research, I knew it had more ram than the Arduino Nano, which was my other option. I hooked a couple of 470-ohm resistances to the input line of each Neopixel Strip. I also added two 470 microfarads capacitors in series. The Adafruit Uberguide for powering Neopixels suggests a 1000 microfarad capacitor, however, I did not have one. All of this was powered by a 5V, 10A power supply. Here’s the schematic below.

Horizonte Schematic

Color

First Gradient display made with Miro

In order to represent the horizon, or at least my view of it, I would have to decide on a color palette to use. I started out doing something like the image below, which gave me a general sense of where to go.

Gradient and sky mood board in Miro

The API, the request, the data

After having seen Tom Igoe’s Connected Devices and Networked Interaction class last semester, I felt fairly confident that making my own API was the way to go in order to access the data at any given time from any device and also to control it. Manipulating the data and processing it before sending it to any device was a priority. I decided to make it in NodeJS since I wanted to keep the possibility of incorporating JS into the web client open. I also mounted it on Heroku, which probably isn’t the best option for continuous communication such as WebSockets but works fine for HTTP requests. As I said before, even though my idea was to keep it as real-time as possible the idea of being a couple of seconds or even minutes off track didn’t mean much. The API consists of two routes: /sun and /now. /sun ill return all of the times for each sun event according to UTC. /now will return the current time.

The data is presented as a JSON file. So everytime we make a request, the server writes an updated JSON file which is sent to the Arduino. You can see the JSON formatted below.

Server console log

To made the request to the server I used the WiFi Web Client example from the Arduino IDE, and later saved it as a JSON using the ArduinoJson library.

Arduino Serial Log connecting to the hotspot, receiving the data, and updating it’s value to the current time.

This allowed me to access each key-value in the JSON easily, and save it to set the parameters in our Neopixels lights function. Below you can see a glimpse of how it went for saving the data for Dawn. I later expanded this to include an SD card in order to save the dat and access it later in case I did not have an internet connection. Mostly as a fail-proof asset.


int dawnR, dawnG, dawnB;
long dawnMillis;
JSONVar myDataObject;

JSONVar apiSunTimesRequest(JSONVar myDataObject) {
  // assemble the path for the GET message:
  String path = "/sun";

  // send the GET request
  Serial.println("making GET request");
  httpclient.get(path);

  // read the status code and body of the response
  int statusCode = httpclient.responseStatusCode();
  String response = httpclient.responseBody();
  Serial.print("Status code: ");
  Serial.println(statusCode);
  Serial.print("/sun Response: ");
  Serial.println(response);

  //Write the Data to the SD Card
  writeToSD(response);

  // parse the string into a JSONVar object:
  myDataObject = JSON.parse(response);

  dawnMillis = myDataObject["dawn"]["time"]["ms"];
  
  // Set the color for each event
  dawnR = myDataObject["dawn"]["color1"]["r"];
  dawnG = myDataObject["dawn"]["color1"]["g"];
  dawnB = myDataObject["dawn"]["color1"]["b"];
   
    return  myDataObject;
  }
}

apiSunTimesRequest(myDataObject);

Form

I went though several iterations in order to get to a form which satisfied my thirst for a minimal horizon representation. I used blender in order to get a dimensionality and feel for what a wanted, and later used Illustrator in order to create the necessary shapes to later cut the shape with wood.

I quickly discarded the circle idea, and rather went for a more “horizon” kind of look. Such as this one taken by NASA:

https://www.abc.net.au/news/image/4688718-3x2-940x627.jpg

When I was satisfied with my 3D model, I decided to makes some illustrator sketches to start building the form . The construction would consist of several wooden layers attached together to achieve the required volume. The ‘Front Side’ , or show side, would have no markings on it, the remaining layers would be the main structure. These would be cut with laser to acquire the best precision possible, and would also allow me to design the frame in order to contain the components out of view.

I also started drawing out possible angles and the necessary depth of the volume, translating into how many layers of wood I would need. I decided on six layers of wood, each with a depth of 6mm. As you can see below, my idea was to make a 45 degree angle in the middle layers in order to place the lights on top

Very illustrative explanation of the position of the lights on the device. Side view above and front view below.
Initial measurements and making Pythagoras proud.
Final calculations.

So at first I just played around with the shape of each of the layers on top of each other:

Playing around.
Displaying each individual layer below the whole structure.

Getting a feel on how I would fit the components inside. As a minimum, the leanest slice would at least have to have 2.6 cms, So that was the measure I used as the scale for the other slices.

Thought about some text on it. Well, maybe not.
Getting there. You can see the cuts for this prototype below.
First attempts failed.
First attempts failed.

After several failed cuts, designs, and a lot of illustrator, I finally arrived at the design below, which divided each of the 5 “out-of-view” pieces to be cut into three separate pieces, each one with a hole for wooden dowels.

The current prototype design.
Current prototype exploded view display.
Current prototype exploded view display.

Production

The Strips

The strips come in a silicon enclosure, so I cut it off to have more access to the strip itself. The enclosure is so nice I actually felt sad about having to cut it. Besides, it cost a lot.

Both strips spread out.
The silicone enclosure is so nice.
Cutting the strips!

I then made a little jig in order to be able to keep my lights a distance away while I still worked on them from my computer. See it below:

Lighting jig solder.
The lighting jig in action.

The structure

First approaches:

Failed design:

Testing the different angles for my lights’ placement using the router table.
Failed print #2
Not as accurate as planned….
Observe the 45 degree angle on the right vs the uncut version on the left.

After 3 different failed cuts, I arrived at this design. Look at that beauty:

So smooth.
This time I kept the pieces together. I would glue them and then take them away.
Clearing the path inside.
Some glue to keep it all together.
It stands by it’s own. Proud parent here.

Putting it all together

I pasted the strips on the device using double-sided tape, and also sent the connecting cables inside the frame to the Arduino compartment as you can see below.

The power,GND, and input cables crawling into the frame.
Fitting the cables from the lights to the Arduino through the frame.
IT LOOKS GOOD!
Gorgeous, the strips fit perfectly. Maybe I should trim the cables though.
Drilling the hole for the power supply plug.
All of it fits snuggly into place.
Debugging the device.

Finishing it up

So….How does it look?

Horizonte.

HUGE THANKS TO:

  • Jeff Feddersen
  • Benjamin Light
  • Schuyler Devos

API Website/ Server : https://horizonte-api.herokuapp.com/

Github API Repository: https://github.com/lacouture100/Horizonte-API

API Website/ Server : https://horizonte-api.herokuapp.com/

Github API Repository: https://github.com/lacouture100/Horizonte-API

Github project Repository: https://github.com/lacouture100/ITP-Time/tree/master/MidTerm

Categories
Computational Approaches to Narrative

NaNoGenMo

…or 55 places the sun wonโ€™t shine on

This assignment is meant to be an entry to the National Novel Generation Month contest. The premise is quite simple: use code to generate a novel of 50k+ words, and submit the novel and code. I will explain in more detail below. As finals are kind of creeping in, I found humor and comfort in this project. Honestly, I never expected it to be so entertaining and exceptionally comical in a not so obvious way (my roommates did not share the feeling). I wanted my NaNoGenMo (National Novel Generation Month) entry to have a magical realism style to it. In a sense, I took this project as a building block towards my final project, and I shall publish a blog post on that next week.

In any case, I decided to use train GPT-2 on a corpus of text that consisted of several short stories from Jorge Luis Borges, such as “The Aleph”, in addition to two novels from Gabriel Garcia Marquez : “A hundred years of solitude” and “Love in times of cholera”. I also attempted adding work from Isabel Allende, as well as Jose Saramago. I had only read these authors in spanish before, and though I found Allende captivating in english as well, it did not feel the same. With Saramago, on the other hand, once I added “Blindness” to the mix the resulting text seemed quite violent. I might use it for my final, however for this long text it seemed unnecessary.


After tuning the model to my taste, (you can find my notebook on the Github repository) I decided to produce several short stories with the same starting prompt: “The sun had not shone on that part of the town for many years.” and cut them off at 55. I later proceeded to fix the style (too many random “/n” popped in the middle of a sentence) using the Atom editor to replace these characters with a standard “space” character. I also trimmed extra words at the end of each small story until I reached a punctuation mark. That way they would appear to be tidier, more of a short story, and less like the author fainted mid-sentence.

In the GitHub repository you’ll find a compiledText.txt which contains the corpus. Novel.pdf contains “55 places the sun won’t shine on”, and you can also see the google collab notebook and the zipped trained model. I leave you with some extracts from the novel:

====================

4

The sun had not shone on that part of the town for many years. The marble houses were on fire. In an instant panic spread through the places where gringos were hiding, dragging their feet and bent over by their neighbors, and the panic became a dragonโ€™s tail as one compact wave ran against another which was spinning in the opposite direction to the direction of the others.ย  At nightfall the survivors were lying face down on the ground, covered with a compact universe of tiny yellow flowers, and the only animal that could think of after all was a maneater. They slept in rectangular arrangements, facing each other in silence. On Friday the same men slept in identical arrangements under the same palm roof, but one night they slept in the same place, under the same bed, and on Saturday they were on the point of perishing under the lash of death. By dawn, they had killed two birds. They had killed a mule and a calf. They each had killed a mule and a calf. It seemed impossible to imagine the number of possible consequences.

====================

====================

22

The sun had not shone on that part of the town for many years. On the following day, however, the inhabitants of Macondo woke up in a kind of angelic stupor that had them ringed by earrings, bracelets, and rings made of bone in order to guess distances and render aid in the building of the church. The inhabitants of Macondo who stayed in the Community until the end of the year 1894 were on the verge of forgetting their sense of social well-being because they could not sleep and felt tormented by the notion of parchments rotting with the irremediable plague of insomnia. Josรฉ Arcadio Buendรญa, without giving up his science, continued his work as if it were the only thing left in the whole town, the decisive in that respect owing not to the natural forces but to the supernatural proliferation of the village.ย 

======================

Categories
Social Innovation Design

Undocumented Immigrants and tech work availability

In the United States, the history of illegal immigration can be described as somewhat turbulent. Albeit it has been sometimes welcoming, sometimes exclusionary (1), it is undeniable that immigrants have been central in the construction of this country. Today we are witnessing several injustices and unfair treatment towards undocumented (and many times documented) immigrants. Many of these injustices are imparted through the systematic approach of political discourse and technological tools. Ironically enough, some of these technologies, which could be considered to be a part of ‘Silicon Valley’, have been created with the labor of undocumented immigrants.

Silicon Valley became the tech hub of the world thanks to a very different set of workers. Unlike the postwar industries that created a middle class from union wages, electronics expanded in the 1970s and โ€™80s through low-cost, often subcontracted, often undocumented labor. Instead of self-aware robots or high-dollar professionals, it was women of color, mostly immigrants โ€” hunched over tables with magnifying glasses, assembling parts sometimes on a factory line, sometimes on a kitchen table โ€” who did the necessary but toxic work of semiconductor manufacturing. Many of the undocumented workers were from Mexico, while many of the documented ones were from there and Vietnam.

Source

The political ideology which has been in power for the last four years in the United States has been more severe in its battle against undocumented immigrants residing in the country. As said in an article about 300 undocumented tech workers arrested on their job-site, these actions “punctuates the push by the Trump administration to target companies employing people who federal authorities say are not authorized to be in the U.S”. (2).

My idea would be to create a way in which undocumented workers could offer their labor(tech is the first that comes to mind) through some tool that allows them anonymity. By anonymity, I specifically mean the possibility for them to work away from a specific physical space, as well as access to some payment system without identity/legal status registration (bitcoins?).

References:

1 – U.S. Immigration Timeline https://www.history.com/topics/immigration/immigration-united-states-timeline – Consulted November 3, 2020

2 – Nearly 300 undocumented immigrants arrested at a tech business in Texas https://www.kusi.com/nearly-300-undocumented-immigrants-arrested-at-a-tech-business-in-texas/ – Consulted November 3, 2020

Categories
Thesis part 1

Thesis Part 1: Research

ITP Portfolio Overview

Horizonte (2020-Fall)

Horizonte main image
Horizonte main image

Tangible Dynamics (2019- Fall)

Multiple users playing with our spheres.
Multiple users playing with our spheres.
https://backend.alvarolacouture.com/tangible-dynamics/

Oneiric (2020 – Spring)

Categories
Time

HORIZONTE – Time Midterm

API Website/ Server : https://horizonte-api.herokuapp.com/

Github API Repository: https://github.com/lacouture100/Horizonte-API

Github project Repository: https://github.com/lacouture100/ITP-Time/tree/master/MidTerm

Introduction

My project is called Horizonte, a horizon clock that adjusts it’s color and light patterns in real-time according to its location and the current time of day. The clock itself is a device that provides a WiFi access point for remote configuration, and an autonomous network-based update feature, which allows the user to customize the clock’s parameters. I wanted to stray away from the sense of being precise, a rather lazy, and non-invasive way of letting you know what part of the day it is. I wanted to acknowledge the shift without precisely getting a sense of what exact moment it is. Consequently, The clock itself gradually shifts from one stage of the day to the next, taking around 5- 20 minutes to shift depending on the event, location, and date. The project uses an API that is based on the SuncalcJS library to acquire the time for each particular ‘sun event’ in the day, such as Dawn, Sunset, etc… For this particular project, at this stage at least, I am dealing with events that happen between Dawn and Dusk. Why? These are the first and last stages of sunlight, and if you are a fan of ” From Dusk till Dawn”, those are vampire-safe times.

This device is meant to be hung on the wall… Specifically, the lights would be pointed towards the wall, as to give a sensation of a “sky” inside a space.

Hardware Used

Power requirements

  • I used a 5V, 10A power supply for both the Arduino and the lights. Same as this one.

Software

  • Arduino IDE
  • NodeJS
  • Suncalc
  • Blender
  • Illustrator

Initial Proposal

Initial mockup idea
First sketches in pen and paper.
First sketches in pen and paper.

Light

I fortunately had a Neopixel strip around right from the start. I tested several colors, patterns, and rhythms in order to get a sense of how I could maneuver the light in the fashion I wanted, starting with something like the video below. Through the tests, I found that maintaining a 45-degree angle served my purpose better.

Testing the Neopixels against the wall

I mainly focused on gradients, however I also tried different approaches, as you can also see below.

Testing different lighting properties with the strips.

I quickly found myself struggling to acquire the control I needed, and after a week or two of trying to make my own gradients based off of Adafruit’s limited examples, I was running out of time to get the other things going. After looking for a while I ran into FastLED, a Neopixel control library developed for Arduino which boasts to be better than the original library. Their examples were very varied, complete, and allowed me to find a couple of useful scripts which I took apart to make my own. Specifically, I heavily relied on the ColorPalette example and the nblendPaletteTowardPalette function found in the docs. The code implemented in the clock goes through nine different gradients, transitioning in and out of them during the day.

The main function takes the following parameters:

  • A current time measure in order to determine which sun event should be activated
  • The start time for the current day loop. It resets at midnight.
  • each of the sun events in millis().
void ChangePalettePeriodically_bottom(unsigned long startMillis,
                                      unsigned long currentMillis,
                                      long interval1,
                                      long interval2,
                                      long interval3,
                                      long interval4,
                                      long interval5,
                                      long interval6,
                                      long interval7,
                                      long interval8,
                                      long interval9
                                     ) {
Gradient using FastLED
Seeing the individual sections on the two strips

In terms of wiring and connections, it was pretty straightforward. I used an Arduino MKR 1010 since it had an esp32. From previous research, I knew it had more ram than the Arduino Nano, which was my other option. I hooked a couple of 470-ohm resistances to the input line of each Neopixel Strip. I also added two 470 microfarads capacitors in series. The Adafruit Uberguide for powering Neopixels suggests a 1000 microfarad capacitor, however, I did not have one. All of this was powered by a 5V, 10A power supply. Here’s the schematic below.

Horizonte Schematic

Color

First Gradient display made with Miro

In order to represent the horizon, or at least my view of it, I would have to decide on a color palette to use. I started out doing something like the image below, which gave me a general sense of where to go.

However, the more I observed the sky the more I could not help but think about what makes each sun stage so unique beyond it’s time. What makes a sunset different than a sunrise? The color? Not really. However, it is different, it feels different. So I felt unsatisfied with my above diagram and expanded it towards the one below.

Gradient and sky mood board in Miro

This one was much better. It made me feel each of the sun events way better and get a more complete sense of the transitions throughout the day. Throughout my observation I also came to the conclusion most of these gradients were two-toned, and in some cases even three, which gave path to the notion of using 2 led strips instead of one.

Finally, after having all of my hardware and LED libraries at hand, I added the FastLED color table to see what I would have to code in order to get these gradients. Image Below. Link to the FastLED color diagram here.

Gradient, FastLED color-code diagram, and sky mood board in Miro

Calculation

My first idea was to make a gradient that would go across the strip, and it would have a fish-lens type of sphere that would travel along the strip throughout the day. However, after working on it for a while and trying out different approaches, I did not quite achieve the effect I wanted. Because even though I could create a custom gradient and modify its position, I did not clearly see a way to make it happen in runtime. So after deciding whether I should invest more time in it in the moment, I changed my strategy. I would show the horizon and it’s particular stage in its entirety, without giving away what passed and what came next.

The calculations made in order to acquire the effect were not so difficult to adapt. However after attempting to do some of them onboard the Arduino, I came to the conclusion ( thanks to Jeff for this one) it was better to make the Arduino as dumb as possible. I would send everything calculated from the Node server I set up and the Arduino would just set the HTTP response body as the parameters for the lights and time synchrony. They were : Which time to turn on and off the device, how long it would take in each of the solar stages, and what time scale would I be following: local or UTC? The first two were easy: Suncalc gave them to me already, I just had to pass it into milliseconds in order for the Arduino to calculate everything in the same scale.

User Interaction

At first I played around with the idea of two or three simple buttons hidden away, maybe even a small OLED screen? (I think I’ve broken 10 of them already) I used Miro app to swiftly design how the user interface would work in terms of what appeared on the screen, what input or option did the user have, and what would happen in the device while all of this happened.

Apparently, the initial idea of maybe having a couple of buttons or a screen was not as straightforward as I thought, at which point Jeff directed me towards the possibility of creating an access point with the Arduino and make the UI and UX in there. Amazing. So that’s what I went for. I used the AP_SimpleWebServer example from the WiFiNINA series of examples to set up my own thing.

The Horizon Hotspot

WiFi Hotspot connected!
Web UI for the Horizonte Clock

In the images above you can see the hotspot being created by the clock, and how the interface is presented. I managed to send some JS in the site in order to grab the local time, GMT, and also modify the CSS to make it look a bit better. Somehow flex doesn’t work all the way, though I might be overlooking something. You can find the code for this one in the “Access point” tab in the Main Sketch. Anyways, once the hotspot is on, you input the credentials for a WiFi network around you, as well as your latitude and longitude. These lase two will later make part of the request for the server in order to get accurate times.

The API, the request, the data

After having seen Tom Igoe’s Connected Devices and Networked Interaction class last semester, I felt fairly confident that making my own API was the way to go in order to access the data at any given time from any device and also to control it. Manipulating the data and processing it before sending it to any device was a priority. I decided to make it in NodeJS since I wanted to keep the possibility of incorporating JS into the web client open. I also mounted it on Heroku, which probably isn’t the best option for continuous communication such as WebSockets but works fine for HTTP requests. As I said before, even though my idea was to keep it as real-time as possible the idea of being a couple of seconds or even minutes off track didn’t mean much. The API consists of two routes: /sun and /now. /sun ill return all of the times for each sun event according to UTC. /now will return the current time.

The data is presented as a JSON file. So everytime we make a request, the server writes an updated JSON file which is sent to the Arduino. You can see the JSON formatted below.

Server console log

To made the request to the server I used the WiFi Web Client example from the Arduino IDE, and later saved it as a JSON using the ArduinoJson library.

Arduino Serial Log connecting to the hotspot, receiving the data, and updating it’s value to the current time.

This allowed me to access each key-value in the JSON easily, and save it to set the parameters in our Neopixels lights function. Below you can see a glimpse of how it went for saving the data for Dawn. I later expanded this to include an SD card in order to save the dat and access it later in case I did not have an internet connection. Mostly as a fail-proof asset.


int dawnR, dawnG, dawnB;
long dawnMillis;
JSONVar myDataObject;

JSONVar apiSunTimesRequest(JSONVar myDataObject) {
  // assemble the path for the GET message:
  String path = "/sun";

  // send the GET request
  Serial.println("making GET request");
  httpclient.get(path);

  // read the status code and body of the response
  int statusCode = httpclient.responseStatusCode();
  String response = httpclient.responseBody();
  Serial.print("Status code: ");
  Serial.println(statusCode);
  Serial.print("/sun Response: ");
  Serial.println(response);

  //Write the Data to the SD Card
  writeToSD(response);

  // parse the string into a JSONVar object:
  myDataObject = JSON.parse(response);

  dawnMillis = myDataObject["dawn"]["time"]["ms"];
  
  // Set the color for each event
  dawnR = myDataObject["dawn"]["color1"]["r"];
  dawnG = myDataObject["dawn"]["color1"]["g"];
  dawnB = myDataObject["dawn"]["color1"]["b"];
   
    return  myDataObject;
  }
}

apiSunTimesRequest(myDataObject);

Form

I went though several iterations in order to get to a form which satisfied my thirst for a minimal horizon representation. I used blender in order to get a dimensionality and feel for what a wanted, and later used Illustrator in order to create the necessary shapes to later cut the shape with wood.

I quickly discarded the circle idea, and rather went for a more “horizon” kind of look. Such as this one taken by NASA:

https://www.abc.net.au/news/image/4688718-3x2-940x627.jpg

When I was satisfied with my 3D model, I decided to makes some illustrator sketches to start building the form . The construction would consist of several wooden layers attached together to achieve the required volume. The ‘Front Side’ , or show side, would have no markings on it, the remaining layers would be the main structure. These would be cut with laser to acquire the best precision possible, and would also allow me to design the frame in order to contain the components out of view.

I also started drawing out possible angles and the necessary depth of the volume, translating into how many layers of wood I would need. I decided on six layers of wood, each with a depth of 6mm. As you can see below, my idea was to make a 45 degree angle in the middle layers in order to place the lights on top

Very illustrative explanation of the position of the lights on the device. Side view above and front view below.
Initial measurements and making Pythagoras proud.
Final calculations.

So at first I just played around with the shape of each of the layers on top of each other:

Playing around.
Displaying each individual layer below the whole structure.

Getting a feel on how I would fit the components inside. As a minimum, the leanest slice would at least have to have 2.6 cms, So that was the measure I used as the scale for the other slices.

Thought about some text on it. Well, maybe not.
Getting there. You can see the cuts for this prototype below.
First attempts failed.
First attempts failed.

After several failed cuts, designs, and a lot of illustrator, I finally arrived at the design below, which divided each of the 5 “out-of-view” pieces to be cut into three separate pieces, each one with a hole for wooden dowels.

The current prototype design.
Current prototype exploded view display.
Current prototype exploded view display.

Production

The Strips

The strips come in a silicon enclosure, so I cut it off to have more access to the strip itself. The enclosure is so nice I actually felt sad about having to cut it. Besides, it cost a lot.

Both strips spread out.
The silicone enclosure is so nice.
Cutting the strips!

I then made a little jig in order to be able to keep my lights a distance away while I still worked on them from my computer. See it below:

Lighting jig solder.
The lighting jig in action.

The structure

First approaches:

Failed design:

Testing the different angles for my lights’ placement using the router table.
Failed print #2
Not as accurate as planned….
Observe the 45 degree angle on the right vs the uncut version on the left.

After 3 different failed cuts, I arrived at this design. Look at that beauty:

So smooth.
This time I kept the pieces together. I would glue them and then take them away.
Clearing the path inside.
Some glue to keep it all together.
It stands by it’s own. Proud parent here.

Putting it all together

I pasted the strips on the device using double-sided tape, and also sent the connecting cables inside the frame to the Arduino compartment as you can see below.

The power,GND, and input cables crawling into the frame.
Fitting the cables from the lights to the Arduino through the frame.
IT LOOKS GOOD!
Gorgeous, the strips fit perfectly. Maybe I should trim the cables though.
Drilling the hole for the power supply plug.
All of it fits snuggly into place.
Debugging the device.

Finishing it up

So….How does it look?

Horizonte.

Behind the Scenes:

What’s missing?

  • Decide on a pattern to set the time and stick with it. Be it UTC or local time.
  • Actually see it throughout the day, a whole run.
  • Implement a failsafe in case there is no internet, or the user inputs the password incorrectly.

Future development

  • Design a more complete Web UI.
    • Customize color on the device from a web customizable gradient.
    • Allow fade control.
  • Design my schematic and board in Eagle. Simplifying and cutting size.
  • Integrate other visualizations based around the concept of a horizon, astronomical horizon, etc.
  • Adjust the shape so it fits my slimmed-out idea.
  • Include an on/off button.
  • Incorporate node-geo-tz – geographical timezone lookup package
    • Scrape Instagram for current color palettes in images and location?

HUGE THANKS TO:

  • Jeff Feddersen
  • Benjamin Light
  • Schuyler Devos

Categories
Desert of the Real: Social VR

Unity VR Sync – Avatar Selector and VRChat world creation

Introduction

This week we looked into rigging the Avatars in Unity with VR IK using the Final IK asset. The process was fairly simple, in terms of just assigning the script to the gameObject which also contained the Animator class. This process also involved setting some control objects which would handle any avatar on scene, as you can see below.

Control Cubes in motion.

Dynamic linking in Unity

The idea of learning this, however, was to be able to dynamically link everything on the scene. Why? When someone enters the game as an avatar, the game has to automatically set the controls for the player, as well as for any other player who is instantiated inside the game. When the game is networked, there is no fixed path or reference point to which scripts or objects can refer too. We have to create these BRIDGES. We create these BRIDGES in order to handle these dynamically linked avatar controls.

  • BRIDGE_ controls are created as a template for any VR headset. When any VR avatar is put into the scene, it will subscribe its body position to the controls. They connect when the avatar is instanciated.
  • These bridges allow us to control our bodies.
  • Below you can see a screenshot of the hierarchy. There are three main sections:
Hierarchy View
  • RED: The Avatars, parented under an empty gameObject.
  • YELLOW: The BRIDGES, which connect to parts of the avatar Rig through the VRIK script. (I attached an image below)
  • BLUE: The dynamically linked controls. The bridges will look for these in order to attach to their positions. There is a BRIDGE for each of the three VR_BodyParts.
VRIK settings.

Character Selection

In order to make the option for the user to change avatar through keyboard input, I created 2 scripts: AvatarSelector, which is attached to the AvatarParent I showed in an earlier image. The second script is AvatarListener, which is attached to each avatar(Alvaro, Satomi_1, and Satomi_2), and it’s parameter can be set to Avatar1, Avatar2, OR Avatar3, which correspond to the ENUM keynames defined in the AvatarSelector script. This means that if you press key 1 in your keyboard, you shall see Alvaro. If you press 2, you shall see Satori_1, and if you press 3 you shall see Satori_2( blue and with a party hat.) You can see the video below, as well as each of the scripts.

AvatarSelector.cs

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using TMPro;
using UnityEngine.InputSystem; 
using System;

public enum AVATAR
{
    avatar1,        //0
    avatar2,
    avatar3
}


public class AvatarSelector : MonoBehaviour
{
    // This will be our array of avatars on scene
    AvatarListener[] avatars;

    // This will be our keyboard input
    Keyboard keyboard;

    // The current Avatar is the onw which will we will see and control. By default it is set to avatar1.
    private AVATAR currentAvatar = AVATAR.avatar1;

    // This will display our current character on screen
    public TextMeshProUGUI avatarText;
    void Start()
    {
        // Set our current Keyboard as an input device.
        keyboard = Keyboard.current; 
        if(keyboard != null)
        {
            Debug.Log("Keyboard found!");
        }
        else
        {
            Debug.Log("Keyboard not found :(");
        }

        // Find all of the gameObject with the AvatarListener component attached. Our avatars.
        avatars = FindObjectsOfType<AvatarListener>();

    }

    private void SetAvatarText()
    {
        avatarText.text = ("You are now " + currentAvatar.ToString());

    }

    public void FixedUpdate()
    {
        // Remember which avatar is active at each frame.
        AVATAR lastAvatar = currentAvatar;

        // If you hit the "1" key in the keyboard, select avatar1
        if (keyboard.digit1Key.ReadValue() != 0)
        {
            // Set the current Avatar to avatar1
            currentAvatar = AVATAR.avatar1;
        }
        // If you hit the "1" key in the keyboard, select avatar1
        if (keyboard.digit2Key.ReadValue() != 0)
        {
            // Set the current Avatar to avatar1
            currentAvatar = AVATAR.avatar2;
        }
        // If you hit the "1" key in the keyboard, select avatar1
        if (keyboard.digit3Key.ReadValue() != 0)
        {
            // Set the current Avatar to avatar1
            currentAvatar = AVATAR.avatar3;
        }

        // Loop through each of our avatars.
        foreach ( var avatar in avatars)
        {
            // If the avatar is not the one we selected, set it unactive.
            if(avatar._avatar != currentAvatar )
            {
                avatar.gameObject.SetActive(false);
            }
            // If the avatar is the one we selected, set it active.
            else
            {
                avatar.gameObject.SetActive(true);
            }
        }

        // Print out everytime you change an avatar
        if(currentAvatar != lastAvatar)
        {

            SetAvatarText();
        }

    }

}

AvatarListener.cs

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
/*
 Attach this to each Avatar. The Avatar Selector class will lokk for this script.
*/

public class AvatarListener : MonoBehaviour
{
    //Select the avatar
    public AVATAR _avatar;

}
Categories
Computational Approaches to Narrative

Bitsy- Time is a garden of memories

I really enjoyed the overall dynamic around Bitsy. A browser-based game development platform with limited processing power manages to give ample room for narrative development. It also reminded me of a MMORPG platform I used to play around with when I was younger. It had a similar general aesthetic. UPDATE: It’s still online! Check it out: Byond.

My story was inspired by the book “The Order of Time” by Carlo Rovelli. You can play it or see the video below.

Bitsy UI capture
Bitsy UI capture
My Bitsy Game

Repository with game file: https://github.com/lacouture100/ITP-Computational-Approaches-to-Narrative/tree/main/Bitsy

I also urge you too check out these other bitsy games: