Body as Reservoir of Secrets, DEMS Cycle 2

For cycle 2, the idea started to take shape in a more concrete manner. I started to develop the costume using coveralls bought on amazon which I tailored myself for a better fit.

The following feature extraction functionalities were in the costume-

1. On the coveralls I stitched on some patches with dark pieces of digitally printed fabric. This increases the overall contrast of the figure, which helps the eyes++ actor and webcam track object movements better.
2. A contact microphone was housed in the chest pocket running up to my neck. An additional elastic band was devised to apply pressure and hold the contact microphone in place.
3. Two iphone Gyroscopes were housed within blue/purple pockets with elastic openings on the arm and calf of the custom.
4. A microphone was blown into whenever a scene change was desired, as a means of activating breath.

Challenges/Feedback

• Composition looked too busy given that more than 10 variables were being fed into isadora.
• My computer’s graphics card seems to be struggling and isadora crashes easily.
• A classmate mentioned that the costume appeared like an astronaut, which is not what I intended.
• I had a headache after wearing the neck piece with the contact mic for 20 minutes because it was restricting bloodflow.

Observations/On-going curiosities

• The class seems to be very interested to map/figure out which visual element was coupled with the various sensors.
• The class seems to be interested view the live generated graphics as a video piece, when I had intended the graphics primarily to be viewed as prints. I am open to the possibility of the video being screened alongside the painting performance.

Cycle 2

Cycle 2

I achieved connecting Isadora and Arduino for Cycle 1 and made just a simple Arduino project like blinking LEDs.
So, I set up my goal for Cycle 2 to try to operate motor(s) by Isadora using a webcam as a motion sensor.

I firstly tried a servo motor. Fortunately, Arduino actor on Isadora already had a feature to control servos so I could easily connect them.
I made a webcam as a motion sensor like we had played in PP3 and made an instant “interactive servo robot” whose arm chases a viewer’s motion.

I also tried a stepper motor. I was a little bit tricky for me as Arduino actor on Isadora doesn’t have a stepper motor operating feature.
So, I re-checked a tutorial Arduino code to control a stepper motor and made the same work flow on Isadora (blinking Low/High to make a magnetic motor spin).
Its motion was not smooth, but it worked!

Probably I need to adjust an interval time between Low/High or some other things to operate a stepper motor smoothly, but anyway I decided to use servo motors for Cycle 3 as I should focus on building a final output now (and servos are easier to control for my project).
But operating a stepper motor by Isadora taught me many things like the idea of translating a written code into an Isadora chart.

My Cycle 3, the final output, would be a small interactive robot using webcam, Arduino, servos, and Isadora for sure.
Feedbacks from the class during Cycle 2 inspired me to make a small but very organistic interactive robot; as I usually make spatial or sculptural scale works, I considered my small Arduino test project (like a tiny servo) just as a small mock-up, but everyone’s reaction suggested me that a small scale thing could have its own communicativeness which a large sculpture doesn’t have. This would be my direction to achieve Cycle 3.

Cycle 1: Blinking Arrows

Going into Cycle 1, I knew that I wanted to make a game of some kind and I knew I wanted to use an alternative control scheme of some kind using the Makey Makey. I was also drawing inspiration from retro games, like Snake and Simon/Simon Says. Aside from this, I didn’t have any idea of what I wanted my final project to look like, so I decided to treat Cycle 1 as a pressure project, in that I would play around with Isadora and my ideas so far, and whatever came from it is what I’d demo for class.

My first step was just figuring out what exactly I wanted my project to look like at the end of Cycle 3, so I decided to just look at some retro games that I could potentially recreate in Isadora within the scope of the class. This research brought me to Dance Dance Revolution (not quite retro but close enough) and the idea of a rhythm game, the mechanics of which felt simple enough to produce before the end of the semester.

I started my patch with a rectangle in each corner of the stage, each labeled with a direction (up, down, left right). I would be using the Makey Makey as my controller (I will decide the actual controls in a later cycle), but these would correspond to the temporary keyboard input for each rectangles action. The rectangle UI didn’t feel very intuitive to me, and I didn’t like how it looked, so I scrapped those and changed them to arrows. At this point, I moved towards creating the game mechanics, starting with getting the arrows to blink based on the input of the user. Below is a picture of the patch for just one of the arrows to demonstrate how I got this to work.

During the in-class demo, I mentioned wanting to use a source of input that would be simple enough that a baby could use it, drawing inspiration from my baby nephew. Someone suggested I use a children’s xylophone as my controller and I really liked the idea, so I will be exploring children’s toys as input methods, most likely in Cycle 3. For Cycle 2, I will me continuing to work on the mechanics of my game.

Cycle 1: Getting Wires Crossed

For my cycle 1, I decided to tackle a basic interactive element I want my cycle 3 to have, a stove. I started my Isadora patch with a top-down view of an electric stove, where I would connect red shapes to slowly fade in and out based on user interaction. I initially tested with a keyboard watcher, but knew I would eventually attach a Makey-Makey to my makeshift stovetop. I used four user actors as the “burners” and used gates to prevent them from being continually turned on and off. Below is a screenshot of one of the burner’s patches.

I also knew going into this process that I want my cycle 3 to be a house-like experience with odd elements thrown in to encourage interaction. I’ve always been interested in the idea of a “normal house but move it slightly to the left.” I’m still deciding if I’m connecting some sort of story to it yet or not, but for the moment I knew something had to happen if the user interacted with the stove a certain way. I made it so a pit would “open up” on the stove top if the user turned all the burners on within 2 seconds. I used the simultaneity actor within Isadora to achieve this.

Here’s a few pictures I took when I first connected the Makey-Makey to the patch. The performance in class was a great experience and I deeply appreciated the feedback others provided. I took note especially that the framing of having physical stove dials helped make the experience more interactive and special, so I hope to include more physical interaction in my next two cycles.

Body as Reservoir of Secrets, DEMS Cycle 1

I see the body as a reservoir of ancestral secrets. I have been coaxing out /imaging these secrets as an abstract painter, through gestures and paint. However, gesture is only one vector in this exploration. The body generates streams of data in addition to gesture that can be harnessed and imaged- such as breath, heartbeat, voice.

I am devising a system that extracts these features and maps them accordingly to the movement/color/scale of visual elements, triggering video or image files. These visual elements come together in compositions that will then have video and print outputs, via the capture stage to image and video actors.

In Cycle 1, I mostly concentrated on figuring out isadora-OSC/gyroscope communications and tuning hardware like webcams and lavalier microphones for feature extraction. The outcome was somewhat successful, but I have generated too many input data streams and are lacking variables to plug these inputs. Feature extraction for hearbeat is not very effective using a lavalier microphone.

For Cycle 2, I intend to:

• Focus on creating more variables to plug the generated inputs into.
• Develop greater flexibility in the visuals generated, so that there is more variance in the composition- composition can range from very simple and very complex.
• Include a contact microphone for heartbeat extraction
• Start to integrate costume into mix.

For Cycle 3, I intend to:

• Develop a clarified live performance dimension to the work
• Figure out how to feed the captured jpegs to a printer

Reflection on Cycle 1

I set up my goal in this class at the end of the semester as “connecting Isadora and Arduino” and “using a webcam as a sensor to operate a motorized object”.

The first step=my goal of Cycle 1 was just simply connecting Arduino to Isadora.
I researched on the web (there are some helpful articles and forums) and found that an application called “Firmata” can play a role to connect Arduino to many kinds of software including Isadora.
This article explains what Firmata is and how to get/use it.
https://www.instructables.com/Arduino-Installing-Standard-Firmata/

We normally need to write a code to operate Arduino, but with Firmata, I can send output signals to or receiving inputs from Arduino without coding process.
I downloaded a protocol code called “StandardFirmata” and uploaded on my Arduino.
Here’s the test of Firmata; by clicking High/Low on Firmata window, I can turn on/off a LED connected to Arduino.

Then, I downloaded “Arduino Firmata Actor” which is an Isadora plugin to Arduino.
https://community.troikatronix.com/topic/7176/arduino-firmata-actor


Now I’m ready to connect the two.
I opened Isadora and set up Arduino Firmata Actor (and also some port connection settings) to connect to my Arduino.
Here, I can turn on/off LEDs on Isadora window (so I’m sending output signals from Isadora to Arduino).

Another test; an input signal from a photoresistor (a sensor detects brightness of light) connected to Arduino is being received by Isadora (sending signals from Arduino to Isadora).
I connected this input to a width of a square, so it’s interacting with light.

This very simple connection test was a very big step for me. The two big magics (Isadora and Arduino) are now syncing!
The next step (Cycle 2) will be connecting a servo motor (and/or a stepper motor??) and trying to operate it from Isadora. Then, I can connect it to a webcam as a sensor, just like we did in PP2 and 3.