Cycle 3: Layering and Gesture: Collective Play

For this third iteration, I decided to set up three digital layers that provided space for play, collaboration, and digital/analog spaces to mingle. My initial idea was to consider how I could introduce the body/model into the space and suggest an opportunity for gestural drawing and experimentation both on physical paper and digitally. As you can see in the image below, participants were actively engaged in working on the paper, viewing what happening on the projection screen, and interacting with one another across these platforms and planes in space. A third layer not visible in the image below is a LIve Drawing actor in Isadora that comes into play in some of the videos below. I stuck with the TT Edge Detect actor in Isadora and played with a Motion Blur actor on the second layer so that the gestural movements would be emphasized.

Note the post-its on Alison’s back below. These were a great surprise as they were translated into digital space and were activated by her drawing and movement. They became a playful, unexpected surprise!

Alex the superhero!
Isadora Patch/Cycle 3
Interaction between three digital layers.
Drawing together across physical and digital space.

I really appreciated the feedback from this experience and want to share some of the useful comments I received as a record:

  • Alison: I loved that Alison shared it was “confusing in a good way” and that she felt like it was a space where she could play for a long time. She identified that this experience was a social one and that it mattered that they were exploring together rather than a solo experience.
  • Katie: Katie was curious about what would show up and explored in a playful and experimental way. She felt some disorientation with the screens and acknowledged that when Alex was using the live draw tool in the third layer, she didn’t realize that he was following her with the line. I loved that this was a surprise and realized that I didn’t share this as an option verbally well enough so she didn’t know what was drawing the line.
  • Alex: Alex was one of the group that used the live draw tool and others commented that it felt separated from the group/collaborative experience of the other two layers. Alex used the tool to follow Katie’s movement and traced her gestures playfully. He commented that this was one of his favorite moments in the experience. He mentioned it was delightful to be drawn, when he was posing as a superhero and participants were layering attributes onto his body. There was also a moment when I said, “that’s suggestive” that was brought up and we discussed that play in this kind of space could bring in inappropriate imagery regardless if it was intended or not. What does it mean that this is possible in such a space? Consider this more. Think about the artifact on the paper after play, how could this be an opportunity for artifact creation/nostalgia/document.
  • Mila: With each iteration, people discovered new things they can do. Drawing was only one of the tools, not the focus, drawing as a tool for something bigger. Love the jump rope action!
  • Molly: How did we negotiate working together? This creates a space for emergent collaboration. What do we learn from emergent collaboration? How can we set up opportunities for this to happen? The live draw was sort of sneaky and she wondered if there was a way to bring this more into the space where other interactions were happening.

This feedback will help me work towards designing another iteration as a workshop for pre-service art teachers that I am working with in the spring semester. I am considering if I could stage this workshop in another space or if using the motion lab would be more impactful. If I set it up similarly in the lab, I would integrate the feedback to include some sort of floor anchors that are possibilities or weights connected to the ropes. I think I would also keep things open for play, but mention perspective, tools available, and gesture drawing to these students/participants who will be familiar with teaching these techniques to students in a K – 12 setting.

I have been exploring the possibility of using a cell phone mounted on the ceiling as the birds-eye-view camera and using NDI and a router to send through Isadora. I’ll work on this more in the spring semester as I move towards designing a mini-version for a gallery experience in Hopkins Hall Gallery as part of a research collective exhibition and also the workshop with the pre-service students. If I can get permission to host the workshop in the motion lab, I would love to bring these students into this space as my students this semester really appreciated the opportunity to learn about the motion lab and explore some of the possibilities in this unique space.


Cycle 2 – Perspective drawing tool

I decided to move from the small handheld book form to the motion lab for my next cycle. I have been working with my own students in the motion lab. I decided to explore the possibility of designing tools for teaching drawing that involve observation, collaboration, and movement in the process. My initial plan involved using 2-point perspective as a starting point. Typically teaching 2-point perspective is a guided drawing kind of process, step-by-step. It is most often experienced using a pencil and a ruler. A teacher guides students through steps for drawing a horizon line, two points, and then proceeds to demonstrate how to draw a simple dimensional cube, often using a document camera to project a larger live capture image of the teacher’s actions. Once students understand the basic process, they are set free to create on their own.

I scaled this process up and envisioned the motion lab as a large format document camera. I set up a drawing space in the center of the space, directly under a birds-eye-view camera. In the image below you can see that I projected the live feed image on the main projection screen and applied a TT edge detect filter in Isadora. In the photo you can see students using the ropes that were each tied to one of the points on our horizon line that bisects the image.

The patch I created was fairly simple, see below:

Isadora Patch

Playing with two camera views layered and both processed through TT Edge Detect

After the initial explorations with the one video in-watcher, I layered another feed from a camera on a tripod that was directed at a whiteboard. This is when participants began to play even more with what was happening on each screen and how they were interacting together in the projection.

Layering with birds-eye-view and camera on tripod. shifting scale by zooming in, turning camera view

For my final 3rd cycle, I want to continue to build on and play with this tool. My initial ideas about pushing it into another phase is to focus on observational drawing and the figure/body in motion. Maybe shifting the output 90 degrees so the drawers can face the large projection screen head on. Using two camera again, but one would be directed at a body moving in space and those drawing would try to capture the movement and gestures on the page. I also wanted to add in the live drawing tool and see how that adds another layer of complexity and interest to the projected image.


Cycle 1: Book to map?

Makey Makey connected to small book prototype, linking into Scratch to trigger audio files.

Exploring the book form and text that is interactive and triggers audio. Questioning the form. Leaning towards a map . . .

How do I design an experience for someone to engage with my research in a way that breaks from the typical dissertation format and reading printed text on a page?

Should the experience be intimate? Like our experience reading a book where it is just us and the text? Or, relational to both the text (object), ideas, researcher and reader?

What structure could facilitate a non-linear engagement with the “text” that is different for each individual?

How does one experience the researcher’s voice?

I considered a book and an accordion-style book but these seem limiting. Is a map an option? What are the aesthetics? How do I include layers? How does the experience facilitate connections across distance? A distancing from the research itself and the researcher?

What do I like about the book format?

              It relates to a way we come to know things through reading.

              A physical book must be held in one’s hand.

              A book has weight. Value. Symbolic of knowledge.

What do I like about a map? The process of reading a map?

It still has boundaries and borders, but can be entered from any point? Could include a hierarchy using scale and volume. 

Can still be held, however can an interactive map be hand-held in a convenient way? (other than a digital map on our phone)? I like the idea of the reader following along with their finger to “trace” the lines of text which also triggers and “generates” the text.

What about a wayfinding format? Like those trailhead maps? Something tactile with “triggers” built in that combine reading with audio that fills in the spaces between maybe? As you follow from one point on a “path” to another, you make a connection and the audio plays and leads you there.

Tactile topographic maps. . .

If this and this, then this. . .

Point A (Idea A) draw line to point B (Idea B), space between how A relates to B (audio).

Trigger through touch. Running your finger over the text or connection line. Touching on two points to draw a connection between them.

Recording: Idea A, connection, Idea B Following the route. Using the text to draw the line between them and make the connection.

Still pre-determined by researcher, in control of what the ideas are, how they relate/connect. Reader is in control deciding what they are interested in making connections between. I could imagine what this might look like in a VR space potentially, but what about a physical space?

How can I present a standard, university-defined, formatted dissertation become more experiential?

What if the “text’s” were not just printed text and audio, but also video and photographic artifacts?

Aesthetics and materials, topography, cast tracing paper, Plexi, watercolor, cut-out text, wires visible? Connections and “behind-the-scenes” transparent, process exposed, making thinking visible, translations, sound and video/photo projection on the “map”? Is there a backtrack? Could there be subtle changes to the backtrack that are affected by the “routing” on the map?

Is it on a wall, positioned on an angle much like a podium at a height about waist-high?

Intermedia map that involves human touch, listening, hearing, and viewing, prompts one to speak?

Is a dimensional, topographic map a good solution? The lines between could be graphite or could be the actual wire that is couched down with embroidery floss. The wires could be “embedded” into the layers of the tracing paper casting so they could be hinted at, and visible, but also underneath the surface. How do you ground the viewer in this case? What if they had to hold a “fake” pen/pencil that connected to the ground and use that to trace? I prefer the idea of using their finger directly to trace though. Maybe they are asked to touch a “key” of some sort as they are also then touching a point and following it with their finger.

How could each point be wired up with a way for a finger to complete the whole circuit?

Some feedback: My next step is to find a simple story to “map” and test out form, does the map form work? physical hyperlinks to audio, video and image files. Connections to braille, reading in different ways, audio books, how we experience making connections between things, our own, personal experiences, choose your own adventure.

Small sample of possible tracing paper topographic map.

PP2: Audio storytelling/Wall Fall

I chose to focus only on the audio for this pressure project. In hindsight, after viewing/listening to the other projects, I wonder if having an image of the Berlin wall surrounding us in the motion lab on the white scrims would have been helpful to site the work so there wasn’t a period of guessing for listeners.

My initial goal was to immerse the listener in the experience of the sound of construction and destruction of the Berlin wall. Having grown up in Germany during this point in history, I have a personal experience with this site and my traveling to and from the site by train was a part of it. That part of the story was lost in my layering process. I think that if I were to do this project again, I would pull back and focus in more on some of the powerful moments. Letting them sit with the listener more and not overwhelm them with so much information that is hard to unpack, especially if one is trying to figure out what the story is about.

I appreciated the feedback and want to think more about how we can tell personal stories that also convey meaning for others, for them to find their own way into the story and a connection to it as well.

In my other work outside of this class, I am struggling to translate from text into audio in a non-linear fashion and provide space for a participant in the work to find both their own entry point and way of navigating the space. Also as a way of connecting with the researcher. Thinking about presenting research as a relational experience. How do I challenge the linear and impersonal format of a dissertation, book, or academic text in a way that provides a valuable alternative experience of the story? I have been using a map as one form to explore, but that is also limiting and has boundaries and borders. I am also thinking about the relationship between printed, read text and verbal, spoken word. Preparing this pressure project helped me think about presenting my own process/experience in a way that is not too overwhelming and has some clarity for those participating in engaging with it. This is an ongoing struggle in my process.


PP1: Breathing the text – Tamryn McDermott

Video of final stage with four scenes. Unfortunately, the sound is missing, had issues recording output stage with sound

For this project, I decided to use some of the digital assets I created (video and audio) for a project I did in the motion lab last spring. I did not know Isadora at the time and so this provided me the opportunity to set up in Isadora from scratch and play with the assets in a new way without relying on someone else to set it up for me.

Initial structure and approach: four scenes, use Text Draw actor, play with integrating video, text, and audio, use pacing in an attempt to affect and engage viewer

Scene 1: Introduce breathing book structure and layer in shape actors with random movement and scale

Scene One. Solarize book video and layers, Include randomized shape actors that change position and scale

In scene one, I had challenges with getting the different shape actors to actually show color variations in the final output even though I had them set up within different user actors. The color remained white no matter what in the final stage display. The layering and actors worked well here although I would have liked to include more randomness in the scene and possibly integrate some of new randomness techniques based on video sensors into the scene.

Scene Two. Two layered videos with randomized video selection of 2 – 15 videos. Four audio tracts starting at different times.

In this scene I used the random actor to select between 14 different videos that would be layered in pairs and change positions, however the random actor favored the number 15 for the video selection field and so most of the time video 15 layered over itself from varying starting positions. Again, with more time, I would have introduced another method of selecting random numbers between 2 – 15. I think it would have been cool to have the audio files trigger this in some way, but I would have to think about this more to figure out the best approach.

In the screenshot below the text draw actors are not working. I had to create workarounds since none of these actors would show up when I exited the scene and returned. There was a bug that prevented it from working on both my laptop and the ACCAD lab computers. Only the video would display on the stage. As a workaround, I generated a .png of the text “stop” and then used that as a shape actor and animated it to correspond with the repeated “stop” audio file. Below are the screenshots from the first and second patches, the second one being the working one.

Scene Three. Non-working text draw actors . . .

Below is the working scene with the substituted image of the word “stop” I wanted to animate.

Finally, in the last scene I wanted to end on a calming note after the stop scene so here is the final scene with again, the text inserted as an image that I animated since the text draw actor did not work for me.

Scene Four. Used picture player with image of text instead of text draw actor.

I learned a lot from this project and hope to be able to get a response from the Isadora team on the text draw actor bug I ran into. I also want to experiment with sensors and introduce more randomness into my work.


BUMP: Willsplosion PP1 by Emily Craver

I wanted to find a pressure project 1 to bump and take a closer look at it to inspire and give me some ideas about how to approach mine as I start contemplating ideas and directions. I was drawn to Emily’s project because she integrated both sound and an element of humor using text into the final experience. I want to use one of the text actor features in mine too. I also found some actors in her patch that I want to experiment with such as the “Explode” actor. In addition, I appreciated her organization within each scene and want to be mindful in my own work to develop an organizational system that works and keeps things clear so I don’t get lost.

Emily’s post left me wanting to see/hear the outcome. This means that when I post mine, I want to share a video of the final outcome much like Tara’s that Katie bumped. What is the best way to set up to do this? How can we export and save the content from a projector?

Willsplosion: PP 1 | Devising EMS (ohio-state.edu)