Cycle 2: It Takes 2 Magic Mirror
Posted: April 12, 2025 Filed under: Uncategorized | Tags: Cycle 2, Interactive Media, Isadora Leave a comment »My Cycle 2 project was a continuation of my first cycle, which I will finish in Cycle 3. In Cycle 1, I built the base mechanisms for the project to function. My focus in this cycle was to start turning the project into a fuller experience by adding more details and presenting with larger projected images instead of my computer screen.
Overall, there was a great deal of joy. My peers mentioned feeling nostalgic in one of the scenes (pink pastel), like they were in an old Apple iTunes commercial. Noah pointed out that the recurrence of the water scene across multiple experiences (Pressure Project 2 and Cycle 1) has an impact, creating a sense of evolution. Essentially, the water background stays the same but the experience changes each time, starting with just a little guy journeying through space to interacting with your own enlarged shadows.
Alex asked “how do we recreate that with someone we only get one time to create one experience with?” How do we create a sense of evolution and familiarity when people only experience our work once? I think there is certainly something to coming into a new experience that involves something familiar. I think it helps people feel more comfortable and open to the experience, allowing them the freedom to start exploring and discovering. That familiarity could come from a shared experience or shared place, or even an emotion, possibly prompted by color or soundscape. Being as interested in creating experiences as I am, I have greatly enjoyed chewing on this question and its ramifications.
I got a lot of really great feedback on my project, and tons of great suggestions about how to better the experience. Alex mentioned he really enjoyed being told there was one detail left to discover and then finding it, so they suggested adding that into the project, such as through little riddles to prompt certain movements. There was also a suggestion to move the sensor farther back to encourage people to go deeper into the space, especially to encourage play with the projected shadows up close to the screen. The other major suggestion I got was to use different sounds of the same quality of sound (vibe) for each action. Alex said there is a degree of satisfaction in hearing a different sound because it holds the attention longer and better indicates that a new discovery has been made.
I plan to implement all of this feedback into my Cycle 3. Since I do not think I will be creating the inactive state I initially planned, I want a way to help encourage users to get the most out of their experience and discoveries. Riddles are my main idea but I am playing with the idea of a countdown; I am just unsure of how well that would read. Michael said it was possible to put the depth sensor on a tripod and move the computer away so it is just the sensor, which I will do as this will allow people to fully utilize the space and get up close and personal with the sensor itself. Lastly, I will play with different sounds I find or create, and add fade ins and outs to smooth the transition from no sound to sound and back.
As I mentioned earlier, the base for this project was already built. Thus, the challenges were in the details. The biggest hurdle was gate logic. I have struggled with understanding gates, so I sat down with the example from what we walked through in my Pressure Project 2 presentation, and wrote out how it worked. I copied the series of actors into a new Isadora file so I could play around with it on its own. I just followed the flow and wrote out each step, which helped me wrap my brain around it. Then I went through the steps and made sure I understood their purpose and why certain values were what they were. I figured out what I was confused about from previous attempts at gates and made notes so I wouldn’t forget and get confused again.
After the presentation day, Alex sent out a link (below) with more information about how gates work with computers, and a video with a physical example in an electric plug, which was neat to watch. I think these resources will be valuable as I continue to work with gates in my project.
Because I spent so much time playing with the gate and the staging, I did not get as far as I wanted with the other aspects. I still need to fix the transparency issue with the shadow and the background, and I realized that my videos and images for them are not all the right size. Aside from fixing the transparency issue, I will probably make my own backgrounds in Photoshop so I can fully ensure contrast between the shadow and background. The main mission for Cycle 3 will be adding in discoverable elements and a way to guide users towards them without giving away how.
The galaxy background is very clearly visible through the shadow, a problem I was not able to fix by simply changing the blend mode. I will likely have to do some research about why this happens and how to fix it.
Pressure Project 3: Soundscape of Band
Posted: April 9, 2025 Filed under: Uncategorized | Tags: Pressure Project 3 Leave a comment »For this pressure project, we were tasked with creating a two-minutes soundscape that represents part of our culture or heritage. I chose to center my project around my experience in band. My original plan had been to take recordings during rehearsals to create the soundscape. None of my recordings saved properly, so I had to get creative with where I sourced my sound.
Upon realizing I did not have any recordings from rehearsal, I had a moment of panic. Fortunately, some of our runs from rehearsal are recorded and uploaded to Carmen Canvas so we can practice the music for the upcoming Spring Game show. I also managed to scrounge up some old rehearsal recordings from high school and a recorded playing test for University Band. I had also done a similar project for my Soundscapes of Ohio class last year, and still had many of my sound samples from that (the first and likely last time my inability to delete old files comes in handy).
Then I had the idea to use videos from the Band Dance, meaning I had to scour my camera roll and Instagram so I could screen record posts and convert them to mp3 files. This process worked quite well, but it was no small task sorting through Instagram without getting distracted. It was also a challenge to not panic about having no recordings, but once I found this new solution, I had a lot of fun putting it together.
I spent about the first hour finding and converting video files into audio files and putting them into GarageBand. I made sure everything was labeled in a way that I would be able to immediately understand. I then made a list of all the audio files I had at my disposal.
The next 45 minutes were used to plan out the project, since I knew going straight to GarageBand would get frustrating quickly. I listened to each recording and wrote out what I heard, which was helpful when it came to putting pieces together. I also started to clip longer files into just the parts I needed. I made an outline so I knew which recordings I wanted to use. It was a struggle to limit it to just what would fit into two minutes, but it was helpful going in with a plan, even if I had to cut it down.
During the next 45-minute block, I started creating the soundscape. I already had most of the clips I needed, so this time was spent cutting down to the exact moment I needed everything to start and stop, and figure out how to layer sounds to create the desired effect. About halfway through this process, I listened to what I had and was close to the two-minute mark. I did not have everything I wanted, so I cut down the first clip and cut another. I did a quick revision on my outline now that I had a better understanding of my timing.
The last half hour was dedicated to automation! I added fade ins and outs in the places I felt were most necessary for transition moments. I wanted to prioritize the most necessary edits first to ensure I had a good, effective, and cohesive soundscape. I had plenty of time left, so I went through to do some fine tuning, particularly to emphasize parts of the sound clips that were quieter but needed to have more impact.
I wanted my soundscape to reflect my experience. As such, I started with a recording from high school of my trio rehearsing before our small ensemble competition. This moment was about creating beautiful music. It fades into the sound from my recorded playing test for University Band last year, where I was also focused on producing a good sound.
There is an abrupt transition from this into Buckeye Swag, which I chose to represent my first moment in the Athletic Band, a vastly different experience from any previous band experience I had. The rest of the piece features sounds from sports events, rehearsals, and the band dance, and it includes music we have made, inside jokes and bits, and our accomplishments.
I wanted to include audio from the announcers at the Stadium Series game because that was a big moment for me as a performer. I marched Script on Ice in front of more than 94,000 people in the ‘Shoe, performing for the same crowd as twenty one pilots (insane!). My intent with that portion was to make it feel overwhelming. I cut the announcement into clips with the quotes I wanted and layered them together to overlap slightly, with the crowd cheering in the background. I wanted it to feel convoluted and jumbled, like a fever dream almost.
In creating this project, I came to realize what being in band really means to me. It is about community, almost more so than making music. Alex pointed this out during the discussion based on how the piece progresses, and I was hoping it would read like that. At first, it was about learning to read and make music. When I got to high school, I realized that no matter where I went, if I was in band, I had my people. So over time, band became about having a community, and it became a big part of who I am, so I wanted to explore that through this project.
Aside from personal reflection, I wanted to share the band experience. My piece represents the duality of being in concert and pep band, both of which I love. There is a very clear change from concert music to hype music, and the change is loud and stark to imitate how I felt going to my fist rehearsal and first sporting events with the band. By the end, we are playing a high energy song while yelling “STICK!” at a hockey game, completely in our element.
Cycle 1: It Takes Two Magic Mirror
Posted: April 1, 2025 Filed under: Uncategorized | Tags: cycle 1, Interactove Media, Isadora, magic mirror Leave a comment »My project is a magic mirror of sorts that allows for interaction via an XBox One Kinect depth sensor. The project is called “It Takes Two”, because it takes two people to activate. In its single-user state, the background and user’s shadow are desaturated with a desaturation actor linked to the “Bodies” output of the OpenNI Tracker BETA actor. When the sensor only detects 1 body (via an Inside Range actor), it puts the saturation value at 0. When a second body is detected, it sets the saturation value at 100. I have utilized envelope generators to ensure a smooth fade in and fade out of saturation.
The above patch was added onto the shadow mechanism I created. I did some research on how to achieve this, and experimented with a few different actors before concluding that I needed an Alpha Mask. The LumaKey actor was one I played with briefly but it did not do what I needed. I found a tutorial by Mark Coniglio, which is how I ended up in alpha-land, and it worked beautifully. I still had to tinker with the specific settings within the OpenNI Tracker (and there is still more to be fine-tuned), but I had a functional shadow.
My goal with Cycle 1 was to establish the base for the rest of my project so I could continue building off it. I sectioned off my time to take full advantage of lab time to get the majority of my work done. I stuck to this schedule well and ended Cycle 1 in a good position, ready to take on Cycle 2. I gave myself a lot of time for troubleshooting and fine-tuning, which allowed me to work at a steady, low-stress pace.
I did not anticipate having so much difficulty finding colorscape videos that would provide texture and contrast without being overwhelming or distracting. I spent about 45 minutes of my time looking for videos and found a few. I also ended up reusing some video from Pressure Project 2 that worked nicely as a placeholder and resulted in some creative insight from a peer during presentations. I will have to continue searching for videos, and I am also considering creating colored backdrops and experimenting with background noise. I spent about 20 minutes of my time searching for a sound effect to play during the saturation of the media. I wanted a sound to draw the users’ attention to the changes that are happening.
Overall, the reactions from my peers were joyful. They were very curious to discover how my project worked (there was admittedly not much to discover at this point as I only have the base mechanisms done). They seemed excited to see the next iteration and had some helpful ideas for me. One idea was to lean into the ocean video I borrowed from PP2, which they recognized, causing them to expect a certain interaction to occur. I could have different background themes that have corresponding effects, such as a ripple effect on the ocean background. This would be a fun idea to play with for Cycle 2 or 3.
The other suggestions matched closely to my plans for the next cycles. I did not present on a projector because my project is so small at the moment, but they suggested a bigger display would better the experience (I agree). My goal is to devise a setup that would fit my project. In doing so, I need to keep in mind the robustness of my sensor. I needed a very plain background, as it liked to read parts of a busy background as a body, and occasionally refused to see a body. Currently, I think the white cyc in the MOLA would be my best bet because it is plain and flat.
The other major suggestion was to add more things to interact with. This is also part of my plan and I have a few ideas that I want to implement. These ‘easter eggs’, we’ll call them, will also be attached to a sound (likely the same magical shimmer). Part of the feedback I received is that the sound was a nice addition to the experience. Adding a sonic element helped extend the experience beyond my computer screen and immerse the user into the experience.
This is a screen recording I took, and it does a great job demonstrating some of the above issues. I included the outputs of the OpenNI Tracker actor specifically to show the body counter (the lowest number output). I am the only person in the sensor, but it is reading something behind me as a body, so I adjusted the sensor to get rid of that so I could demonstrate the desaturation. Because it saw the object behind me as a body, Isadora responded as such and saturated the image. The video also shows how the video resaturates breifly before desaturating when I step out and step back in, which is a result of the envelope generator. (The sound was not recording properly, please see above for sound sample.)
My score was my best friend during this project. I had it open any time I was working on this project. I ended up adding to it regularly throughout the process. It became a place where I collected my research via saved links and tracked my progress with screenshots of my Isadora stage. It helped me know where I was at with my progress so I knew what to work on next time I picked it up and how to pace myself across this cycle by itself and all three cycles together. I even used to to store ideas I had for this or a future cycle. I will continue to build on this document in future cycles, as it was incredibly helpful it keeping my work organized.
Pressure Project 2: Unveil the Mystery
Posted: March 3, 2025 Filed under: Uncategorized Leave a comment »For this pressure project, we had to unveil a mystery. Going into it, I had a general idea of what I wanted to create, but I faced many struggles during execution. I spent a good portion of my 7 hours doing research into different actors I would need and how to use them. I think I got so focused on the technical aspects that I pushed the storytelling aspect to the side and didn’t flush the story out as much as I wanted. Overall, though, my project was successful and I learned a lot.
I wanted to hide little mini mysteries throughout my project, so I built three scenes for the user to explore before getting to the end. Each scene has a base set of actors that I build on to customize the scene. I created a little avatar to move around the stage with a Mouse Watcher actor and added a sound effect (unique to each scene) that played when the avatar hit the edge of the stage. The avatar also completed a different action in each scene when the user hit a button on the Makey Makey interface I made.
This is the Makey-Makey interface I made for myself, which allowed me to use it single-handed. I planned to make another one that was more like a video game controller but ran out of time.
In the intro scene, a digital city, the avatar jumped when the key was hit, and after 7 jumps, it inflated to take up a large portion of the scene, then quickly deflated. In the ocean scene, it triggered fish to “swim” across the scene so it could swim with the fish. In the space scene, the avatar teleported (an explosion actor was triggered to scatter the avatar’s pixels and reform them). Every scene had another key plugged into a counter actor, and after that key was pushed so many times, it jumped to the next scene. This project was designed to make the users curious to push all the buttons to figure out the mystery. A great suggestion I received was to put a question mark on the action button, which my peers agreed would be more effective than having it unlabeled.
I received a lot of positive feedback on my project, both verbally and through reactions. There were several moments where they were surprised by something that happened (aka a feature they unlocked), namely when the avatar first exploded in the space scene. I was surprised by this because I didn’t think it would be a big moment, but everyone enjoyed it. They went into it with an unlock-the-mystery mentality and were searching for things (oftentimes things that weren’t there), so they were happy to find the little features I put in for them to find before reaching the final mystery, which was that they won a vacation sweepstakes. They said the scenes felt immersive and alive because I used video for the backgrounds. Again, the space scene was a big hit because the stars were moving towards the user and was more noticeable than the more subtle movement of the ocean scene. There were multiple moments of laughter and surprise during the presentation so I am very pleased with my project.
The main critique was that the mystery was a bit confusing without context, and I do agree with that. One suggestion was to add a little something pointing to the idea of a context or raffle to add context for the final mystery, and another was to progress from the ocean to land to beach to get to the vacation resort or something along those lines. My original idea was to have portals between scenes and end with a peaceful beach vacation after traveling so far, but I ran out of time and didn’t dedicate enough time to telling the story that I ended up throwing something in at the end. They did say the final scene provided closure because the avatar had a smile (unique to this scene) and was jumping up and down in celebration.
I hit a creative rut during this project so the majority of it was improvised, contributing to the storytelling problem. I started by just making the avatar in the virtual stage because making a lil guy seemed manageable, and it sparked some ideas. I decided on a video game-esque project where the avatar would move around the scene. As I brainstormed ideas and started researching, I had a list of ideas for what could happen as the user navigated through my little world. I spent about 20-25 minutes on each idea on the list to figure out what worked. This involved research and (attempted) application. Some things didn’t work and I moved on to try other things.
I broke this project up into several short sessions to make it more manageable, especially with my creative struggles. This gave me time to sit and process/reflect what I had already done, attempted, and wanted to do. I was able to figure out a few mechanisms I had struggled with and go back later to make them work, which helped me move the project along. One time-based challenge I did not consider was how much time it would take to find images, videos, and sound effects. The idea to add these sonic and visual elements occurred much later in the process, so I did not have a lot of time left to adequately reallocate my time, so I do think I would have been better off creating a storyboard before diving right in to better prepare myself for the project as a whole.
Pressure Project 1 – Interactive Exploration
Posted: February 4, 2025 Filed under: Uncategorized | Tags: Interactive Media, Isadora, Pressure Project Leave a comment »For this project, I wanted to prioritize joy through exploration. I wanted to create an experience that allowed people to try different movements and actions to see if they could “unlock” my project, so to say. To do this, I built motion and sound sensors into my project that would trigger the shapes to do certain actions.
Starting this project was difficult because I didn’t know what direction I wanted to take it, but I knew I wanted it to have some level of interactivity. I started off small by adding a User Input actor to adjust the number of facets on each shape, then a Random actor (with a Limit-Scale Value actor) to simply change the size of the shapes each time they appeared on screen. Now it was on.
I started building my motion sensor, which involved a pretty heavy learning curve because I could not open the file from class that would have told me which actors to put where. I did a lot of trial-and-error and some research to jog my memory and eventually got the pieces I needed, and we were off to the races!
A mockup diagram of which section of the motion sensor is attached to each shape. The webcam is mirrored so it is actually backwards, which made it difficult to keep of which portion of the sensor attached to which shape.
Figuring out the motion sensor from scratch was just the tip of the iceberg; I still needed to figure out how to implement it. I decided to divide the picture into six sections, so each section triggered the corresponding shape to rotate. Figuring out how to make the rotation last the right amount of time was tricky, because the shapes were only on-screen for a short, inconsistent amount of time and I wanted the shapes to have time to stop rotating before fading. I plugged different numbers into different outputs of a Wave Generator and Limit-Scale Value actor to get this right.
Then it was time to repeat this process five more times. Because each shape needed a different section for the motion detector, I had to crop each one individually (making my project file large and woefully inefficient). I learned the hard way how each box interacts and that not everything can be copied to each box as I had previously thought, causing me to have to go back a few times to fix/customize each shape. (I certainly understand the importance of planning out projects now!)
I had some time left after the motion sensor was done and functional, so I revisited an idea from earlier. I had originally wanted the motion sensor to trigger the shapes to explode, but realized that would likely be overwhelming, and my brain was melting trying to get the Explode actor plugged in right to make it work. Thus, I decided on an audio sensor instead. Finding the sweet spot to set the value at to trigger the explosion was difficult, as clapping and talking loudly were very close in value, so it is not a terribly robust sensor, but it worked well enough, and I was able to figure out where the Explode actor went.
I spent a lot of time punching in random values and plugging actors into different inputs to figure out what they did and how they worked in relation to each other. Exploration was not just my desired end result; it was a part of the creative process. For some functions, I could look up how to make them work, such as which actors to use and where to plug them in. But other times, I just had to find the magic values to achieve my desired result.
This meant utilizing virtual stages as a way to previsualize what I was trying to do, separate from my project to make sure it worked right. I also put together smaller pieces to the side (projected to a virtual stage), so I could get that component working before plugging it into the rest of the project. Working in smaller chunks like this helped me keep my brain clear and my project unjumbled.
I worked in small chunks and took quick breaks after completing a piece of the puzzle, establishing a modified Pomodoro Technique workflow. I would work for 10-20 minutes, then take a few minutes to check notifications on my phone or refill my water bottle, because I knew trying to get it done in one sitting would be exhausting and block my creative flow. Not holding myself to a strict regimen to complete the project allowed me the freedom to have fun with it and prioritize discovery over completion, as there was no specific end goal. I think this creative freedom and flexibility gave me the chance to learn about design and creating media in a way I could not have with a set end result to achieve because it gave me options to do different things.
If something wasn’t working for me, I had the option to choose a new direction (rotating the shape with the motion sensor instead of exploding them). After spending a few hours with Isadora, I gained confidence in my knowledge base and skill set that allowed me to return to abandoned reconsidered ideas and try them again in a new way (triggering explosions with a sound sensor).
I wasn’t completely without an end goal. I wanted to create a fun interactive media system that allowed for the discovery of joy through exploration. I wanted my audience to feel the same way playing with my project as I did making it. It was incredibly fulfilling watching a group of adults giggle and gasp as they figured out how to trigger the shapes in different ways, and I was fascinated watching the ways in which they went about it. They had to move their bodies in different ways to trigger the motion sensors and make different sounds to figure out which one triggered the explosions.
Link to YouTube video: https://youtu.be/EjI6DlFUof0