Independent Study with Makey Makey
Posted: May 6, 2026 Filed under: Uncategorized Leave a comment »Building on my skills and experiences from the DEMS class I took last semester, I wanted to use some independent study time to continue experimenting with Makey Makey to keep developing ideas for my MFA thesis project in dance. At the end of last semester, with the tools I had been experimenting with in DEMS such as Makey Makey, OSC, Live Video, NDI camera, Isadora, I combined my interests in movement and technology in my 2nd year showing performance and presented my work to my dance colleagues and professors. In that showing, I had used Makey Makey as a bridge between touch and memory/answer/response. After receiving feedback and progressing my ideas, I steered toward another trajectory in my approach where I started experimenting with absurdity and humor to reach the depth. This was also a progression I was able to mentally achieve this semester after the experience I devised with honey in DEMS last semester. Letting myself include objects in my artistic works with curated associations allowed me to keep asking and answering the question “What is the experience?” with each trial.
This semester, I also took a composition course called Rebel Innovations with Abby Zbikowski, where I had the chance to work on a new solo piece. In that piece titled “Am I too much pink?” I used several objects. Some of them are pink sunglasses that make you see the world in pink when you put them on, another pair of sunglasses with pink flowers all around the frame, tiny sunglasses that is too small for a human head, an ornament shaped like a unicorn, and a pink spatula shaped like a guitar.
Using objects, humor, and silliness, the piece implicitly explores themes of queerness, childlike disinhibition, neurodivergence, and ways of seeing/perceiving, through my and the audience’s interaction with the objects.
For my Independent Study, I decided to pursue my intended scores within this new context. My work for that class didn’t have the deep media systems component engrained other than using my phone to play music/sound and someone else’s phone to use the flashlight as a “spotlight”. But I remained interested in incorporating media systems because I am still interested in composing my final MFA thesis project with technology incorporation. So I worked with Isadora and Makey Makey to discover what the possibilities were. My main question was “How can Makey Makey and Isadora help me with building a bridge between object and meaning through a technology interface?” I remained interested in pursuing touch as a trigger for a response. My objects were mostly plastic and weren’t gonna conduct electricity the way I would need them to. So once again, I used the help of the conductive pencil to complete the Makey Makey connection. I programmed an Isadora patch with 9 scenes.
Using the keyboard watcher, text draw, trigger delay, jump, wave generator that’s set to random, colorizer, spinner, dots, and enter scene trigger, I designed/devised an experience where seeing the image of an object would prompt the interactor to touch the object, the object being touched would trigger a text or video to show up and those objects, texts and videos would accumulate in more questions for the interactors for them to reflect on internally.
Even though my initial Independent Study score also included exploring the relationship between camera, movement, and OSC, in this new context, I built a new relationship between movement and technology through pre-recorded videos and specific timestamps in the videos where that specific action would occur, asking the interactor to trigger the next scene through touch via Makey Makey when they encounter a specific movement.
As I have one more year left to develop my final piece, I am still in the building, questioning, asking, and responding phase. My explorations are finding micro shifts and deepening through experiments. With this Independent Study, I managed to accomplish my own scores and create a more solid foundation for my future creations involving objects, movement, Isadora, and Makey Makey.
Cycle Two: A 3D Movement-Based Sound Explorer
Posted: May 5, 2026 Filed under: Luke Buzard | Tags: Cycle 2, FluCoMa, MaxMSP, MediaPipe, touchdesigner Leave a comment »Going into cycle two, I immediately knew that I wanted to take the ideas I had done in 2 axes of movement in cycle one and translate it to 3D. In order to do this, I needed to rework the analysis portion to plot all the samples in 3 dimensions, as well as figure out a new way for the user to see their movement within the corpus of sounds. For several days, I tried to do the analysis section on my own, but found myself unable to figure out how to plot them in a 3D space as I’m not great with Jitter, Max’s visual package. However, just about when I was getting ready to give up, I found someone who had already done exactly what I was trying to do.
I won’t get too in-depth on how this works. But in short, it reduces the MFCC analysis data to 3 dimensions, as opposed to 2 in cycle 1, unpacks all that data into a matrix with 3 dimensions, and then that matrix data is used to generate an OpenGL environment with all the samples plotted in a 3D space. Then, depending on whatever point the camera is closest to in the Jitter window, the sample that correlates with that point is played back. This camera, however, was driven with WASDQZ, so I had to implement a new system of control that allowed this 3D corpus explorer to interface with the Google Mediapipe data I was receiving from TouchDesigner. I did this by scaling the data to the 3D space, smoothing it out some, and sending it a as list into the position attribute of the jit.gl.camera object.
Zoom-out of the 3D corpus space, each dot represents a sample; dimmer dots are deeper on the Z-axis
Results
Unfortunately, I don’t think my vision for this cycle translated to reality as well as I’d hoped it would. One of my biggest issues was that I left on an option that locked the camera to (0,0,0) in the space. This caused a major decorrelation between the movements of anyone using it and the movement in the space. Everyone seemed to struggle with “navigating” the corpus space much more than in the 2D iteration. Once I got rid of this attribute during our discussion afterwards, people understood what was happening better right away. I think the camera was also just a little glitchy since MediaPipe isn’t the *best* at detecting depth, which caused it to jump around wildly. I think if I were to continue with this 3D idea, I would have to incorporate a depth sensor in some way so that all 3 dimensions are accurate.
In my first cycle, I also included the MediaPipe landmark camera feed in the projector image as a sort of monitor for movement. In this one, I made the decision not to include it, as I’d hoped the movement in a 3D space would translate more intuitively to the user. However, I was really surprised that most everyone actually missed its presence. I think the playfulness of the little dot stickman moving around on screen must have provided a nice counter to the cacophony of sound. For this reason, I made a point to include the landmark skeleton in the final version.
Here’s a short recording of Alex using the system. As you can see, it was not very precise. Additionally, something in the analysis created a lot of sample points with that shrill high pitch sound that frequently plays. I don’t know where this came from, but it taught me to always check the samples when I’m working with tools like FluCoMa.
While this cycle was a bit of a fail in meeting the goals I had, I think it actually gave me a chance to step back afterwards and examine what the goals I wanted to achieve with these cycles were. Out of all of them, the largest was to create a group experience. This setback caused me to really narrow in on that idea for the 3rd cycle, which I think worked out much better in the long run.
Cycle 3 – Solo 2: Electric Boogaloo
Posted: May 3, 2026 Filed under: Uncategorized Leave a comment »Hello! Welcome to the official successor of Solo: Paper Plate DJ.
I won’t go into much detail on the pre-production and programming for this iteration, as not much has changed on that end; refer to the first post for that. The most important update to the project is in the controller; we are no longer using a paper plate (woohoo). We used a bucket (2 of them to be exact).
There were 9 inputs wired into the two buckets: 3 based on taps and 5 on holds. Originally, the player was to wear my watch with the conductive alligator clip attached, which worked pretty well for me, but stressfully, during the presentation, it was not working for Zarmeen. Michael came in with a save by lending me a bracelet designed to do exactly what my jimmy-rigged watch was meant to do (be a bracelet that makes the user conductive). The user finger-tapped their way through the visuals, each labeled with a lyric from the song (what the visuals were connected to).
While this controller was a big upgrade over a plate and closer to my vision of evoking a sense of musicality and rhythm in the user, it had its flaws:
Flaw #1. Nobody could remember the lyrical labels, nor did they look at them during their performance.
SO yeah, turns out it’s hard to read and hit buttons that make visuals appear at the same time. I was not shocked at this feedback, as I found myself not reading them as I played, but I also know the song, so I wanted to test it anyway. I’ve decided to incorporate the lyrics in another way that will hopefully be more effective (check my final post for that reveal).
Flaw #2. The buckets are janky
The buckets are janky. They are a bit ugly and easily disconnect from their inputs (the Makey Makey)… I will try my best to make this less the case in the next cycle.
Flaw #3. The meaning of the song did not matter to the users/was not clear.
Because no one remembered the lyrics and felt they were mostly preoccupied with making the coolest visual combos possible. I hope that by incorporating the lyrics into the visuals, the user will take heed of them.
Okay, time to be more positive.
My classmates reported feeling immersed in the process and as if they were performing or learning an instrument; some even expressed nervousness when in the “spotlight”. This accomplished one of my goals: to simulate the feeling of, well, performing a solo. I really enjoyed watching the unique interactions as people took turns. My classmates were oohing and aahing, and trying out different techniques to garner different visual combinations. I am lucky there are so many actual musicians in the bunch. Now I just need to accomplish some sort of meaning-making process in this experience.
Pressure Project #3 – Tomato Tales
Posted: April 16, 2026 Filed under: Uncategorized Leave a comment »From my understanding this pressure project was to create an audio-based experience tied to a cultural subject close to you. We only had 5 hours total to work on it. Maybe due to the amount of work, I had ahead of me at the time, in this course and others I decided to go with the first idea that came to me. I thought of tomatoes.
In a design course taken in the prior semester, we did an on visual mapping exercise. We had to make a map about tomatoes within the class period, anything about tomatoes. Most people chose cooking processes or the life cycle of a tomato as an agricultural good. I tried to find a connection between tomato sauces, as I thought of mole (the sauce) and how every culture around the world seems to have their own version of tomato sauce…so where did tomatoes come from and how did they get to Italy?
It was colonialization, because of course it was. But Tomatoes originate from South America in the Inca Empire and flourished in the Aztec Empire.
It’s not the prettiest chart but I spent most of the exercise learning about tomatoes.
So, I made an audio experience using tomatoes as the means of interaction. While we weren’t allowed to show any visuals through the first playthrough I planned to reveal the control panel for the sounds with a second playthrough (which was allowed). I prompted the audience to try to piece together a narrative taking place, before and after the second playthrough.
As for the audio I built a library of 15th century Inca instruments, Aztec instruments and 16th century Spanish instruments. I began the recording with a sound of seeds rattling in a bag, followed by Inca instruments, gradually joined by Aztec performances showcasing the migration and meeting of two cultures. Aztec celebratory performances take the lead for a bit, to indicate the prosperity of the empire and cultivation of the crop. What follows is a change of lighting and sound. Waves and grumbling can be heard, followed with heavy footfall. A Spanish guitar plays a riff and is met with an Aztec Death Flute (scary ass sound) but is quickly cut off by the boom of a musket. This was to show way weapons (guns) from Spanish and other Western colonial forces were able to lethally end even the mightiest of native empires. To conclude waves and ship sailing sounds ensue again; a ren faire sounding band performance plays (apparently the sound of Italy at the time) along with chopping noises (the tomato is being used).
Shout out to Luke to basically nailing the message on the head when I prompted everyone to guess.
The recording process was very stupid. I used the Makey Makey and their soundboard app, however it doesn’t have a feature that lets you record in browser. So, I set up a microphone beside my speaker, and I am so shocked it didn’t sound horrible. I realized I more made a tactile activity for myself and gave far more cerebral activity for my audience (thinking game).
I chose this topic because colonialization is the primary driving force that has led to the life I live today. Colonialization is what makes me an American and not some random person in Ireland or Germany right now (where my ancestors are from). Colonialization is what provides us with iPhones, clothes…bananas most stuff really. My partner is here because her ancestors were stolen from their country and brought here as slaves. Behind almost every invention, creature comfort, and privilage that makes up the West is covered in a history of bloodshed… Including spaghetti if you follow the threads enough. This isn’t something to make people feel bad or anything. Knowing this type of thing is impotrant to me, how much violence makes up the creature comforts we have today. Who am I harming or partcipating in the harm of by consuming in the empire I live in now? How do I react to learning this things?
The good news is, I don’t really like tomatoes anyway.
Mole is pretty good though.
Cycle One: The Movement-Based Sound Explorer
Posted: April 13, 2026 Filed under: Uncategorized | Tags: cycle 1, FluCoMa, MaxMSP, MediaPipe, touchdesigner Leave a comment »As I began thinking about what I wanted to make my cycles about, I found myself gravitating towards a question I had previously been interested in when beginning to work on my senior project at the beginning of the year: How might technology allow for the creation of new modes of musical interface, where the relationship between audience and performer is almost entirely dissolved?
My primary resource was Max/MSP, as I know it best of all the computer music softwares, and I find it very useful for the development of new ways of making music.
Going into this cycle I knew one of the central resources that I could use to help me answer this question was Google MediaPipe–a real time motion capture software that uses webcam input as opposed to dedicated hardware/software that requires mo-cap suits. This allows for systems which anyone can easily interact with, even without knowing how the system works or what each mo-cap landmark is controlling. I handled this part of my patch in TouchDesigner, as the Max integration of MediaPipe has some difficulties I don’t have time to get into here.
My main goals for this cycle were to create an interface to interact with sound that was fun, interesting, but also left room for potential emergent behavior when left in the hands of different users.
The second major piece of software I used was the Fluid Corpus Manipulation (FluCoMa) toolkit for Max/MSP (also available in SuperCollider and Pure Data). This toolkit uses machine learning software to analyze, decompose, manipulate, and playback a large collection (or corpus) of samples. I initially chose this piece of software as one of its modes of playback is a 2D plotter which can map two different aspects of the sample analysis on to an X and Y axis. I thought this would be a perfect interface for MediaPipe control as the base 2D plotter uses mouse input, which I found to be detrimental to using it as an “instrument.”
I had initially wanted to expand the idea of the 2D plotter to a 3D one, as I felt being able to interact with the patch in a 3D space would be much more natural. However, I found expanding the logic to work in 3 dimensions was a much more difficult task than I’d thought, so I decided to stick with the 2D plotter for this cycle.
Results
I thought that I was mostly very successful with the goals I set out to accomplish. Everyone wanted to try out the patch, which I thought was a testament to the “fun” and “interest” aspects of it. The controls were also quickly picked up on, which was a goal of mine, as I’m interested in systems that audiences can interact with regardless if they’re conscious of the mechanics of that interaction or not. I was most interested to see how different people had their own unique ways of interacting with it as well. Chad, for example, was really trying to make something rhythmic and intelligible out of it, while others were going all over the place, or looking for specific sounds.
Some missed opportunities that I want to expand on in future cycles is the use of the Z dimension in controlling the playback of samples, as well as the use of multiple limbs to control playback. As you can see in the video, users were somewhat restricted in how they could control the patch by the Z direction not doing anything, as well as the fact that only the right hand could trigger sounds. By expanding this idea to 3D, instead of two, and allowing for the use of multiple limbs, I think it’ll give people more freedom in how they interact with the corpus of sounds.
This was the first real project I’ve done with FluCoMa, and thus I learned a ton about its mechanisms, particularly the storage of non-audio data in buffers. This is a concept used a lot more in environments like SuperCollider or Pure Data, as Max has some other objects for storing that kind of information. However because of the way the machine learning tools in FluCoMa work, it needs to store all of the information it may need in RAM. This was also the first project I’ve done sending OSC data between different apps on my computer, which had a bit of a learning curve as I discovered OSC data sends as strings, instead of floating point numbers. This didn’t create any real difficulty, as the conversion took no time, but it did make me aware of an important aspect of using OSC (particularly with Max, as certain objects process strings/floats/integers differently).
Cycle 1 -Solo (but at this moment) Paper Plate DJ
Posted: April 9, 2026 Filed under: Uncategorized Leave a comment »What is up?
Here is my cycle one post 🙂
I started this cycle with the hopes and dreams of creating a live performance experience that challenges the user to piece together a story from a song with lyrics they can’t understand. This branches from two research interests/questions.
1. How can interactive technology facilitate meaning-making and engagement with works of art?
2. How can designers best utilize interactive/immersive experiences to invoke a sense of power within their participants?
I tapped into my own personal lived experiences to explore these questions; I feel the most powerful when moving to music and playing rhythm games. So I set out to recreate that feeling of being pretty good at a rhythm game. Demonstrated in the brainstormed documents below 🙂
I chose a song that is entirely in Japanese, with the knowledge that no one in my class knows Japanese. I sectioned off a translation of the lyrics to attach to 4 different inputs that align with the imagery described. I also used the music video as a reference. I also planned to buy buckets. The user drums on various parts of the bucket that are labeled with lines of lyrics from the song. They will keep the tempo in the area of the drum where they believe the lyrics are being sung.
I decided to create the control scheme first – focusing entirely on that before setting up the physical controller.
This test was accompanied by instrumental music. I didn’t want to reveal the song yet, and wanted to focus on the physical reactions with the controls to influence how I construct the controller in cycle 2. Most of the songs were slower-paced, but once a faster one came on, Chad (shown in the video above, but the specific moment wasn’t captured on film) stood up and rapidly switched between visuals, which contrasted with the slow, exploratory manner he had before when testing out the various interactions. The song I chose is a bit funkier and faster-paced than the music played before, so I think it will add some excitement to the interactions.
I received feedback that having the grounding element be the user holding their thumb to the center of the plate felt more natural than the watch, and avoiding tangled wires. I also received feedback on the controls being confined to the desk. I agreed as I plan to have the control be in the center of the room, but I didn’t have that prepared for this cycle… So on to the next…
Cycle 1: The (bad) Friend
Posted: April 7, 2026 Filed under: Uncategorized | Tags: cycle 1, touchdesigner Leave a comment »The Score
My idea for this cycle was simple (or atleast it seemed so in my head): make an AI-powered interactive experience where the user shares a space with an AI ‘presence’. It lives on a screen, but its there for you and it listens to whatever you have to say – or dont have to say. The score: a participant enters a space, speaks naturally, and the environment responds to the quality of what they shared through a particle system. No text output, no voice back. Just the space changing around them. The framing I gave participants was: “this is a friend you can talk to.” That framing is what became the main problem.
Resources
- TouchDesigner for the visual/particle system
- Python + PyAudio for microphone input
- OpenAI Whisper for speech-to-text transcription
- Claude API to interpret the speech and return atmospheric parameters (brightness, movement, weight, density) as JSON
- OSC to pipe values from Python into TouchDesigner
- Orbecc depth camera for body tracking (ceiling-mounted, blob detection)
- Motion Lab
- A Michael for troubleshooting (1)
Process and Pivots
I wrote a python script that takes user input through the microphone, then uses OpenAI Whisper for speech-to-text transcription. It then sends the speech to claude in order to parse it according to the system prompt I gave it, which were metrics like emotional register, weight, intensity etc. The python script in turn sends these metrics to touchdesigner through OSC. Inside touchdesigner, I made table DATs that were storing the values of the incoming signals in order to apply those values to the visual system (a particle system). The values were suppsoed to effect the movement and color of the particle system.
I initially built my system on MediaPipe for body tracking, but then when I shifted the system to the motionlab, I had revelations. The system worked fine for a laptop but for it to work in an open space and a big projection screen, it would need a camera directly in front of the participant’s face (and the screen) to work, which sounds horrible for an immersive experience. So, I switched to blob detection through the Orbecc ceiling camera. That took a while to get right. It wouldn’t even detect me and I couldn’t figure out why so I made the very obvious assumption that it hates me lol. Turns out it needs something to reflect off of and I was wearing all black.
The original prompt to Claude was trying to do an emotional analysis, as in read how the person was feeling and respond to that. At some point I rewrote it to just read the texture and quality of what was shared, not the emotional content. That was actually the most important design decision I made: the difference between “I understand you” and “I am here.” The particle system was jerking between states and it felt mechanical, so I also had to apply some smoothing for it to not act crazy.
What Worked, What Didn’t, What I Learned
What didn’t work: ALOT. I think apart from the framing of the system, I had not realized the amount of time I needed to properly do this. I had only gotten a limited amount of time in the MOLA so I was only able to troubleshoot the projection and not run through the whole pipeline. I did not anticipate alot of things as they went wrong the biggest example of this would be the lag. There’s bad bad latency in the pipeline (mic → Whisper → Claude → OSC → TouchDesigner) and it was long enough that participants got confused. They’d speak, nothing would happen, they’d speak again, then two responses would arrive at once. A few people got genuinely frustrated. The “friend you can talk to” framing made this much worse because it set up an expectation of conversational timing that the system couldn’t meet. Lou said it was a bad bad friend. Like one of those people who keep looking at their phone when you’re trying to talk to them.
What worked unexpectedly: The observers. People watching someone else use the system felt something – specifically, they felt empathy for the participant who was being poorly served by the AI. That observation became the most interesting research finding of the whole cycle.
What I learned: Time is the biggest resource, and you have to plan according to it. Instead of trying to force all of your bajillion ideas into the time that you have. Also, framing matters more than you think it does! Had the same system been framed a different way, I would’ve gotten away with it, but since I had framed it a specific way, there were specific expectations.
Pressure Project 2: One for All, All for one
Posted: March 9, 2026 Filed under: Pressure Project 2 | Tags: Pressure Project, Pressure Project 2, touchdesigner Leave a comment »I named my cell One for All, All for One because it is built around the idea that an individual, and communities as a whole are constantly shaping each other. The cell itself is a constan conversation between the oneself and the communal archive. It takes a live video feed and layers it over a slideshow of images showing communities and people from different parts of the world.
Then interactive sound enters the picture. A glitch effect driven by audio input levels determines how much the live video overlay fractures. The louder the audio, the more the live layer breaks apart and reveals the slideshow underneath. Alongside this, I built in an internal LFO paired with an Edge TOP to create a rhythmic pulse, something I called a “heartbeat”. Even without external input, the system works fine and feels alive.
The structure is modular and layered, and honestly not that complicated. There is a live video input, and a media player (which controls the slideshow) which plug into a switch. The output of the switch goes into a glitch system, and a pulse system. Each could be replaced without breaking the overall logic. The audio input, live video input, and the signal (LFO) are designed so that they could be overridden by an external network signal as well. The cell has its own system, but it is designed to connect, following the true concept of one for all, all for one.
Reflection:
When all the cells assembled, things became unstable. Signals were constantly dropping and connections were dying. For a while, I thought something was wrong with my cell because nothing would show up (it was a problem with the input signals I was getting). When I finally got it to work, very interesting emergent behaviors appeared. The glitches danced to different rhythms. Video overlays ended up in very interesting stacked outputs. It was interesting because I did design my system while being aware that it had to be plugged into a bigger system. However, I did not envision the results I got during testing. What I controlled alone became either amplified or distorted by others. The network did not just combine outputs. It reshaped them. I think where my careful planning fell off was the heartbeat. I had not accounted for the fact that other cells can have signals of different types. Instead of a steady pulse, I got an irregular signal input, which changed the whole heartbeat effect. At first it felt like something went wrong. My cell was no longer just reacting to my inputs. It was reacting to everyone. That is exactly what One for All, All for One means. Each cell affects the others. Each signal influences the collective behavior. My cell had a life of its own. In the network, it learned to respond, adapt, and sometimes surrender to the collective.
Project File: pp2_Zarmeen.zip
Pressure Project 2 – The Flipper
Posted: March 3, 2026 Filed under: Uncategorized Leave a comment »Description
The Flipper is a TouchDesigner patch that uses an audio input to create video, and uses a video input to create audio. When used in a network this “cell-block” acts independently by creating entirely new audio and video, instead of just modifying the audio and video it receives. Its modularity lies in its ability to provide other users in the cell-block network with new sources of audio and video, that are themselves generated from other audio and video over the network.
Collective Documentation
Pending
Individual Documentation
Overview of my cell-block’s network. This is connected to three inputs and outputs on the outside of the container, which connect to other cell-blocks on the network. While there’s a lot on screen, it breaks down into a few simple sections.
This portion of the network takes in audio from over the patch through the in_audio CHOP. The envelope, math, and audioparaeq objects slow the stream of data and boost high frequencies, respectively. This then is turned into a spectrogram and is sent directly to a chopto TOP.
This portion processes that audio spectrum into a new visual. Starting in the bottom left, I use a series of TOPs to create a flow-like visual, which is then composited with the spectrum. This new visual is colored using a series of ramps and a look up TOP. The ramps are cycled through using either an LFO or an input from in_osc over the network. An example of the visuals this produces is below.
Lastly, this portion of the patch processes video received over the network from the in_video TOP (or in this case, a camera input) into audio. While I didn’t get quite as interesting an audio output as I wanted, I still think I was effective in transforming video to audio. The video that is received gets sent directly to a topto CHOP, which reads RGB values over the X and Y planes of the video. the following objects then reduce the amount of data, and turns those waves into a stereo audio signal by the merge CHOP. This wave is given an envelope by the math objects (I attempted to control this with another osc input but failed to make it work) and is sent out over the network. An example of audio is included below.
Reflection
Since I knew I wanted to flip the audio and video signals inside my patch, the independence of the cell-block was semi-inherent the entire time I was working on it. In order to ensure it was connectable with others, however, I needed to ensure that whatever the patch did was interesting enough, while still clearly using audio and video to influence the opposite output, so that it didn’t just seem like I was generating something entirely new.
I made choices about what to include and exclude primarily by trying to figure out what I could accomplish that was reasonably within my ability, but still interesting. For example, I’ve worked with spectrogram imagery in the past, so I knew I would be able to incorporate that the easiest. On the opposite end of that, I attempted to integrate FM synthesis into the audio part of my patch to get some more interesting sounds. However, with my inexperience in TouchDesigner, I found it really difficult to make FM work, so I chose to exclude it.
One thing that surprised me was how even if cell-blocks didn’t work “perfectly” together, they still were able to have some sort of interaction, even having unexplainable interactions. I was also a bit surprised how underutilized the OSC data we were sending was. I know I was personally having difficulty in doing something interesting with the OSC signals, but it was interesting that it was a widespread problem. I think this might come from the fact that the other signals we were working with were both very tangible. Since the OSC input was just a number, I think we were a bit less motivated to find an interesting way to use it, as opposed to the audio and video, which we could immediately do interesting things with.
I think we didn’t have quite enough time to experiment with combining our cell blocks in different ways for a lot of emergent behavior to emerge. But one that I enjoyed seeing was how the visuals would layer together through 2, 3 or 4 cell-blocks. I thought that all of the cell blocks were interesting on their own, but the most interesting visuals were created through the combination of several together. This relates to Halprin’s cell-block framework through the idea that we can each create our own module that does its own thing, but the most exciting behaviors only emerge once we begin to combine the different cell-blocks, and experiment with how they feed into each other.
Download Patch
Pressure Project #2 – Transcendence through Snares
Posted: March 3, 2026 Filed under: Uncategorized Leave a comment »- Description of my cell-block:
- Independently, my cell-block uses the snare of the audio provided to cycle through a set of mouth shapes to simulate lip syncing (albeit not realistic lip syncing). It also takes the video input and through Ramp and Displace, warps the image based on the “mid” registered from the audio as well. Without outside input the audio used is “Position Famous” by Frost Children, and the video is a looping timelapse POV of a subway traveling underground. This was to create a sense of motion and exhilaration (the movement of the subway and displacement), and playfulness (the lip syncing).
- Collective documentation:
- Video/photos of the assembled system: Admittedly, I forgot to take footage of the showcase. I was a bit more nervous about this project, worried about everything working properly with the other cell-blocks. Once it was my turn, I only focused on presenting my work. I plan to reach out to classmates to see if they recorded footage.
- Process reflection:
- The cell-block was self-contained but, on the exterior, was connected to incoming TOP and CHOP inputs and fed those inputs out. So, on its own, the block would play as planned, but once external audio and video were fed in, they would inherit the effects of the previous media. There were some issues with feedback loops when testing this out, but mostly, it worked. The lips were a last-minute add-on and therefore independent… so no matter what, the lips stayed on screen; how it reacted depended on the audio input.
- I chose to control the level of flashiness and movement in my visuals. It’s easy to fall into producing loud and flashy imagery with programs like TouchDesigner or even After Effects, however, I try to use media responsibly, and I also didn’t want to give myself a headache. I’ve made materials that are hard for photosensitive people to take in, and while some others loved the chaotic visuals, I wasn’t satisfied knowing a group of people wouldn’t be able to watch it (and enjoy it).
- I was surprised that a lot of people didn’t use audio that contained many snares (or used much audio at all)… I was also surprised that everything worked together for the most part (if you can’t tell, I was nervous).
- When combined, everyone’s work offered me something new. I would combine with Luke’s when I wanted the most cohesive combination, I would combine with Zarmeen’s when I wanted to destroy everything (or use her audio), I combined with Chad’s because I wanted to appear on his channels more, and I combined with Curtus when I wanted to see a dragon.
- This project was a new way to envision Halprin’s cell-block method, but in a strictly digital realm. The goal was to have every block exist on its own and influence others (multiply the possibilities of the content produced). I think we mostly did that, although networking still feels stressful to me; I at least know how it works (sort of).
- Individual documentation: