Cycle 3
Posted: December 11, 2024 Filed under: Uncategorized Leave a comment »Resources
Motion Lab
- Projectors and screens
- Visual Light Cameras
- Overhead depth sensor
Castle
Blender
Isadora
Adobe Creative Suite
Poplar Dowel Rod
MaKey MaKey
Arduino Board
Accelerometer/Gyroscope
Bluetooth card
Score
Guests are staged in the entryway of the Motion Lab. They are met by the “magical conductor” who invites them to come in with a wish in their hearts.
As guests file into the completely dark room housing only an unlit castle the song “When You Wish Upon a Star” from the Pinocchio soundtrack begins to play.
Projections onto the castle are delayed by approximately 10 seconds. Projections start slow, progressively showcasing the full range of mapped projection capabilities (i.e. isolating each element of the castle.
Stars shoot across the front of the castle corresponding to the first two mentions of “when you wish upon a star”
The projections steadily increase in intensity.
As the novelty of the intensified projections wears off, the magical conductor enters the scene to demonstrate the capabilities of the magic wand, introducing a new dimension to the experience.
The magical conductor offers the magic wand to guests who can try their hand at interacting with the castle.
The music fades along with the projections.
The Peppers Ghost emerges from the castle entrance and thanks guests for coming. Fades out.
This score outlines the behavior of my relatively low-tech approach to creating a magic wand. As a physical object, the magic wand required only a poplar dowel rod and a little bit of imagination.
Valuation
Embrace the bleed – I spent hours during cycles 1 and 2 trying to tighten my projection mapping and reduce bleed around the edges. During our feedback session for Cycle 2, one of my classmates pointed out that the bleed outlining the castle on the main projection screen actually looked pretty cool. I immediately recognized that they were right, the silhouette of the castle was a striking image. It took time for me to fully embrace this idea.
Simplify to achieve balance – I started to hear Alex’s voice ringing in my ears over the course of this project. This was the most critical lesson for me to take away from this course. I dream big. Probably too big. I always want to create magic in my projects, I have an impulse to improve and expand and
During cycle 3 I made several conscious simplifications to achieve balance, i.e. to deliver a magical experience within the allotted time.
I removed the stone facade projection from the castle. This was a neat effect, but the lines in the stone made imperfections in the mapping readily apparent. The payoff for the audience was marginal and diminished quickly. Removing this element allowed investment in features that made the experience more immersive and engaging.
I simplified the function of the magic wand (which had already been simplified several times from the original concept of a digital fireworks cannon). Prior to the final iteration, the concept for the wand was to embed a small Arduino chip, an accelerometer, and battery to capture movements and trajectories, sending signals to the computer over bluetooth that would trigger events in Isadora. The simplified concept used the Motion Lab’s depth sensor to detect activity in a range over the top of the castle. The active areas were cropped to isolate the effect to specific areas of the castle. An auxiliary effect was added to the Isadora interface with a button that enabled the operator to trigger a shooting star when guests tried to perform magic tricks that were not programmed into the system.
Early concepts for what would become the Magic Wand called for a blaster. At various times the blaster was to house some combination a Nintendo Wii remote, a phone using Data OSC, or an Arduino board with a gyroscope/accelerometer and bluetooth card. Other conceptual drawings outline the operation whereby the pull rope connected to a PVC pipe on an elastic band inside the blaster would close a circuit to send a signal to the computer to fire a digital firework along the trajectory of the blaster. (More drawings will be uploaded upon retrieval of my notebook from the Motion Lab).
I simplified the castle, removing several intended features from the castle including multiple towers, parapets, and roofs. These features would have been visually striking and several towers were already cut and shaped, but each additional tower increases complexity and time costs. Truthfully if I had not broken the castle twice (once on the night before final installation in the motion lab and again on the way into the building the morning of final installation) I likely would have added a few additional towers. It’s not clear whether this would have substantially improved the final experience.
The most difficult simplification was cancelling plans to include a Pepper’s ghost in the doorway – this was the hardest to part with because the potential payoff to guests was huge. Other features like windows with waving character silhouettes were also difficult to cut because they would have improved upon the magic of the experience.
Manage the magic – I created gigabytes of interesting visualizations, 3D models with reflective glass and metal textures, and digital fireworks for this project and it was very tempting to bring everything in and have the experience run at full intensity throughout, but this approach burns too quickly. People need time to adapt and recover in order for escalations to be effective. Metering out the experience requires a measure of discipline.
This score shows an emotional journey map of the planned experience prior to dropping plans for the Pepper’s Ghost in the week prior to the performance. Ultimately removing the ghost still allowed for escalation of emotional intensity and a meaningful journey, where diverting time to building it would have put the project completion in jeopardy.
Simplify the projection mapping – Prior efforts to simplify my approach to projection mapping had been unsuccessful and at this point multiple projectors could no longer be avoided. I decided to change my approach to do much of the mapping in the media itself to reduce the amount of mapping necessary at installation.
I used multiple approaches to support this effort with the expectation that some may fail, but that diversification would ultimately lead to the greatest chance for success. First I took photos of the castle from the projectors’ vantage point. Next I measured the distance, rotation, and orientation of the projectors in relation to the castle to model the space in Adobe After Effects. Finally I built a 3D model of the castle in Blender, hoping that I would be able to import the obj files into Isadora and use my projections as textures. This was a pipe dream that did not work, but I did manage to create something interesting that I couldn’t figure out how to actually use in time for the final presentation.
A cool effort that turned out to be only marginally useful in the end. Given more time I would love to use this OBJ file to reflect things in the environment like fireworks that could create a really interesting and cohesive experience.
I used images of the castle to help build digital models.
Early attempts to model the castle using Adobe After Effects and position cameras to correspond with the projector positioning were time consuming and came close to approximating the environment, but ultimately unsuccessful. Using the measurements taken from the motion lab helped close in on actual positioning, but the angles and exact positioning proved elusive.
The most useful asset proved to be line tracings from the perspective of the motion lab projectors that I could pull into After Effects to create graphics that were very close to the actual dimensions of the castle. These were much easier to map onto the castle and only require minor distortions to account for imperfections in my photography approach.
Performance
I think that many of my theories were confirmed during the performance. The flow worked well. Nobody mentioned the removal of the digital stone facade. Classmates enjoyed playing with the magic wand even during the feedback session. The silhouette of the castle on the main projector made for a striking visual and gave an impression of being larger-than-life. The magical conductor role was looked upon favorably. Obviously the Pepper’s ghost was not mentioned because it was a secret closer, but I suspect that if well executed it would have fit well with the overall theming and magic.
One major oversight that would have been embarrassingly simple to execute is the addition of an audio indicator for successful magic wand interactions. I added these in post to demonstrate how it might have changed the experience.
What could have been…