Cycles 2+3 (Maria)
Posted: April 30, 2021 Filed under: Maria B Leave a comment »Intent
The overall intent for my cycles was to create an interactive experience using the LEAP motion. For the first cycle and a lot of the second, I focused more on the types of interactions that would happen and their intuitiveness. Starting in the second cycle, I started thinking more about how I could use a narrative to tie these interactions together and give them meaning. My overall goal was to explore what it means to design UX for motion interaction devices.
It took me awhile to land on a story I liked and made sense to me. I pulled inspiration from escape room plots, different video games, movies, and the simple aesthetic that naturally arises in Isadora by using simple shapes. I ended up taking a lot of influence from the game “There is No Game,” which was really fun in the way it broke the 4th wall with the users. I combined that with the simple, almost 8-bit aesthetic to make a story where the person playing the game is taking part in ‘user testing’ for a new, immersive video game and ends up getting trapped. Then, depending on which ‘door’ they choose, they either win, lose, or break the game
Process
I made a wireframe of my storyboard in Miro, writing out what narration and game objects were in each scene, and how it responded to different actions of the user.
Next, I went into Isadora to mock everything up. The individual activities I made in cycle 1 and the beginning of cycle 2 proved really helpful because, even though I didn’t use all of the exact activities, I used a lot of similar bits of code that I was able to pull and apply to the new ones. The original goal was to do a second pass-through and add better assets, but I ran out of time.
In the first scene, I made a hand tracker that tracked the angle and position of the player’s hands using a LEAP motion and a 3D projector. I used a series of Inside Range Actors and calculators to track when the hands were inside the boxes, and when they were it sent a signal to jump to the next scene
The second scene was just a copy of the button press scene I made in cycle 1.
The third scene used a Live Drawing actor to make the circles, and the circle trigger on the Leap Motion Watcher actor to set of sequential triggers that gradually made the scene darker as the user drew circles
Scene 4 was a bunch of chaos that was made to be the ‘glitch’ in the program. I initially didn’t have sound in this scene, but adding it made a huge difference in recognizing that the program was glitching out. I was really happy to see that users recognized that the situation was an ’emergency’ and everyone who played it attempted to press the ’emergency button’ that was introduced a few scenes earlier (yay successful UX design!).
If you look closely you can see a super mario bros level in the background (music from the game is also playing) — this is an ‘easter egg’ that gives users a hint for later in the game!
If the users pressed the button, they were taken to this scene. Basically acknowledging that the user pressed the button, but it did no good because the button was out of order. If the user didn’t press the button, the patch automatically jumped from scene 4 to scene 6 after about 8 seconds.
This is the scene where user choice comes in. There is a yellow circle (a ‘flashlight’) that responds to the position of the users hand movement. Hidden in the darkness are 3 ‘portals’: a pacman ghost, a galaga ship, and a mario ‘?’ block. Each of these portals leads to a different ending! The light turns green and a green message pops up when the user holds the light directly above the object, and if they hold it for 3 seconds it takes them to the associated ending.
To make the ‘flashlight’ in Isadora, I used a chroma key. Basically, when the foreground layer is a specified color (in this case yellow), it reveals the background layer that holds the hidden object, BUT if the foreground layer isn’t the specified color, you only see the foreground layer. To keep the flashlight from disappearing completely, I slightly offset the flashlight color from the ‘key hue,’ making the flashlight slightly transparent and still revealing the background.
To send the user to the appropriate ending when their hand/the yellow dot was inside a specified range, I used the same method as I did back in the first scene with the hand tracking, but I made a different one for each of the three hidden icons.
Challenges
One of the biggest challenges of this project was choosing/creating a story that would give meaning to and make sense with the kinds of interactions I wanted to make. A lot of the stories I was coming up with were a lot more complex than I had time for or just didn’t really make sense with the context of these interactions. I found myself succeeding when I just picked a story and ran with it. Even though the initial story I ran with didn’t end up fitting with the interactions exactly, it helped me come up with a similar story with a different context that made a lot of sense with the LEAP motion-based interactions!
Another challenge I faced was the attempt to make motion-based interactions with the LEAP Motion in Isadora that (1) Worked, and (2) made intuitive sense to the users. When I was originally storyboarding, I came up with all these interactions like pinching to grab, clapping, etc. that just didn’t work. I ended up sticking to more basic motions like moving your hand around that was associated with a dot on the screen or moving your hand up and down to push a button.
Adding on to the challenge of making motion-based interactions that works and made intuitive sense, it was hard to get a feel for how the users would respond to the prompts without user testers, and I could’ve really benefitted from getting more frequent feedback from others. One big example of issues with this was the button press scene. Some users didn’t really understand how they were supposed to ‘press’ the button without physically pressing it. They would try pressing their hand towards the screen or just pause in confusion. Once they figured out how they were supposed to press it (downward motion with your hand like you’re pressing an invisible button on the desk in front of you), users often tried to press it really fast and/or continued the motion until their hand hit the table. This made a lot of sense upon seeing others do it, but it was never something I realized when creating the patch on my own. I knew that the LEAP wouldn’t detect my hand motion if I moved too fast and that it would lose the position of my hand if I set it down on the table next to the device, so when I was testing it I was sure to move my hand slow and keep it in range–but people who have never used the device before have no way of knowing that!! I’ve come to really enjoy thinking about how to design/problem solve for these intricacies and edge cases that come up when designing for motion detection 🙂
Another challenge I faced was just how long each of these scenes seemed to take, and it really started to make me feel bogged down and unmotivated because I felt I wasn’t getting anywhere. It was additionally frustrating because I was working on a project using another visual coding platform at the same time, and it was hard trying to shift my brain back to working in Isadora. However, as the project went on, I found it a lot easier because I had more ‘tricks’ up my sleeve (bits of code that I had used in other scenes) that I could use in future scenes! I also found it a lot easier to do when I had a more detailed plan. Something like a post-it note or two in Miro spelling out exactly what I wanted to accomplish in the scene and, even better, writing out the basic logic/actors I’m going to use to accomplish it.
Accomplishments
I was really proud of the intuitiveness I was able to accomplish with some of the interactions. The top two were the user knowing to press the ’emergency’ button when the screen started glitching out, and the scene where the message asks if the user “has any control over their environment” and the user has to figure out what to do with the yellow orb they can move around with their hands (this one could’ve been smoother with some more testing, but I liked how it introduced a sort of puzzle to the users). Basically I was just really excited about anything I could get the user to do without explicitly telling them to do it!
I was also really happy with the narrative I ended up coming to, and the way it engages with the users. I really enjoy how it breaks the fourth wall and gives the user the sense that they are an integral part of the story that has an impact on what happens. If I had a cycle 4,5,6, etc, I would love to push this further and give the user even more sense of choice and decision-making.
Finally, by the end of this project I felt like I had a really good handle on Isadora and figuring out new things to accomplish with it. Although I probably won’t be using Isadora much in the future, I can totally see this way of thinking benefiting me when using other coding languages!! I may even go and try to teach myself java 🙂
Observations/Takeaways
I feel like there are a lot of ‘observations’ sprinkled throughout this article, but I’ll try to sum some up here!
- Even though Isadora isn’t necessarily built for my field, it serves as a great learning tool that helped me realize some of the creative powers of coding!
- Planning is KEY, and I need to work on it (take advantage of Miro!)
- Reflecting is also key to learning, so I’m glad I’m doing this 🙂
- Pick something and run with it! Just because you make a decision now doesn’t mean you can’t change things later down the road
- Engaging multiple senses (even just adding sound) adds SO MUCH to an experience
- Giving users a sense of choice makes them more engaged
- You have to give users a REASON to do something (which is why having a narrative is key)
- Coding (even visually) is a design in itself. It is so satisfying and exciting when you get something to work! I love the thinking and logic that goes into it, and I want to keep on doing it!
- I really enjoy finding creative ways to engage users!
- I really enjoy the problem-solving process of making interactions intuitive to users