Pressure Project 3: Expanded Television

I had a lot of ideas for this pressure project but ended up going with an expanded version of my first pressure project. I thought it would be really fun to use a Makey Makey to an actual TV remote into something that can control my Isadora TV. If I used sticky notes and conductive pencil drawings, I could fake pressable buttons to change the channel and turn it on and off.

To me, the hardest part of using a Makey Makey is always finding a way to ground it all. But I had the perfect idea of how to do this on the remote: because people usually have to hold a remote in their hand when they use it, I can try to hide the ground connection on the back! See below.

This worked somewhat well but because not everyone holds a TV remote with their hand flat against the back, it may not work for all people. You could use more paper and pencil to get more possible contact points, but this got the job done.

For the front buttons, I originally also wanted the alligator clips to be on the back on the remote, but I was struggling to get a consistent connection when I tried it. I think the creases at the paper bends around to the back of the remote cause issues. I’m pretty happy with the end result, however. See below.

For Isadora, I created a new scene that was the TV turning on so that people could have the experience of both turning on and off the TV using the remote. The channel buttons also work as you would expect. The one odd behavior is that turning on the TV always starts at the same channel, unlike a real TV which remembers the last channel that it was on.

I also added several new channels, including better static, a trans pride channel 🏳️‍⚧️, and a channel with a secret, a warning channel, and a weird filter glitchy channel. Unfortunately, I cannot get Isadora to open on my laptop anymore! I had to downgrade my drivers to get it to work and at some point the drivers updated again. I cannot find the old drivers I used anymore! It’s a shame cause I liked the new channels I added… 🙁

The static channel just moves between 4 or so images to achieve a much better static effect than before and the trans pride channel just replaces the colors of the color bar channel with the colors of the trans flag.

The main “secret revealed” I had was a channel that started as regular static but actually had a webcam on and showed the viewers back on the TV! The picture very slowly faded in and it was almost missed, which is exactly what I wanted! I even covered the light on my laptop so that nobody would have any warning that the webcam was on.

There was also a weird glitchy filter channel that I added. This was inconsistently displayed and was very flashy sometimes but other times it looked really cool. Because of this, I added a warning channel before this channel so that anyone that can’t look at intense things could look away. When I did the presentation, it was not very glitch at all and gave a very cool effect that even used the webcam a little bit (even though the webcam wasn’t used anywhere in that scene…)

The class loved the progression of the TV for this project. One person immediate became excited when they saw the TV was back. They also like the secret being revealed as a webcam and appreciated the extra effort I put in to covering the webcam light as well. In the end, I was very satisfied with how this project turned out, I just wish I could show it…

Cycle 3: Failure and Fun!

My plan for this cycle was simple: add phone OSC controls to the game I made in previous cycles. But this was anything but simple. The first guide I used ended up being a complete dead end! First, I used Isadora to ensure that my laptop could receive signals at all. After verifying this, I tried sending signals to Unity and nothing! I tried sending signals from Unity to Isadora and that worked(!) but wasn’t very useful… It’s possible that the guide I was following was designed for a very specific type of input which the phones from the motion lab were not giving, but I was unable to figure out a reason why. I even used Isadora to see what it was seeing as inputs and manually set the names in Unity to these signals.

After this, I took a break and worked on my capstone project class. I clearly should have been more worried about my initial struggles. I tried to get it working again at the last minute and found another guide specifically for getting phone OSC to work in Unity. I couldn’t get this to work either and I suspect this was because it was for touch OSC which I wasn’t using (and also didn’t need the type of functionality). I thought I could get it to work but I was getting tired and decided to sleep and figure it out in the morning. I set my alarm for 7 am knowing the presentation was at 2 pm (I got senioritis really bad give me a break!). So imagine my shock when I wake up, feeling rather too rested I should say, and see that it’s nearly noon! I had set my alarms for 7 pm not am…..

This was a major blow to what little motivation I had left. I knew there was no chance I could get OCS working in that time and especially no way I would get to make the physical cardboard props I was excited to create. I wanted to at least show my ideas to the class so I eventually collected myself and then quickly made a presentation to show at presentation time.

The presentation showed off the expanded control that I wanted to add had I been smarter in handling the end of the semester. The slides of the presentation can be seen below.

I then had an idea to at least have some fun with the presentation. Although I couldn’t implement true OSC controls, I could maybe try to fake some kind of physically controlled game by having people mime out actions and then use the controller and keyboard to make the actions happen in the game!

To my surprise, this actually worked really well! I had two people control the direction of the ship and shield by pointing with their arms, and one person in the middle control the vertical position of the ship with their body. I actually originally was going to have the person controlling the vertical position just point forward and backwards like the other two people, but Alex thought it would be more fun to physically run forwards and backwards. I’m glad he suggested this as it worked great! I used the controller which had the ship and shield direction, and someone else was on the laptop controlling the vertical position. The two of us would watch the people trying to control the ship with their bodies and tried to mimic them in game as accurately as we could. A view of the game as it was being played can be seen below.

I think everyone had fun participating to a degree although there was one major flaw. I originally had someone yell “shoot!” or “pew!” whenever they wanted a bullet to fire, but this ended up turning into one person just saying the command constantly and it didn’t add much of anything to the experience. I did originally have plans that would have helped in this aspect. For example, I was going to make it so there could only be 2 or 3 bullets on screen at once to make spamming them less effective, or maybe have a longer cooldown on missed shots.

In the end, I had a very good time making this game and I learned a lot in the process. Some of which was design wise but a lot was also planning and time estimating as well!

Cycle 2: Space Shooter Expanded

I originally planned to implement phone OSC controls into my Cycle 1 Unity game for this cycle. I would then use the final cycle to refine the gameplay and expand the physical aspects of the controls. Holding a phone is pretty boring, but imagine holding a cardboard shield with a phone attached to it!

However, I realized after some testing that I would need to do some slight gameplay reworks first. This was because after I saw the projection capabilities of the motion lab, I learned that the amount of area that can be covered is severely limited. This is especially true in the vertical direction. After seeing this, I decided to change the way enemies are spawned in. Instead of appearing in a circle around the player, they would appear at the horizontal edges of the screen. Given the size of the projection, I think this will be easier on the player. It still felt limiting in a way though, so I decided to allow the player to move vertically up and down as well.

These were significant changes, so I needed to redesign the game first before adding the phone OSC elements. While going through this process, I added a few new things as well. The overview of changes is as follows:

1. Enemy Spawner
The Enemy Spawner was changed to spawn enemies at the horizontal edges of the screen. This is actually much simpler than the circle spawning it was doing before

2. Player
The Player was given the ability to move up and down on the screen. This means the player class now has to look for more inputs the user can give. There is another consideration though, and that is that the player cannot be allowed to move off screen. To stop this, a check is done on what the player’s next position is calculated to be. If this position is outside the allowed range, the position is not actually updated.

3. Enemies
After implementing player movement, I discovered a major issue. The player can just move away from enemies to avoid them! This trivializes the game in many aspects, so a solution was needed. I decided to have enemies constantly look towards the player’s position. This worked exactly as expected. I decided to make enemy projectiles not track the player however, as this created additional complexity the people playing the game would have to account for.

4. New Enemy: Asteroid
Sense all enemies track the player now, I thought a good new enemy type would be one that didn’t. This new enemy type is brown instead of red and cannot be destroyed by player bullets. The only way to avoid it is to move the ship out of the way. To facilitate this gameplay, when an Asteroid enemy is spawned, it heads towards where the player is in that moment. This also ensures the player cannot be completely still the entire game.

5. Shoot Speed Powerup
After the recommendations I received from the Cycle 1 presentation, I decided to add a powerup to the game. This object acts similar to the projectile objects, in that it just travels forward for a set time. Except when it collides with the player ship, it lowers the cooldown between shots. This allows the player to shoot faster. This powerup only travels horizontally, again to add more variety to object movement.

The final result for this cycle can be seen below.

After finishing the game, I got a great idea. I remembered back to people attempting to play the game for cycle 1, and many of them found it overwhelming to control the ship direction and shield direction. But this new version of the game added an entirely new movement direction! That’s when I decided to turn my project from a single player experience to a multiplayer experience. I would have one person control the vertical movement of the ship on the keyboard, one person hold the left side of the controller to use the left joystick, and one person hold the right side of the controller to use the right joystick and trigger. This way, each person only really has one task and thus it should be a lot easier to keep track of.

However, once I tested this I ran into a major issue. The keyboard and controller could not move the ship simultaneously. It seemed like control was being passed between either keyboard and controller, and so they couldn’t happen at the same time as was needed to control the ship. After much testing, I found that if I had two input listeners, one for each type of user input, simultaneous control could be achieved!

I was running out of time for this cycle, and given the major reworks I made to the game, I decided to push phone OSC controls to the next cycle.

After presenting the game and having the class play it, I received very positive feedback! Most people liked how much easier it was to play when the tasks were divided up. The team effort also allowed them to achieve much higher scores than they were able to get for cycle 1. Two people holding one controller was a little awkward though, and the phone OSC would help with that. Ideally, it should feel like they are all part of a team that is controlling a single spaceship.

Cycle 1: Space Shooter

I knew that the interactive system I wanted to make for the cycle project was some kind of video game using Unity. I also knew that I wanted it to be a simple 2D shooter game where the player controls a ship of some kind and fires bullets at enemies. However, because this class is about interesting ways you can interact with systems, I thought using a controller would be boring! My initial plan was to use the top-down camera in the motion lab to track the player’s movement. This would turn the player’s body into the controller. Then, instead of a standard screen being used, the gameplay would be projected onto the floor. That way it was almost like the player was in the game, without having to use any VR! The game would have looked something like the classic game Space Invaders, shown below.

My plan was to have the person controlling the ship have a light on their head and another light they can activate. The camera would use the light on their head to track their movement and the activated light to fire a bullet. I realized there was likely a major flaw with this design though, and that was that the camera in the motion lab does not have a big enough view to accommodate the space I wanted. I also hadn’t considered the role that perspective would play on the camera. This made me move away from Space Invaders and more to a game like Asteroids, shown below.

The idea is to lock the player in the center of the screen so that the person controlling the ship can stand directly under the camera. This makes the camera angle less of an issue while still allowing the person to control the direction the ship is facing with their body. An additional interesting result of this is that the person in control cannot see behind themself like you could watching a screen of the entire game, which adds an additional interesting layer to the gameplay. Because the ship loses the horizontal movement seen in Space Invaders, I decided to add a shield mechanic to the game as well. The idea is that some enemies would shoot projectiles at you that you cannot destroy with bullets and must block with a shield. The person controlling the ship would use one arm to shoot bullets and one arms to block projectiles.

With that in mind, my goal for this cycle was to create the game working on a controller. The two joysticks would control the ship and shield direction and the right trigger would fire bullets. There would be two enemy types: One that is small and fast, and one that is bigger, slower, and can fire projectiles.

The main Unity scripts made were:
1. Player
2. Player Projectile
3. Shield
4. Enemy
5. Enemy Projectile
6. Enemy Spawner
7. Game Manager

I don’t want this post to just turn into a code review, but I’ll still briefly cover the decisions I made with these scripts.

1. Player
The Player checks for user input and then does the following actions when detected:
– set ship direction
– set shield direction
– spawn Player Projectile (the Player also plays the projectile fired audio clip at the same time)

2. Player Projectile
The Player Projectile moves forward until it either collides with an enemy or a timer runs out. The timer is important as otherwise the projectile could stay loaded forever and take up resources (memory leak). If the projectile collides with an enemy, it destroys both the enemy and the projectile.

3. Shield
The Shield is given its angle by the Player. It does not check for collision itself, the Enemy Projectile does that.

4. Enemy
The Enemy is actually very similar to the Player Projectile, it moves forward until it either collides with the player or a timer runs out. If the enemy collides with the player, it destroys the player which ends the game. There is also a variant of the Enemy that spawns Enemy Projectiles at set intervals.

5. Enemy Projectile
Similar to the Player Projectile, the Enemy Projectile moves forward until it either collides with the player or a timer runs out. If the projectile collides with the player, it destroys the player which ends the game. If the projectile collides with the shield, it destroys itself and the game continues (nothing happens to shield).

6. Enemy Spawner
The Enemy Spawn is what spawns enemies. Because enemies just move forward, this script calculates the angle between a random spawn point and the player and then spawns an enemy with that angle. The spawn points is a circle around the player. Every time an enemy is spawns, the time between enemy spawns is decreased (i.e. enemies will spawn faster and faster as the game progresses).

7. Game Manager
The Game Manager displays the current score (number of enemies defeated) as well as plays the destroy sound effect. When an enemy is destroyed, the interaction calls to the Game Manager to track the score and play the noise. When the player is destroyed, the interaction calls to the Game Manager to play the noise and display that the game is over.

The final game result for this cycle can be seen below.

Some additional notes:
The music is another element in the scene but all it does is play music so it’s pretty simple.
The ship is actually a diamond taken from a free asset pack on the Unity Asset Store. Link
The sound effects and background music were also found free on the Unity Asset Store. Because of the simplicity of the game, retro sounds made the most sense. Sound Link Music Link
Just after the entire game was done, I closed and reopened Unity and all collision detection was broken! I ended up spending hours trying to figure out why before creating an entirely new project and copying all the files over. So annoying!

After presenting to the class, I was initially surprised at how much some people struggled with the game. I knew it was difficult to keep track of both the ship and the shield using the two joysticks, but I didn’t consider how nearly impossible this was for people who had never used a controller. Otherwise, the reaction was fairly positive. One note that makes sense was to possibly try to differentiate between the two enemies clearer as they are the same exact color and that can be confusing. There were also some cool suggestions such as adding powerups to the game. It was also suggested that maybe instead of trying camera tracking, I could use phone OSC to have the players control the ship instead. This seemed like a much better idea and so I decided to investigate that for the next cycles.