Cycle 3 – Seasons – Min Liu

Based on the second cycle, I further developed the experiential system for children to learn about geography & season in this final cycle. I built a spherical interface with makeymakey, copper tape and conductive paint. Participants can touch specific locations on the globe and trigger different interactive environments featuring local geographical and climatic characteristics.

I searched for countries and places that is now (late April) in Spring, Summer, Autumn and Winter. I choose Tokyo (Japan), Amazon rainforest (Brazil), Mount Maceton (Australia)​, and Yellowknife (Canada) correspondingly and marked them on the globe. I used Kinect as input device this time and it worked much better than Webcam. In the autumn and the winter scenes I had created in the previous cycle, participants can interact with falling leaves and snows. I built the other two scenes this time. In the Spring Tokyo scene, the Kinect tracks people’s head and Sakura flowers will fall right above participants’ heads on the screen. In the Summer Amazon rainforest scene, rain falls on people and bounces off.  I will explore more interactive visual effects in the future. I also added audios to each scene which made the experience more immersive.​​​​​​ Here are the scenes I built in Touchdesigner:

In the final presentation, I was happy that participants explored the physical globe and engaged in interacting with different virtual environments both individually and together. They said the experience was fun and educational and they didn’t even notice that they were learning. Also, children will be excited about this experience. It’s a very COSI thing.

Here are some points to improve about this system. First, there needs to be some ambient light to make people brighter and more detectable, thus improving the sensitivity of interaction. Second, the spring Tokyo scene is too bright, and the flowering interaction is too slow. Third, there can be some explanations for participants to know that they are experiencing the weather in the area in current time.

Here is the recording of the experience:

Cycle 2 – Seasons – Min Liu

For this cycle, I wanted to create interactive virtual environments that contribute to embodied learning. I chose Seasons as the theme of this project. Participants can interact with nature things in specific seasons using their bodies. For example, spring & flowers, summer & rain, autumn & falling leaves, winter & snow.

I worked with Touchdesigner this time since Isadora was not friendly to what I wanted to do. I used webcam as the input device. In Touchdesigner, the NVbackground actor can remove the background and the Optical flow actor can track body movements. By ParticleGup, different kind of interactive particles can be created. Since I was not familiar with Touchdesign, I spent a long time figuring out how to transit from one scene to another in time order which is very easy to realize in Isadora. In this cycle, I only created the autumn and winter scenes. I am still learning about other interactive visual effects using different body languages, and how to trigger sound by movements in Touchdesigner. This is the Touchdesigner patch:

I was excited to install it in the motion lab. To avoid shadow, Oded told me that I could project the image onto the back of the screen, and people stand in front of the screen to interact with virtual environments. It’s magical to me seeing both digital figures and human shadows reflected on the curtain from people standing before and behind the curtain.

Participants were happy to interact with snows and leaves. But due to the limitation of the webcam, they must stand in a specific area, or they were out of the screen. I will try Kinect next time to see if I can solve the problem. I also learned that lighting is important because it decides the quality of the real time input video.

Shadrick gave me some nice advice to further develop this learning system. First, I can add some fun filters to digital figures, like wearing them hats in winter.  Second, different countries have different climates characteristics. I can build environments based on different regions. For the next cycle, I will also combine media-enhanced objects and environments together.

Here is the recording of the experience:

Cycle 1 – Tomatoes – Min Liu

My research interest is designing for joint and interactive educational experience for family visitors in informal learning environment like museums and zoos. In this project, I explored how physical and digital materials can work together to enhance sense-making, and how individual and collaborative interaction can motivate learning.

I chose tomato as the learning theme. In the default scene, participants will see a tomato garden with birds and some insects. They can wave their hands to get rid of the animals. A background music of nature winds, insects and chimes is on. A green tomato, a red tomato and two leaves (one is fresh, and one is dry) are controllers. I hope this can create a sense for participants that they are in a tomato garden holding tomatoes. When the participant holds the green tomato and a leave, he/she will see the growing process of tomatoes. When the participant holds the red tomato and a leave, there shows the rotting process. When two participants hold different tomatoes together, they will see microscopic structures of tomatoes.

In the test in class, players were surprised when they triggered different video clips about tomatoes. They thought these clips were interesting and well-chosen. One participant said that the rounded style of videos makes the experience intimate. Players needed to keep touching the tomatoes along their watching the video. I wished this setting could stimulate people to make some connections between the sense of touching physical materials and the digital contents on the screen. However, the feedback was that the connections were not strong there. Players just concentrated on the videos and separated the touch sense. So, how to design integral interactions with physical and virtual materialities needs further exploration. 

I tested the system again with a ten-year-old kid at home. To be honest, I failed to act as a good guidance and companion. I couldn’t help telling him what to do to trigger things and explaining the system mechanism. During the experience, the boy told me that he had watched a similar video before at school. It’s about a seed growing into a big tree. When the boy and I held the two tomatoes and watched the video together, he asked me questions like which part of the tomato is observed under microscope. This system is successful in establishing conversations between children and adults.

In the next cycles, I will keep exploring: 1. How can participants interactive with the system autonomously without verbal or written instructions? 2. How to build the connection between physical and digital materials? 3. Other creative learning and collaborative interaction ways. 4. Installations in big space.

Pressure Project 3 – The Story of Mulan – Min Liu

In this project, I retold the story of Mulan, a woman hero in Chinese history who disguised herself as a man and fought on the battlefield instead of his father. As a Chinese, I found it hard to tell a cultural story in English. The language itself affords meanings and I am not a good translator. That’s the main reason why I chose the story of Mulan. I knew there were several versions of movie about this story and two of them are in English, which could be good resources for me. I chose the Disney version. I reorganized the story to different parts (start, development, climax, end) based on the poem, the ballad of Mulan, an old poem that almost everyone can recite in China. Then I found audio resources (background sound, music, dialogues) for each part and tried to make the music transform smoothly between each session. As a totally new hand in compiling audio and video files, it’s not easy. Alex asked about the choice of the Disney version. I admitted that some of the scenarios and dialogues in the movie was funny and awkward. Actually, the short poem is more vivid and inspiring, and that’s why I added the poem in the video. I am learning to be a good storyteller of my culture. I am also happy that this project gave me an opportunity to learn about Au and Pr. I will incorporate audios in the following projects because sound is so powerful. Here is the final work:

Pressure Project 2 – Personality Test – Min Liu

This project asked us to real a mystery using Isadora and Makey Makey. I wished that the building steps to the secret could have strong connections to the final mystery itself, so I thought of the personality test. Different people may have different feelings about the same sound due to their characters, and their answers reveal the personality mystery. I set up three questions and four personality types. Each question has two answer options. Testers put the alligator of the corresponding color into the bottle to choose an option. This can be a fun interaction way than just pressing the keys.

The most difficult part of this project was deciding the mathematical logic behind the personality test and setting it in Isadora. For now, I didn’t know how to feed data from one scene to another. I set a different key to each answer option. And I built a separate scene to deal with the logic. Actually, it is interesting but kind of strange to calculate a personality. I also tried to interpret different sounds with abstract elements like lines and dots. I am going to learn more about video effect actors and tools for transmitting data in Isodora. This is the Isadora patch for calculating personalities:

I was a proud of the final outcome. But I also met some issues in the test in class. First, the first scene about game instructions is too easy to be triggered by movement… so many testers missed this scene. Second, in one test, two alligators twisted together and trigger a scene unexpectedly, which disrupted the experience. So, it’s important to sort out circuits. This is the test video at home:

Pressure Project 1- Congestion – Min Liu

In weekends and holidays, there are many people shopping in groceries and congestion often happens in checkout area. For this pressure project, I observed people, environment and technology in the self-checkout area of the Giant Egle grocery store to have better understanding of the traffic happens here, and brainstormed some ideas (benevolent intervention) to solve this issue.

I took some photos in this area. Based on past experience and my observation today, I draw this environment map which shows the flow direction, congestion pattern and their relationship to physical environment.

The journey map helped me organize and visualize custom’s behavior and emotional change during self-checkout process. It also shows some design possibilities to enhance shopping experience.

To avoid congestion and enhance self-checkout experience, we can change the environment layout (IDEA 1) and improve the efficiency of interaction with the self-service machines (IDEA 2). Re-designing a self-service machine is a big project. A lot of things need to be improved including the user interface, packing area, scanning system… More automatic scanning system can avoid troublesome process of finding the barcodes or inserting the item number. (Alec said Kroger had tried similar checkout system in some pilot stores.) Then I suddenly think of why we must wait and checkout in a specific area and why we cannot just pay while we are shopping. And I came up with the third idea -a self-service app, which I thought it’s time and money saving. Alex said that this app can be malevolent if it’s designed to intrigue consumption. It’s true. It’s important to think about possible issues causes by technologies.

Project Bump

Pressure Project 2 (Sara) | Devising EMS (ohio-state.edu)

I enjoy this pressure project. The little game Bottle Lighting motivate exhausted people spending too much time in front of screen to do wrist stretching exercise. Prolonged use of devices like smart phone and computer can cause many physical injuries like poor vision, neck pain … It is a good idea to remind people of the possible issues and help them in a gamification way.