{"id":6222,"date":"2026-10-21T19:07:41","date_gmt":"2026-10-21T23:07:41","guid":{"rendered":"https:\/\/dems.asc.ohio-state.edu\/?p=6222"},"modified":"2026-10-21T19:07:41","modified_gmt":"2026-10-21T23:07:41","slug":"between-the-signal","status":"publish","type":"post","link":"https:\/\/dems.asc.ohio-state.edu\/?p=6222","title":{"rendered":"Between the Signal"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">In this project, I used webcam cameras and MediaPipe hand tracking technology to create an interactive TouchDesigner experience. The main limitation is that it\u2019s not possible to use a keyboard or mouse; therefore, I decided to use the participants\u2019 hands as the interface. Different hand movements trigger visual changes, and the participants gradually realize that their own bodies are actually controlling the entire environment, thereby revealing another layer of meaning associated with that environment.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Concept and Mystery<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The core idea of this project is that the cameras are not merely used to display the participants themselves, but act like a damaged communication system that attempts to connect with another space. Initially, participants cannot fully understand the effects of their hand movements. As they continue to experiment with different gestures, the visual effects change, and those hidden digital worlds begin to become visible.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Therefore, this puzzle isn\u2019t presented in the form of a traditional puzzle; instead, it allows participants to learn about this system through their own bodies. Their hands become the bridge between the physical world and the hidden digital world.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I chose to use hand tracking technology because I wanted this interaction to be more immersive. Clicking the mouse feels like controlling software, while moving hands in front of the camera makes the interaction more direct. Participants can immediately see the impact of their actions on the system, thereby making this process more personal.<\/p>\n\n\n\n<figure class=\"wp-block-video\"><video height=\"720\" style=\"aspect-ratio: 1280 \/ 720;\" width=\"1280\" controls src=\"https:\/\/dems.asc.ohio-state.edu\/wp-content\/uploads\/2026\/10\/record_DTPH4S68.mp4\"><\/video><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Story and Audio<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The atmosphere was inspired by Hideo Kojima\u2019s Death Stranding 2: On the Beach. I was especially interested in its themes of connection, isolation, and uncertainty about whether connecting two different places is always a positive thing.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I used music from Death Stranding 2 because I wanted the sound to make the experience feel like the participant was discovering an unfamiliar space rather than simply testing a hand-tracking program. The slow and atmospheric music also helped connect the different visual transformations into one continuous experience.<\/p>\n\n\n\n<figure class=\"wp-block-video\"><video height=\"544\" style=\"aspect-ratio: 960 \/ 544;\" width=\"960\" controls src=\"https:\/\/dems.asc.ohio-state.edu\/wp-content\/uploads\/2026\/10\/fc719a56e4a3cb261137e4242f3cd0f3.mp4\"><\/video><\/figure>\n\n\n\n<figure class=\"wp-block-video\"><video height=\"544\" style=\"aspect-ratio: 960 \/ 544;\" width=\"960\" controls src=\"https:\/\/dems.asc.ohio-state.edu\/wp-content\/uploads\/2026\/10\/396526a12eee9c161cc23c065852ba53.mp4\"><\/video><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Reflection<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">During the demonstration, people were particularly interested in how gestures control various changes. They also asked about the stories behind this experience. These feedbacks made me realize that the interaction itself is understandable, but the descriptions of the related stories can be conveyed more clearly.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If I continue to develop this project, I will add more demonstration elements, so that each gesture can reveal different aspects of the hidden world. Additionally, I will make the sounds more directly related to the participants\u2019 movements, so that both visual and auditory systems can respond together to body movements.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this project, I used webcam cameras and MediaPipe hand tracking technology to create an interactive TouchDesigner experience. The main limitation is that it\u2019s not possible to use a keyboard or mouse; therefore, I decided to use the participants\u2019 hands as the interface. Different hand movements trigger visual changes, and the participants gradually realize that [&hellip;]<\/p>\n","protected":false},"author":141,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-6222","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/dems.asc.ohio-state.edu\/index.php?rest_route=\/wp\/v2\/posts\/6222","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dems.asc.ohio-state.edu\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dems.asc.ohio-state.edu\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dems.asc.ohio-state.edu\/index.php?rest_route=\/wp\/v2\/users\/141"}],"replies":[{"embeddable":true,"href":"https:\/\/dems.asc.ohio-state.edu\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=6222"}],"version-history":[{"count":1,"href":"https:\/\/dems.asc.ohio-state.edu\/index.php?rest_route=\/wp\/v2\/posts\/6222\/revisions"}],"predecessor-version":[{"id":6231,"href":"https:\/\/dems.asc.ohio-state.edu\/index.php?rest_route=\/wp\/v2\/posts\/6222\/revisions\/6231"}],"wp:attachment":[{"href":"https:\/\/dems.asc.ohio-state.edu\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=6222"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dems.asc.ohio-state.edu\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=6222"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dems.asc.ohio-state.edu\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=6222"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}