Throughout the weeks, different components of the prototype — snow visuals, motion tracking and the snowboard — were developed concurrently.

SNOW VISUALS

The visuals were gradually developed through p5.js as we learned how to use the program along the way. It was a playful processs of trial and error, experimenting with the code that we built to produce generative visuals that fit our vision.

Initially, our first snow visual created was a basic snowfall against a plain background. We were encouraged by our lecturers to go beyond something that looked realistic and try something that was more expressive of the calm/chaos dichotomy.

Calm Snowfall

Blizzard

MOTION TRACKING

Although we did not integrate motion tracking into the final prototype, it was still a significant part of the process and it would be necessary for the full-scale vision. We had to take time to learn how we could use PoseNet to achieve visual effects through motion tracking.

With the help of our lecturers, we learnt how to tell PoseNet to identify specific body parts. Consequently, it was easy to produce visuals when 2 people interacted by holding hands as we could activate visuals when two pairs of wrists in close proximity were detected.

Our developments of motion tracking is documented here.

  • Ellipses shrink and grow when skeletons are detected within a range of X & Y coordinates.
  • Snow starts to fall when a person is detected within a range of X & Y coordinates.
  • Shapes form when two left wrists are tracked.

SNOWBOARD MOCKUP

To complete the interactive experience, we needed a component that would direct users to rely on each other. This required us to artificially incorporate wobbliness into a snowboard mockup. Therefore, we tested out a few different iterations to see what would work best. In the end, we realised using tennis balls below the mock snowboard gave the participants the best experience with the installation.

  • Snowboard Prototype Version 1

  • Snowboard Prototype Version 2

  • Snowboard Prototype Version 3

  • Snowboard Prototype Version 4

USER TESTING

We presented some of the snow visual drafts to lecturers and we got to hear what impressions they had.

Nadine: Plain snowfall is boring and that it doesn't need to be 100% realistic. There is room to play with expressing snow. She suggested exploring functions and effects such as lens flares and light properties.

Joe: In Week 12, Joe suggested visuals that were more three-dimensional since we were simulating the experince of snowboarding down a hill. Therefore, the snow should be coming closer to the users' faces.

For this physical component of the installation, our friends and us tested out being on the snowboard prototypes where we used different objects to make the snowboard wobbly.

Shalom: I tested out the different versions of the prototypes and realised that the dumbbells only created lateral movement.

Rakshitha: She really liked the version with the tennis balls because she had to try to balance herself against all her directions. She even found it fun!