Brainstorming and exploring
After the lecture (13/10) on the technical aspect of ML and the framework we’re using me and Anton sat down to check out the code and explore it. We wanted to explore what we can do with sound as output instead of using a screen, and made this little sketch to try that out:
When the wrist distance is less than 100 pixels the note will play. This is made through connecting a P5 sound library to our code. This sketch proved to be quite generative, because through this we now have ideas of trying out how the sound will change depending on how you move your wrists and maybe even how fast you move them. Our metaphor for this ideation is using our body as an instrument. We wanted to see if we could make it play two notes depending on where our wrists were and came up with this sketch:
This opened up for ideas such as creating the illusion of a space with sound. Maybe recording the eye-distance to know how far/close the person is to the screen. This will then impact the sound. Whether it gets louder/quieter or changes pitch due to distance from the screen. This might invite to action through having low sound that makes you want to get close to hear better -> computer responds by getting louder -> eventually loud enough to repel the user from getting any closer. Maybe we could make it more nuanced through taking into account with which speed you move. Our vision for this is to create an interplay between human reacting to the sound the computer makes and the computer reacting to the movements of the human.
With this being our “primary” idea at the moment to explore and experience, I also got a completely different idea inspired by the interactive hologram in Iron Man (which I somehow always seem to come back to...). This idea is a bit less developed and talked about since we feel like the illusion of space through sound might open up for more nuanced and interesting aspects. Either way - this second idea is about manipulating an object on the screen through gestures.
Instead of using your cursor to manipulate a 3D object, how would it be to use gestures?
It would probably feel more tangible and coupled, but that’s just prejudices - it’s nothing that we’ve actually tested and experienced yet. This is something I might save for future projects as well, as I believe it’s not so far away to be implemented. Say you’re looking at an object on your screen, maybe a shoe or a piece of clothing and you’d want to easily view it from another angle. It could probably done with smaller gestures than I’m displaying in the GIF above. But of course, this is more of a concept and that’s what we’re trying to stay away from. Another reason not to go further with this idea is the question of why complicate interaction with gestures when it can so easily be done with the mousepad? I haven’t been able to answer this yet, only with “because it’s more fun” but that’s not enough, I want to be able to elaborate on the why before we take a chance with this.
I would still like to keep this idea in the back of my head in case the first idea doesn’t give us any interesting insights (even though I believe it will). I feel like it’s good to have a few different ideas to be able to fall back on.
