Vocal Visuals: audio and visual telepresence
Eliot Ball and Ricky Young
We bring to you our finding and studies of how a pair of people interact through a point in an axis controlled by the sounds they make.
And talk about the final advances that brought us to a point where we could test this project.
Stereo Amplification and Dual Screen Routing
The stereo amplifier that we purchased (the LEPY 2020A) was key to the success of the oscilloscope system. Prior to purchasing an amplifier, the waveform was so small the display appeared as simply a quivering line on each axis. The amplifier increased the amplitude of the signal enough to provide each axis with a waveform capable of taking up the entire screen. In addition to this, the amplifier came with a built in voltage transformer. It seems as though this transformer as provided the coil within the television with the proper amount of signal, therefore solving the problem that plagued many of our previous iterations. Because the television was being fed the proper amount of voltage, the system didn’t fail or automatically should down.
With the issue of the television and waveform displays solved, we moved on to the problem of how to make this installation into a dual screen set up. We took advantage of some home studio equipment we had available. Using an audio interface (a Focusrite Scarlett 18i20), we were capable of developing a series of routing paths that would split input signals, meaning each microphone could be sent to multiple sets up televisions at once (up to 8 in fact.) We then purchased a specialty microphone cable that converted a standard XLR jack into a stereo RCA output. Each stereo amplifier requires a stereo RCA input, so this cable was paramount in transferring the standard microphone jack into something capable of being received by the amplifiers.
Within the audio interface’s software, we assigned these specialty microphone cable as outputs 1+2, and a monitor speaker for each TV as outputs 3+4. We then instructed each microphone signal to be sent to their corresponding specialty cables, as well as sending each microphone to the speakers. Because the specialty cable resulted in a stereo RCA jack, we then routed ONE of each stereo pair to either amplifier. The resultant signal path gave each television a display of both microphone inputs.
a diagram showing the wire we used to split the XLR audio signal from the microphone into left and right RCA signals.
diagram of the full system with pictures of each element
User Studies and Results
With all systems solved and ready for installation, we ran a series of user studies in order to observe how participants would interact with the interface. We specifically wanted to see how a diluted sense of each other’s presence might affect a user’s interaction with the opposite person. Would such a scenario enhance two strangers attachment do one another? What might occur when two people familiar with one another? We wanted to see what trends we could observe based upon the relationships of the two people, and also see how much cooperation was fostering through interacting with the interface.
In addition to letting the two participants hear both of their voices while speaking, we also engaged in an alternative version of the experiment that prohibited the users from hearing one another. This involving cutting the microphone input to the speaker, pumping white noise to each user, and giving each person a pair of ear plugs. We observed the differences between a purely visual stimulus and one with both audio and visual cues.
While the studies were occurring we also recorded each interaction on both microphones and with a video camera on each participant. With the recordings handy, we can pump the original audio back through the interface, effectively creating a simulation of their initial dialogue.
After each pair finished their session, we gave them each a questionnaire in order to gauge their reaction to the experiment. The questions focused on their own feelings during their experiment, their engagement and connection to their partner, and also if they felt the interface enabled or inhibited a meaningful interaction.
Discussion of Results
After observing and recording the user studies, we began to notice several trends develop within the experiments. Many reacted positively to their partners role in the interaction. Each group had a different dynamic, but in general pairs (who often did not know each other prior to the installation) would sing together, hum together, or make wacky noises together. They observed the visual output of their voices, augmenting the sounds they made to play with the shapes that appeared on scream. Some would coordinate efforts, each producing a different noise to facilitate a unique shape. Others attempted to match notes in order to make a “perfect circle” or to at least make the waveform stand still for a brief moment.
There was a minority of participants who seemed to feel mildly uncomfortable with the experience, choosing to talk in short bursts and observe their individual axes move horizontally or vertically, never fully giving themselves to the interface. Yet it should be noted that there was never any disinterest. Those who felt uneasy engaging with their partner still interacted with the interface and expressed enjoyment in the outcome. Still others began to engage before laughing too hard to continue, not an ideal but still very satisfying outcome.
In general, observing one's partner through the abstracted lens of diluted physical presence, meaning a distillation of an individual into merely one or two perceptible variables, seems to have an effect on the user's inhibitions and sense of humility. In a way the installation acting as an ice-breaker of sorts. Here are some quotes from the questionnaire that reinforce that notion:
What effect did the interface have on two strangers interacting with one another?
“I could communicate with someone that I didn’t know through something that I wasn’t familiar with, but it was actually working.”
“It was just an instant moment of collaboration.”
“I feel as if I went on a blind date spontaneously, the TV allows for a level of intimacy that you might not have ever expected with the person you’re doing the experiment with.”
“I didn’t know my partner, but the oscilloscope allowed us to have fun together and be silly in a way that we wouldn’t have been able to without the oscilloscope.”
How did the interface contribute to the interaction with the other user?
“Well, it was the limitation of interaction that really defined the connection.”
“Took away the human element. It felt like the future of communication.”
“It was kind of like a battle to see who could get the longest or highest line between the two of us. But it was cool to see the mix of the same pitch and volume.”
“It made me question his intentions or emotions.”
“Technology typically doesn’t require more than one person to activate, so the act of collaboration makes more a dependence we tend not to expect/encounter.”
Further studies will continue using these oscilloscopes. More intense scrutiny regarding participants relationships to one another will further develop and uncover trends of human interaction and meaningful connection. Though in general we are very happy with the methods in which people cooperated and created together, forming small visual artifacts that are quite literally a composition of their combined efforts. More work to come, stay tuned.