With the artist nominations just announced, we wanted this next post to take a critical look at the nomination data and consider the results. Remember this heatmap of check-ins over the weekend?
At first glance, most people looked at this map and assumed the majority of nominees would be coming out of the hottest of hot spots, but the resulting nominations are surprising because they run counter to these assumptions. Compare to the heatmap of nominations:
You can see what’s happened here—nominations were dispersed throughout. Even though an artist may have been in a high density building or even a hotspot, the nominations in those areas were dispersed among the many studios open. Was there something in the traffic patterns and the resulting metrics that meant a confluence of events could occur—was there a sweet spot?
Corinne Hoener, one of our web developers, is going to show us a couple of examples:
There were 1708 artists in 758 buildings with open doors for GO. Of those buildings, 188 housed multiple studios. If we look at the data on our top ten artists, six of the ten were in multiple-studio buildings, and five of those six were in buildings with 8-15 studios, such as 925 Bergen and Invisible Dog (both had 15 studios). Naturally, this led us to wonder if 8-15 studios is a “sweet spot” in terms of having enough studios to draw visitors to your building, but not so many that visitors are overwhelmed by choice. If we crunch the numbers, it certainly looks that way.
On average, an artist’s studio saw 30.35 check-ins, but artists in buildings with 8-15 studios had higher than the average with 44.09 check-ins. On either side of this sweet spot, there were fewer check-ins; artists in buildings with fewer than 8 studios had 22.92 check-ins and buildings with more than 15 saw 40.94 check-ins. Studios in buildings with 8-15 studios also received about two more nominations than average. That might not sound like a lot, but with tight margins like we have, two additional nominations can make all the difference.
Speaking of nominations, it’s important to remember that an individual visitor can check-in to as many studios as they want, say 100 or more, but they can still only nominate up to three artists. So if we’re digging into the numbers behind our top ten artists, we have to consider the number of visitors to their studios. And again, we can see a sweet spot effect for those buildings with 8-15 studios. Among our top ten artists, studios in buildings with 8-15 studios saw on average 223.25 unique visitors. These buildings had, on average, 12 studios in them, so there is an average visitor-to-studio ratio of 18.6. Contrast this with two mega studio buildings, which had an average of 72 studios in them and an average of 429.5 visitors—nearly double the average visitors that the studios in 8-15 buildings saw. However, since they also had about 6 times the number of studios, that leaves them with an average visitor-to-studio ratio of only 6.0. There were far fewer visitors relative to the number of studios in these major buildings. And as we pointed out earlier, fewer visitors means fewer potential nominations.
There’s also some interesting data related to intra-neighborhood traffic, the visitors going to see artists in their own neighborhoods. Across all the neighborhoods, an average of 32.69% of check-ins came from visitors who lived in or near that neighborhood. However, across the nine neighborhoods represented in our top ten, on average 42.07% of check-ins came from people who live in or near those neighborhoods. In fact, seven of those nine neighborhoods had intra-neighborhood percentages that were above average. It’s possible that this factor combined with a sweet spot (or even the high intra-neighborhood traffic alone) can help to boost nominations in a rally effect. Intra-neighborhood stats have now been added to the raw data, so you can take a look.
We should recognize that the artists who are in the top ten are there for a reason. Sweet spot or no, many artists were nearby (some even in the very same building), but it’s the work of these ten that resonated. Also, we didn’t see either sweet spot traffic or the rally effect cause a “sweep,” or dominance of one area or building in the nominations. Even so, as Sharon and I think about possible future iterations of the project, we wonder how can we avoid the sweet spot and continue to level the playing field.
Simply put, in order to lessen the effect of sweet spots, we need a critical mass. With 1,708 artists, we think we’d need at least 17,000 eligible voters going to at least five studios each. Though we estimate we had 18,000 voters, only 4,929 were eligible to nominate artists; many of our participants were just visiting studios without checking in or nominating. We both feel that the high bar to become an eligible voter is critical and non-negotiable in future; we would also want to continue encouraging all kinds of participation (voting and non-voting). Given what we saw this year, we think the metrics to get us to statistical significance and wisdom of crowds decision-making would be a tough uphill climb at any point.
As Sharon looked at the results, she found interesting work not only at the top, but also among the middle of our long tail (and beyond), so as we think about future iterations of the project, we find ourselves asking about participation models that could surface even more work for consideration.
The good news is, this first go round has given us an incredibly rich place from which to start. As we consider the future of GO, we can experiment with various ways to move forward using this year as a solid foundation.
