Integration Day
On Monday morning we got working as a group on the integration stage of Kitchen Hands. For the uninitiated, this means integrating the ZED camera hands with the Vive camera and the rest of the game scene. This was actually our second integration day, but our first one was entirely taken up by getting past windows errors and other mishaps, so I won’t bore you with those details.
Things traveled much more smoothly than I expected. We were actually able to get a working prototype running pretty quickly, and so had a lot of time to get some much needed experimentation going; figuring out some green screen details, lighting settings, and some physics interaction details.
Initial Implementation
As mentioned in one of my earlier posts, the ZED camera hands implementation has some finicky details in rendering to different displays (see the tail end of this post). This issue really bugs me, but it hasn’t been pressing enough to warrant exploring a solution just yet -- this is just a prototype after all, and a “stuck together” solution is just fine. I digress. After fluffing around for a while, I was able to get the scene “working” just fine though.
Sorry for the lack of quality screenshots, it was a long day.
Findings
Something to notice is that the hand doesn’t look anything like it should. Recall the original hand in my example was much more true to life:
The problem here is that the chromakey calculation used to discard the greenscreen pixels is quite poor. As a consequence we had to fiddle a lot with the brightness and ended up sacrificing a lot of the fidelity in the process. In fact, a black screen actually performed far better than the green screen, but I’ll talk more about fixing the green screen in a later blog post.
A further consequence of the greenscreen calculation that I hadn’t really thought about is that it’s really bad for performance when the calculation is wrong. The problem is that the problem space (number of pixels / vertices) dramatically increases when we don’t cull the greenscreen. All the more reason to get this right.
The final thing point of consideration was of course the performance. The commonly accepted framerate for a VR application is over 90 FPS. We’re not quite there yet, especially with the hardware of the test machine (GTX 1070). It was a little disorientating running at ~30 (with low settings), and definitely not ready for long periods of use. There’s always work to be done on this front, but to me this just emphasizes that it’s an important thing to work on.
You can find a video of our integration testing here.














