#vtuber tech

[ziiro]

Been reworking my vtuber model lately. First it was materials to be shiny (from VRM format to VSF format), and then it was getting the eye tracking/expressions dialed in with ARKit. I'm pretty happy with the state of it.

The eye is tracked to one of my eyes, so I can look around, blink, squint, widen. Very exaggerated, very animated.

Not shown in this is that the eye also flickers when I'm talking with different shades of red.

[gremlingirlsmell]

i was trying something new, there's a facial tracker now in NVidia Broadcast. and a program that can translate it into iFacialMocap protocol

it looks really cool and promising but when i connected it with VSeeFace it was lagging like hell and could only track the general facing direction. no expressions whatsoever. but that might be a me / my model problem

buuut it's cool that an iPhone might not be required in the future for detailed tracking!!!

[ziiro]

I got an ultraleap leap motion 2 last week, I preordered one on a whim over a month ago when they announced it because I was looking at one to tinker with anyway.

I am not at all impressed with this device. For something that should be very good at dedicated hand tracking, it's completely inconsistent and nearly useless outside of broad gestures.

The tracker has decent idea of where your hands are and oriented, but it stops there - it usually has no idea what your fingers are doing outside of a mitten-like vague idea of what they should be doing. (Fist? Open? Curved digits?) You cannot raise individual fingers while completely curling the rest.

The software is also pretty bad - I have to run it in inverted mode in order for it to detect my hands at all. I'm hoping this is something they can fix over time with updates.

Also the USB C cable they provided is like 3ft long. Why.