I spent this easter running through a ‘quick’ photogrammetry workflow. I’m interested in what I can manage with free and open source tools.
Some ramblings on my process after the break :)
I took a 45 second video on my phone and with a vlc filter split it into 104 very poorly lit, framed and arranged photos. I’ve been trying to find a tool that works well for this sort of thing. Lots of options, many of which are very spartan research code, a few very expensive proprietary offerings and a few that seemed to fit my bill.
Regard3D is interesting and I’ve had some minor success with it once before, this time I used COLMAP and discounted VisualSFM due to licensing limitations.
Generating a sparse point cloud is easy enough in both programs. If you’ve ever tried to run a surface reconstruction of a sparse point cloud though you have probably seen the blobby monstrosities it creates.. This time I decided I wanted to bite the bullet and run a proper dense point cloud. It took around 5 hours for everything to solve in COLMAP. I’m going to investigate different settings and try and speed things up. The iteration time on all these settings is pretty rough, but the more I work them out the better that will be too.
After hours of stereo solving in COLMAP, an hour fusing, twenty minutes meshing I finally had something to look at. The cloud was pretty spotty and the mesh was bad, so I took the dense cloud into meshlab for tidying up. I’ve acquired a lot of respect for the litany of things meshlab can do, after you get past the clunkiness of it compared to most other 3d viewers.
To tidy up my dense cloud I used the transform tools to get it aligned to the grid, set camera to ortho and snapped to side views. I selected loose noisey data as i went and deleted it. When I was done I had most of a frame.
The dense cloud isn’t joking with its name, it is dense. 4 million points are too many to manually marquee select my way through. I ran a poisson disk operation to get some resampled points and had a go with meshlabs surface reconstruction. Part way into this I wasn’t getting the results I was after and switched over to Cloud Compare, another program I only just learned about a couple days ago. In the end I just used meshlab to align the dense cloud and crop it to a managable rectangle of points.
Cloud Compare has a nice denoiser. It did a good job at fixing some of the stray points around the nose and eyes but did end up destroying some data in the border and mouth.
When I was happy enough with my cloud I tried CC’s implementation of poisson surface reconstruction which after a few tweaks produced a mesh I was happy enough with. I exported this mesh as a PLY to preserve vertex colouration and opened it in instant meshes. I solved for a medium poly reconstruction and exported it out.
In blender I imported both the decimated mesh from instant meshes and the original ply. Some quick and dirty UVs on the lowpoly and a vertex group masked decimate to reduce the non mask geo I got some baked of vertex colour, tangent normals and AO. I decided finally to send these meshes to substance painter and use the ID map baker to get the vertex colours and apply them via fill layer to the base colour.
Overall this has been an interesting process. I’ve learned a few things about making sure my dataset is worthwhile and am going to further tweak the settings in the dense reconstruction stage, as well as try the other solver in COLMAP. Next up, I want to get outside and find some nice ground to photograph and solve (especially because substance designer has some cool scan processing i’m keen to try)!
