Experiments

Here I will infrequently post some snippets and info about the things I'm working on. Which is mostly related to my UI framework, Meek. @marcus_eckert

[marcus-experiments]

On to the web!

I'm currently spending A LOT of time redoing my website and trashing a lot of experiments in the process. I’ve rewritten a lot of ideas of Meek in JavaScript and am now using it to see what I can do with it on the web. Here is one of the canvas experiments that I probably won’t use for the website.

http://marcuseckert.no/experiments/motion/motion.html

This stuff is not browser tested at all, btw!

[marcus-experiments]

Questions and answers to questions on Meek

So, it's been a interesting couple of days with a lot of people reaching out, asking questions, checking in and being all and all fantastically supportive and kind. Thanks for that!

So let me address some of the questions that keep popping up.

A lot of people have asked about the roadmap for Meek. At this point I don't have any plans to open source it or to commercialize it. Sorry. The design of the framework is still somewhat in flux and I'm still making changes to the foundation, albeit the contours of a viable logic for the scene graph and the object responsibilities are starting to emerge. But obviously, it's a shitload of work and at some point I should probably figure out if I should invest more time and what that would look like, but I guess that's more a "personal problem". And then, who knows, it might show up in one way or another. But it's not something I'm actually thinking about right now.

Some were also wondering what the point of all this is. Well, the framework places a premium on the user being able to define meshes, points placed in 3D space that make up triangles which in turn make up more complex objects (like a curtain mesh or a carpet mesh or circles or anything really). You can outfit these objects with certain properties and shaders that influence the appearance and behavior of said objects and this allows to build things that you traditionally don't see in interfaces. UIKit, for example, revolves around rectangular objects and does most of its heavy lifting on the CPU (as far as I can tell) which definitely has its benefits and makes some things a lot easier, but it also poses a number of limitations which I'm looking to circumvent with Meek.

Meek does a lot of work on the GPU, is written in C++ and based on OpenGL, so it's also very much geared towards performance. I'm trying to use as few third party plugins as possible and if I do use any, I try to use a solution that works on many platforms. At the end of the day the idea is to make this cross-platform. I started developing it on Android and Windows but have now shifted more towards iOS because I just know a lot more about iOS. But I do intend to bring it back to Android at some point, although it's certainly in a distant future at this point. 

For guides, tutorials and the like... it's really difficult to do. The stuff on this blog is basically the distilled form of what I've been doing the past 2 to 3 years. It's in a sense the journey of a million little steps. Tiny things and ideas building on other tiny things and ideas that eventually form a bigger whole. So you can't take just the curtain mesh for example and explain it (or sell it) as a closed off system, because it isn't a closed off system and ties in with the whole framework. I do hope, though, that I'll have some more time in the future to give you guys more insight into how these things work and explain things a little better. We shall see.

If you have any specific questions, e-mail is usually the way to go. Sometimes it takes a while for me to get back because... life. But I generally do try to answer the mails I get.

[marcus-experiments]

One more curtain mesh...

It should probably be said that the things on this blog are not exercises in design, they're technical exercises . And they're experiments more generally, hence the name.

The curtain mesh for example was not something I thought about putting into an app, actually. Because I do think it's somewhat tacky and generally a bit much. You could probably get away with it in an app for kids or something. It is, however, also satisfying to use and people seem to like it, so that's in a sense the ideal outcome of one of those posts - to rethink things. 

The curtain mesh has quite a lot of variables one can change and experiment with. You can obviously increase folds, change their subdivisions vertically and horizontally, change the way their width changes from left to right, change the way they form, the way how the curtain bunches up and many more things. Sometimes things end up looking unrealistic but still sort of interesting. Here is one of those examples.

And one more. Playing with the curtain mesh settings. https://t.co/I4o0JdJ8Ad

— Marcus Eckert (@marcus_eckert) 25. März 2014

[marcus-experiments]

Curtain Mesh with Video

The curtain mesh is probably a lot nicer to use with video. It sort of struck me how few things people do with video in the UI department. Obviously, video is somewhat resource intensive but I did find it to be still surprisingly performanent to map the video onto a mesh and play with it. There's still a ton of crazy things one could do with video and meshes, and I will probably revisit this at some point.

The curtain mesh revisited, this time with video. Running on Meek for iOS. https://t.co/2w1EmTRuwx

— Marcus Eckert (@marcus_eckert)

25. März 2014

[marcus-experiments]

Curtain Mesh

A colleague showed me this concept pic and I thought I might try my hand at actually implementing it. It was surprisingly straightforward and way less work than the last experiment. There's one more iteration of this to come.

Edit: Here's the concept pic. Found it.

Playing with a curtain mesh on my framework Meek for iOS. https://t.co/6LhzuPcr34

— Marcus Eckert (@marcus_eckert)

17. März 2014

[marcus-experiments]

Different kind of carpet mesh animation

It's the last post on this stuff.

Different colors and another type of animation. https://t.co/h47euedVru

— Marcus Eckert (@marcus_eckert) 11. März 2014

[marcus-experiments]

More Carpet Meshes

Here's what a whole animation looks like.

Animating a whole carpet mesh https://t.co/kOuQThbtad

— Marcus Eckert (@marcus_eckert) 11. März 2014

[marcus-experiments]

So, here is a more in depth video about this carpet mesh builder GUI. It's one of those things that are impossible to do in AE but reasonable to do in code (well, ok...), so I thought it might be a nice test scenario to build the framework around. 

The GUI for creating this animation touches on a lot of things that are vital to building a standalone UI framework, for example some of the not so easy gesture interactions, a way to save information, it needs proper aliasing (fake, in this case), shader handling, transparency sorting, normal creation, touch handling and a bunch of other things that a UI framework can't do without (well, normal creation maybe it could do without or the transparency sorting). 

The final animations are not of a quality I would put in an app. You would need to make the timing better and you would also need to have some non-fake anitalising for the 3D edges, which is actuallly mostly trivial but beside the point here.

So the GUI basically allows you to trace a picture and place a kind of hose-shaped mesh around it with bones in-between the segments that will later be used to drive the animation. You can save everything to an XML format and you can subdivide and smooth the mesh.

There are two basic modes in which the final mesh can then be used. You can animate the rotation of the bones and then it will curl up. But you can also animate the scale of the bones and then it will look more like a brush painting-in the paths (there are better ways to make this latter animation, though). And then there is the time when I misplaced the bones and it just randomly looked like cloth flying away. So I also included that in the video.

[marcus-experiments]

Animating a carpet mesh

So, that's what the animation looks like. There is another kind of animation it can be used and I'll most more on that in the next couple of days. 

And that's what it does:) Among other things... https://t.co/xBjM8Iso6W

— Marcus Eckert (@marcus_eckert)

10. März 2014

[marcus-experiments]

Carpet Mesh Maker for Meek

Som here's what the thing below is for.

Creating a carpet mesh for Meek on iOS https://t.co/T78mSkweWR

— Marcus Eckert (@marcus_eckert)

10. März 2014

[marcus-experiments]

This is the GUI for a specific type of animation I want in my upcoming app. An app that I haven't started, yet, just, you know...

Anyway, it runs on Meek in C++ and iOS. Not only does is facilitate making the animations I want to make, it also served as an elaborate test case for my framework. I used it to add functionality, to test functionality, to test performance and so on and so forth. 

The icons and the general design - it's just intended to be functional and for me only, really. So be aware.

I will post a video and some more info on what it actually does in the upcoming days, hopefully.

[marcus-experiments]

Here’s a walkthrough through the main parts of the prototype. It also shows the two log-in animations and the commenting system.

For a proper app you would obviously flesh out the design a bit more (and replace the placeholder smileys etc) but for the time I had I think it came out pretty nicely.

[marcus-experiments]

Prototype 3

 And here's an impression of the spinners in the app. Just some details, you know.

last part: spinners https://t.co/zvOa2QXouv

— Marcus Eckert (@marcus_eckert) 20. Februar 2014

[marcus-experiments]

Prototype 2

Here's some examples of the transitions in the app and the way panels react to touch.

more playfulness https://t.co/xBHqQUvNBS

— Marcus Eckert (@marcus_eckert) 20. Februar 2014

[marcus-experiments]

Prototype

Some footage from a prototype I did a couple of months after my game was released. It still uses the old version of Meek running on Objective-C. The new stuff will all run on C++.

The app was basically a video viewing app and I was supposed to come up with the interface, design and some nice interactions. There are a couple of nice examples in there, I think, the scrub indicator for example.

From the archives: prototype from a year ago https://t.co/jMMICqUJ84

— Marcus Eckert (@marcus_eckert) 20. Februar 2014

[marcus-experiments]

More AE Shapes to Meek

Here's a bit from the mesh behind it all. The bezier points are the actual points from AE, they don't move because only the one point in the middle is really animated, but in theory (and practice), they would:)

Some more playing around w AE shape animations on Meek for iOS... https://t.co/GjOsE0KwLv

— Marcus Eckert (@marcus_eckert) 9. Februar 2014

[marcus-experiments]

After Effects Shapes to Meek

When I made my game (äähhh, Wide Sky) I got asked a lot if I somehow had a pipe into After Effects. I didn't and I still don't, really. 

I can do a lot of things in code that I can do in AE but often times doing things in code is almost as fast, with the advantage that you don't then have to replicate it yet again. It's also easier to change parameters on the fly and there are also a lot of things AE just doesn't do but that you can do in code (animating meshes, for example).

Making a "proper" pipe from AE to native code would come with quite an overhead and you could obviously only port some very small parts of AE functionality to work at runtime on a mobile device. I don't think it makes a lot of sense, in most parts.

But some things obviously are much easier to do with a GUI. Animating bezier paths is one of those things. And that's why I made a small pipe for that specific thing. The AE part is a script that basically grabs the bezier points from AE, along with the keyframes and the associated interpolation and converts it to data that my framework understands. 

The goal is to make the animated objects infinitely scalable and the animation adaptable. You could for example do one of those "pull to refresh" things which wouldn't really work with prerendered animations since they're only defined at the frames you rendered out. You could also grab the bezier points in code and then throw them around according to user behavior (scrolling etc.).

The shapes are also outfitted with fake antialiasing and are then rendered on the graphics card.

The trickiest part was probably finding out what AE means by "influence" and "speed" when it comes to interpolating the shapes between keyframes. Properly triangulating and filling the shapes -> also not that easy.

In the video below you can see what happens when the step at which the bezier paths are interpreted is being changed. 

After Effects Shape Animation to Meek on iOS pipe. Allows to create shape animations in AE and dynamically ... https://t.co/zVsBr7ZN4O

— Marcus Eckert (@marcus_eckert)

9. Februar 2014