Building Ducky

The development log from the people behind Ducky.

[useducky]

Long time, no post!

We’ve had a busy *checks* three months (sorry!) and so haven't had too much time to work on new software (or, evidently, write posts here).

Here's what has gone down:

App Engine shut off its OAuth1 support. Which means we had to roll out an update for 2cloud to move it to OAuth2. That update is pending right now, and should be out tomorrow. Whee!

We wrote about our horrible track record on shipping. TL;DR: if we don't have at least a prototype of Ducky by 1/1/16, we're going to refocus our efforts on actually maintaining 2cloud, and reexamine our plans for Ducky. Which I, personally, have no desire to do. So here's to hoping that we can have a functioning prototype of Ducky in the next 7 weeks. It doesn't have to be pretty, but it does have to work.

I live in Washington now! That's pretty cool, right?

Because I live in Washington now, that means Second Bit has to become a Washington LLC. And because the jerks that own secondbit.com want thousands of dollars for it, I'm taking this opportunity to rebrand. Second Bit is about to become Impractical Labs, LLC. There's a story there, and maybe I'll tell it some day. Not today.

Once I get this "update 2cloud" stuff out of the way, I'm going to set my sights on a working prototype of Ducky for ASAP. The amount of work involved in that feels scary, but I could probably get a prototype out in like two weeks, if I had to. I'm just stubborn about maintainability and not putting junk into production. So I'm hoping I'll be able to put mostly well-engineered stuff in production before I call it a prototype. We'll see.

[useducky]

So last time we talked about our new subscription flow, and a bit about events.

Then we went silent for two weeks.

That’s because at the end of August I’m moving across the country (which I expected to be doing at the end of November, instead, and it got moved up a bit) so Ducky development has stalled out a little. It basically is reduced to those selfish hours of time that I can steal for myself to develop software I can believe in. And those blocks of time are getting rarer and rarer.

Fortunately, after the move, we go back into Ship Mode™, so we should start cooking with gas again in a few weeks. Apologies for the deadness until then.

When we come back:

We need to get our PostgreSQL backup working with our hosting provider, which is something I’ve been talking about for roughly a year now. It’s not an unbearable amount of code, so I just need to buckle down and write it. Though I’m not looking forward to telling the project maintainers that they need to adopt a Code of Conduct before I’ll contribute the code back to them.

Kubernetes 1.0 was released, so we need to get a cluster stood up to use that, which means resolving the code storage stuff I wrote about previously.

We need to get NSQ set up on Kubernetes, now that we have all the tools. I’ve written most of the software needed to accomplish this already, it’s just a matter of tying it all together.

Once all that’s accomplished, we have all our infrastructure worries taken care of, and we’re cooking with gas. Then we just need to finish application development, the Chrome extension, and the Android app. I’m still hopeful for a release this year.

[useducky]

Last night we managed to get events set in order, which allows us to do some very neat stuff on our backend, and makes our infrastructure so much easier to manage.

We’ve still got to create a listener that will create a subscription for the user, as we talked about earlier. We’ll also need a listener that will update our subscription for us, when an email address gets verified, so the user automatically will receive receipts. I’m a little bit worried about the whole “adding a subscription” thing, because it can have a race condition.

See, NSQ is what’s called delayed processing. It usually is pretty quick, but the subscription may not actually be created for a minute or even a few minutes after the account is created. If the user enters payment details and selects a plan in that time, we can’t update their subscription with the new information, because they don’t have a subscription. Whoops.

The way out of this may be as simple as:

When the account is created, we send the message.

When the listener receives the message, it uses the API to create a subscription.

When the user enters their payment information and selects a plan, the system tries to make a new subscription for them. If that fails because a subscription already exists, they ignore it and move on.

That way, each user is guaranteed to have a subscription relatively close to the time their account was created. We could even update the subscription to start when the account was created. (Note to self: do that). But that’s the important part, anyways. It doesn’t matter if we try twice, because only one will succeed, and we can just update instead of creating a new subscription. I think that’ll work nicely. Good talk, everyone.

In other news, I did have a pretty great idea: if we use a lua client for NSQ our auth proxy (nginx) can push a message to NSQ every time an authenticated request goes through. We can then have a listener retrieve these messages, debounce them, and use the API to keep track of when people, their devices, and the specific login they used were last active, which helps us sort things sensibly so the most-used ones float to the top.

So that’s exciting. It’s always great when you find new and exciting ways to do cool stuff, and I love it when an architecture just comes together.

I know I’m updating this blog a lot over a short amount of time. Don’t worry, it won’t last.

But hopefully it’s enlightening: a lot of work, planning, and careful thought goes into how these things get built.

[useducky]

For those of you not up to speed, and since I’m jumping into this story while it’s already in progress, let’s do a quick catch-up. Right now, we have figured the following stuff out:

The main thrust of this thing will be the Android app and the website. Chrome packaged apps are woefully inadequate for building an actual application with, so we’re going to stick with what we’ve always had: a tiny Chrome extension that doesn’t offer any sort of onboarding experience or anything. Just log in, receive links, and send links. Android and the website are how you’ll set up and administer your account. Which is not what we wanted, but them’s the breaks.

We’re doing monthly or yearly subscriptions, in a choose-your-own-adventure kind of way. By which I mean we’ll have plans, and you’ll get to pick which one you want. Yaaaay.

So here’s where we are with that:

That’s our registration page! It works! You can sign up! I know the buttons are mismatched, I’m going to work on fixing that I’m just doing other stuff right now.

Once you sign up, you’re asked to set up a subscription and payment method:

And that also works… sort of. It’s not behaving exactly how we want it to right now. Sure, it sets up a subscription in our database, and a subscription in our payment processor, and it’s secure (except that the page isn’t served over SSL do not enter your credit card info into anything not HTTPS) but it’s super easy to use the current setup to get a free trial forever. Like, “close the window when you see this screen” easy. Our free trial starts ticking when your subscription is created, so if we wait for you to enter your payment information before we set up your subscription, you’re in control of when your free trial starts, which probably shouldn’t be the case.

Also, this method means that we need to try to make you enter your credit card 31 days before we actually need it, and that you can’t really use the app before handing over payment info. First of all, ew. Second of all, that historically has given companies problems with their adoption rates. Probably because the “ew” factor.

So yeah. We’re working on updating that part.

We’re also distracted by that site I just linked to, the code.secondbit.org one. It’s the self-hosted version control (if you don’t know what that means, it means it’s where we put our code to make sure we don’t lose it) we set up. Right now, it’s running on a cluster that is only powering it and getsby (which is a community project of ours). Clusters are expensive, so we’re going to put it and the Ducky services all on the same cluster. But we’re trying to update the software install we have to be a bit more elegant, let us do a few more things, and match Mozilla’s setup because their setup is very nice and much more thought-out than ours. So we’re also distracted by that.

Fortunately! We’re nearing completion on both those projects. Which means we get to move on to the actual meat of Ducky, starting with keeping track of your devices.

Unfortunately! I have a super busy rest-of-July, between family-and-friend visits, birthdays, work, and things-to-do-before-I-move. So I may only finish (or not even finish :( :( :( ) those two things.

We’ll find out! It’s going to be an adventure.

[useducky]

Let’s Do This

We’ve been working on Ducky for a long, long time. Too long. We’ve called it a lot of things. But it’s time to ship now, and we’re sprinting towards the finish line.

Sprinting. Crawling. Whatever. Point is, we’re committed to shipping as soon as we possibly can. We feel some urgency.

We were trying to launch a beta by July 6th, 2015. That, uh, clearly didn’t happen. But… well, we thought it probably wouldn’t. But hey, we had fun trying!

So we’re starting this development journal in media res because, frankly, I’m enamoured with the idea of a development journal, kind of like an experiment log. Just to write about what we’re doing, what we’re thinking, what’s working and what isn’t working. Pictures. Whatever.

I just want to tell our story.

Who is “we”? “We” is me. My name is Paddy. I’m the poor sap that has worked on this thing for years now, just because. I own the company making Ducky. And I’m the only person writing its software. Today. I hope there will be more people, eventually.

So that’s it. Welcome to Ducky’s development log. I can’t promise everything from here on out will be fully explained for people who aren’t programmers, but that’t probably OK.