2cloud Blog

The blog of the 2cloud team. Announcements, ideas, previews, and other fun stuff will be posted here.

[2cloud]

Congratulations

We would like to wish congratulations to our very own Paddy Foran for his wedding today!

[2cloud]

So who wants an App.net invite?

Email [email protected] if you want an invitation for a free App.net account!

[2cloud]

This one screen will generate more hate mail than anything else we do.

[2cloud]

Guess what we're working on today at Second Bit HQ?

Peter, Peter, come out and play!

[2cloud]

Awesome Friendly Buffalo Person Nick Quaranto posted this on the Signal vs. Noise blog, and we had to share.

We've thrown out and rewritten our server seven times over the life of this project. We've drastically changed our technology. What remains the same is the loose community that has slowly built around the service.

That wonderful, awesome community.

[2cloud]

We've been talking a lot about the focus of the project, its future, and what our vision is. Dylan then got to work on a cool new interface for the service. While these are still just his rough first passes and things are likely to change, we're incredibly excited about them.

[2cloud]

Where our parent company is silly and ridiculous and lacks professionalism. Because we felt like it.

We didn't do it on the 2cloud account because almost 500 people follow that, so it didn't seem like a good place for experimenting.

Anyways, we'll stop talking about Twitter and go back to work. But now you know.

[2cloud]

[2cloud]

You Are Who You Are

One of the complaints we heard longest and loudest from users of 2cloud was that the authentication flow was terrible. That's because it was. It was a hack: browser redirects leading you through an OAuth 1.0a dance, because that's the only option App Engine gave us.

We've been working on the 2cloud server stuff for a while now, and left the authentication as OpenID for all this time. It was all we needed to test the service out, and we reasoned that we'd come back and add in whatever we needed to support Android later. We never planned on leading users through a crappy, hacked-together browser flow, but we didn't know what server-side code was required to use the tokens returned by Android's AccountManager—the built-in list of accounts you've added to your device.

Tonight I took the time to find out, and have successfully implemented a test application that will retrieve a Google account token from your device and send it to our server, which will match it up with your account, create an account, or report an invalid token. There's a lot to be done on it still, but it's a very strong foundation to build on. Logging in to your device will be as simple as selecting the Google account you want to use from a list.

I worked hard to make sure your Google account would be matched across both OpenID and OAuth2 authentication schemes. The same Google account gets you credentials for the same 2cloud account.

I also laid the groundwork for future authentication mechanisms. There doesn't seem to be a good reason that you can't, for example, use your Facebook account to log in. Or your Twitter account. Or even your custom OpenID endpoint.

For our initial release, we're going to only accept Google accounts. As we iterate, however, we're going to work hard to make sure that you can authenticate yourself in whatever way you choose. You are not a Google account, and we know that.

[2cloud]

What's Going On?

It has been a while since we've issued a missive from Second Bit HQ about the status of 2cloud. We had a private beta, an alpha, really, in early December, then shut up. It's six months later--shouldn't we have something to report?

Yes. Yes we should. So here's a long-overdue report.

Let's talk a bit about what we're working on and trying to accomplish. We've made the decision to leave behind our previous platform-of-choice, App Engine. We love App Engine, and think the world of it, but it wasn't a fit for 2cloud. We grew a lot as a project on App Engine, and we helped App Engine grow, but it's time for us to part ways. The reasons we're leaving App Engine behind come down to user experience. Too many crucial parts of the application are completely out of our control on App Engine. I've had to reply to too many bug reports with an apologetic "nothing we can do, just try again" message. If we're going to ask users to pay us for this service (we are), we need to be able to give them a user experience that is worth their money. If we can't guarantee that, we can't take their money. If we can't take their money, we can't keep the server online.

Once we made the decision to leave App Engine, a lot of the problems the project faced were solved, replaced by scary new problems:

Servers crash, systems fail, things go wrong. This is a universal truth. No project is without points of failure. Before, on App Engine, other people took care of those failures. Now it's our job. We've been busily working on systems that are designed to be resistent to failure. All of our systems are designed from the ground up to enable redundancy. We've been coming up with plans and tools that will notify us when things go wrong and will let us react and recover in a timely manner. We can't promise perfect uptime, but we can build a system that is designed well from square one. This is not an easy undertaking--I'm not a systems administrator, I've never done devops work before. I'm learning as I go along, and some of the best have been kind enough to give me advice. I've gotten disaster recovery training, advice on redundancy, deployment strategy advice... We're learning to make deploying quick, disasters rare, and recovery timely. All of this takes a lot of time to learn and build an infrastructure around.

There are some bad people out there. Our users aren't just entrusting us to keep the service running, they're entrusting us with their data. Things like usernames and passwords, credit card details (those really make us nervous), and even the links being sent--these are all targets for data thieves, and we don't have the brilliant minds at Google protecting them anymore. We're busily running security audits on our code and trying to come up with new ways to keep your information safe.

You're still the same you. Just because we're leaving App Engine behind doesn't mean that you should have to stop using your Google account to log in. You have too many accounts and passwords already, you don't need us adding another. But we no longer get the benefit of built-in access to those accounts; we need to implement our own sign in process to handle that. Figuring out how to allow you to log in using your Google account, while keeping your account credentials secure, was a bit of a challenge.

All these things take time and software to do. We're busily writing open source libraries for different APIs and new tools for managing and running cloud infrastructures.

It may seem like we're ignoring the lean startup methodology of launching a minimum viable product. That's true, we are. We're taking the time to build an infrastructure that most would consider extravagant for a launch. We're doing it for good reason, though: there are thousands of you waiting to hit it. We don't get a soft launch, we don't get a slow and steady growth curve that lets us ramp up our infrastructure as demand grows. From day 1, our infrastructure has to be able to deal with an onslaught of users.

The second part of this is the people involved: I'm currently working for a fantastic company, which consumes quite a bit more of my time than school did. My business partner, Tino, is currently finishing up grad school, and so doesn't have a whole lot of time. The time available to us to engage with the community, answer support questions, and work on the numerous problems we face is limited.

So we beg your patience. We beg your patience as you wait for us to release. We beg your patience as we take a few days to get to your support request. We beg your patience as you run into that bug that seems like it will never be fixed.

We see all those things. We want them done as badly as you do. But we're not willing to trade today's problems for a new batch tomorrow by rushing a release and botching it. We know we're losing users as some of you drift away to new solutions. That's okay. We understand. Do what's best for you.

But we need to do this right the first time, and we're not taking any chances with that.

[2cloud]

Magical

Since I wrote the first version of 2cloud, I've always had a favourite moment. It's the moment where things first start working, where I pick up a random device and use it to tell another random device to open a link. And it does so. Instantly.

That's magical.

I just finished my third pass at writing a system (the first used a particularly naive polling algorithm that made servers cry, the second used Google's Channel API, now I'm using WebSockets). I know how the sausage is made for each of those: for the first, it's an insane amount of HTTP requests to a specific URL. For the second, it's a combination of an insane amount of HTTP requests and some black magic with keeping the connection alive, among other things (XMPP gets worked in there, too). For the third, it's a protocol that explicitly allows what I'm doing, that was built for what I'm doing, that both my webserver and my browser support. Nothing in these three solutions is more magical than the Internet itself.

And yet, every time I get the first google.com link to pop open on some device across the room, it's a huge rush and I inevitably end up showing anyone who will watch, bemused at my excitement.

A lot of people have asked why we don't just offer a free version that doesn't pop links open automatically, which would drastically reduce our costs and allow us to keep a majority of our userbase. I'm not ruling that out, but this is my big objection. The magic is a large part of what I love about working on this; if users don't get a chance to see that, I've denied them something wonderful.

[2cloud]

New Blog!

As we move towards the Bessie release, we're taking a look at how we're managing the project, and making changes where appropriate. One of those changes has been the establishment of a new blog.

We're hosting this on Tumblr, so it's easier for us to do some fun things--things like posting previews of upcoming features, things like posting videos. It also makes it easier for any of you who have Tumblr accounts to follow along and share our posts. Finally, it lets us answer questions you may have, along with all of Tumblr's other social features.

The old blog will continue to function for now. We may phase it out at some point in the future, but we'll be sure to post if that's happening.