Jeesty Endeavor

Cutting edge cloud apps and strategy from Austin Texas the live music capital of the world.

[jeesty]

Docker, Mesos, Spark &c

By now you've heard about Docker, but you might not know that Appsoma has been using it since the very early days. Scalable process isolation is critical, and combining Docker with Mesos helps achieve scale for both persistent and batch applications. The last ingredient is real-time processing, achieved with Spark, suitable for everything from DAGs to SQL to Machine Learning. Yes, we're looking at a Lambda architecture, and just starting to investigate Kappa...

[jeesty]

Big Data Analytics is "Software" not "Scripts"

Sure, you can hack together a bunch of scripts to process your data - even "big data". You get a result the script just... hangs around. What if you want that analytic processing to be re-usable, or to reproduce your results later? What if you need to share it with others, guarantee it's stability, and upgrade or derive from it over time? Suddenly it isn't a throw-away script, its "software". And software means a lot of things: version control, stable releases, easy publishing and validation testing. For analytic software it also means reproducibility and a clear record of exactly what was done to get those results. Not to mention compliance to internal or government standards. Scientists and scientist/programmers are seeing more and more data coming their way, processing it with with ever-more-complex analytics pipelines. Some of them are already well down this path, while many more are still learning. As the analytics complexity grows a "software" orientation will be one of the key tools to keep it under control.

[jeesty]

Big Data - you still have to move it around...

Maybe you have a big fat pipe. Maybe Google fiber is coming to your area (it's coming to mine). Or maybe the prospect of moving 4tb of data is just plain daunting. Sure, technologies like S3 browser (which does UDP blasting) and Aspera (proprietary) help you utilize the pipe you've got more efficiently than pure TCP alone. But you're still stuffing an elephant in a drain pipe. This is a key challenge for bioinformatics computing, and a key obstacle to adoption of off-premises technologies. Recognizing this, Appsoma enables hybrid computation - on premises or off, using bare metal or cloud or grid. I would argue that companies planning on having you upload all your data to them simply... assume to much.

[jeesty]

Security is the Cloud's Missing Link

Back in the 1970's when I was a child I knew a fellow at Bell Labs. He was working on the problem of security in Unix. All sorts of innovative ideas were flowing: root users, sudo, authentication certificates. In the late 1990's the web faced security as a fundamental problem, and Certificate Authorities arose as a key solution, making web browsing of sensitive data a possibility. Here we are in 2013 and the cloud faces the same problem: Virtual Machines are great, but they all use 1970's security methods, that is, they know how to secure themselves but root access to the machine exposes everything. Enter Cloud Passage - a company that acts as a kind of certificate authority by centralizing security and policy away from individual VMs. Nice solve - and a critical piece of the Cloud puzzle.

[jeesty]

Biosciences and the infancy of cloud and big data

Certain biosciences companies, especially those working with genomics or proteomics, face massive data processing challenges. Although the cloud can help them manage costs, the overall immaturity of the technology means humans still do a lot of manual tweaking. Ping times from high CPU cloud farms to big data repositories should be minimal, yet the methods for making this happen typically involve plugging cables into physical boxes and hand-optimizing proprietary database query code. Not very cloudy, and not very easy. Biosciences is missing a magic third ingredient at the intersection of Cloud and Big Data.

[jeesty]

The old cliche "The customer is always right" has subtle tones where software development is concerned. But one durable theme is that your product hasn't maximized it's Quality (ability to serve customer needs, ability to be sold, etc) until customers have used it. Better still, put yourself in their shoes by watching them use it - then you'll really know. How quickly are you moving to top quality?

[jeesty]

Entrepreneurs are learners...

Are you "learning at the speed of pain"? Or are you a great listener, seeking the advice of experts and using "smart trust" to rapidly make decisions? Buzzwords aside, these are crucial concepts for entrepreneurs: When you can't know the outcome, make the lowest-cost mistake and learn from it. Where on the scale do you fall?

Not Learning Expensive Mistakes Lean Mistakes Learning from Experts

[jeesty]

Curation is the new Search

What delivers better results than search? For many things the answer is "curation."

Pinterest has already proven the point, and now a flock of companies are following suit, using curation for everything from finding great wine to music to art.  Search has been  fragmenting, with sites like trulia taking real estate search share, yelp for local food/entertainment and of course youtube for video.

How granular will serving people's search needs become? Have you seen startups aiming at curation?

[jeesty]

Fall seven times, stand up eight.

[jeesty]

Shibboleth unites, and distresses

Did you know huge federations of institutions (mostly educational) accept log ins from each other? That system is called Shibboleth, and I recently helped a cloud coding provider connect into that "federation".  But it was a very distressing experience!

So here is a tutorial to help you get through Shibboleth installation, with a sample Ruby/Rails/Passenger implementation as well.