#srslynothipsters

[webtech-cs294]

Hadoop: The Mother of Distributed Computing

Last Wednesday, Eric Baldeschwieler, founding CEO and CTO of Hortonworks and lead architect of the Hadoop framework, talked about Hadoop and the history of its making.  To start, he described Hadoop as a framework comprised of two parts:  YARN, the computation layer where all deployed jobs get evaluated, and the HDFS, the Hadoop distributed file system.  This dual architecture was meant to create a framework that was scalable (able to handle petabytes of data), reliable (resistant to component failure), flexible, and economical.

One of the greatest breakthroughs Hadoop provided was its ability to provide faster, more reliable data storage solutions, at a 100-times fraction of the price.  The core insight was to use commodity computers instead of huge mainframes.  It turned out that buying thousands of computers and organizing them into racks of servers was much more cost effective and faster than designing and maintaining complicated mainframe computers (although Eric did mention racks were potentially more power consuming; there are trade-offs to any kind of metric system).  However, what the commodity-computer-approach gained in speed and cost, it lost to reliability.  Any of the thousands of computers could fail at a time; even a 99% success rate means a 40% failure rate at 50 computers.  Consequently, Hadoop was built from the ground up, taking for granted failures happened and architecting itself robustly against such failures.  At the time Hadoop was introduced, most other competitors did not factor in reliability, which made Hadoop stand out even more.

When Eric lead the Yahoo! Team effort to take Hadoop from prototype to full fledged production software, he designed it to fulfill two main purposes.  First, Hadoop was built to enable fast applications to crawl and index the web for the Yahoo! Search engine.  It was built to sort and organize data, process, extract links and words, and fetch and store billions of web pages, and perform operations on all that data.  And second, Hadoop was built to have monitoring at all levels.  This would allow big companies like Yahoo! and later Facebook and Amazon to monitor everything in real time, react to failures/bugs, and provide personalized experiences (and targeted ads).  What the Hadoop architecture essentially enabled was a way to offload some computational analysis away from the database, making all the information simpler and easier to scale.

While Hadoop may not seem as spectacular today as it did in the past thanks to new competitors like Apache Spark, Hadoop set a lot of important precedents, and remains incredibly relevant today.  It is still being used for advanced computational analysis (think Machine Learning and advanced business analytics).  Hadoop is also open-source, and like other open-source software, takes a guerilla-warfare approach to designing enterprise software, striving for optimal interactivity, speed, and ease for the web.

[webtech-cs294]

Distributed systems with Finagle

This week, we heard from Marius Eriksen from Twitter about Finagle, a utility for managing distributed computing systems. Eriksen’s presentation centered first around how modern systems work. Because of the scale of modern systems, we treat distributed datacenters like a single “computer”. But, as Eriksen noted, they make a terrible computer, for a number of reasons. Not only is partial failure—having some component fail but have the whole system keep running­—possible, it’s also extremely common. There are also a bunch of layers on top the normal ones, like complicated networking.

As a result, frameworks like Finagle exist to manage these kinds of distributed systems. Finagle is a distributed remote procedure call (RPC) system. RPC is a generic term for making the ability to make procedure calls across a network just like on a normal machine. Finagle is intended as a high-performance implementation of RPC, designed to be run across systems and across datacenters. If one makes use of some form of service architecture ­— meaning, essentially, modules that are distributed ­— It allows making procedure calls from one service (distributed module) to another using a uniform and consistent API by abstracting away failures and system specifics. The only downside to such a framework is that it restricts you to a particular set of APIs, and gives you less control; but that’s the case with any framework. Unfortunately, I don't have enough experience with distributed systems to really understand how Finagle compares to other frameworks like it.

[webtech-cs294]

HTTP/2 in Summary

Last week’s blog posts covered the talk given by Brad Fitzpatrick about HTTP/2.

Problems with HTTP/1.1

For over a decade, the internet has been powered by HTTP/1.1, but as web applications become more complex, and reliability and speed become more important than ever, it becomes quite clear HTTP could use some revamping. Here are the main issues that groups highlighted about HTTP/1.1:

HTTP/1.1 might open up too many TCP connections for multiple requests, and this could hurt network performance. For example, every resource on the server (.css, .js, etc.) has to be sent one by one, so there is a lot of unnecessary overhead from communicating back and forth for individual assets.

There is no way for the server to initiate communication with the client, so the client has to start the talking first every time.

HTTP headers are large and clunky, and HTTP/1.1 only supports compression on the HTTP body.

Advantages of HTTP/2

Instead of plaintext, data is sent over binary, which is more efficient and easier to parse. In addition, binary is less error-prone since it doesn’t have to deal with things such as whitespace handling, capitalization, line breaks, etc.

HTTP headers can now be compressed using a compression scheme called HPACK.

HTTP/2 supports preemptive pushing, which allows the server to send the client resources without the client having to initiate communication every time. This reduces the back-and-forth exchange that we see today with HTTP/1.1, thus lowering round-trip time.

Asynchronous requests mean that multiple requests can be handled simultaneously. This removes a lot of the back-and-forth overhead.

Disadvantages of HTTP/2

Binary protocols can be difficult to debug and understand.

Proxies and virus scanners can break HTTP/2 communication until the protocol becomes more ubiquitous.

Currently, the HTTP/2 spec is only supported over HTTPS.

QUIC

Another alternative to HTTP/1.1 is the Quick UDP Internet Connections (QUIC) protocol developed by Google. Instead of TCP (which HTTP uses), it is built on top of UDP. Since it supports multiplexed connections over UDP, provides reduced connection and transport latency. In addition, it was designed to provide security protection equivalent to TLS/SSL.

All in all, HTTP/2 has many advantages over HTTP/1.1, and its problems mostly stem from it being relatively new. In the future, we can expect to see HTTP/2 making the web a faster and better place for users and developers.

[webtech-cs294]

On Wednesday, Brad Fitzpatrick from Google talked about the history of HTTP and the recently-finalized HTTP/2 spec.  Some of the key goals that came out of the HTTP/2 effort was to maintain high-level compatibility with HTTP 1.1 (so that the new spec did not break old functionality), to introduce new Data Compression technologies (i.e. requests in binary instead as strings), to load page elements in parallel over a single TCP connection, and allow server push technologies.  This allowed HTTP/2 to solve a lot of the problems of HTTP 1.1.  For example, in HTTP 1.1, a key problem was that page elements could only be loaded six at a time, leading web developers to come up with hacks such as instantiating multiple servers to load more resources than six in parallel, and cramming multiple resources into one file (e.g. all sprites on one sheet, or multiple files of JavaScript compressed into one) in order to reduce the number of resources to load.  These solutions often were horrible in that they defeated the point of caching (changing one sprite meant re-caching the entire sprite-sheet).  HTTP solved this by introducing connection streams that allowed virtually unlimited parallel transfer of file over HTTP asynchronously, with tremendous latency gains.  Furthermore, HTTP/2 introduced new server push technologies that would allow servers to push content to the client if both the client and server agreed, enabling WebRTC technologies such as websockets to function more naturally and less hacky (i.e. without the need for client polling, etc.).  

One of the biggest criticisms of the HTTP/2 spec that Brad mentioned was that most browsers implemented only the Encrypted form of HTTP/2.  The problem was that often sending HTTP requests through an unencrypted connection was often reliable (note that even if a connection has a 99% success rate, sending 50 packets would mean a ~50% failure rate, which is unacceptable).  So it makes sense that browsers only implemented the Encrypted form—it would mean more reliable communication, which is important especially since new clients and servers using the new HTTP spec would likely be communicating asynchronously and continuously, and continuously dropped packets could compromise performance. 

[webtech-cs294]

Summary: Deployment Tech.

Last week’s blog posts discussed Brendan Burns’ talk on deployment technology.

Strengths

Most groups discussed the advantages of Kubernetes’ declarative approach: rather specifying how to keep a deployed system in a production-ready state, one only specifies what the production-ready state is, and Kubernetes automatically keeps the system in that state.

Some groups mentioned the power of Kubernetes’ networking layer: since each pod is assigned a unique IP, there’s no need to deal with weird bugs resulting from dynamic port allocation.

Almost everyone touted Kubernetes’  integration with Docker as a big plus.

Weaknesses

One group mentioned that Kubernetes’ documentation didn’t really make clear why it’s useful.

Another group observed that the declarative event loop might be less efficient than an imperative system, in part because the declarative approach permits less control over low-level optimizations.

Comparisons

As mentioned above, many groups also discussed Docker, which is one of the foundational elements of Kubernetes: a Docker VM is self-contained, stripped-down virtual machine, that allows users to modularize an app and all of its dependencies and configurations for easy deployment.

Other groups mentioned CoreOS, an operating system (based off of ChromeOS, apparently) designed to incorporate Docker’s design goals into an actual OS, rather than just a VM.

One group compared and contrasted Kubernetes and Ansible. They noted that while the two projects have different goals­—the former focuses on cluster management and the latter focuses on host configuration­—they share a number of design principles. For example, they both use a declarative description of state rather than an imperative one, and they both focus heavily on modularization.

All in all, while relatively few of us have real experience managing the types of systems Kubernetes is designed for, everyone seems pretty excited about the potential utility modularized deployment systems have.

[webtech-cs294]

Kubernetes and the Cluster Management Revolution

This week we had Brendan Burns, staff software engineer at Google, talk about distributed system management and his project at Google - Kubernetes.

Kubernetes is an open source cluster management system that manages containerized applications across multiple hosts. It also has basic mechanisms for deployment, maintenance, and scaling of applications.

Kubernetes Architecture

Here, I will talk about two of the main architectural components of Kubernetes - the API server and pods.

Before we get into the nitty gritty, it is important to note is that one of the key methodologies of Kubernetes is the use of declarative primitives. This allows the system to easily maintain the state requested by a user. For example, let’s say a user requests that 3 containers be running at any given time. If Kubernetes detects that the number of containers running falls below 3, it will automatically spawn new running containers. Likewise, if the number is more than 3, it will kill any extra containers. This declarative nature allows Kubernetes to have self-healing mechanisms that makes it much easier on the cluster maintainer. No one likes being woken up in the middle night because of some server error.

API server The REST API server serves up the main Kubernetes API and allows the outside world to communicate with Kubernetes. Some examples of what you can do are scheduling pods (explained below) to worker nodes and retrieving pod information (where they are, what ports are they using, etc). This is the main hub for orchestration of the distributed system.

Pods Another main unit of the Kubernetes architecture is the pod. Pods are basically a group of tightly coupled containers that are scheduled onto the same host. They serve as distinct, containerized environments that serve as the basic units of scheduling and deployment. The containers within a pod also share the same fate and resources such as storage volumes and IP addresses.

What this means for developers Historically, ramping up clusters of servers and maintaining them have been extremely challenging. Often times, this causes developers to spend more time servicing their server architecture than actually focusing on application development. Up and coming cluster management systems like Kubernetes and CoreOS aims to solve this problem. With Kubernetes, many of the complications of maintaining a distributed system are abstracted away, allowing developers to adopt a more application-centric ideology instead of having to worry too much about their cloud infrastructure. This is an exciting time for developers. Open source systems like Kubernetes are making cluster and distributed system management easier than ever before.

[webtech-cs294]

Leveraging other services to your advantage

Heroku started off as a giant monolithic codebase, with a heavy dependence on RabbitMQ, Noah Zoschke said on Wednesday. And there is nothing fundamentally bad with giant monolithic RabbitMQ based apps, except when your software hosting platform starts getting millions of apps it has to host, and RabbitMQ support starts to taper away. The problem is, while building monolithic software ensures great interoperability and potentially better reliability, micromanaging every aspect of the software (for e.g. database management, infrastructure management, integration testing, and log analysis) is extremely expensive, tedious, and impractical.

As Heroku began to scale, Noah Zoschke and the rest of the guys at Heroku were forced to rethink how they structured and maintained their large infrastructure. And one of the biggest realizations they discovered during the process of restructuring was to fully embrace the “Software as a Service (Saas)” philosophy—that it was more efficient and practical to outsource specific parts of Heroku’s infrastructure handling to other software that specialized in handling that one thing. It was better to use a piece of software that one company spent years maintaining and perfecting to implement a feature than trying to implement the feature yourself an spread your own resources too thin.

The unix philosophy comes to mind: “Do one thing, and do it well.” Heroku was, at its core, a software hosting platform, and foc]us on providing the best Platform as a Service (PaaS) it could be. Implementation details were necessary, and key modular aspects of the infrastructure could be outsourced to other SaaS’s.

To be sure, there are some downsides to using Anything as a Service. For one, there may not be any piece of software out there that implements the exact functionality you are looking for, and hence you will have to perform some tweaking/adjusting yourself (but SaaS is booming, so this will become less likely as time continues). In addition, using an Anything as a Service involves becoming somewhat dependent on that service. Your uptime/reliability becomes dependent on all SaaS uptime/reliability. And if one SaaS cuts support or makes a breaking change to its codebase, your own code could be in danger of breaking.

To protect against the downsides of SaaS, Noah recommends using software that has been out for a while and has had a chance to mature (i.e. the SaaS is production-ready and not in a alpha/beta stage). Furthermore, the SaaS should have an active and responsive technical support group.

Noah and the rest of Heroku believe SaaS is the future of software engineering for the web. Though not without downsides, using SaaS products can ensure excellence in efficiency, and allow a company to focus on implementing its key vision. And Heroku embraces this ideal fully, using its own service, Heroku, to host parts of itself. Inception? Nope, just the SaaS methodology in action.

[webtech-cs294]

Ready, set, GO!

In our last class, we heard from Rob Pike, a Google engineer who helped create—and currently works on­—the Go programming language. Go was released a few years ago, and has since received significant attention following its stable 1.0 release in 2012.

During his talk, Pike focused on the core design principles behind Go. In typical Google fashion, Go was conceived as a language willing to depart from established conventions for the sake of scalability. In this case, scalability meant a language could be used in projects with huge numbers of developers and high performance requirements. As a result, Pike’s focuses have been twofold: readability of code, and performance of the language. The Go team looked to occupy a sweet spot in code readability: make the syntax verbose enough that it’s immediately clear to a reader what the code does (unlike, say, Lisp), but not so verbose that it becomes cluttered and tedious to type (like C++). This led the Go team to introduce a few features in the language not seen in most other mainstream languages. First, there are syntactic elements: type declarations must follow the expression, for example. This is a simple change that improves the readability of complicated type declarations—an example taken from the Go website, for example:

int (*(*fp)(int (*)(int, int), int))(int, int)

becomes:

f func(func(int,int) int, int) func(int, int) int

There are also semantics: only capitalized names are exported from a module, and every import is namespaced.

Go also made some important strides in terms of performance. The most obvious (and extremely useful) feature is the concurrency support baked into the language. Whereas in other systems languages concurrency is an afterthought (because many of them were conceived well before the rise of multi-core processors), Go has the concept of goroutines, which are a kind of abstraction over threads, built into the language with the “go” keyword, which runs a function in another thread, and other keywords for joining and managing threads. The Go team also made some major improvements in order to dramatically cut down compile-time in large projects, most notably by reducing the fanout of Go code; that is, the ratio of bytes of human-written Go code versus bytes sent to the compiler. This was achieved by reducing code duplication amongst modules that import the same dependencies.

Having played around with Go for some toy projects, the buzz around Go seems well-founded. While some changes are initially disorienting (re-ordered type declarations, for example), you quickly come to get used to them, and eventually even appreciate them.