#pao305

[webtech-cs294]

HadoopMapReduce(Input=<Blog Posts>, Output=<Summary>)

Last week Eric Baldeschwieler talked with us about Big Data with a focus on Hadoop. This week, groups mainly focused on two aspects of Eric’s talk: Hadoop and its strengths, and the growing big data technology stack.

Hadoop is built on top of:

Common - the common libs that Hadoop uses

HDFS - the Hadoop Distributed File System, used for making many disks across many machines work together

MapReduce - the computing aspect of Hadoop, takes a lot of data and does work on it

Hadoop has a few strong points:

Cost effective - built to work on commodity components, Hadoop (and HDFS) was described by Eric as one of the cheapest storage options, with a free compute cluster thrown in

Scalable - it is very easy to add on more machines to a computing cluster which can Hadoop can immediately take advantage with little additional work

Reliable - Hadoop is failure tolerant, if a node goes down Hadoop can redistribute the task assigned to that machine

Open Source - As an open source project, Hadoop has many contributors and additions are built on top of it by many different companies and groups

However, Hadoop has a few weak points as well:

Single point of failure  - if the master node goes down, all the information on jobs is lost

Writing to disk - after each step in MapReduce, the results are written to disk which provides good durability and recovery but is also slow

Boilerplate code - to get a job up and running, you often need to write the same boilerplate code and it gets even worse when you have to run many jobs for a single data processing task

In the wider context, Hadoop has been the mainstay in the data processing stack, but there are many additions and alternatives to Hadoop that is machine processing more efficient, intuitive, and scalable.

Pig and Hive - SQL-like syntax built on top of Hadoop to provide a more familiar and expressive abstraction to the developer

Spark - an alternative to the MapReduce framework was takes advantage of in memory speed and streaming

Mesos - cluster management tool

[webtech-cs294]

Concurrency in Summary

Last week, we learned about concurrency and scaling with Marius Eriksen discussing the architecture of Twitter’s Finagle system.

Programming a datacenter-level application application designed for high availability for millions of users is a much different beast than what most programmers are used to. Here are some of the differences students identified:

Highly distributed – code is often interacting with many different servers and databases

Frequent partial or complete failure of components – when a node goes down it needs to know how to recover without any human intervention. The distributed system also needs to know how to deal with node failures and how to redistribute tasks

Fewer tools to test for correctness and efficiency – distributed systems is much harder to wrap one’s head around so writing code that will work in a parallel environment is hard, and there are few tools like debuggers and static checkers

These reasons strongly motivate Finagle, an extensible remote-procedure call framework for high-concurrency servers. A remote procedure call allows one machine to call a function on another machine (or on the same machine from a different address space). Several groups provided an overview of the benefits and disadvantages of using an RPC system, using Apache Thrift and Akka as examples:

A uniform abstraction in a heterogenous environment –The RPC abstraction is transparent to whether the subprocedure is being run on a local or remote machine.

Smooth over some exceptional states, such as network timeouts, disk failures etc – this addresses the issue of component failures: if a component has crashed the RPC will handle these cases and still provide the expected functionality

Language agnostic – RPC systems provide static language code generation for clients and servers; this makes it easy for clients and servers, implemented in different languages, to communicate with each other without having to worry about the specific quirks of each language.

A commonly mentioned disadvantage – an RPC system incurs overhead for network traffic as well as marshalling and unmarshalling the data into a common format, such as JSON or binary

However, the Finagle systems provides many other benefits than a generic RPC system, and in fact can often sit on top of services like Thrift:

an elegant Scala abstraction – in Finagle, the response to an asynchronous operation is a “Future”, which is a first class citizen in Scala

In-depth analytics – since Finagle sits between all interacting components, it can gather statistics in a consistent manner to provide operators with information on failures, latencies, and load. Other Twitter services such as Ostrich and Zipkin aid in stat collection and reporting.

In summary, as anyone who has taken an operating systems class can attest to, building and debugging concurrent, distributed systems is extremely challenging. Often, it is impossible to test for correctness, so developers must design for correctness – and don’t forget about efficiency too! Systems like Finagle aid developers by defining clean abstractions for building highly concurrent and highly available server applications.

[webtech-cs294]

Super (Ha)Duper

Hadoop and MapReduce are so widely known in the tech world that they need little introduction. Used for “big data” processing, Hadoop has become the go to tool for small startups to big companies alike. This week, we had Eric Baldeschwieler speak to us about Hadoop in the context of the ever evolving tech world.

Hadoop has 3 main components: the Hadoop FileSystem for storage, Hadoop MapReduce for computation, and Hadoop Yarn to tie it all together. The MapReduce way of data processing has many advantages which have been lauded for many years:

Cheap - because it runs on commodity hardware, it is very easy to build large data stores and computing clusters for relatively cheap. Eric pointed out that simply using HDFS is the cheapest option per petabyte compared to NAS and SAN, plus you get a computing cluster for free along with all that storage

Scalable - Hadoop is distributed so it is almost trivial to simply add a new node to your cluster and use it within seconds. Scaling up or down is as easy as choosing the number of nodes you want to work with

Reliable - If a node goes down, Hadoop will re-run the task that was running on that node on a different node. Intermediate data is also written to disk so if there is a catastrophic power outage, you will probably be able to boot back up and get back to business relatively quickly

However, Hadoop isn’t the pinnacle of big data tools; there are many other tools which alleviate the pain points which Hadoop suffers from.

Apache Hive - SQL is not dead, in fact it is so useful that Facebook developed Hive with it’s SQL-like HiveQL to make using MapReduce easier. Instead of having to write a lot of boilerplate MapReduce code, just write a SQL statement and Hive will take care of the rest

Apache Pig - similar to Hive in that is provides a SQL-like abstraction to MapReduce, Pig also provides User Defined Functions (UDFs) which can be written in multiple languages so you can do almost anything using simply SQL statements

Apache Spark - why write everything to disk? Disk is slow and memory is fast, so just stream everything. Spark, developed at Berkeley, provides a much faster way to perform batch data processing. Whereas MapReduce performs a map tasks, writes everything to disk, then reads from disk, performs a reduce task, and writes the final output to disk, Spark simply reads, maps, reduces, writes. Spark also doesn’t confine you to the map then reduce way of processing, what used to require multiple map/reduce iterations can now be done in much fewer passes using Spark

There are many more tools out there and even more in development. Eric mentioned many of these tools and talked about how the big data stack has greatly evolved over time. From simply having MapReduce and HDFS we now have the ability to do streaming data processing (Spark) using SQL-like syntax (Spark SQL) in live time while doing large, disk-backed batch processing in the background, all running on commodity machines that we can scale up or down using Amazon Web Services. The world of big data is constantly evolving and more efficient, effective, and intuitive tools are being developed as we speak.

[webtech-cs294]

finagle is fin-able

Marius’s talk this week was about Finagle, a protocol agnostic RPC system. For our blog post, we decided to take a step back and examine what an RPC system is, in particular by looking at Apache Thrift, how it works, and what it is useful for. We then briefly describe how Finagle’s features are an improvement of the abstractions afforded by Thrift.

Generally speaking, a remote procedure call (RPC) allows one system to trigger a subroutine to be run on another (potentially different) system. The nice property of RPC systems is that this abstraction/handling is the same whether the subroutine is to be run on the local machine or a remote machine. RPC systems are often used for client/server services, like the toy example Calculator we saw.

In Thrift, services are defined by a special definition language, which is implementation agnostic. This allows you to have clients and servers which span several languages. Thrift currently supports popular languages such as Python and Java, as well as some more exotic ones like Erlang and Smalltalk. It’s pretty neat to run Thrift and the code generators spit out a language specific client that encapsulates the state and actions in the canonical way for the language, e.g. JSON object or an OOP class.

For marshalling and transporting the data between machines, Thrift supports a variety of protocols including JSON, a human readable format, and an efficient binary protocol. This heterogeneity of protocols is part of the pain-point Finagle aims to address. In a request to Twitter’s enormous infrastructure, several different protocols for failure detection, load balancing, etc. Finagle clients sit between a server and a client and normalize for differences in the protocols, and allow for better management of components and uniform data collection interfaces for monitoring and logging.

There are many other features of Finagle that set it apart from existing systems including it’s fault tolerance capabilities, elegant Scala Future/Service interfaces, and visualization plugins. To learn more, we found the user’s guide to give a great high-level overview: http://twitter.github.io/finagle/guide/

[webtech-cs294]

HTTP/2 and You

Brad Fitzpatrick from Google spoke with us this week about the new HTTP/2 spec currently in development. HTTP/1.1 is widely used all over the internet but it has many drawbacks which are mitigated or fixed in the new HTTP/2 protocol.

One of the most prominent differences between HTTP/1.1 (and below) and HTTP/2 is the way the data is represented. HTTP/1.1 uses text; most people who are familiar with web development are used to seeing information like “GET /” or “POST /form/handler” displayed in the server logs. However, HTTP/2 uses a binary representation with Huffman encoding to transmit data. One of the chief benefits of this is smaller size; instead of having to send many bits just to describe the HTTP method type, we only have to send a few bits which describe the method type. In this binary representation HTTP/2 uses HPACK which is a form of compression where different frames and paths are mapped to binary encodings. Utilizing Huffman encoding, widely used characters are represented using only a few bits whereas very rare characters require more bits.

Another very useful aspect of HTTP/2  is allowing for multiple open connections between the client and server; whereas HTTP/1.1 only allowed for 6 simultaneous connections, the new spec allows for any number as long as the client and server negotiate this number beforehand. Brad showed us a very convincing demo where the server introduce some fake latency and we could see how a picture renders. With HTTP/1.1 and 200ms latency, it was clear that the picture was only loading a few tiles at a time and it took an annoyingly long time to load. With HTTP/2 the all the tiles were transmitted at once so the whole image loaded after only 200ms. This effect was amplified when the latency was increased to 1s, a 60 tile image loading in 1s is much more satisfying than 10s.

A final feature of HTTP/2 which we thought was very interesting was the PUSH_PROMISE frame; this gives the client resources that they didn’t even ask for. If a server knows that when you load the root URL, you’re probably going to also want a certain stylesheet and JS file, then it can just send those in the response and when the client asks for them they’re already there. This reduces the number of required round trips between client and server to just 1.

Brad also mentioned that Google is developing and using another protocol internally called QUIC. HTTP/2 mainly operates in secured (HTTPS) mode and this is the direction which much of the web is moving. In order to set up the secured connection, 3 or more full round trips are required. With latency in the hundreds of ms, this can be a lot of overhead just to create a connection, and then you have to add on the latency of the round trips required to send page data for the initial render. With QUIC, all this overhead is compressed into 1 round trip. Even better, if the client has talked to the server before the connection is instant. QUIC uses UDP rather than TCP. UDP is usually used for streaming or gaming but Google uses it because it allows them prototype quicker and experiment more. Since using UDP for web browsing isn’t widely supported, currently only Google Chrome browsers speaking to Google servers currently support QUIC. Currently, over half of all requests from Chrome to Google servers is using QUIC, and Google hopes that some features of QUIC will migrate over to TCP and TLS so that they can become more widely supported.

[webtech-cs294]

Kubernetes Ideas in Ansible

One of the related deployment technologies listed alongside Kubernetes this week is Ansible. Ansible (and other related tools such as Puppet and Chef) are IT automation engines to manage server provisioning, configuration, management across many servers. Where as Kubernetes focuses on cluster management, Ansible focuses on host configuration. Despite their different focus, when exploring more about Ansible we were able to see many of the same design principles that went Brendan described about Kubernetes.

The first similarity we noted is the importance of a declarative description of the state. That is, the input to the systems should be *what* the state of the system should be, not necessarily *how* to do it. This allows for a great deal of flexibility in how the system operates underneath the covers, and for server systems, how often human intervention (which is expensive) is necessary. For example, if a cluster of three computers has one failure, a declarative system can reconcile the observed state to reach the desired state, where as an imperative system may be left waiting for instructions on how to proceed. We see declarative descriptions exemplified in Ansible in the form of Ansible Modules, which allow the administrator to simply specify a resource and its desired state. For example, in Ansible instead of saying on reboot run “/etc/init.d/apache start”, you would indicate in the configuration that the web server service should always be up.

The second common principle we observed between Kubernetes and Ansible is modularization. Kubernetes is all about self-containment and modular design for launching applications. This abstraction frees the developer from worrying about interactions of their application with the underlying OS/physical machine, etc. In Ansible, machines are divided into playbooks based on similar operations to be performed on them. For example, you may have a playbook full of data-crunching machines, and a playbook for database servers. Furthermore, the Module system described earlier isolates independent operations from each other. For example, a playbook’s ssh module is independent from those machines’ user permissions module. These ideas seem fairly simple, but we’d imagine that when dealing with an enormous number of heterogenous machines, being able to reason about the configuration of machines in this clear way is extremely valuable.

We found this week’s talk especially interesting. While we’re not sure if we’ll be launching and managing thousand-server clusters anytime soon, the talk explained two key principles that may come in handy for future web technologies we decide to implement: the flexibility of declarative descriptions, and the importance of modularization for simplicity.

[webtech-cs294]

Everything as a Service

Noah Zoschke from Heroku spoke with us this week about building the platform-as-a-service tool Heroku and more generally about using infrastructure-as-a-service apps to make better apps, faster.

Heroku started as a sort of Google docs for Rails Development; it was also a single monolithic codebase which using RabbitMQ at it’s center. As development progressed, the team realize that the deployment platform was really what made Heroku useful, and that their infrastructure was not scaling well. Instead they decided to remove RabbitMQ and restructure their infrastructure to leverage more services built by others.

Along the way, they also developed two very important documents: “The Twelve-Factor App” and “HTTP API Design Guide.” Both of these documents describe guidelines for how to develop scalable, efficient, and maintainable codebases. In 12factor, some key takeaways for us were:

Declare and Isolate dependencies - gone are the days where there was a complex machine setup involved every time a developer wanted to work on an established project; instead all they have to do is type in “pip install -r …”, “npm install”, or “bundle install” and all of the app’s dependencies are right there. This makes sharing code across machines much easier and takes away the overhead time required to set up each developer’s workstation.

Backing Services - treating backing services as attached resources, not as resources on the same machine, helps decouple the server from other dependencies which makes it portable and swappable. If you’re developing on a SQLite server locally and have an externally hosted MySQL server backing your live site, all that needs to be done is to switch out some URLs and everything is good to go.

The HTTP API Design Guide outlines best practices for creating APIs that are useful to both humans and machines. Noah talked about the importance of creating both human-readable docs and machine-readable JSON schemas because both are going to have to deal with your API. One thing which the document advises is creating executable examples; from our experience, this is what sets helps API consummation the most. One service which we have found a lot of companies using is Apiary which makes it very easy to build API documentation and live examples, and for users to immediately understand and test the API. 

Noah mentioned a lot of external services which help make developing, monitoring, and maintaining our applications must easier. IaaS apps like Nagios, PagerDuty, Splunk, Rollbar, and many more take away a lot of the burden of having to develop internal services to take care of these infrastructure challenges. A lot of the time it makes more sense to use a product which a company specializes in than to try to roll your own product and use valuable developer time to create something that might not be as robust or bug free.

Heroku was developed to be extensible and to provide a marketplace of apps which you can add to your own. There are a lot of other PaaS applications such as Amazon’s ElasticBeanstalk and RedHat’s OpenShift. All of these provide similar services to Heroku and differ in small implementation details and pricing. We have used Heroku for many of our apps and have been very happy with it, but perhaps it would be useful to investigate these other services if we ever needed to go beyond the free single dyno option that Heroku provides.

[webtech-cs294]

Playing With Golang

In his talk, Rob Pike told us all about the design principles which went into the design of Go, namely: readability, scalability, suitability, and toolability. However, the presentation only gave us a few snippets of actual Go code and syntax. We decided to complete the Go tutorial and see how much of Pike’s principles were were able to see exemplified, with a particular emphasis on readability and scalability.

One thing that is fairly clear from both Rob’s talk is that a great deal of thought went into the design decisions. Pike mentioned that his team deliberated greatly before making the relatively simple distinction to use capital letters to denote public members of packages. Another decision which we initially thought was arbitrary and a bit weird at first was reversing the type name and variable name (e.g. `pi int` instead of `int pi`) . However there’s a detailed explanation found [here](http://blog.golang.org/gos-declaration-syntax), which demonstrates how this choice was made to aid readability in complex function declarations. Indeed, many of these characteristics of Go are improvements over less than ideal choices in C, such as reasonable initialization of variables and requiring explicit cast statements for any type conversions.

Some design choices of Go aren’t as easily justifiable for us. As an example, consider the [naked return statement](https://tour.golang.org/basics/7), which the documentations only to be used in short functions. It seems like for the sake of readability and standardization, it would make sense to always make the returned values explicit. Another small is the removal of the `while` keyword, and instead using `for` all the time, even though we think the former makes programmer’s intention more clear.

All in all, we feel Go’s features make it a great improvement as a systems languages and developer tool in general. For example, Goroutines, with their built in message passing, seem like a very liberating approach to concurrency. As an example of the latter, `gofmt` is an amazing idea! As Pike mentioned, having and enforcing one canonical way to format source code eliminates an entire class of confusion and disagreements that arise among developers.

Another language that appears to be tackling the same type of problem as Go is Mozilla’s Rust language. It shares many similar features as Go such as garbage collection and supporting closures. However, Rust’s high functional programming structure seems like it lends itself to the type of dense constructions Pike described as “too hot”. Personally, we’re a fan of the elegance of constructs like higher order functions and list comprehensions, but certainly agree they can be taken too far. It’s interesting to see that despite their similarities, Go and Rust have distinct personalities as languages. We’re looking forward to watching the growth and evolution of both languages, and it’s worth nothing that their success will depend on many factors including the developer community, and supporting organizations (Google vs. Mozilla).