#sad team

[milesisamitakifan]

My new background ladies and gentlemen

[webtech-cs294]

Hadoop – The Big Data Monger

 Last week Eric Baldeschwieler, founder of Hortonworks and former Yahoo employee, came to talk to our class about processing big data. In particular, his lecture focused on the creation and goals of Hadoop which is perhaps the most popular big data processing tool available to developers.

 One of my favorite parts about Eric’s keynote was his description about Yahoo’s motivation to create Hadoop. The project’s inspiration was two-fold: For one, the project had internal purposes. Yahoo wanted to rebuild its web infrastructure and optimize Yahoo search to make it the superior search engine. By the early 2000s, many search companies had created their own version of MapReduce in efforts to crawl and subsequently map the web. Granted, the jobs were clunky and only running on 15-20 nodes, but conceptually they were all trying to do something very similar: break down and summarize data. Yahoo figured it made sense to devote a team who’s only ambition was to make this process better. The second motivation was more business / ego driven. Yahoo wanted to boast they had a strong technical skill set in house that was capable of developing complex, thorough software. In fear of looking obsolete next to companies like Google and Microsoft, Yahoo felt pressure to grab its stake in the open source community.

 Fortunately for the rest of the industry, Eric took Hadoop from the “prototype it was to what it is today.” Although the system started as an internal tool, it quickly began to appeal to external sources. Software companies saw the business value in the ability to analyze and deduce information quickly from data they had collected. Hadoop was affordable, efficient, and reliable: no extra hardware was required, nodes could be added or removed from a job with little effort, and each node itself was self healing and produced comprehensive logs that could be used in an invent of a failure.

More than 10 years after Hadoop’s creation, it’s hard to imagine an industry without a defacto open-source MapReduce library. Websites are only able to A/B test front-end components because data about millions of user sessions and page clicks can be interpreted. Features like autocomplete/type ahead only exist because there was a tool that was computationally capable of mapping and aggregating billions of web searches and words. Search engines can know process the results of crawling trillions of web pages thanks to the power of Hadoop.

Clearly Hadoop does some things really well, which is why it is so popular both in industry and academia. However, the creation of Hadoop has lead to the development of many open source projects which attempt to fix things Hadoop doesn’t do very well. Pig, Hive, and Spark are some of the more popular projects built by Apache and build on top of Hadoop’s infrastructure. Perhaps the coolest of the three is Apache Spark, which was developed here at UC Berkeley. Spark aims to leverage in-memory primitives to process data rather than disk based storage. This approach makes a lot of sense and is summed up well by Professor Hellerstein (currently the CS186 instructor): “Why waste time writing and reading from memory? That’s slow. Streams are fast. Let’s just use streams for intermediate data instead of storing everything, then we can just store the results”* Dr. Hellerstein may be oversimplifying the ambitions of spark, but his lighthearted tone does a good job to summarize how eager the open source community is to build code to extend the already huge potential of Hadoop. When developers put in as much effort and time as was put into Hadoop, it inspires the entire community to contribute.

 *paraphrased from a CS186 lecture

[webtech-cs294]

Last week’s blog posts discussed Finagle, an RPC system for the JVM written by developers at Twitter. Many of the posts last week discussed the topic of distributed computing and the challenges associated with building a large, efficient, scalable distributed system.

Many posts discussed the sheer issue of scale with regards to today’s web 2.0 companies. Millions and millions of requests need to be serviced every day, and without a scalable architecture, this is not possible. 

Many groups listed many features that Finagle provided that helped to make scaling easy:

Protocol agnosticity

Asynchronous

Finagle’s powerful session layer

Load balancer

Many groups also highlighted some of the weaknesses of Finagle, including:

RPC basis

Costs associated with deciding to use a structured system like Finagle

One post also highlighted the “8 fallacies of distributed computing”, which I thought was really interesting. To quote: 

The network is reliable - Many problems, from physical hiccups, to hardware and software, and even security, can arise, rendering a system that does not account for some sort of error useless.

Latency is zero - Latency, the amount of time it takes for data to move from one place to another, is nontrivial and must be accounted for when developing distributed systems, as (for example) even over ethernet, accessing data from another computer is still much slower than accessing local memory.

Bandwidth is infinite - Bandwidth, the amount of data we can transfer at a time, is limited by two main things: the fact that as it grows, the more we try to deliver at a time over it, and that larger packet size often leads to more errors.

The network is secure - Despite advances in security, attackers have also gotten more sophisticated, and therefore no network is secure, and threat modeling is needed to determine what security measures should be taken.

Topology doesn’t change - The topology of the networks is likely out of the system architect’s hands once it goes into production. That is,  IT can move servers in and out of the network, and clients can introduce new computers or devices into their networks.

There is one administrator - In most enterprise applications of distributed systems, there are likely a plethora of administrators, each with his own expertise, which may not even be associated with you (if your application is deployed on the Internet), which can lead to issues if any problems (which you then need to pinpoint) arise.

Transport cost is zero - This can be interpreted two ways. The first is that moving from application to transport level takes time because we have to serialize the data (which reinforces the “latency is zero” fallacy). The other is that the distributed network is not free to set up; routers, computers, bandwidth, etc. all cost money.

The network is Homogeneous 

In general, groups seems excited by the prospect of trying out a service like Finagle!

[webtech-cs294]

Concurrency and scaling

Scaling companies is really, really hard. As the number of teams and products explode, it becomes increasingly difficult to standardize on just one language or technology. Indeed, diversity in an engineering ecosystem can be a good thing if engineers are choosing different languages to meet different performance and development needs.

Take, for example, Facebook search. This system needs to be highly performant in order to reduce the latency of queries, but PHP (the language of the original site) is not nearly fast enough. If you utilize SOA and build a search API in C++, now you have to build clients for each language you use. The Facebook website uses PHP, the mobile apps use native frameworks, and the public API may use yet another language. Building modules for each language you use in production for every new service you introduce to the ecosystem is a huge time sink for your dev team.

Enter Thrift. Thrift is a system that allows you to define a service interface in a language-neutral file. Thrift will then automatically generate static code for any one of the target languages. Thrift is one example of an RPC (remote procedure call) framework. An RPC acts as an interface to call procedures in another address space. This allows application developers to focus on their application code without worrying about how to communicate with services in a scalable way. In addition, Thrift maintains DRYness by writing all the buffering and I/O logic in one place rather than have each application team write it individually. It is an example of the Strategy design pattern outlined by the Gang of Four.

That being said, Thrift (and really an RPC in general) is just like every other piece of software and therefore isn’t always the perfect solution. The power of an RPC is definitely alluring, but it comes at a cost. Extra RTT (round trip times) are implicitly added to every program you write because the RPC has to make a request /response to an entirely different computer running different code.

While Thrift may be overkill for smaller teams, it’s easy to see how it can help both quickly-growing and large-scale engineering organizations maintain a fast pace of development without wasting developer man-hours duplicating code. Facebook, which is a large and mature company, created Thrift and uses it extensively, as does Uber, a startup which tripled its engineering headcount in less than a year.

Marius Eriksen spoke about the use of Thrift at Twitter. Twitter’s stack actually utilizes a library called Finagle that aims to help you manage the different pieces of a distributed system. Since Finagle’s code is protocol agnostic, implementing services in new languages is relatively easy. When Twitter set out to build Finagle, their goal was to develop a single implementation of the basic components of network servers and clients that could be used for all of their protocols.

Finagle implements connection pools, failure detectors, fallover strategies, load-balances, and back-pressure techniques. By providing implementations for a lot of features that modern, highly concurrent, and distributed systems need, Finagle makes implementing a network service with much more ease and safety.

[webtech-cs294]

http/2 is the new sliced bread

The Internet is the world’s biggest hack. The web, and the protocols that power it, was designed before we could even imagine the rich, dynamic websites we enjoy today. And while these websites have more functionality than ever before, they’re also massive, requiring huge numbers of TCP packets to transport. Each packet requires an ACK, resulting in even more requests. This means that loading sites with HTTP 1.x, which itself uses TCP, is becoming more time-consuming than ever. HTTP2 is the first major revision of the HTTP protocol, and it’s based on a protocol called SPDY/2 which Google developed specifically to reduce latency in web requests. It is managed by IETF’s HTTP Working group, which currently maintains the HTTP protocol.

After SPDY started gaining traction with implementers (e.g. Chromium, Mozilla, Opera) and showing significant improvements over HTTP/1.x, discussions started to happen about HTTP/2. SPDY manipulates HTTP traffic by compressing, multiplexing, and prioritizing traffic. SPDY makes it so that only one connection per client is required. In addition, TLS encryption is everywhere, making SPDY traffic more secure, and transmission headers are compressed, as opposed to just being sent as plaintext. In addition, SPDY requires SSL/TLS for security, and allows servers to push content to clients as opposed waiting for requests to come in. SPDY was very promising and showed a lot of significant improvements, so when the time came to design HTTP/2, SPDY was used as a base for the design.

Let’s take a look at some of the major problems web developers are running into with HTTP 1.x.

Every HTTP requests contains a header, which itself takes up a huge amount of data on each request.  HTTP2 compresses headers, using a lookup table to shrink common paradigms to integers. A typical webpage may take, in aggregate, up to 7-8 round trips just sending packet headers. With http/2 it’s possible that all packet headers could be sent within 1 round trip, and while that doesn’t really make sense in practice (as it wouldn’t make sense to send all the packet headers at once) it does how big of a difference header compression can make packet size.

Downloading a single modern webpage takes multiple HTTP requests to download the associated JavaScript and CSS files. HTTP2 allows the server to proactively send data to the client without them requesting it. For example, if a client requests a webpage, then the server can send back the HTML in addition to all of the associated JavaScript and CSS that the server knows the client will need. This will lighten the load on networks and servers. It also makes a lot of sense because it agrees more with the fundamental principals and ideas TCP was built on. This feature hopes to eliminate the practice of opening up multiple TCP connections from the same source, which has become a very popular hack and takes advantage of bandwidth at the cost of steal bandwidth from other protocol services.

HTTP is a text-based protocol yet the implementation of the protocol was never strictly defined (what do we do with extra spaces? What if something is misspelled?). HTTP2 solves this problem by using binary, which has the added advantage of being more lightweight.

People more familiar with networking will understand that HTTP/1 suffers from head of line blocking, in which a router / switch is unable to process multiple packets with the same destination within the same clock cycle. Http/2 aims to reduce issues like head of line blocking by minimizing the number of round trips a connection has to make. For example, http/2 does away with the “3 way handshake” connection set up that has been an HTTP standard. In a mobile driven Internet http/2 prioritizes speed, which is encouraging.

Http/2 makes a lot of sense, but it will take a long time before it is globally adapted. Still, it is nice to see that multiple browsers (including Chrome and Firefox) have already implemented support for http/2. I think as web developers we’re all excited to reap the benefits of http/2, though it’s difficult at this point maximize the new architecture at an application level.

[webtech-cs294]

Docker and Kubernetes

This week, Brendan Burns came to talk to us about Kubernetes, which is a way to effectively manage Docker containers. 

First, let’s talk about Docker. Docker is an open-source platform that allows developers to abstract away the pain of setting up development environments on different systems. In the past, websites were relatively simple: LAMP stacks running on a single server. Now, not only are the toolchains much more complex but they need to run in a variety of environments: OS X, Windows, and Linux, plus in development, test and production modes. Docker defines a standard format to package up dependencies, libraries, and frameworks into a “container”. Infra teams know how these tools are packaged up inside the container, while developers know how to handle containers by using the standard Docker commands to manage processes. Docker allows the user to abstract away differences in operating systems and the underlying infrastructure.

So how is Docker different from a virtual machine? Well, you might have two different virtual machines, one for your QA setup and a separate guest OS that mimics prod. Each of these machines could be several gigs. With Docker, each container is composed of an application and its dependencies. Each container runs in its own process, sharing kernel resources with other containers. Many companies have moved to service oriented architectures, and each service will have its own container. As you can imagine, even small companies can have a lot of containers running in on many different machines. Managing this type of complexity is where Kubernetes comes in.

Kubernetes is a piece of software “developed by Google for managing containerized applications in a clustered environment.” More colloquially, Kubernetes is an open source cluster manager made by Google. It provides the scheduler layer that cluster computing requires, which is responsible for delegating work to different machines. Google’s had two intentions when it decided to distribute Kubernetes. Google was interested in both sharing the information it had gained through internal experiences with clustered machines and was learning interested in learning from developers within the open source community. Kubernetes has done a lot to progress the conversation about cluster container management within the developer community.

Perhaps the coolest part of Kubernetes is that it also provides a powerful, restful API to easily communicate with your clustered container application. The API allows developers to manipulate the three main components of the kubernetes platform: pods, services and replicationControllers. It is quite cool that web developers will be able to leverage familiar skills and goes to show that the team working on Kubernetes is committed to building software that is easy to transition to (in hopes of making its own open source community even stronger).  To find out more about the API, follow a link to the github ReadMe here -> http://tinyurl.com/ljvdngv

[webtech-cs294]

Sad Team post 9 - creative title

This week, we had the distinct pleasure of having Noah Zoschke, a platform architect at Heroku, come speak to us about building scalable systems. Noah knows a lot about building scalable systems, since he helped transform Heroku from a monolithic Rails app into the distributed and highly scalable system it is today.  Most companies never get to this point, but by heeding Noah’s advice when you begin building a system you can avoid pain and suffering should you be lucky enough to hit it big.

While many services exist to lessen the pain associated with building up a large system, the degree to which these services help varies. IaaS (Infrastructure-As-A-Service) providers like Amazon EC2 provide the lower-level infrastructure for your service. On EC2, you can pay for server use and configure them to your liking. PaaS (Platform-As-A-Service) providers like Heroku offer services at a much higher level, giving you less control, but (to most) an easier time setting things up. For example, setting up a Heroku app if often just as much as filling in a config file and then pushing to a Git remote. With EC2, many aspects of the server must be configured (e.g. installing packages, user access), and if things go wrong, you don’t have Heroku support or a nice error message telling you what the issue is (as you do on Heroku).

Noah made it clear that service oriented architecture is the future. Many people in the industry are preaching the Service Oriented Architecture gospel these days. Most of these people argue that a service oriented architecture is more maintainable and more scalable than traditional monolithic architecture, and those people would be right. However, Noah made a fantastic point that I had never heard before when discussing the merits of SOA: using SOA allows you to easily plug in SaaS products. Engineers can be adverse to SaaS products. “Why pay for that service when I could bang together my own version in a couple of days?” Noah’s point is that SaaS, especially mission-critical services like monitoring, metrics, and on-call responsibilities, represent years of domain expertise and product learnings that you don’t have. Save yourself the time and heartache and leverage existing solutions for these problems. You can always build additional, custom solutions (for metrics, especially) on top of the SaaS’s API. Until then, use Graphite for monitoring and PagerDuty (the industry standard) for on-call alerting.

Noah’s point about using SaaS products to plug gaps in your domain expertise also leads to another, more subtle point: use libraries and services for what they are supposed to do. Many fast-growing companies fall into this trap. Their early engineers are very comfortable with a particular technology, build out the system using that technology and try to scale it as their company grows. Inevitably, that setup reaches a breaking point as the engineers push their preferred technology beyond what it was designed to do. For Heroku, that technology was RabbitMQ. RabbitMQ is a perfectly acceptable messaging system, but as Heroku’s rocketship began to take off, RabbitMQ wasn’t able to scale to meet Heroku’s needs. Eventually, it became a single point of failure for the entire system. Instead of recognizing the tremendous growth and building systems in anticipation of this growth, Heroku engineers were busy fighting fires and hyper-optimizing RabbitMQ so Heroku would actually function.

Eventually, Heroku realized that they could push RabbitMQ no further and switched to Redis. Since you can have multiple redundant Redis buses, it gave Heroku’s system the durability it needed for people to rely on them as a hosting solution. Noah’s point? Although it can be hard, you and your engineering team need to constantly be asking yourselves if the technology you’re using is the right tool for the job. If you don’t, you could just be digging yourself into a deeper hole.

One final point Noah made was about the importance of availability and uptime of the service you are providing. Although industry standard is now officially 5 nines (99.999% uptime annually, which translates to 5.26 minutes of downtime a year), Heroku is famous for struggling with maintaining high availability. Last month, Heroku had an availability percentage of 99.992%, which is up from 99.98% a year earlier. Although a magnitude of 10 doesn’t sound like a huge difference, there are several major, international websites that depend on high availability and to whom a magnitude of 10 really does matter, including Toyota and Macys. Noah also emphasized that PaaS providers like Heroku are victim to the instability of any service it provides/utilizes. For example, if MailChimp or New Relic goes down, it can potentially bring down your own Heroku application or even the Heroku server. As the final step in the pipeline, PaaS providers are unfortunately unfairly blamed for a number of failures that they themselves aren’t directly responsible for. Noah discussed that part of the importance of the analytics platform Heroku provides its users with is to clearly point the finger and blame other parties / services when applicable. While it is certainly more difficult for companies like Heroku to maintain high availability than most sites it is encouraging to see Heroku’s prioritization of low downtime and subsequent steady improvement in availability over the last 2-3 years.

[webtech-cs294]

The sheer scale of Google is something to be in awe of. When you make a search query on google.com, the total number of results are returned, along with the time it took for the result to get back. A simple query for “hello” results in 793 million results over 0.41 seconds. Google deals with billions of queries like these every day, and making sure results are delivered consistently and quickly to users all around the world is by no means an easy problem. Clearly, it makes sense to invest in tools that help alleviate this problem of writing code for large-scale applications, tools like Go.

As described by Rob Pike, Go is a language built for software engineering. It is a language built to facilitate writing software for large systems. To this end, many of its features were designed just for that. gofmt, gofix, and smart dependency management are just some of many examples of ways that Go has been designed to increase developer productivity and help maintain four important properties: readability, scalability, suitability, and toolability.

One of Go’s biggest advantages over other languages is that it was built with concurrency in mind, providing first-class constructs to handle concurrency. There is only one language that handles concurrency as eloquently as Go: Scala. In many ways, Scala and Go are like yin and yang. Rob Pike intentionally kept Go simple like its spiritual ancestor C, which is useful when you have large codebases with many contributors, each with their own programming styles. On the other hand, Scala gives you much more syntactic sugar to play with. This means that Scala code can be much more concise than similar code in Go, but the ability to complete tasks in so many different ways can be a curse when many developers, each with their own styles, are working in the same codebase.

Although syntax is the most visible trait of any programming language, these philosophical differences can be found in the language implementation as well. On almost every issue that is central to language design, Scala and Go take different stances. At this time, Go does not have support for generics (although the developers are open to adding them if they can find a way to do so easily). Scala, which encourages functional programming, relies on generics heavily. Go is heavily typed, but Scala supports local type inference in some cases. Go does not support type inheritance at all, whereas Scala uses implementation inheritance.

As you can see, the creators of Go and Scala have very different ideas of what a language’s purpose is, and that has influenced the design of their respective languages. Which language is “better” will depend on the size of your organization, the problems you are trying to solve, and your personal philosophies on what a language should be.

Although not mentioned directly by Rob Pike, it is worth mentioning that Go has several disadvantages. First and foremost, Go’s youth is one of its biggest deterrents to prospective engineers. Go has only existed for 3 years, which feels like but an instant relative to C++’s 30 year tenure. Developers may be hesitant that the language is too infantile to really be adopted at a massive scale. Another possible disadvantage is the Go Garbage Collector. In Go, garbage collection is handled for the developer. While most of the time this is considered quite convenient, it has its limitations in terms of optimality. Lastly, Go still lacks package depth, most notably in its failure to include a UI toolkit, which may or may not be a deal breaker for some developers.