Technology by FTBpro developers

Developers, Designers, Testers passionate about delivering awesome products

[ftbpro-engineering]

Migrating to iOS8 - the challenges we encountered, and how we dealt with them

When you google “iOS8” the first sentence you'll see is “iOS 8 is the biggest iOS release ever.” And it sure is. When we first tried to compile our code for iOS8 in Xcode 6 we thought it would instantly work and we’d see our beautiful app running on an iOS8 device with no trouble at all. And then we hit “run”.

Oh, how naive we were. The app crashed often, and when it did run everything looked bad. The scaling was wrong and things just weren’t working.

Most of our efforts went into these tasks:

New NSObject properties

Update all SDK in the project to iOS 8 (and 64bit) support

Urban Airship SDK iOS 8 support

Configuring the project and code to 64bit

How to migrate Xcode project to support “real scale” for the new iPhone 6\6+

Auto Layout transform issues

New NSObject properties

Guess what? In iOS8 the NSObject class now contains 4 other properties. Properties, not methods, and here they are:

hash superclass description debugDescription

We had a “description” property in one of our classes that caused the crashes.

Update all SDK in the project to iOS 8 (and 64bit) support

Most SDKs and 3rd party libraries have an iOS8 compatible version. If one of the SDKs you are using have not been updated in a while, it might be a good opportunity to replace your outdated SDKs with one that is compatible.

Urban Airship SDK iOS 8 support

Updating the SDKs will probably require some work, one SDK in particular we had to adjust the integration with is the Urban Airship SDK.

Urban Airship is a service we use to manage our push notification process. Since some of the API regarding push notifications has changed, it was of course inevitable to make some adjustments to the integration with them.

Some functions were removed like this method:

registerDeviceToken:(NSString *)token;

that as has been replaced with this one:

appRegisteredForRemoteNotificationsWithDeviceToken:(NSString *)token;

Also, the call to this method       

[[UAPush share] registerForRemoteNotificationTypes:(UIRemoteNotificationTypeBadge| UIRemoteNotificationTypeSound |UIRemoteNotificationTypeAlert)];

has been replaced with

[UAPush shared].userNotificationTypes = (UIRemoteNotificationTypeAlert| UIRemoteNotificationTypeBadge | UIRemoteNotificationTypeSound); [UAPush shared].userPushNotificationsEnabled = YES; [[UAPush shared] updateRegistration];

Another change was to the

[UAPush shared].userNotificationTypes = (UIRemoteNotificationTypeAlert) UIRemoteNotificationTypeBadge | UIRemoteNotificationTypeSound);

function to register the device’s token. But in iOS8 devices, this function was not called. So we added this function:

(void)application:(UIApplication *)application didRegisterUserNotificationSettings:(UIUserNotificationSettings*)notificationSettings;  

which had to be called first in iOS8 devices.

Configuring the project and code to 64bit

Another struggle we had to deal with was including 64bit support in our app. As stated in the Apple developer page: https://developer.apple.com/news/?id=10202014a, starting from Feb 1, 2015 new submissions to the App Store must include 64bit support.

The first thing you have to do is change the architecture to “standard architectures (armv7, arm64)”. Note that Apple no longer requires support for armv7s since they claim it differs from armv7 in so little that it’s not really necessary.

Next, in our app we sometimes used int and sometimes NSInteger, and often casted between them. Since in 32-bit NSInteger is 32-bit and in 64-bit, NSInteger is 64-bit, we just went ahead and replaced “int” with “NSInteger”, keeping in mind that it can’t be casted to int.

How to migrate Xcode project to support “real scale” for the new iPhone 6\6+

At this point we noticed that the app was still auto scaled in iPhone6/6+. To make sure, we checked what was the screen width in iPhone 6 and saw it really was 320 instead of the expected 375. This made the views and fonts look too big and bulky.

Apparently the solution for this problem is to add launch images for iPhone 6 and iPhone 6+. When the new launch images were added to our Xcode project, Xcode automatically stopped doing “automatic scaling” and allowed us to adjust the elements on the screen as we like.

Before this addition, Xcode auto scales your app for the new devices, which caused the problem and made all objects bigger than you might want them to be. After this fix, our app was able to fully use the extra screen real estate the new iPhones have to offer.

Here is an example of how it looked with auto scale, and how it looked after the launch images addition: iPhone 6+ with auto scale                                        iPhone 6+ with real scale

                     side note:

For us, the auto scale also caused the iPhone 6+ feed to have this weird, unsmooth scroll. Be sure to go over all of your app at this point, if you had constraints that assumed all devices have a fixed width of 320, this will need to be changed because it’s no longer the case.

Auto Layout transform issues

One more big change we had to do has to do with the Auto Layout transform issue. In iOS7 and iOS8, changing the transform of a view behaves differently.   pop up view before the fix: iphone 5s iOS7                                                        iPhone 5s iOS8

                    We found that in iOS7 a transform change causes an immediate call to layoutSubviews, and in iOS8 it does not. So a simple solution could be adding [view layoutIfNeeded] right after the transform change, but we decided to go with a safer solution.

Before the fix, the code that adds the popup was something like this (this is of course partial and simplified):

[view addSubview:popUpView]; popUpView.center = view.center; view.transform = CGAffineTransformScale(CGAffineTransformIdentity, 0.1, 0.1); [UIView animateWithDuration:0.4/1.5 animations:^{    view.transform = CGAffineTransformScale(CGAffineTransformIdentity, 1, 1); }];

So what we eventually did was instead of adding the view and setting its center, we added the view and added centerX, centerY, width and height constraints between the 2 views:

popUpView.translatesAutoresizingMaskIntoConstraints = NO; NSLayoutConstraint *centerXconstraint = [NSLayoutConstraint constraintWithItem:view attribute:NSLayoutAttributeCenterX relatedBy:0 toItem:view.superview attribute:NSLayoutAttributeCenterX multiplier:1 constant:0];NSLayoutConstraint *centerYconstraint = [NSLayoutConstraint constraintWithItem:view attribute:NSLayoutAttributeCenterY relatedBy:0 toItem:view.superview attribute:NSLayoutAttributeCenterY multiplier:1 constant:0]; NSLayoutConstraint *widthConstraint = [NSLayoutConstraint constraintWithItem:view attribute:NSLayoutAttributeWidth relatedBy:0 toItem:nil attribute:0 multiplier:1 constant:view.width]; NSLayoutConstraint *heightConstraint = [NSLayoutConstraint constraintWithItem:view attribute:NSLayoutAttributeHeight relatedBy:0 toItem:nil attribute:0 multiplier:1 constant:view.height]; NSArray *constraints = [NSArray arrayWithObjects:centerXconstraint, centerYconstraint, widthConstraint, heightConstraint, nil]; [view.superview addConstraints:constraints];

And now it looks good in both iOS7 and iOS8!

pop up view after the fix: iphone 5s iOS7                                                                 iPhone 5s iOS8

                    * Written by Meshi Haiman

[ftbpro-engineering]

Mobile Apps - 4 Mandatory Tools for Every Mobile Developer

1) Crashlytics

Crashlytics, acquired by Twitter in early 2013, is the best tool for crash reporting. Until Crashlytics came around the biggest challenge for mobile developers was their inability to detect and fix crashes by real users in the open.

There were some tools that were able to send a basic crash report but because the code is compressed when building an app in Google or Apple stores, there was no way to detect which class really caused the crash. Moreover, on mobile some bugs happen only on certain devices depending on the vendor, Android version, App version and more. There was no way for developers to know all this data before Crashlytics came along.

Enter Crashlytics, with an easy integration on Android and iPhone we are now able to monitor all the crashes (that are unfortunately always there) in production. The crashes are sorted by priority, for each crash we can see:

1) Distribution by device.

2) Distribution by Android version

3) And most importantly the full stack trace with the real class names and row numbers. Crashlytics was able to achieve this very smartly by including dedicated plugin for Xcode and Eclipse, which takes care of uploading the symbols file automatically and seamlessly.

2) Google Analytics

Having analytics on your mobile apps is mandatory, and if that's the case, why not use the most reliable tech company on earth? Ohh, and it's also free.  Google Analytics (GA) for mobile have easy integration for both Android and iPhone. By just inserting one line of code to your app (together with the SDK of course), you'll be able to immediately track: 

1) Active Users

2) Sessions

3) New vs Returning Visitors

4) Session Duration

5) Users device details including device model and operating system

6) Many more 

However, there's much more than just those parameters, with GA you can also define specific events that happen in your app such as: "user read article", "user successfully connected to Facebook", and so on. By defining the key events or KPI of your app, you'll be able to track funnels like how many users did those specific events? Which country are they from? What are the effects of some changes you've made in the app on specific key events? Maybe distribute your users into 2 groups and A/B test to decide which group converts the best?

3) Appirater

Google and Apple Stores give a lot of weight to the app rating. Most app developers don't ask their users to rate them at the store, and that's a big mistake. Having a rating over 4.0 can dramatically improve your app visibility at both stores. But how should you ask your users to do it?

1) You don't want to disturb a user who just installed your app with this sort of msg. Maybe only in the 3rd or 4th session?

2) Maybe it's better just to wait X days until prompt?

3) We want to remember the users who already got the "rating prompt" so we won't bother them again anytime soon.

4) Maybe the best time to prompt the rating is not when the app is just opened, but later on when a user is doing some significant event?

As you can see, even though it may sound simple to just show a dialog with text and link to the store but like anything else, to do it properly you need to invest some thoughts and implement functionality. 

Luckily, Appirater is an open source project that exist for both iPhone and Android which have all the functionality I just described, built in. Plus, it has vast multi-language support in case your app is targeted to more than just English. 

We use Appirater to ask all our users to help us by rating at the store and currently we have a rating of 4.6 on both stores with tens of thousands user ratings. 

4) App Annie

App Annie is a free service that allow you to track downloads from both Google and Apple stores. You might ask, why should you use it if all the data already exist for you at Google or Apple developer console? 

First, App Annie saves all your data forever (unlike Apple in the past). It has a much better user interface which allows you to analyze stats easily. Moreover, you can compare your download numbers to your category average, and most importantly, you can get the full picture of your app stats in terms of its rating in each country and you'll be able to see where and when you app is being featured.

Also, App Annie is a great tool to analyze any app from Google or Apple, You can look at your competitors and better understand their ranking, rating and where they are featured.

* Written by Oded Regev

[ftbpro-engineering]

Getting The Best Out of Logstash for NginX

Note: If you're not familiar with Logstash, please watch the introduction video to get the hang of inputs, filters and outputs and how they operate within Logstash. I do assume you have the basic knowledge about Logstash in this tutorial.

Logstash was a real game changer to our monitoring capabilities at FTBpro.com. We're using it to aggregate and analyse every system which produces log files in our infrastructure. The setup may take some time, but it is certainly worth the while.

Among all the logs we aggregate using Logstash, we also aggregate our Nginx logs, which I am sure a lot of you would like to but just don't know exactly how. I wanted to share our configuration and how exactly we set things up to make it work. It has been working for a few good months now with incredible results.

So this is a general sketch of our system:

The Server

The server has 4 main components which make it work:

Redis: Used in order to receive and store events form the web servers

Logstash Indexer: Takes events from the Redis and pushes them into ElasticSearch

ElasticSearch: Gets events from the Redis and stores them

Kibana Web: A very cool web application which nicely presents data stored in ElasticSearch

You will have to install Java, Redis and ElasticSearch to have it ready for action. Afterwards, just follow this 10 minutes walkthrough to get the Logstash agents up. Pay attention that you will have two Logstash agents: one to move events from the Redis to the Elasticsearch (Indexer), and another Logstash web process which holds the Kibana Web interface.

After you have everything installed, the server.conf file is really simple:

input { redis { host => "127.0.0.1" data_type => "list" key => "logstash" # We use the 'json' codec here because we expect to read # json events from redis. codec => json } } output { stdout { codec => rubydebug} elasticsearch { index => "logstash-%{+YYYY.MM.dd}-%{type}" protocol => http } }

As an input, we have redis which operates on localhost. We're reading the "logstash" key of it which holds a list of events. As an output, we have ElasticSearch and rubydebug just to be able to watch the log files of the agent if things mess up.

The Clients

The clients (Web servers) have only a Logstash Agent which reads the NginX access log and sends it directly to the Redis on the Logstash Server. The Logstash agent here is the same one we installed on the Server earlier. So get back to that 10-min tutorial and install the Logstash agent on your web server. Pay attention that the Logstash Web (Kibana) is not needed on the Web servers so don't run it here.

After getting the agent installed, we need to set up the NginX logging to log all the data we care about. We found that the default NginX log format does not supply us with all the data we'd like to have, so we altered it a bit. This is the log format definition on our NginX servers:

log_format ftbpro '$http_host ' '$remote_addr [$time_local] ' '"$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent" ' '$request_time ' '$upstream_response_time';

The differences from the default log are:

We added a HTTP host to the log. This is because one NginX server may serve multiple hosts and we'd like to know on which one the request operated.

We omitted $remote_user variable from the default log - we saw no use of it.

Added $request_time to have response time of static files served by NginX.

Added $upstream_response_time to have response time of our application server (Unicorn)

Now that we have all the needed data in the log file, all that needed is to ship it right to the Redis on the Logstash server. This is how the agent.conf file looks on our Web Servers:

input { file { path => **YOUR NGINX ACCESS LOG FILE PATH** type => "web_nginx_access" } } filter { grok { type => "web_nginx_access" match => [ "message", "%{IPORHOST:http_host} %{IPORHOST:clientip} \[%{HTTPDATE:timestamp}\] \"(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})\" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{QS:agent} %{NUMBER:request_time:float} %{NUMBER:upstream_time:float}", "message", "%{IPORHOST:http_host} %{IPORHOST:clientip} \[%{HTTPDATE:timestamp}\] \"(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})\" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{QS:agent} %{NUMBER:request_time:float}" ] } date { type => "web_nginx_access" match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ] } geoip { type => "web_nginx_access" source => "clientip" } } output { stdout { codec => rubydebug } redis { host => **REDIS_SERVER_IP_HERE** data_type => "list" key => "logstash" } }

A few points worth mentioning about it:

The input, as expected, is the NginX access log file.

Grok

The Grok filter is one of the most useful filters included in Logstash. It allows you to analyse each log line and fetch fields like client_ip, request and response code from it, by matching the log line against regular expressions. We're using it here to analyse the NginX log line and to extract all the fields we'd like to get out of it to ElasticSearch. Logstash ships with a bunch of regular expressions set up for you so you can easily start matching log lines without even writing regular expressions by yourself. You can find the list of given regular expressions here. For example, the first part of the string %{IPORHOST:http_host} means we're matching the pre-defined Grok pattern IPORHOST and want to have it as http_host field in ElasticSearch. So simple. Getting the hang of Grok is really a matter of practice. There is also a great web tool called Grok Debugger which allows you to try Grok Patterns on inputs you define. The reason that there are two patterns (lines 12, 13) is that Grok will try to match the first and then the second. If NginX serves a static file it won't have $upstream_response_time. So we need one Grok pattern to "catch" requests that went to upstream (this is the first pattern) and one Grok pattern for static requests served (this is the second one).

Inside the Grok pattern we can explicitly mention data types on fields we would like to analyse as numbers. For example

%{NUMBER:request_time:float}

means that request_time field will be indexed as float on ElasticSearch and that we would be able to run numeric functions on it like mean, average etc, and to show it on the graph. This is extremely important because otherwise the response_time would have been indexed as a string and not a number and we couldn't properly display it on a graph. NUMBER is, again, a pre-defined Grok pattern.

The date filter allows us to set the date on the event sent to ElasticSearch. By default logstash puts the time the agent read the log line as the timestamp. This is not good for us because NginX writes the log line after the request has been processed, so the time the log line was written is not the time in which the request arrived to our server.

The geoip filter adds geographical fields on the event. This will later allow us to ask questions like "From which country I got the most requests?"

The output is pretty self explanatory: Send the events to the Logstash Server Redis and to STDOUT for debugging purposes.

Results

The top left graph shows us HTTP responses by time. Each response time - 200OK, redirects, not-found and errors are colored so we can distinguish between them. Dragging the mouse from left to right inside the chart will drill down on the dragged time frame on all of the graphs on the dashboard.

The top right graph shows us only HTTP errors (5xx) responses by time. If we see a bump on this graph it usually means something went wrong. Again you can drag your mouse to drill on the time frame you wish to focus on.

The two bottom left graphs are showing top URLs hits and top user-agents. Clicking on the magnifying glass on the right to each of them will filter all the graphs on the dashboard by this url or user-agent.

The bottom right pie chart shows us the distribution of response codes. Clicking on each section on the pie will filter all graphs on the dashboard to show only requests with the chosen response code.

The left chart simply shows us mean response time.

The right map shows us geo data of requests. The more requests, the darker the color of the country of the map. Clicking on a country will filter all the graphs on the dashboard to show only requests from this country. So simple, but yet so powerful.

Summary

So what do we get out of it? Endless possibilities to analyse and drilldown our web server activity logs. We are able to instantly answer so many questions we couldn't before. Questions like:

1. What are the top URLs in terms of hits? 404 responses? 2. What agents get the most errors? 3. From what countries are we getting most traffic? 4. What are the slowest responses and to which URLs?

The limit is your imagination and getting immediate results is a matter of a click.

So yes, go get Logstash now. It is a great project, really well maintained and most important - will elegantly provide you with answers you had no way to get before.

* Written by Erez Rabih

[ftbpro-engineering]

THE SLIDES: Growing X20 without spending an extra penny on hosting

"A year ago we started a big change in FTBpro. We completely changed the visual design, moved to a single page architecture and started exploring new ways to minimise load on our servers - both when serving our actual website, and mobile API responses.We'll focus on how scaling considerations are now an integral part of our architecture, which enabled us to serve 20x more traffic than we did 1 year ago, with the same setup and with no additional costs."

The presentation we gave at Google Campus TLV on the 20th of May 2014 based on http://tech.ftbpro.com/post/78969626647/growing-x20-without-spending-an-extra-penny-on-hosting

[ftbpro-engineering]

Single Page Applications Done Right

Single Page What?!

Not too long ago, FTBpro.com moved its website from classic web architecture (CWA) to a single-page application (SPA). For those who are less familiar with SPA, the major differences between the two are as follows:

First client request:

CWA: The server returns a full HTML page

SPA: The server returns a set of templates and data which is then rendered to the full HTML page, by Javascript, on the client side.

Subsequent requests

CWA: The server returns a full HTML page

SPA: The server returns only the data needed to display the request. Since all templates are already on the client side from the first request, no HTML/CSS/JS should be delivered.

To put it in simpler words - while in CWA the client gets a full HTML page on each request, in SPA the client gets everything needed to be able to render the HTML by itself on the first page load, so that subsequent requests might be carried in a very fluent and rich manner. It makes your website more of an application and less of a site.

Pros & Cons

SPA relies heavily on Javascript to perform actions that were server-side only a few years ago. It means that a large part of the "heavy lifting" involved in creating web pages is now the responsibility of the client, for better and for worse. Let's go over the advantages and disadvantages of SPA over CWA.

Advantages:

User experience is much richer and more fluent.

Lower server load because HTML rendering is done on the client, and subsequent requests only require the specific data needed to render that page.

Definitive separation between client and server - server acts more like an API.

Disadvantages:

SEO: Search engines (Google etc...) do not execute Javascript when they index your site. Since in SPA pages are rendered on the client side by Javascript you need to find a way to supply search engines with a full HTML page.

Caching: Since pages are rendered on the client side, you can't cache full HTML pages on the server end. This also impacts CDN services as what they cache is only templates/data and not full HTML pages.

Performance: As said, SPA is heavily client-side dependent. It means that your site might load fast on some clients while being painfully slow on others. This becomes a large factor as old mobile devices might have a rough time rendering the pages and introduce large load times for your site.

No Compromises

We knew that we must build our web site as a SPA in order to stay relevant and provide our users with the best possible experience. We also knew we will have to overcome those disadvantages if we wanted to be able to keep our SEO rank high and our users happy. There are several solutions to these problems like PreRender but they are more focused around SEO solutions and less on the other two. We wanted a solution that would enable us to:

Be indexed by Google bots and other search engine crawlers.

Cache full pages HTML on the CDN and on our server cache.

Improve user experience on old mobile devices, which were suffering from the heavy Javascript execution a normal SPA requires.

We figured out that what we really need is a hybrid between CWA and SPA: We want to return fully rendered HTML pages on the first request while subsequent requests behave just like normal SPA. If we could achieve that target, we would enjoy all the advantages of SPA with all the disadvantages eliminated because:

Google bots and other crawlers will easily index our pages since the server returns fully rendered HTML pages.

There is no problem to cache full HTML pages on the server and the CDN will also have a full HTML page cached.

Since we render HTML pages on the server no additional work is needed on the client side (at least for the first request) for the page to be shown. This decreases the dependency on client hardware which mainly impacts old mobile devices.

Since there was no product or service fulfilling all three, we set out on a journey to create one ourselves. The idea was to use PhantomJS,  which is a headless browser capable of running Javascript, to render our SPA pages. The journey ended with 3 separate open source projects: phantom_renderer, phantom_manager and phantom_server. Each has its own important role in the process of producing a full HTML page for our client.

How Does It Work?

Client (user) requests a URL - assume it is /posts/3 from the web server

The web server, via phantom_renderer asks phantom_server to render /posts/3

phantom_server requests SPA version of /posts/3 from the web server

The web server returns the SPA version to phantom_server

phantom_server renderes the SPA to a full HTML page

phantom_server returns a full HTML page to the web server

The web server returns a full HTML page to the client

A quite simple flow, which ends in a full HTML page being sent to the client. Let's talk a little bit about each of the components and what are their exact roles in the system.

Phantom Renderer

Phantom Renderer is a ruby gem which integrates with rails controllers and communicates with the phantom_server in order to produce a full HTML page. It is responsible on two important things:

Caching: when full HTML pages are returned by phantom_server they are being cached on the server cache so that they won't have to be produced again.

Managing phantom_server responses: In case of bad (5xx) response from phantom_server a regular SPA page will be returned to the client. The same is applied if phantom_server does not respond in a given timeframe (configurable). This way, even if something bad happens to phantom_server we fallback to a regular SPA.

Phantom Manager

At the beginning we were having a lot of problems with PhantomJS processes over time. Some of their memory got bloated, some of them crashed and some of them just stopped responding. We had to come up with a solution to make those processes more stable or at least monitor them somehow. Phantom Manager does just that: It manages a predefined amount of PhantomJS processes behind an NginX server. For example, when one PhantomJS process stops responding it will remove it from the Nginx Configuration and re-add it only after it raised another process instead. This way we keep a constant pool of PhantomJS processes which are always ready to render a page.

Phantom Server

Phantom Server is a collection of technologies making the whole rendering process happen. This is how it works:

An NginX server listening on port 80

phantom_manager manages PhantomJS processes on ports 80xx. It syncs every action it does on PhantomJS processes with the NginX Config.

Each PhantomJS process runs a customized version of rndr.me

Requests are delegated from NginX directly to PhantomJS processes on ports 80xx

Conclusion

The system has been up and running for about half a year now. It is all open-source and ready to deploy. If you have a SPA site, and experience the same issues as we did, give it a try. We will be more than happy to help with any setup issues you encounter.

Written By Erez Rabih - Web Developer @FTBpro ([email protected])

[ftbpro-engineering]

Growing X20 without spending an extra penny on hosting

If your website is a social network then this post is probably not for you. If you have a blog, a news site or an e-commerce site, it might!

This shot from NewRelic compares the load on the server on the cyan line with the pageviews on the yellow line before and after a push notification is sent. 

We acheived these great results by a fanatical use of CDN and integrating it really deep into our servers. We constantly recited that disconnecting the correlation between pageviews and CPU will allow us to scale without really scaling hardware.

A year ago we started a big change in FTBpro’s website. We have changed the design totally, moved to a single page architecture and started exploring new ways to minimise load on our servers. Later we have implemented the lessons we learned and the methodologies we have developed on our mobile API. On the outside, the result is FTBpro.com site and mobile app as you know it today. We had two goals in mind - make the user experience faster and lower the load on the servers. This post is about the latter. 

Disconnect the correlation between Pageview and CPU

We need to make sure CPU power is not wasted - twice, thrice and more for the same data. Why SQL twice when you know nothing changed…

If a page is called twice in a reasonable timeframe don’t rebuild it. This can be achieved via full page caching in one of three levels:

Application Level (e.g. Rails, Java, PHP)

Middleware (e.g. Varnish)

by CDN

The first and second approaches are slightly easier to manage as they are contained within your own server but they don’t eliminate the correlation between Pageview and CPU - they do take it to minimum. If the first page took 200ms to render the cached version could be returned at 1ms or even less. With CDN the cached version is not even hitting the main server so the correlation can be defined as “disconnected”. The downside of using a CDN is usually the hassle of choosing one, setting it up, and get a good contract - there are tons of CDNs out there and many small parameters to distinguish between them.

The naive approach for full page caching is setting an expiration time on a page (e.g. 15 minute) so once every X minutes it is expired and the CDN takes the freshest version from the server. That’s okay and very easy to manage but it has two disadvantages:

Data updates don't appear straight away to the user.

There will still be correlation between pageviews and CPU, even if low one.

In order to overcome the second disadvantage we can just set the expiration time to never ;-) but now our page will surely be obsolete at some time - as any editorial update won’t be reflected. This can be solved by using a CDN provider that has purge or load APIs. In the server layer attach an expiration event to the classes that are in charge of updating data to these pages (e.g. Post#after_save in a Rails app). Most CDN providers has these APIs but there are two important criteria that differ from one to another:

Speed: some CDNs purge at 200ms, some at 1min, some at 45min

Purge criteria: some CDNs allow purge by exact URL, some allow REGEXPs, some force you to “tag” each URL in the HTTP response headers and purge by those tags (much more work but can give the best results in a few cases)

So what did we do?

Configured the CDN to keep all our pages for ever, never expire.

Modified URL structure of the APIs (mobile & web APIs) to be in a pattern that is purgeable. For example - our CDN couldn’t purge based on query string parameters; We had to modify URLs to have a restful structure and /feed?team=arsenal had to change to /feed/arsenal.

Added “expirators” to our different models. Whenever a post is saved we expire it’s URL & the URLs of feeds that should contain it. e.g. Updating a post about a game between Arsenal and Barca will expire the url of the post itself, Arsenal’s feed, Barca’s feed, Premier League feed and La-Liga feed (both on mobile and web)

Before sending push-notification on a post, it is automatically preloaded to the CDN. At these times we can get up to 100k requests a minute to the website and none of them is reaching our servers.

Added an Application level full-page caching layer with MemCached after realising CDN is constructed of many different independent servers which will all hit our application server if they don't have the cached version, creating a real load on them.

What happened?

The user experience became much better because all requests are served from light&fast CDN servers that are geographically near the user.

We use the exact same server resources for 100m pageviews we have today as we used for 5m pageviews we had 7 months ago.

We chose a CDN that fit our needs. We pay them a small fraction of what we payed our former CDN with a x20 increase in load.

That's a good opportunity to praise Edgecast, the CDN which we use. They have exceeded our expectations in every parameter:

Amazing quality of service. They respond fast to emails, they are available on the phone and just stay there and give service for as long as it takes.

Technology. Their user interface is a bit sluggish but it allows us to really go crazy and set different configuration rules based on our wild url structure. And they purge fast - a few seconds to 1-2 minutes per purge. 

Great price. That wasn't the main criteria in choosing CDN but it happened to be very affordable nonetheless.

by Dor Kalev, CTO @ FTBpro

[ftbpro-engineering]

Ruby 2.1 - Our Experience

We've recently moved FTBpro's Ruby on Rails servers to the newest Ruby version on earth - Ruby 2.1. It has been running on our production servers for the past two weeks. Our stack includes: MySQL, MongoDB, Rails 3.2, ElasticSearch Memcached and Redis. We wanted to share our experience of making this change.

Incompatibilities

1. First thing you encounter when you move to Ruby 2.1 is the non-working net/http module. As explained here, Ruby 2.x Net/HTTP library asks for gzipped content by default, but does not decode it by default, which makes some JSON parsing of HTTP requests break. This breaks koala gem, right_aws gem and many other gems which relies on JSON HTTP communication to operate. The solution to this is a small patch to the net/http library. We have put it in our config/initializers/a_net_http.rb so that Rails loads it upon boot. The patch:

require 'net/http' module HTTPResponseDecodeContentOverride def initialize(h,c,m) super(h,c,m) @decode_content = true end def body res = super if self['content-length'] && res && res.respond_to?(:bytesize) self['content-length']= res.bytesize end res end end module Net class HTTPResponse prepend HTTPResponseDecodeContentOverride end end

UPDATE: Looks like this is a specific bug with right_http_connection which monkey-patches Ruby's net/http and breaks it. You can read more about it in this thread.

2. All of the mongoDB users on the room pay attention: current stable version 0.12.0 of the mongomapper gem does not support ruby2.x. We upgraded our gem version to 0.13.0beta2, and in combination with the net/http patch in bullet 1 it works like a charm.

3. If you are a fan of the debugger gem you'll have to say farewell. It does not support ruby2.x in any manner and causes nasty segmentation faults with long outputs. The good news is that there is a very good replacement: byebug gem. Its interface is almost similar to that of the debugger gem so you'll feel right at home, and it works well with ruby2.x.

4. If you're using imagesize gem to determine the height/width of images you'll have to find a replacement. We already had Rmagick in our gemset which includes image dimensions retrieval so we just used it.

5. We had a weird bug with the BigDecimal library in ruby 2.1. Here is the output of the exact same code under ruby1.9.3 and ruby 2.1:

#ruby 1.9.3 require 'bigdecimal' ; require 'bigdecimal/util'; (0.5.to_d / 0.99.to_d).to_f # => 0.505050505 #ruby 2.1.0 require 'bigdecimal' ; require 'bigdecimal/util'; (0.5.to_d / 0.99.to_d).to_f # => 0.0

Don't know how to explain this but we're lucky to have a test suit for this module because we'd never discover it until it got to production.

UPDATE: Wasn't aware of it but apparently BigDecimal division is a known bug in Ruby 2.1. You should check out this list for more info.

This concludes the changes we had to make to our code so it runs well under Ruby 2.1. Not much, but is it worth the hassle?

The Effects

We observed three, very prominent, improvements in Ruby 2.1 over 1.9.3:

1. Load times are significantly lower. And by "significantly" I mean about forth of the time. The larger your environment is, the larger the difference of the load time. We were nothing less than amazed by this:

* Deployment time dropped from 14 minutes to approximately 5 minutes. This is due to the many rake tasks we run while deploying. We make about 15 deployments to our QA servers daily. That's 135 minutes, a little more than two hours saved per day for developers waiting for their version to arrive on the QA server.

* Build time by Jenkins CI was reduced from 14 minutes to about 6 minutes. This has shorten the time from opening a pull request to a successful / failed build notice and made the feedback loop a little more bearable.

* Every run of binary that requires rails environment to be loaded takes now forth of the time. This are the measurements I made on our environment:

Ruby 1.9.3: bundle exec rails runner 'puts "a"' 41.06s user 2.23s system 98% cpu 43.916 total

Ruby 2.1.0: bundle exec rails runner 'puts "a"' 11.07s user 2.04s system 94% cpu 13.823 total

It saves a lot of waiting time for our developers when running rails server / console and various rake tasks.

2. Garbage collection times dropped from 100ms to almost 0ms. This is our New Relic graph for garbage collection. The vertical line marks the deploy which moved us to ruby 2.1:

3. We had a severe problem on deployments during high traffic hours - we'd just go down from time to time while unicorn workers were restarting. Ruby 2.1 amazingly mitigates this problem since environment load time is 1/4 now and Ruby 2.1 GC is copy-on-write friendly which makes unicorn better handle forking. You should definitely read this article which explains how Ruby 2.x affects Unicorn's forking mechanics.

Weird Stuff

The only thing we can't explain yet since moving to Ruby 2.1 is some strange unicorn master process behaviour: When we restart it, it always starts off with a different amount of memory size. As a result, every unicorn restart causes the mean response time to differ in about 100ms. Here are two graphs where the vertical line represents a unicorn restarts:  Unicorn Restart - Bump Up Unicorn Restart - Bump Down

This is the only thing we feel uncomfortable with moving to Ruby 2.1.

You Should Also

Ruby 2.1 has some killer advantages over Ruby 1.9.3. It will make you daily operation a lot faster than it is today and can even help you overcome or mitigate other infrastructure problems you're having. The changes we had to do to move to Ruby 2.1 are really minor comparing to the benefits we got from it. There is no reason to stay behind - make a step forward to Ruby 2.1.

by Erez Rabih, Head of infrastructure @ FTBpro

[ftbpro-engineering]

From Illustrator to a web font. Creating custom scalable web-icons.

State of affairs: We want to switch from cup-up PNG sprites to an icon font and replace the .PNG sprites. Trying to achieve that with minimum effort and if possible no additional software we searched for a web-app to help us.

  Research:

There is a ton of software and free or demo apps that can help you get your way and add some nice pre-designed vector icons to your project (like fontawesome). However, being a perfectionist using pre-designed icons did not satisfy me. What I wanted is a way to translate the exact vector icons I developed in illustrator and photoshop into a web-font and that seemed to be impossible without expensive font-editing software.

  Solution:

Then we found icomoon app. This baby can take custom .SVG files, combine them with pre-designed icons from several open-source libraries and export them into a custom web-font. Then the session can be stored and downloaded as a .JSON file, making it possible to edit in the future. All free, no strings attached. This is perfect!

(Note, the font-export button is at the bottom.)

  This is how we do it now:

Now I open a sheet in illustrator, exporting every icon to a separate .SVG file and importing them in icomoon.

Front-end developers call every icon by its code like so:

<a class="prev ficon icon-arrow-left"></a>

And voila, now we have scaleable and custom icons all over the website, fitting mobile, tablet and desktop. This saves us a lot of work in development and cutting PNG's is now history.

Check them all out here: www.ftbpro.com

— Mark Levinson

Designer at ftbpro.com

[ftbpro-engineering]

Push Notifications Explained

There are two types of users at FTBpro: writers & readers.

Our readers want quality content about their favorite team & league.

When we have content that might interest our readers we don't want them to miss it.

The writers on the other hand would like to get their content read by as large audience as possible.

Fullfiling both these needs is the essence of FTBpro, one effective way we found to accomplish that is through mobile push notifications.

Recently we started a project to rebuild our infrastructure for sending PN, the rest of this post is about this new system.

First thing first: what do we exactly need?

We have 21 apps on the App Store and 21 more on Google Play, whenever we send PN it should arrive to all of them.

Every mobile user is a fan of one team from several leagues we support. 

In addition each user can choose the language in which to consume the content on the app.

Now it's not our intention to spam our mobile users with PN they may not like, to prevent that users should only get PN with content about their favorite team and written in language they can read.

We need the ability to send both immediate PN mainly for breaking news, and also scheduled PN.

Another key requirement is the ability to customized the message & schedule time of the PN per team & league basis, so if we had a post about a lose of Real Madrid to Barca we would like to send a different message to Real fans than Barca's.

How do push notifications work anyway?

Generally speaking each mobile platform has its own way to send PN.

For iOS devices it's Apple Push Notification Service (APNs), for Android devices it's Google Cloud Messaging (GCM). When you want to send PN you have to talk to these services and they in turn send the PN to the mobile devices.

Both APNs and GCM have their own protocols for sending PN, Interfacing directly to these services can be quit tedious. We use Urban Airship.

Urban Airship (UA) is a service that provide us with a convenient way to manage PN for both iOS and Android.

Every app on any platform makes one logical app on UA, we have 21 of those. Using UA the task of sending of PN is reduced to making HTTP POST request to their api.

One very useful feature of UA is tags, tags are just labels that can be associated with any device. The cool thing about tags is that you can tell UA to send PN to all devices associated with one or more tags.

How do we use it? When a user opens one of our apps he is being registered to UA with two tags representing he's favorite team & league as well as his language . For example a Chelsea fan in English will be registered with 'team_4_en' and 'league_1_en' tags. Having the tags setted up this way allows us to tell UA to send a PN only to fans of Barca in Spanish for instance.

UA's api provides us with two endpoints for sending PN: '/push' & '/schedules'.

Making POST request to '/push' will result in immediately sent PN, we use this endpoint for PN that need to go out 'now'.

POSTing to '/schedules'  schedules a PN to be sent at a later time, this feature saved us from implementing a scheduling solution on our own.

Ain't nobody got time for that

Sending PN to all our apps envolves lots of HTTP requests. Making these requests takes time which our web server don't have, instead we make these requests in the background using Sidekiq.

Sidekiq is background job processing framework for Ruby, it uses threads for its workers giving it advantage over other frameworks, such as Resque that uses one process per worker.

When sending PN the web process enqueues one Sidekiq job for each one of 21 apps, then on a dedicated server 21 Sidekiq worker threads are processing the jobs. each such job is making appropriate requests to UA api for one app and then updates back the status for that app.

The effect of this setup is that when we send PN it arrives to all our apps (almost) at the same time.

Persistence

Shai Kerer had already stated that we strive to use the right tool for the job whenever possible.

We store all the PN related data in one MongoDB collection. each document contains canonical data on PN to one app with embedded documents that includes team or league specific data. an example of such document is:

{ "app": "aston_villa", "post_id": 617803, "targets":[ { "message": "Transfer Talk: Tottenham Set to Battle for FC Porto Midfielder", "locale": "en", "scheduled_time": null, "team_id": 17, "status": "sent" }, { "message": "Transfer Talk: Aston Villa Set to Battle for FC Porto Midfielder", "locale": "en", "scheduled_time": null, "team_id": 2, "status": "sent" } ] }

Using MongoDB allowed us to store the data as we perceive it and not be penalized by expensive joins.

Conclustion

For the few months the new system is live, it has been working smoothly.

Using Urban Airship saved us both time and effort. It allowed us to focus on our specific needs instead of implementing GCM & APNs protocols for sending PN.

We have introduced MongoDB to our ecosystem that will be utilized in future projects.

So overall, you could say it was a good project :)

Gashaw Mola, Web Developer @ FTBpro.com

[ftbpro-engineering]

Count von Count - A real-time counting database!

FTBpro is all about user generated content. Our articles are written by Football fans around the world. Their incentive for writing over and over again is the exposure they know they will receive. They are motivated by the number of reads, comments, likes, tweets or shares their articles will receive. For this reason, these, and many other counters, are very prominent across our site and mobile apps.

Along with that, we started working on a new gamification project. The requirement here is that for each action a user makes on our site or mobile app (e.g, reading an article, writing an article that gets featured, sharing on a social network) - he gets a score. 

This compels us to count many different actions for each user on the site and calculate the score - live.

If you have read one of our previous posts, you probably know that we have been dealing with the counting issue for quite a long time.

At the early days of our startup, we used to store the numbers in a MySql database. This meant the number of reads of an article was stored in the article's table. As our scale grew, and the load of the database was increasing, we moved to another solution: extracting the counting to a dedicated Nginx server. 

In this solution, every time an article was read we initiated a request to our counting server with the relevant parameters. The Nginx server logged all the requests to its access.log file, and we had a script running every minute that aggregated the numbers from the recent requests. After the aggregation, the script updated our main app server with the numbers, and they were saved to the same MySql database.

This is no longer good.

Let's examine it from the end. This counting system stored the numbers in MySql database. Relational database may be good enough in the simple cases of counting reads of a an article, but how can we store a leaderboard? Or all the countries the readers of each article come from? Well, of course there are some tricks that can help you do it, but it's much easier to save this kind of a data in NoSQL manner.

We would also like the information to be available live. We don't want to count (boy, this word appears a lot in this article) on background processes for manipulating our data to the relevant format. 

To sum up, we need a live counting system based on some kind of a NoSQL database. It is also has to be scalable (more than hundreds of requests per second) and reliable.

That's why we developed Count von Count

It is based on OpenResty, an Nginx based web service, bundled with some useful 3rd party modules. OpenResty turns an nginx web server into a powerful web app server using scripts written in Lua programming language. It still has the advantage of the non-blocking I/O but it also has the ability to communicate with remote clients such as MySQL, Memcached and also Redis. We are using Redis as our database for this project, leveraging its following features: 

EVAL command evaluates a Lua script in the context of the Redis server. Lua? again this Lua? Yep, this magical language is supported both by Nginx and by Redis. It is also the language for writing addons for World Of Warcraft. It allows us to write all the counter logic in a lua script, which is preloaded to Redis, and is evaluated from the Redis module of OpenResty.

Sorted Set datatype is great for leaderboard data modeling. We extensively use it for storing any kind of leaders data, such as top writers and most-read articles. We have different keys for daily, weekly and monthly leaderboards, and each read action makes an update in all of them.

Bitmap datatype helps us count real time metrics in a space efficient way. We use it to count the number of daily active users on our mobile applications. Here you can read more about using it.

Ttling helps us clean the database from irrelevant objects. 

Pipeline requests speed up the whole thing.

Putting it all together

In a different server from the app server we have an OpenResty service up and running waiting for counting requests. We make requests to this server both from our client side and  app server, each time we want to +1 or -1 a counter. Based on Nginx EmptyGif module, we return an empty pixel to each request.  Each request holds the action we want to count and extra relevant params. For example, when a user shares a post, the following request is made: http://<counting_server>/post_share?user=700&post=900&author=15&team=arsenal. Since the server returns a gif, the request can be invoked using <img src=...> html element. 

When the Nginx receives the request, he triggers a very minimal Lua script using the LuaModule. The script just parses the request arguments and evaluates a lua script that was preloaded into Redis. The request is also logged to Nginx's access.log. All the Redis updates are made inside the Redis script to save connection overhead.

The Lua Redis script is a bit more complex and is responsible for updating all the the relevant keys for the given action. For instance, if we take the previously mentioned post_share action, we need to update the number_of_shares field in the following hashes: user_700 key, post_900 and team_arsenal.

For cases of unexpected failures or downtime, we developed a log player, that "plays" the access.log files and updates the relevant Redis data models.

Using the data

Count-von-Count offers an API for retrieving the data. Since we need to show live counters across our site, we wrote a javascript module that collects all the counters in a page, queries our counter server API for the numbers and updates all the counters on the page. In this way, we always show live numbers, as you can see on our user page and post page. 

Open Source Project

We've put a lot of effort in making this project open source. What and how is being counted is configured in a json file, making it extremely easy to embed this project wherever you need. No single line of code is needed! Check it out at https://github.com/FTBpro/count-von-count

We have been using count-von-count in production for several months and we really satisfied with it. It receives millions of requests per day and thousands of requests per minute at peak times.We use it wherever counting is needed., i.e, Player of the Month Widget , Top writers leaderboard and writer's profile page.

I wish to give a big kudos to the maintainer of the OpenResty project - Yichun Zang. Yichun is also the administrator of the OpenResty Google Group where you can get lots of information about this powerful project.

To learn more about this project, you can watch this short video from DevconTLV conference.

Happy counting,

Posted by

Ron Schwartz, Software Developer @ FTBpro.com

[ftbpro-engineering]

The Dark Side of Single Page Applications @ Rails Israel 2013 http://www.slideshare.net/DorKalev/the-dark-side-of-single-page-applications

[ftbpro-engineering]

How to manage a multilingual webfont

State of affairs: we have a multi-lingual web-project. We needed a freeware web font, free or cheap software for font editing and a way to do it all from our office, without any third parties.

We started off FTBpro.com with a cool web-font in mind - Days. The font had the most potential and our website's design relied on it heavily. The font was never tested for language compatibility, up until the point front-end development was ready, and this is what gave us the most trouble.

Here is Days font:

  Problem One—Days does not have the european characters such as À ß Æ Ñ etc.

Solution: I found a freeware software called Type light that has basic editing capabilities and added the missing characters one by one. This is an ongoing process and I still add a letter here and there.

Typelight has very limited editing capabilities (version 3.2) but I managed to sketch the needed serifs and combined different letters to make the missing one's. I made a photoshop test-sheet to check if the letters are rendering:

Example: Combining existing letters to get the needed Æ character. Far from perfect, but good enough for small titles:

Example: Extending the character 3 to a germansharp s:

  Problem Two—missing characters change the font of a whole word. 

When the Days font was rendered in a sentence with a missing character, the whole word would render in a default font (Times New Roman) which looked something like this:

Solution: define a (standard) fallback font similar to the title font: in our case  Helvetica and Arial Bold did the job.

font-family: "OpenSansCondensedBold","Helvetica-Bold",Arial,sans-serif;

  Problem Three—the font has enough characters in OTF but they are not rendering well when put on web.

To be exact this is a problem with conversion. Fontsquirrel web-font generator has many options for converting a font to a web-font kit, and some of them remove characters like the german sharp s (ß). Another problem we were having is with kerning (space between individual letters).

Solution: Finally it took us quite a white to get the web font to render well. Our own Alon Idelson went out for a hunt and found this online web font converter: http://www.fontconverter.org/ which made the job well. Fontsquirrel webfont generator did not do a good job, in case you wondered.

  Conclusion:

Web fonts are definitely the future of web typography. There is already a ton to pick from. While using a freeware font is seductive, the consequences can be difficult to handle. Make sure you know what languages you are going to deal with in your project in the future, inspect the glyphs (characters map), and make sure the font you are using has them.  Posted by: Mark Levinson, Designer @ FTBpro.com

[ftbpro-engineering]

Be Proud Of Your Commits

In FTBpro.com, we have a nice procedure of merging new code to our master branch. We use GitHub's neat Pull Request (PR) feature to gain two big advantages:

It is easy to see the commits and the files changed in the branch compared to master. It is a very good tool for code reviewing.

Our Jenkins CI server automatically makes a build of the merged branch to verify all the specs run, and marks the PR with success/failure accordingly.

I want to extend the first point a little bit more.

Going through code review is not always an easy experience: It may have taken the programmer days or weeks to produce this pull request. A lot of effort was made to meet a deadline or even free hours spent on finishing the job, and now his/hers PR is not approved due to lacking code quality. So what a developer should do before submitting a PR to be reviewed by his peers?

Look at the the diff and ask himself one, very simple but yet powerful, question:

Am I proud of the code I delivered? If I were looking at this diff, would I think it is an excellent code? Once this question is positively answered you know you made your best to submit high quality code. It may still get some rejects or corrections from peers, but that's only natural - the more eyes, the better your code gets.

Being proud of your code is very subjective and changes from person to person, of course. How can you really know that the code you're proud of, will be appreciated the same by someone else's point of view? You can't, but you should endorse yourself a set of rules which makes you feel good about the code you write, and try to apply them every time you submit new code.

I'm going to present you with my set of rules. Some of them may fit you, others may not, with some you may agree and with some you may disagree. This is what makes me feel good about the code I write.

Working Code Is Nothing To Be Proud Of

I think that the strongest thought that sits in the back of my mind while writing code is this one. Once you change your state of mind from writing code that works, to writing an excellent code that works, you can never go back. You always seek where you can improve it so that it stands out as an excellent piece of code.

Think about it this way: Every 15 years old boy can probably submit a code that does exactly what you are trying to do. So what makes you, a mature and experienced developer, better? What knowledge do you have that this boy does not, and how is it applied in your code? What makes your code above average?

Be honest with yourself answering these hard questions and make sure you have an answer to at least one of them when submitting the code.

Become A Writer

Grady Booch, author of Object-Oriented Analysis once said something very powerful: "Clean code reads like well-written prose".

All modern, high-level programming languages strive to allow the developer write code that reads like well-written prose. From all the languages I'm familiar with, Ruby wins the prize in this category. You should always strive to write code that reads like plain English. Hide the implementation details as far as you can. Use meaningful yet simple names for you classes, functions and variables. Before submitting a PR go over the code you've written and make sure it reads like well-written prose.

Believe me, when you go over your own code and it looks like plain English, the satisfaction will keep you going this way forever on.

Test Your Code

Tested code has its obvious advantages. Once the test suite is done, refactoring is a piece of cake. Extending the module or class is easier since the developer can be sure nothing old is broken. It is even a good documentation tool since reading the test suite must convey the purpose of each public/interface method you present.

Apart from these obvious advantages there's another one which lays underneath the surface: Writing tests is a process in which you, the one which creates the system, takes the role of the client which uses it. Seeing your code from the client's perspective is a whole different matter. You get insights that you couldn't get in any other manner. Suddenly, instead of thinking "what would be the easiest way to implement this feature" you start thinking "what would be the easiest way for the client to use this system" and that makes your improve your interface, method naming and code modularity.

Be sure to test your code before submitting it. It will definitely make it better.

Refactor When Possible

Often we alter or extend a module that was not written by us. Most of the time we're thinking to ourselves: "If only I were the one writing this code, it would be 10 times better" but the timetable and deadlines make us give up on refactoring it. I know that an overall refactoring process can take days or even weeks for large modules, and there isn't always time to perform them. But even in large modules like these, small, precise refactorings can make a difference: You can change a method or a variable name to a more meaningful one, eliminate small code duplications, extract a few code of lines to a well named method or even put one break statement that will save us going over a whole collection when the object we wanted was already found. All of these does not require much time, and affect only small portions of the code.

Always leave cleaner code behind you.

These are the things that makes me proud of my commits. When I apply them I feel like I contributed to a better, more maintainable code base. The feeling of being proud of your commits is an addictive one and that's a great thing: Once you're there, you can never go back.

Now go, make yourself proud :)

Like this post? You should definitely check out this one.

posted by Erez Rabih, Backend Developer @ FTBpro.compainting by Nila Ward

[ftbpro-engineering]

The Dark Side of Single Page Applications

The story of all the pitfalls we had while transferring FTBpro.com from the good old web to a Backbone single page application... and all the great solutions we came up with.

[ftbpro-engineering]

Dont do_something and return !

I never understood why I dislike the Rails approach to do_something and return. I always wrote return do_something. This is how I used to develop Rails since 2005 and only saw the do_something and return approach in the last year.

"That's the Rails way!" they said, "eveybody's doing it!" and actually it does look nice and more or less like proper english.

Lately we've seen a bug here in FTBpro.com that helps me justifying my dislike.

We had this code in place:

def show set_obj_by_id show_rss and return if rss? render_via_phantom(cache_key: MemcachedKey.for(obj, locale)) and return end

and for some rsses we saw render_with_phantom errors. Refactoring to this style helped:

def show set_obj_by_id if rss? show_rss and return else render_via_phantom(cache_key: MemcachedKey.for(obj, locale)) and return end end

How could that be, isn't it just the same logic just styled differently? Well, of-course not. How does "and" works?

Let's look at the method Koko

def koko nil and return return true end

It will always return true because "and" never evaluates the second parameter if the first one is nil/false.

We went to look what happens in our render_rss method and a Voilà - we have the same pattern there:

def show_rss @posts = PostsRssFeed.for(obj, locale, params) @show_full_text = params[:text] == "full" @rss_title = obj.name render 'singlepage/shared/league_team' and return end

Either Rails render method returns nil and its return isn't called or the return just returns nil so show_rss certianly returns nil and the show method does "nil and return" that don't trigger the return method and moves on to the next line.

#FAIL

So one would suggest using the pattern do_something and return when you're perfectly sure do_something returns a value that does not evaluate to false but its an assumption that can't be implied by reading the code and thus it should not be used.

If you have to use return use "return do_something" - but it's usually (not always) better to use simple if condition like we demonstrated above.

Posted by:

Dor Kalev, CTO @ FTBpro

[ftbpro-engineering]

11 Steps To Make Your RSpec Specs Awesome

https://gist.github.com/erez-rabih/6697511

Preface

Everyone agrees that writing clean ruby code is essential. There are many blog posts about refactoring your ruby code to make it more readable and maintainable. As tests become a major part of our programming routine, I feel there is a large place for improvement in the way we write and maintain our test-suites. Many ruby projects consist of more rspec lines than actual ruby code, and that requires us to treat our test code with as much care as we treat our actual ruby code, if not even more. Writing specs may be considered by programmers as a burden or a chore if not done the right way. That may cause some programmers to write less specs for their code or to even totally give up on it. Luckily, RSpec provides us with all the tools necessary to create efficient, great looking and fun-to-write specs for our code. I'd like to go over a process of refactoring a test-suite written in RSpec for a small library I've recently written in FTBpro.com.

The Domain

In FTBPro, we have posts written by users, and a dynamic set of reaction templates that allows the user to react to that post. A post has a locale and a type, reaction templates has a locale and an array of types which they fit to. When a new post is created, we want to attach the most suitable reaction template to it:

It must have the same locale as the post.

Its types array must include the post's type.

From the set of matching reaction templates by rules 1 and 2, choose the one which was created the latest.

I came out with the following interface to do this job: FindReaction.for(post).attach While FindReaction.for(post).reaction holds the reaction which was found - I mainly exposed this for tests purposes. Now the funny part about this blog post is that I'm not going to post here the code which implemented this logic, we shouldn't even care about it. What I'm going to do is to present you with a messed up test suite for this logic, and lead you through the process of making it a blast. This test suit contains 4 simple specs which describe the behavior we expect from FindReaction:

describe FindReaction do #Spec 1 it "should return no reaction template when there are no reaction templates" do post = create :post, locale: "en", type: "basic" reaction = FindReaction.for(@post).reaction reaction.blank?.should == true end #Spec 2 it "should return no reaction template if reaction templates exist, but non fitting" do post = create :post, locale: "en", type: "basic" reaction1 = ReactionTemplate.create locale: "de", subtypes: ["basic", "top_x"] reaction2 = ReactionTemplate.create locale: "en", subtypes: ["slideshow", "top_x"] reaction = FindReaction.for(@post).reaction reaction.blank?.should == true end #Spec 3 it "should return the right reaction if it exists" do post = create :post, locale: "en", type: "basic" reaction1 = ReactionTemplate.create locale: "en", subtypes: ["basic", "top_x"] reaction2 = ReactionTemplate.create locale: "en", subtypes: ["basic", "top_x"] reaction3 = ReactionTemplate.create locale: "en" , subtypes: ["slideshow", "top_x"] reaction = FindReaction.for(@post).reaction reaction.blank?.should == false reaction.should eq reaction2 end #Spec 4 it "should change posts reaction_template when attaching it" do post = create :post, locale: "en", type: "basic" post.reaction_template.nil?.should == true reaction = ReactionTemplate.create locale: "en", subtypes: ["basic", "top_x"] FindReaction.for(post).attach post.reaction_template.nil?.should == false post.reaction_template.should eq reaction end end

Although these specs pass they're not optimal. Let's follow these rules of thumb to make this spec a blast!

Rule #1: Keep your specs DRY - use before blocks

The first line of each spec is identical - we create a post. It should be extracted to a before block so we don't have to mention it over and over again.

Rule #2: Use before :all instead of before :each when possible

Why should we create the post before each spec runs? If you're not changing your model's attributes between specs, you should definitely use before :all. A note on this: unlike before :each blocks which are wrapped in a DB transaction and thus getting rollbacked after the spec is executed, before :all blocks have to be cleaned explicitly so that you don't fill your test DB with garbage.

Rule #3: Use build instead of create when possible

We're using FactoryGirl to create a post. Creating is an expensive operation: The post has to go through validations, we save it to the DB and in many cases it requires other models to be created as well. For example, if part of a post validation says it has to have a user than we'll create a user as well. By building a post instead of creating we save all this time and get the same outcome.

Rule #4: Use describe to create nested specs groups. describe should relate to an operation or an object.

We're actually testing here 2 different methods of FindReaction. Specs 1-3 describe the reaction method while spec #4 describes the attach method. Each of these methods should have its own describe block.

Rule #5: Use variables instead of constants to keep your specs from becoming too fragile.

In lines 19-21 we create reaction templates and we want their locale to match the created post's locale. Instead of using the string "en" again and again we should use post.locale This makes your spec less fragile so that if someday someone changes the post's locale, he won't have to change the reaction templates creation as well.

Rule #6: Use RSpec's magnificent be_ syntactic sugar

RSpec has many syntactic sugars that makes your specs more readable. Let's look at line 6 for example: reaction.blank?.should == true When an object reacts to some_method_name? RSpec allows you to write the test in the following way: object.should be_some_method_name Or in our case: reaction.should be_blank

Rule #7: Use create! and save! instead of create and save

When creating or saving models to your DB, use create! and save! instead of create and save. The difference is that the bang version of the methods throws an exception if the operation fails, while the non-banged versions just return false. I wasted a lot of time debugging failed specs only to find out the model was not saved or created due to validation or another problem. By using create! and save! you'll be sure that the model was created successfully and it exists for you to test it, otherwise an exception will be thrown and you'll know about it immediately.

Midway

After applying these  7 simple rules our spec following already looks much better:

describe FindReaction do before :all do @post = build :post, locale: "en", type: "basic" end before :each do @post.reaction = nil end describe :reaction do #Spec 1 it "should return no reaction template when there are no reaction templates" do reaction = FindReaction.for(@post).reaction reaction.should be_blank end #Spec 2 it "should return no reaction template if reaction templates exist, but non fitting" do reaction1 = ReactionTemplate.create! locale: "de", subtypes: ["basic", "top_x"] reaction2 = ReactionTemplate.create! locale: "en", subtypes: ["slideshow", "top_x"] reaction = FindReaction.for(@post).reaction reaction.should be_blank end #Spec 3 it "should return the right reaction if it exists" do reaction1 = ReactionTemplate.create! locale: @post.locale, subtypes: [@post.type, "top_x"] reaction2 = ReactionTemplate.create! locale: @post.locale, subtypes: [@post.type, "top_x"] reaction3 = ReactionTemplate.create! locale: @post.locale, subtypes: ["slideshow", "top_x"] reaction = FindReaction.for(@post).reaction reaction.should_not be_blank reaction.should eq reaction2 end end describe :attach do #Spec 4 it "should change posts reaction_template when attaching it" do post.reaction_template.should be_nil reaction = ReactionTemplate.create! locale: @post.locale, subtypes: [@post.type, "top_x"] FindReaction.for(post).attach post.reaction_template.should_not be_nil post.reaction_template.should eq reaction end end end

You might think to yourself "OK, this looks great", but I've got a surprise for you: The interesting part is only ahead of us. Let's take the extra leap to improve our specs even more.

Rule #8: If/When rule of thumb

When there is a "if" or "when" in your it "should ..." clause, there's a hidden context in it. Looking at spec #2 our it clause looks like this: it "should return no reaction template if reaction templates exist, but non fitting" There's an ifin it, and that's exactly what we're looking for. If or when mean you're describing a state. These states should be extracted to a context block with a before block describing the state. After applying this rule spec #2 should look like this:

#Spec 2 context "reaction templates exist, but non fitting" do before :each do reaction1 = ReactionTemplate.create! locale: "de", subtypes: ["basic", "top_x"] reaction2 = ReactionTemplate.create! locale: "en", subtypes: ["slideshow", "top_x"] end it "should return no reaction template" do reaction = FindReaction.for(@post).reaction reaction.should be_blank end end

We should apply the same to specs 1,3 and 4

Rule #9: Use subject/it clauses to keep your specs DRY'er and cleaner

We can easily identify that specs 1,2 and 3 all test the same result: FindReaction.for(@post).reaction This can be easily extracted into subject {FindReaction.for(@post).reaction} phrase which does miracles to our test code.

Rule #10: One assertion per test case

I like to have only one assertion in each spec. This way if the spec fails, you immediately know what went wrong and don't even have to look at the detailed output of the failure. It also keeps your specs clean and focused on what they're supposed to test. Spec #3 has two assertions in it, in the first we check that a reaction is found and in the second we check what specific reaction was found. These are two completely different assertions and deserve different specs. The one assertion per spec rule is easily achieved with combination of Rule #9, we'll see why right away.

Rule #11: Use RSpec's expect {} block to describe a change in the state of an object

In spec #4 we test that the post's reaction_template has changed from nil to the attached reaction. RSpec has a nice and clean way to describe these kind of state changes: expect { some_operation }.to change{something}.from(initial_value).to(final_value)It is much cleaner and more readable this way. This is the result after applying rules #8-#11 to our spec:

describe FindReaction do before :all do @post = build :post, locale: "en", type: "basic" end before :each do @post.reaction = nil end describe :reaction do subject { FindReaction.for(@post).reaction } #Spec 1 context "there are no reaction templates" do it {should be_blank } end #Spec 2 context "reaction templates exist, but non fitting" do before :each do reaction1 = ReactionTemplate.create! locale: "de", subtypes: ["basic", "top_x"] reaction2 = ReactionTemplate.create! locale: "en", subtypes: ["slideshow", "top_x"] end it {should be_blank } end #Spec 3 context "fitting reaction templates exist" do before :each do @reaction1 = ReactionTemplate.create! locale: @post.locale, subtypes: [@post.type, "top_x"] @reaction2 = ReactionTemplate.create! locale: @post.locale, subtypes: [@post.type, "top_x"] @reaction3 = ReactionTemplate.create! locale: @post.locale, subtypes: ["slideshow", "top_x"] end it {should_not be_blank} it {should eq @reaction2} end end describe :attach do #Spec 4 context "attaching a reaction to post" do before :each do @reaction = ReactionTemplate.create! locale: @post.locale, subtypes: [@post.type, "top_x"] end it "should change posts reaction template" do expect { FindReaction.for(@post).attach }.to change{@post.reaction_template}.from(nil).to(@reaction) end end end end

Conclusion

So what have we earned in this refactoring process? I’ll point the main advantages of the new code:

Our test-suite runs faster

Our test file is more readable

The failure output provided by RSpec is better

We're not repeating ourselves between specs

It is extremely easy to extend the specs to describe new features

I find it much more enjoyable writing specs that looks cool (And you must admit, it does look cool).

I know that even this code can be optimized and refactored, but I had to stop somewhere :) I'll be glad to hear your insights and thoughts.

More sources: Refactoring from good to great by Ben Orenstein

By Erez Rabih @ FTBpro.com

[ftbpro-engineering]

Represent data in a natural way

(or - how i'm no longer afraid to use multiple DBs in one project)

Now, at FTBpro, we are using 3 different types of DataBases - MySql, ElasticSearch, Neo4j, and we are considering the option of adding Mongo to the party for a new upcoming project.

(* Actually we also use Redis, but it is used more for caching  then a persistent DB.)

But things weren't always like that!

At the beginning there was only SQL, and all was good.

But then came the "join tables",

And the need to add "just one tiny itty" new column to the table, 

And another one,

And another :-O

And so on... you get the point...

This kind of approach can work just fine, and people have been doing it for long time. but if you stop being scared, and open your mind for new possibilities, you may just be amazed of what the world has to offer :-)

Here is a real life example (although i exaggerated a bit): At FTBpro we have "users" and we have "Teams", and a user can follow few teams.

Classic SQL implementation will tell us to create a new table, lets call it "user_follow_team", and have a foreign key for user, and a foreign key for team. simple enough.

But suddenly, after this feature has been lunched, the evil product manager wants to add a new option - a user can also be a "fan" of a team. so now what? lets add a new "is_fan?" column to our table. Ok, so we solved that. but a week later he decides a user can also follow a "league"! have you got no mercy? 

Now lets try it with GraphDB, specifically with Neo4j:

Our users will be represented as nodes, and so are the teams.

A user is now following a team? no problem - just create a relationship between that user and that team, we'll call that relationship "follow".

(in cypher - Neo4j's query language - it is represented as - 

(user)-[:follow]->(team), pretty straight forward)

We want to add a "fan" relationship, no problem, just add it!

A user can now follow a league? no problem, league is also a node (like team and user), so just create the "follow" relationship between the user node and the league node!

Thats the beauty of it - its dynamic! no schemas, no need for redundant data, and the data representation is very intuitive.

For sites that implements social-network-like features, GraphDB is a perfect match! representing relationships between objects, using relationships! mind blowing :-)

Yes, it does mean that in some cases we need to query two different databases to get all the data we need, but it also means the load on each DB is smaller... And beside, Neo4j does a great job on indexing the data and is build to handle large data and growth without degradation of performance.

So don't be afraid to use different DBs according to your needs. representing data in a "natural" way is not only easier on the mind, but might also make your life easier on the technical level.

And special thanks to the great guys at GrapheneDB, which not only provides us with Neo4j hosting, but are also very responsive, give great support, and even helped us with the data modeling.

Check out http://www.neo4j.org/ to get you started. it has a lot of great video tutorials and guides and everything you need to get started.

If you are using Ruby like we do, also checkout the Neography and Neoid gems.

Posted by: Shai Kerer, backend dev @ FTBpro