Vacation Labs Blog

A "boot strapped" startup based out of Goa. Building an e-commerce platform for online travel. Doing interesting web projects on the side for some cash.

[vacationlabs-old-blog]

Create infographics

If you're interested, the data for the graphs was pulled directly out of our git repository using this shell script

[vacationlabs-old-blog]

Brilliant creative by AdRoll.com — a re-targeting platform for online ads. It gets the message across AND demonstrates their product!

PS: I had visited their website to research something on Facebook re-targeting.

[vacationlabs-old-blog]

Spotted this awesome comeback from MakeMyTrip. Context: Cleartrip's obvious and tad unoriginal dig at MakeMyTrip on April Fool's Day

@mkshitij @cleartrip Oh we loved it,usually once we acquire smaller companies,we like them to keep a similar UI as ours #clutterflutter

April 1, 2013

Now, why didn't they respond with a "public release" on their blog on 1st April itself? Would've been hilarious!

[vacationlabs-old-blog]

Installing mysql2 gem on Mac OSX

Just spent 20 mins trying to install the mysql2 gem on my Mac OSX dev machine. Here's the solution for anyone else who's stuck with the same problem.

The gem requires the mysql client library (libmysqlclient) which it wasn't able to find. Apparently, the client library is provided by the mysql-unified-dev fink package [1] which was broken (failing with a compilation error) [2]

So, I downloaded the MySQL Mac OSX tarball, extracted it into /usr/local/mysql, and installed the gem. And voila! It worked!

$ gem install -v 0.3.11 mysql2

[1] I prefer fink over homebrew

[2] Error message:

gcc -DHAVE_CONFIG_H -I. -I../include -I../include -I../include -I/usr/local/fink/include -DDBUG_OFF -Wimplicit -Wreturn-type -Wswitch -Wtrigraphs -Wcomment -W -Wchar-subscripts -Wformat -Wparentheses -Wsign-compare -Wwrite-strings -Wunused -O3 -fno-omit-frame-pointer -g -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DIGNORE_SIGHUP_SIGQUIT -c longlong2str_asm.c longlong2str_asm.c:29:7: warning: type specifier missing, defaults to 'int' [-Wimplicit-int] char *longlong2str(longlong val,char *dst,int radix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../include/m_string.h:240:38: note: instantiated from: #define longlong2str(A,B,C) int2str((A),(B),(C),1) ~^ longlong2str_asm.c:29:20: note: instantiated from: char *longlong2str(longlong val,char *dst,int radix) ^ longlong2str_asm.c:29:7: warning: type specifier missing, defaults to 'int' [-Wimplicit-int] char *longlong2str(longlong val,char *dst,int radix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../include/m_string.h:240:42: note: instantiated from: #define longlong2str(A,B,C) int2str((A),(B),(C),1) ~^ longlong2str_asm.c:29:33: note: instantiated from: char *longlong2str(longlong val,char *dst,int radix) ^ longlong2str_asm.c:29:7: warning: type specifier missing, defaults to 'int' [-Wimplicit-int] char *longlong2str(longlong val,char *dst,int radix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../include/m_string.h:240:46: note: instantiated from: #define longlong2str(A,B,C) int2str((A),(B),(C),1) ~^ longlong2str_asm.c:29:43: note: instantiated from: char *longlong2str(longlong val,char *dst,int radix) ^ longlong2str_asm.c:29:7: error: expected parameter declarator char *longlong2str(longlong val,char *dst,int radix) ^ ../include/m_string.h:240:49: note: instantiated from: #define longlong2str(A,B,C) int2str((A),(B),(C),1) ^ longlong2str_asm.c:29:7: error: expected ')' longlong2str_asm.c:29:7: note: to match this '(' char *longlong2str(longlong val,char *dst,int radix) ^ ../include/m_string.h:240:36: note: instantiated from: #define longlong2str(A,B,C) int2str((A),(B),(C),1) ^ longlong2str_asm.c:29:7: error: conflicting types for 'int2str' char *longlong2str(longlong val,char *dst,int radix) ^ ../include/m_string.h:240:29: note: instantiated from: #define longlong2str(A,B,C) int2str((A),(B),(C),1) ^ ../include/m_string.h:234:14: note: previous declaration is here extern char *int2str(long val, char *dst, int radix, int upcase); ^ longlong2str_asm.c:29:7: error: parameter name omitted char *longlong2str(longlong val,char *dst,int radix) ^ ../include/m_string.h:240:38: note: instantiated from: #define longlong2str(A,B,C) int2str((A),(B),(C),1) ^ longlong2str_asm.c:29:20: note: instantiated from: char *longlong2str(longlong val,char *dst,int radix) ^ longlong2str_asm.c:29:7: error: parameter name omitted char *longlong2str(longlong val,char *dst,int radix) ^ ../include/m_string.h:240:42: note: instantiated from: #define longlong2str(A,B,C) int2str((A),(B),(C),1) ^ longlong2str_asm.c:29:33: note: instantiated from: char *longlong2str(longlong val,char *dst,int radix) ^ longlong2str_asm.c:29:7: error: parameter name omitted char *longlong2str(longlong val,char *dst,int radix) ^ ../include/m_string.h:240:46: note: instantiated from: #define longlong2str(A,B,C) int2str((A),(B),(C),1) ^ longlong2str_asm.c:29:43: note: instantiated from: char *longlong2str(longlong val,char *dst,int radix) ^ longlong2str_asm.c:31:39: error: use of undeclared identifier 'val' return longlong2str_with_dig_vector(val, dst, radix, _dig_vec_upper); ^ 3 warnings and 7 errors generated. make[2]: *** [longlong2str_asm.o] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 ### execution of make failed, exit code 2 ### execution of /tmp/fink.o46y_ failed, exit code 2 Removing runtime build-lock... Removing build-lock package... /usr/local/fink/bin/dpkg-lockwait -r fink-buildlock-mysql-unified-5.0.96-1 (Reading database ... 65817 files and directories currently installed.) Removing fink-buildlock-mysql-unified-5.0.96-1 ... Failed: phase compiling: mysql-unified-5.0.96-1 failed

[vacationlabs-old-blog]

Our experience with using HTML5 as a replacement for Flash

We needed to build a short animation, for a client's website, which explained a rather complex concept within 60 seconds to a "lay audience" [1] We had never built an animation before - neither in Flash nor in HTML5. There would've been a learning curve with either choice. With the exception of newer iOS devices Flash would "just work" whereas HTML5 would be more of an uphill battle. However, somehow we felt more "in control" of HTML5. We would be building stuff using technologies that we were pretty good at - HTML, CSS, & JS. So, with a clear bias towards HTML5 we tried seeing if we could make it work... ## Overall browser support for HTML5 There are a lot of different web technologies that are grouped together and have been given the common name of HTML5. For our animation, we didn't need all of them. Probably just the `

` element, CSS3 transitions + 2d transformations + animations + gradients, along with various other CSS3 properties like `text-shadow`, `box-shadow`, `border-radius`, and multiple backgrounds. A quick glance at [Html5Readiness.com](http://html5readiness.com/) and [Can-I-Use](http://caniuse.com/) gave us the following: * `

` & `border-radius` were supported by all the major browsers except IE8 * CSS3 animations + transitions were not support by Firefox <= 3.6 and IE <= 9 * CSS3 2d transformations were not supported by IE8 * Multiple backgrounds were not supported by IE8. So, basically Firefox <= 3.6 and IE <=9 could cause issues with the HTML5 animation we wanted to do. ## Browsers used by our audience Next, we dug up the visitor stats for the website using Google Analytics. * IE8 was used by 10.3% of our visitors * IE9 was used by about the same number -- 10.3% * Firefox 3.6 (and lower) was only 0.4% So, it was safe to say that about 80% of our visitors would be able to see the HTML5 animation without any hiccups. ## Picking from available HTML5 animation tools Next, we needed to pick a tool / library that would allow us to rapidly prototype the animation video over multiple iterations. Most of the stuff out there basically falls into two buckets: * Javascript libraries that help you manipulate various CSS properties against a timeline -- eg. [jsAnim](http://jsanim.com/), [$fx](http://fx.inetcat.com/), etc. * Complete animation suites -- the HTML5 versions of Flash Builder, eg. [Adobe Edge Animate](http://html.adobe.com/edge/animate/), [Tumult Hype](http://tumult.com/hype/), etc. While we would've loved to work close to the metal with JS libraries, we would've not been able to iterate so quickly. We took a look at Tumult Hype, but had to pass because it was only available for Mac OS X. We downloaded the beta version of Adobe Edge Animation v0.16 (then it was called only Adobe Edge) and while it had its quirks, it looked like it could help us get our job done. So, we took the plunge into building an animation video using HTML5 using a tool that was still in its infancy! ## Issues with Adobe Edge Animate **No progressive loading:** One of the first issues that we came across was that animations built using Adobe Edge were "monolithic" in nature, i.e. there was no way to progressively load the components of an animation. We wanted the animation to start as soon as the components required for the first few frames had been fetched by the browser (with the rest of the stuff being fetched in the background). Unfortunately, there is currently no way to do this in Adobe Edge. **No spriting:** To make matters worse, Adobe Edge was making about 50+ individual request to fetch all static assets from the server. I guess they didn't get around to implementing auto-spriting and JS bundling in their early releases. So, we had to do it manually. We lived with the 50+ requests during the initial prototyping phases, but for the final release we built image sprites manually. In the Edge builder each element on a frame was basically the same image-sprite cropped to a different area. **Weird JS loading issues:** We probably encountered a bug in the (custom built?) async loader used by the Adobe Edge "runtime" [2] to load its JS dependencies. About 40% of the time the animation failed to start because of certain JS objects not being defined (due to dependencies not loading in the correct sequence). We solved this by putting all the JS dependencies into a [Rails manifest file](http://guides.rubyonrails.org/asset_pipeline.html#manifest-files-and-directives) to kill two birds with one stone -- circumvent the bug AND reduce the number of server requests. //= require require //= require edge_includes/jquery.easing.1.3.js //= require Slide_1_edgePreload.js //= require edge_includes/edge.0.1.7.min.js //= require edge_includes/edge.1.0.0.min.js //= require edge_includes/json2_min.js //= require Slide_1_edge.js //= require Slide_1_edgeActions.js We had to fool the async loader by loading a single blank JS file so that it would trigger the event that started the animation. aLoader = [{ load: "/assets/blank.js"} ]; **Weird Google Chrome bug:** While technically not an issue with Adobe Edge Animate, we did end up hitting some bugs with Google Chrome leaving behind artifacts after CSS3 transitions. A known bug, this was to be fixed in later versions of the browser. ## The Other 20% -- Non HTML5 browsers While we were pretty happy with the HTML5 animation - only 80% of our audience could see it. 20% of them were still left in the lurch without the animation taking them through a key piece of communication. We turned towards [conditional IE comments](http://www.quirksmode.org/css/condcom.html) to help us deliver a "degraded" version of our spanking new animation to browsers unable to handle it. The degraded version was a simple slide show of static images (using [SlidesJS](http://slidesjs.com/)) consisting of the most important animation frames. <%= render :partial => "animation" %> ## In the end... We were impressed with what Adobe has achieved with Edge Animate. They have now released v1.0 of the entire [Edge suite](http://html.adobe.com/edge/animate/) for *free*. (Although we hate their attempt at a private app store with the ["Creative Cloud"](https://creative.adobe.com/apps) - rant for a separate post!) As with any new technology, we faced quite a few teething issues, but we're sure these will get sorted out with time -- as HTML5 support in browsers gets better and the tools to author rich content using these standards mature. ------------- [1] You can view the final result at [Magic Bus' homepage](http://www.magicbus.org). Magic Bus is an Indian NGO, which works towards alleviating poverty by driving behavioural change in children. [2] Calling it a "runtime" makes it sound like some fancy VM, but it's basically a JS file that can take the animation definition (define in plain ol' JSON) as input to produce an animation in a container div. The Adobe Edge Builder basically represents the animation video as a well-defined JSON.

[vacationlabs-old-blog]

Automated backups in under 30mins

Once you've gotten your application into production, do not procrastinate in setting up automated backups for important data. You'll be thanking yourself the day your cloud-server instance dies on you. Or after a botched-up DB migration job. Or probably an image processing script that wipes out all images uploaded by your users. Believe me, there are hundreds of things that can go wrong with your production data -- apart from an actual system crash. There's really no excuse for not setting up automated backups early-on. Especially when you have nifty tools like the [Backup gem](https://github.com/meskyanichi/backup) readily available. It took us under 30mins to get started with the Backup gem from scratch and to configure it to do the following: 1. Every hour, take an SQL dump of our Postgres DB and save it to Amazon S3 while keeping last 7 days worth of backups. 2. Every hour, sync the images folder (user uploaded images) to a bucket in Amazon S3 Here's what's missing from our setup: 1. To ensure against something (or someone) accidentally deleting images, we should probably setup a way to take fresh backups (not sync) of the images folder every week. Or modify the sync behaviour to only upload new images, not delete them. 2. A way to restore backups automatically -- while the Backup gems does a brilliant job of automating your backups, it doesn't have any support for automated **restores**. Even though we haven't suffered any data loss in production, we've already used the backups to construct the production environment locally -- to test a few complicated DB migration scripts before rolling them out to production. If you aren't already backing up your data, get started with the [Backup gem](https://github.com/meskyanichi/backup) now! It even generates configuration file templates for you through its [generator scripts](https://github.com/meskyanichi/backup/wiki/Generator)! It'll might even take less than 30minutes! Here's how we've set it up... ## The Setup Here's a quick illustration of how the Backup gem works at a *conceptual level*. This may not be a true representation of how the data actually flows within the system.  You want to periodically get data from your data sources into your backup storage. Along the way you may want to process your data by one or more "steps" -- compressing, encrypting, splitting into smaller chunks (to ease the process of transferring large backup files), or cycling (retain backups for the last 3 months). Let's start with the entries that landed up on the crontab:

# m h dom mon dow command 0 */1 * * * /bin/bash -l -c 'rvm gemset use b2b && cd /home/nanda/b2b/current && backup perform -r lib/backup/ -t db_backup' >> /tmp/backup.log 2>&1 0 */1 * * * /bin/bash -l -c 'rvm gemset use b2b && cd /home/nanda/b2b/current && backup perform -r lib/backup/ -t image_backup' >> /tmp/backup.log 2>&1

The `db_backup` and `image_backup` scripts ("models" for the Backup gem for some strange reason) are set to run every hour. (Side rant: Why can't cron ship with sensible defaults for the shell it fires up? It took me 10 minutes - out of the total 30 minutes - fiddling with the various `bash` and `rvm` settings to get this running properly!) ## Database backup Here's our config file for backing up the production Postgres DB as an SQL dump (the data source) to Amazon S3 (the data storage), while retaining 168 previous backups (cycling), compressing the SQL dump with bzip2, and splitting it into chunks of 250MB if the size crosses that limit. Finally, we've set-up up email notifications for letting us know in case something goes wrong while performing a backup.

Backup::Model.new(:db_backup, 'Main database backup') do # Split the backup file in to chunks of 250 megabytes # if the backup file size exceeds 250 megabytes split_into_chunks_of 250 ## # PostgreSQL [Database] # database PostgreSQL do |db| db.name = "name" db.username = "username" db.password = "password" db.host = "localhost" db.port = 5432 # In case you want to skip some tables or backup only few tables # db.skip_tables = ["skip", "these", "tables"] # db.only_tables = ["only", "these" "tables"] db.additional_options = ["-xc", "-E=utf8"] end ## # Amazon Simple Storage Service [Storage] # The Amazon S3 are picked up from the global config if not specifically mentioned here store_with S3 do |s3| s3.path = "/" # The cron job runs this backup job every hour. The following line asks # the Backup gem to retain the last 168 backups, i.e. 7 days worth of hourly backups s3.keep = 168 end ## # Bzip2 [Compressor] # compress_with Bzip2 # Notify me by email if anything goes wrong... notify_by Mail do |mail| mail.on_success = false mail.on_warning = false mail.on_failure = true end end

## Image backup via sync Here's how we're **syncing** all the images uploaded by our users (photos linked to various trips) to Amazon S3. Note, this **syncs** the `public/system/images` directory with a bucket in S3 instead of blindly **uploading** all the images each time the backup is performed (it would be really expensive to upload potentially many GBs of images every hour, or even every day). Only new photos that have been added to the directory since the last backup are uploaded. What this also means is that, if any photos that have been **deleted** from the directory are also deleted from S3 (potentially dangerous).

Backup::Model.new(:image_sync, 'Image sync for public/system/images/') do ## # Amazon S3 [Syncer] # Mirroring: # # When enabled it will keep an exact mirror of your filesystem on S3. # This means that when you remove a file from the filesystem, # it will also remote it from S3. # # Concurrency: # # # sync_with Cloud::S3 do |s3| s3.access_key_id = "access ID" s3.secret_access_key = "secret key" s3.region = "ap-southeast-1" s3.bucket = "bucket-name" s3.path = "/image_backup" s3.mirror = true # `concurrency_type` may be set to: # - false (default) # - :threads # - :processes s3.concurrency_type = false # Set `concurrency_level` to the number of threads/processes to use. # Defaults to 2. s3.concurrency_level = 2 s3.directories do |directory| directory.add "public/system/images/" end end notify_by Mail do |mail| mail.on_success = false mail.on_warning = false mail.on_failure = true end end

## Common settings Finally, here's the common settings file for defining things that are common across various backups -- email settings, AWS credentials, GPG credentials, etc.

## # Global Configuration # Add more (or remove) global configuration below Backup::Storage::S3.defaults do |s3| s3.access_key_id = "access key" s3.secret_access_key = "secret key" s3.region = "ap-southeast-1" s3.bucket = "bucket name" s3.keep = 168 end Backup::Notifier::Mail.defaults do |mail| mail.from = 'Backup Error ' mail.to = '[email protected]' mail.address = 'smtp.gmail.com' mail.port = 587 mail.domain = 'vacationlabs.com' mail.user_name = '[email protected]' mail.password = 'password' mail.authentication = 'plain' mail.enable_starttls_auto = true end ## # Load all models from the models directory (after the above global configuration blocks) Dir[File.join(File.dirname(Config.config_file), "models", "*.rb")].each do |model| instance_eval(File.read(model)) end