Natacha Codes @natachaspringer - Tumblr Blog

How a SlackBot made us more productive...

All of our engineers are now using Slack. The issue we were facing was that engineers asked for support in Slack rooms rather than using our clunky ticketing system. Support was done faster but we were losing all traceability of the work that was being done. I figured that our best solution was to build a SlackBot for this. Users would simply add /support in front of their requests and a ticket tagged support would be created. Once the request has been completed, our engineer just had to use /closeTicket [ticket number] to close that request in Slack without having to even open our Ticketing system.

#devops slackbot productivity

DevOps is a Mindset

Right after graduating from the Flatiron NYC web Fellowship, I was extremely fortunate to have been selected for a fellowship at Fog Creek Software.

When I started receiving job offers, my Fog Creek mentor, Jude Allred, offered extremely helpful insights into the pros and cons of each position. Eventually I accepted an offer with Dow Jones as a DevOps Engineer. It was both exciting (because devops is all new to me) AND scary (because devops is all new to me). Jude helped calm my nerves by putting me in touch with two of his colleagues at Trello: Jack, who works on Continuous Integration, and Quentin, who handles DevOps. Those conversations were priceless for a newbie like me and very inspiring. They also showed me how cool DevOps could be!

Jude also pushed me to automate everything, and one day it "clicked": DevOps is really a mindset about automating your infrastructure configuration, building the tools to monitor your infrastructure, and collaborating with other engineers to design the best solution for your organization.

To start preparing for my new job I've been writing AWS scripts, for example, to upload files to cloud storage (S3) and configure visibility using the Ruby AWS SDK, like this:

It’s going to be hard to say "goodbye", everyone has been super nice and interesting to talk to, and having lunch all together really fostered interesting conversations! Fog Creek not only provided a great space for me to work from every day, it also helped me become a better developer by giving the guidance and confidence I needed to start a new job in a new field.

Thanks to all the Creekers !

DevOps Tools part I : Getting started with Chef

After learning how to build webapps using Ruby on Rails, i have started looking at other tools that use this language.

I came across Chef - it is a tool that simplifies the way you perform infrastructure automation and configuration management tasks. Chef is a way to turn infrastructure into code which makes it ideal for continuous deployment. It allows you to quickly deploy entire environments instead of only single applications. It uses Ruby and handles configuration by packing details into what it calls recipes.

Chef is made up of several components that have specific roles:

-Chef server : It stores recipes as well as configuration data. It is the central machine that other machines ('the nodes') will use for deployment configuration.

-Chef nodes : they represent the target for deployment. A node could be any physical, virtual or cloud machine that is configured to be maintained by a chef-client, an application used to communicate with the chef server.

A Chef workstation is where the chef configuration details are created or edited. The code is then pushed to the Chef server where it will be available to be easily deployable to any node.

I tested out Chef on a CentOs virtual machine and was amazed at how easy it was to use.

Very similarly to Rails commands you run this command to generate a cookbook named webserver:

chef generate cookbook webserver

A cookbook provides structure to your recipes and enables you to more easily reference external files, such as our web server's home page. In essence, a cookbook helps you stay organized.

Then to generate a homepage template you'd run :

chef generate template webserver index.html

In this newly created file (index.html.erb) you could write a static hello world or use erb templating to add dynamic information about your server.

Then you will need to update the recipe to enable and start Apache and to reference your template.

package 'httpd' service 'httpd' do action [:start, :enable] end template '/var/www/html/index.html' do source 'index.html.erb' end

You are now ready to run your webserver locally with this command :

chef-client --local-mode --runlist webserver

Check your local host ! Voila ! You automated your local web server ! Another cool thing you could do is generate dynamic web pages by using erb templating, for example your index.html.erb file could contain this :

<h1>hello from <%= node['ipaddress'] %></h1>

You can dynamically add all sorts of information about your server such as the node's IP address, total installed memory, number of CPU's etc...

I enjoyed learning more about Chef and I am looking forward to start using it more and setting it up in the context of continuous delivery for various web apps.

Merge Sort Algorithm in Ruby

Merge sort is is a comparison based sorting algorithm also known as the 'divide and conquer' algorithm.

Conceptually, it consists of :

-divide the unsorted list into n sublists, each containing one element

-repeatedly merge sublists to produce new sorted sublists until there is only one unsorted sublist remaining.

Here's one example implementation using Ruby code:

def merge_sort(m) return m if m.length <= 1 middle = m.length / 2 left = m[0,middle] right = m[middle..-1] left = merge_sort(left) right = merge_sort(right) merge(left, right) end def merge(left, right) result = [] until left.empty? || right.empty? if left.first <= right.first result << left.shift else result << right.shift end end result + left + right end ary = [7,6,5,9,8,4,3,1,2,0] p merge_sort(ary) # => [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

Project Euler - Problem One

In order to develop my programmatic thinking I decided to tackle those problems one by one with Ruby or Javascript...

"If we list all the natural numbers below 10 that are multiples of 3 or 5, we get 3, 5, 6 and 9. The sum of these multiples is 23.

Find the sum of all the multiples of 3 or 5 below 1000."

My answer using Ruby:

(1..999).select{|x| x%3 ==0 || x%5 ==0}.reduce(:+)

(1..999).select{|x| (x % 3 == 0) || (x % 5 == 0) }.inject{|sum,n| sum+n}

=> 233168

My answer using Javascript :

var total = 0;

for (var i = 0; i<1000; i++) {

if (i%3 == 0 || i%5 == 0) {

total += i;

}

console.log(total);

=> 233168

#Project Euler

Alternative to using StockTwits API

While working with StockTwits API we encountered a major issue. Stocktwits would only allow 200 requests to their API per hour. We were very often going over this limit since we wanted to get the latest stocktwits every seconds or so....

To get around this issue my teammate Mandy Yeung wrote a nifty scraper to get the data we were interested in and then ran the code every 10 seconds using the setInterval() function in Javascript. The result is the same and works very well.

def stocktwits_urls urls = [] self.stocks.each do |s| urls << "http://stocktwits.com/symbol/#{s.ticker}" end urls end def visit_urls messages = [] stocktwits_urls.each do |url| doc = Nokogiri::HTML(open(url)) doc.xpath('//ol/li').each do |item| messages << item.values end end messages end def stocktwits_feed msgs = [] visit_urls.each do |m| message = {} full_message = JSON.parse(m[0]) message[:time] = Time.parse(full_message["created_at"]) message[:avatar] = full_message["avatar_url"] message[:body] = full_message["body"] message[:user] = full_message["user"]["username"] msgs << message end msgs.uniq.sort_by{|hsh| hsh[:time]}.reverse! end

The function stocktwits_feed is called in a partial and then here's the javascript code to reload the partial every 10 seconds :

$(document).on('page:change', function (){ setInterval(function() { $('.stocktwits').load('/users/stocktwits'); }, 10000); });

POSTMAN

When i started building apps with various APIs, i quickly realized that testing was key. I came across this nifty tool, a Chrome app called “Postman”.

Postman is a REST client that allows you to be more efficient while working with APIs.

For example was app was listening for POST requests from another server to fire a text message. So I used POSTMAN to make those requests, you can even pass various params that you define in their interface.

I hope this will help you in your projects !

Building a map and solving new problems

I love maps and saw some pretty cool maps that i wanted to try to reproduce. The first part of my project was to locate the right data.

In France we have an institute dedicated to geographical institute called IGN (National Institute of Geographical Information). All the data they provide is free and can be used for any projects (even commercial).

Unfortunately the format of the data is shapefile (compressed with 7-zip) - geospatial vector data. D3 (the javascript library used to visualize the map) is not able to use it, so the data had to be converted to GeoJSON.

The best tool for the job is a converter available on line called "my GeoData converter" which converts shapefile to GeoJSON.

Ok so armed with my data I started writing a script (inspired from this tutorial), I spent some time trying to figure out why my GeoJSON wouldn't load...then realized I had to set up and use a local web server to see it in the browser. I set up MAMP which was pretty straight forward.

My next problem was to show off my map into my blog.

To first get around this problem i used github to host my project right off my github repo (very easy just add a gh-pages branch on your repo), the result can be seen here, but i really wanted to embed it in my tumblr and got a CORS (cross origin resource sharing) error from the github server so i had to set up my own NGINX server (on Digital Ocean of course) and enable CORS on it for the script to load on Tumblr.

The last thing is thatTumblr wouldn't let me embed the stylesheet but you can still see the result below (click on it and play with it !) :

IFTTT Hacking

I just came across IFTTT and found out it was a very cool website that everyone could use !

I quickly realized that it had some limitations and that the possibilities were restricted to some pre-set 'recipes' as IFTTT calls them.

After doing some research I found this repository :

ifttt uses wordpress-xmlrpc to communicate with the wordpress blog. It causes ifttt to be fooled into thinking of this as a genuine wordpress blog. This 'webhook' then submits a post request to any specified url.

I set up a rails server that i deployed on Heroku (http://natachapostserver.herokuapp.com/) to test this theory. I then coupled it with a call to the Twilio API to send me a phone call in French to let me know that it is time to eat when a tweet from @DUMBOfoodstruck is being sent. Ok this sounds a little complicated but here's a little diagram to explain it all :

There's probably easier ways to get a phone call when a tweet is made, but I got curious when I learnt about IFTTT and wanted to test its limits. It was also a great exercise for me to deploy a rails server and play around with the Twilio API.

The future of distributed computing is NOW...

While developing small apps I realized that shipping code to a server is very hard...You need to set up your server with an OS, copy your file securely etc...It probably takes longer to ship code to a server than a case of wine from France. This is mostly due to the heterogenous nature of web apps, which have databases, background workers, web front end...etc...

Before Docker, you had to build a virtual machine with its own kernel and OS with your app to have your app running on a server.

Now with Docker, you can build your code into a container-like structure, and then push it to any server running docker... and it will run identically everywhere.

Docker was born last year and 400,000+ downloads, 300+ contributors, and 10,000+ GitHub stars later, Docker is now one of the most popular open source projects in the world.

The easiest solution for a newbie web app developer is deploying on Heroku, but I quickly realized that locking into heroku limits what you can do and can become expensive.

Now I can just use Dokku, a mini Heroku in its own Docker container. Dokku is similar to Heroku in that you can deploy to a remote server. The difference is that Dokku is built to deploy to a single, personal server and is extremely lightweight. It can also be cheaper than using Heroku and will give you more flexibility.

And best yet Digital Ocean, my favorite cloud server host is now preloading its servers with it !

This field has been moving so fast and is really being revolutionized.

CoreOS, a re-architectured Linux distribution system using Docker (and originally formed from Chromium OS), has just released its beta version last month. CoreOS enables warehouse-scale computing on top of a minimal, modern operating system that is the perfect lightweight host for Docker. Utilizing CoreOS as a Docker host will help squeeze more performance out of servers and should make things cheaper for us web developers.

SQLite or PostgreSQL

A relational databases are databases that hold a collection of tables of data items. SQL is a special purpose programming language designed to manage the data in a relational database.

The three major open source relational database management systems (DBMS) are:

- SQLite

- MySQL

-PostgreSQL (most advanced)

Since we'll be mostly using SQLite and PostgreSQL here are their pros and cons.

SQLite

When an app uses SQLite the integration works with functional and direct calls made to a file holding the data instead of communicating through an interface. This makes SQLite extremely fast and efficient.

Its supported data types are : NULL, INTEGER, REAL, TEXT, BLOB.

Advantages : File based, great for developing and even testing.

Disadvantages : No user management, no way to tinker with for additional performance.

PostgreSQL

PostgreSQL is the most advanced opensource RDBMS.

It supports a LOT of data types.

The advantages of using PostGreSQL are:

-its strong community

-strong third-party support

- it is extensible (it is possible to extend it programatically with stored procedures)

-it is object oriented (with support for nesting and more)

It disadvantages are :

-its performance/speed - for simple read-heavy operations it can be overkill and appear less performant than its counterparts such as MySQL

-unless you require absolute data integrity, it is overkill for simple set-ups.

-replication : unless you are willing to spend time and energy, achieving replication is not simple with PostgreSQL

Why Digital Ocean is awesome...

First of all it is CHEAP.

I wrote a program for the common good and realized that deploying it would cost me money.

I was dumbfounded by this concept so i decided to go ahead with my project and searched for the best solution to deploy it. At first I used Heroku for free but then realized that if I added dynos (a dyno a lightweight container running a single user-specified command), the fee would quickly add up !

I came across Digital Ocean and my life changed forever ! Digital ocean is a service that provides cloud hosting built for developers. For 5$/month you get a server (they call it a 'droplet') and get to do whatever you want with it. It's definitely more time consuming than Heroku but you do get to set your server any way you'd like !

They give you total control of your server and just like your favorite sleazy motel, they do charge by the hour so using a 512MB server just to test something for a couple of minutes will only cost you $0.007 !

Also the interface is very beginner-friendly.

After you created your server and set up your SSH key(preset on your droplet's control panel), so you can also connect to your server from your computer by typing :

ssh root@[your.ip.address.here]

And Voila !

Figaro simplified...

I had a problem....How to showcase my code on GitHub without giving away my authentication parameters like my email account credentials or API keys ? Our instructor Steven Nunez told us to use Figaro. And then my investigation began...

Figaro is....

No wait...Figaro is a Ruby gem that parses a Git-ignored YAML file in your application and loads its values into ENV, in other words it is a practical alternative to setting environment variables in the Unix shell.

The gem also gives you a syntax to access the environment variables as Figaro method calls.

How do I use Figaro ?

In your gemfile add :

gem 'figaro'

Then in Rails run

$ rails generate figaro:install

(that's when the file config/application.yml is created and added to .gitignore) In this new file, you can add environment variables as key,value pairs. For example :

GMAIL_PASSWORD: Your_Password

And then in your app you can access them by using :

ENV["GMAIL_PASSWORD"]

If you’re deploying to Heroku, the gem gives you a rake task that will set all the environment variables on Heroku.

rake figaro:heroku

There are other alternatives to solving this problem but using this popular gem is definitely the simplest way...

ROT13 encryption in Ruby

ROT13 is a substitution cypher where each letter is substituted with the letter 13 letters after in the alphabet. It is one of the oldest encryption method and was created by Julius Caesar in ancient Rome. It is also very convenient because the same method can be used both for encoding and decoding (there are 13x2 letters in the alphabet)...

There are a fews gem (rot13,crypt-rot13...) that were developed but I wanted to be able to use Ruby to develop a ROT13 method from scratch.

On learnstreet.com, which is a website that teaches people how to code they suggested this solution :

def rot(str) rot13 = "" lower_case = "abcdefghijklmnopqrstuvwxyz" upper_case = lower_case.upcase lower_case = lower_case.each_char.to_a upper_case = upper_case.each_char.to_a char = str.each_char.to_a for i in 0...char.length if char[i] == ' ' rot13 += ' ' elsif (char[i] == char[i].upcase) loc = upper_case.index(char[i]) idx = (loc+13)%26 rot13 += upper_case[idx] else loc = lower_case.index(char[i]) idx = (loc+13)%26 rot13 += lower_case[idx] end end return rot13 end puts rot("natacha")

But that just seemed a tad too long for my taste...there had to be a better way ! I finally decided to experiment with the tr Ruby method :

"natacha".tr!("abcdefghijklmnopqrstuvwxyz", "nopqrstuvwxyzabcdefghijklm") #=>"angnpun"

After more digging, i realized that the 'tr' method could also handle ranges of characters.

"natacha".tr!("a-z", "n-za-m") #=>"angnpun"

And finally, I added capital letters to the ranges as well :

"natacha".tr!("A-Za-z", "N-ZA-Mn-za-m") #=>"angnpun"

Now I am looking forward to start looking at various ways to establish rot13 encryption in other computer languages! Javascript-here I come !

Optimism...

Today in class we had to read this very interesting blog post and Blake our instructor asked us to write about it.

To me, the most interesting part of the post was :

"Let’s think about those three things whenever we look at someone or something. Let’s think of the positives as personal, general, and permanent. Let’s think of the negatives as impersonal, specific, and temporary. Let’s actually go out of our way to inject these three things into our discussions and debates and flame wars."

I will try to apply those principles during my journey at the Flatiron School and beyond...

Great factorial method for Ruby

I was surprised to find out that there is no factorial method in Ruby. I finally found this on the internet and it looks beautiful !

def factorial(n) return 1 if n.zero? 1.upto(n).inject(:*) end

Going from development to production...

When you deploy on Heroku, do not forget to migrate your database as well (heroku run rake db:migrate)....Otherwise you'll be spending some time figuring out why you app doesn't work on Heroku despite your code being 'fine'...

Trending Blogs

Recently Viewed Blogs

Natacha Codes