#ruby-003

[flatironschool]

Porting SQLite3 Data to PostgreSQL Database in Rails 4

The following is a guest post by Joe O’Conor and originally appeared on his blog. Joe is currently in the Ruby-003 class at The Flatiron School. You can follow him on Twitter here.

This past week I’ve been building a database with almost 1,000,000 records across five tables. So far, it has been just fine to use the Rails default, SQLite3, to manage it. That’s no longer the case, as I need to start thinking about the deployment environment for my application, and I have also been running into some issues with my SQLite3 database locking as I try to add records. Thus, I have ported my data over to PostgreSQL. The process is straightforward, but I did run into a few hangups and did not find a great guide to cover my situation. After some struggling, I got my new PostgreSQL database up and running with all of my data. Here’s how I did it.

First, I had to create a dumpfile from my sqlite3 database. I had been doing this already as quick way to back up my data. I read that it might be more convenient to dump it into a different format, like CSV, that is better understood by other databases like PostgreSQL, but I used the normal SQLite3 command and found that the manual adjustments from there were not cumbersome.

This is the path to my database: db/development.sqlite3. The .dump command breaks down your database into SQL statements, including everything you need for your schema. Then you indicate where you want to direct the dump using >. I created my dumpfile in the same directory and added a timestamp. Since the dump gives you SQL statements, the file extension should be .sql.

This file is sufficient to reseed another SQLite database, but PostgreSQL databases have some differences in syntax and datatypes. At first, I looked around to find a script or an application to manage these changes for me. My advice is: don’t bother. Finding one is hard enough, then you have to learn how to use it. A simple find/replace in your dumpfile will solve it just fine. This is what I had to change:

I only had to update this last one because I had entries in “description” columns that were too long. According to this very helpful post on StackOverflow, you may also need to change your boolean values into a type that is compatible with PostgreSQL. My booleans were already formatted correctly. For reference, here’s the list from the PostgreSQL documentation:

“Leading or trailing whitespace is ignored, and case does not matter. The key words TRUE and FALSE are the preferred (SQL-compliant) usage.” docs

The next step is to take out the SQLite-specific statements. At the top of my dumpfile, I had PRAGMA foreign_keys=OFF;. This tells the database to not enforce foreign key constraints, but PostgreSQL handles these differently. I took out the statement with no issue.

At the bottom of my dumpfile, I had a statement like this for each of my tables:

INSERT INTO "sqlite_sequence" VALUES('ks_users', 227393);

These tell SQLite where to set the next value for the autoincrementing primary keys of my tables, which in the example isks_users. PostgreSQL does the same thing, but in a different way, using sequences, which are special database objects designed for this purpose. We’ve already told our PostgreSQL database to use sequences when we changed our primary keys toSERIAL instead of INTEGER with AUTOINCREMENT, so the next step is just to take out all references tosqlite_sequence tables. (If you need to add new records to your database after the conversion, you’ll need to make one other change that we will talk about a bit lower down.)

If you haven’t created the PostgreSQL database yet, follow these steps:

Update config/database.yml to look something like this:

Start the database server with this command (if you’ve got the default settings)

$ postgres -D usr/local/var/postgres

Create the database

$ rake db:create

Don’t run the migrations. All the necessary statements are in the dumpfile.

At this point you should be able to fill your PostgreSQL database with your SQLite data with this command:

$ psql -d my_database_development -W < db/sqlite-dumpfile201311231200.sql

This invokes the PostgreSQL binary. The -d flag tells it to open a database with the name that follows (in this case “my_database_development”; the name of your database can be found in config/database.yml). Then -W specifies to prompt for your password. This password will be the password for your OSX user account. The last part of the command specifies which file to use to feed in to your new database. This will be the SQLite dumpfile that you created and then modified.

There’s one more step you need to do. If left as is, PostgreSQL will try to add records with the primary key starting at 1, because we removed the sequence information from the dumpfile, as it pertained specifically to SQLite. You won’t be able to add new records to the database because the primary key for new records won’t be unique. It is easy to update the sequence information manually, and you may also be able to include the correct statements into your dumpfile prior to loading it into your PostgreSQL database. Here’s the manual version:

Go into the PostgreSQL interactive terminal with your database

$ psql -d my_database_development

I recommend looking at your schema next, so that you know the names of all the tables you have (database=# represents thepsql prompt — the commands to run come after the =#)

database=# \d

You can also check to see where your database will try to start autoincrementing from (but this is not necessary)

database=# SELECT nextval('ks_users_id_seq');

Set the starting point to be one after than the last record in the table. Be sure to use single quotes for the arguments here

database=# SELECT setval('ks_users_id_seq', (SELECT MAX(id) FROM (ks_users)+1));

Repeat this process for every table that has an autoincrementing primary key See this post for a more detailed explanation of this process.

And that’s all. You should be able to access your database with all of your old data and add records with no trouble.

[flatironschool]

JavaScript vs. Ruby

The following is a guest post by Amanda Himmelstoss and originally appeared on her blog. Amanda is currently in the Ruby-003 class at The Flatiron School. You can follow her on Twitter here.

After beginning JavaScript last Thursday, I’ve been thinking about the differences between this language and Ruby, whom I’ve gotten quite intimate with these last 7 weeks. Coming from Ruby, JavaScript isn’t all that different, yet the syntax is unfamiliar and some logic differs slightly.

typeof vs class

To determine what type of value something is (a number, a fixnum, a string, an Object, an array, etc), in JavaScript you write typeof and then then the object; in Ruby, you append .class to the end of the object.

=== vs ==

In JavaScript, the triple equals sign helps determine if two objects are the same exact object (having the same typeof and the same value). For example: 3 === 3 returns true In Ruby, that is accomplished through the double equals. (3 == 3 returns true)

JavaScript has its own double equals, which shouldn’t be confused with Ruby’s: double equals in JavaScript only determines if the values match. For example: "3" == 3 returns true

parseInt(); vs .to_i

In JavaScript, to turn a string “10” into the integer 10, you write parseInt("10");. In Ruby, you write "10".in_i.

Incrementing and Decrementing

To increment or decrement by 1 in JavaScript, you can write ++ or -- (for example, var x = 2; x++; returns 3. In Ruby, it’s x += 1. You can write this as well in JavaScript (which is useful if you want to increment with numbers other than 1), however, you have the handy ++ and -- as well.

Functions

Functions in JavaScript are the same as methods in Ruby. The syntax of declaring a function vs. declaring a method is slightly different. In Ruby, it’s :

In JavaScript, the same logic can be written as:

Objects

Objects (similar to Ruby’s hash), declared as variables, are a way of organizing key/value pairs. For example, var cat = {color: "grey", name: "Spot", age: "12"}; To access the value of an associated key, you can call that on the variable name: cat.name returns "Spot". You can also easily delete a key/value pair by writing delete cat.name. To declare a new key/value pair, write: cat.gender = "male";. This appends this new data point to the end of the Object.

Arrays

Both Ruby and JavaScript have Arrays, which are more of less logically the same; however, the Object Array functions (JavaScript) and the Class Array methods (Ruby) have different syntax. For example, to find the index of an element in an array, in JavaScript you write arrayName.indexOf("element");. In Ruby, this is accomplished by array_name.index("element")

each

The method each in Ruby is a handy way of iterating through an array of elements in order to change or manipulate them. JavaScript as well as an each function that does the same thing; however, the syntax is slightly different:

And in Ruby, the same is written as:

Falsey Values

One big difference between Ruby and JavaScript is the falsey-ness of certain values. Like in Ruby, null (in Ruby, it’s nil) and false will return false, however, some truthful values in Ruby will return false in JavaScript, like 0, empty strings (""), and undefined.

Object Oriented Programming

JavaScript, like Ruby, can be object-oriented, meaning that objects, their associated methods/functions, can be neatly contained. In JavaScript, a Prototype is much like Ruby’s Class. To create a new Prototype in JavaScript:

The maruCat object is considered a new instance of the Cat prototype. Prototype is essentially the same as Class in Ruby. Note that this in JavaScript is the same concept as self in Ruby.

[flatironschool]

Hide Your Keys, Hide Your Tokens ... Unless Deploying to Heroku

The following is a guest post by Greg Eng and originally appeared on his blog. Greg is currently in the Ruby-003 class at The Flatiron School. You can follow him on Twitter here.

In that case, you have to do a little more than just hide them.

Typically, an API will require a key or authentication token before serving data downstream. It’s logical to avoid committing any of these to a public repo on GitHub, so I often found myself falling into a pattern. I would make hidden files in the assets folder of my project, then .gitignore each one.

It would result in a folder structure that might look like this:

Then in the class that controls all of the API calls, I would do something like this:

I set each relevant piece to a constant to be used in methods throughout the API interactions.

This way feels a little clunky, but it works. As long as you remember to add each hidden file to the .gitignore, it keeps the credentials safe. But as a project grows and it accumulates more authorizations, it becomes a little difficult to manage.

It wasn’t until my group tried to deploy an app to Heroku that we discovered the better way. Heroku makes it simple to deploy a Rails app, but as a result it lacks a lot of the bells and whistles. As a result, it is impossible to log in to the server in a shell to upload each individual hidden file described above.

In order to accomplish the same level of security and function, we can instead use Heroku config variables that will be read at runtime. Heroku has some great documentation on this topic.

It’s very easy to set up from the local shell. Thanks to my teammate David Bella, we were able to figure out what we needed to do immediately.

Just navigate to the root of your project directory and run the heroku config:set command.

For example:

That’s all there is to it! If you want to see your setup at any point, just type heroku config to see an output like this:

This is great for production, but how can we get the same level of organization and clarity for our development environment? My classmate Scott Luptowski told us about the dotenv gem that assists with that exact problem.

Start by including gem 'dotenv-rails' in your Gemfile in the appropriate groups. We only use it in :test and :development. Then, make a .env file at the root of your project directory. In it, include the assignments for each of the keys, tokens or passwords you want to call.

For example:

Rails will be able to call these variables anywhere in the application by using this pattern: ENV['API_KEY'], ENV['AUTH_TOKEN'], ENV['PASSWORD']. The hard coded keys will stay out of your code and you will have one organized place to manage all the necessary information. Just remember to also add the .env file to your .gitignore. The app should now work seamlessly in both local development and production Heroku environments. A simple solution for an important problem!

[flatironschool]

23 years ago...Tim Berners Lee proposed to invent the web

The following is a guest post by Sam Yang and originally appeared on his blog. Sam is currently in the Ruby-003 class at The Flatiron School. You can follow him on Twitter here.

Pretty amazing.

http://www.w3.org/Proposal.html

[flatironschool]

5 Simple Tips/Lessons From Gem-Building

The following is a guest post by Stephanie Oh and originally appeared on her blog. Stephanie is currently in the Ruby-003 class at The Flatiron School. You can follow her on Twitter here.

Two posts ago, I dropped a hint about a basic ruby gem I was trying to build. At the time, building one’s own gem(s) seemed to be the de rigueur object of our collective fascination at Flatiron. I guess that’s because we were all itching to start building things, and we hadn’t yet begun to learn Rails.

So I’m a little late to our gem-party, but I’m so happy I gave it a go (better late than never, right? Cue: mental reminders about ‘unrealistic self-expectations’). Because inevitably, the thing you learn most about through the process of gem-building — as is the case with absolutely anything you’ll ever build with code — is git.

Only joking.

But really, I did actually learn an unexpected amount more about git. And with that said, since everyone loves lists, here’s a list of 5 simple tips and lessons from my personal experience with ruby gem-building:

  1. Just start with the RubyGems Guide. Like, seriously.

Don’t waste hours debating your starting point. Just start with the RubyGems Guide. Having done precisely the opposite — I ditched the guide in favor of a Treehouse video tutorial then got confused and ended up coming back to the guide — I know now that the RubyGems Guide is actually the easiest thing to follow. It walks you through the process of building a simple “Hello World!” gem, and without having to worry much at all about project directories or structure, you can build and ship your first ruby gem in about 15 minutes.

2. Then try a tutorial, like this one

Once you’ve familiarized yourself a bit with basic directory organization and in which directories stuff should go, you can then follow along with this Treehouse tutorial to build your real gem. The main reason I suggest you wait until you’ve practiced with the RubyGems Guide first before diving into a tutorial like this one is that the latter instructs you to run this simple command as your starting point:

bundle gem <the name of your gem>

This will perform a bit of magic for you — not unlike the Rails/DHH magic we have at our disposal with the rails g scaffold <the name of your project> command — and instantly create a bunch of directories with files that have placeholder text and stuff in them. While this might sound totally awesome, it can also be confusing to use this as your starting point because you won’t know why things have been set up the way they have. Which is to say, it will be harder for you to customize stuff to suit your needs — because you won’t know what’s going to cause the whole thing to break and what’s okay to change.

So build your directories and files from the ground up yourself first (following the RubyGems Guide), THEN take advantage of shortcuts like bundle gem <the name of your gem>.

3. Don’t forget, you’re using git

If you do go off of the bundle gem shortcut, be aware: it initializes a git repo in your project directory automatically.

Yes. Look at the last line! Initializing git repo... it says. Somehow, I didn’t pay attention to this detail, so later down the road when I pushed my first version of the gem up to rubygems then instantly realized I had to make an edit and push up a new version, I ran into a huge wall. I kept getting error messages that said something along the lines of “[some_file_name, some_other_file_name] are not files”. And I thought “WHY??! WHAT DO YOU MEAN THEY AREN’T FILES? I’VE SAVED EVERYTHING, I CAN SEE THEM IN MY DIRECTORIES, THEY EXIST! OF COURSE THEY’RE FILES!!”

Well. Turns out, my computer wanted me to do a git add . and git commit before I could proceed with pushing up new gem versions. This is because the bundle gem <the name of my gem> shortcut initialized a git repo for me, which meant I had to start tracking my changes. The fact that I’d forgotten this and/or hadn’t paid attention to the activity in my console from the get-go didn’t change the fact that I now had to answer to git.

4. Make sure your gem-name isn’t already taken

The problem I ran into above (and the subsequent 3 hours of de-bugging) was all due to this basic mistake: I’d forgotten to make sure my gem name wasn’t already taken. So after building everything and trying to run gem push <my gem>, I was promptly denied. This forced me to have to go back and make a lot of changes, including RENAMING MY GEM.

So just check first on RubyGems.org that your gem name hasn’t already been taken! And don’t assume, like I did, that it’s necessarily something more complicated — like an error with my API-key (I ran around in circles trying to ‘fix’ that for a while, when that was never the issue to begin with). It’s really easy to get excited and just start building, and equally easy to jump to crazy conclusions as to where and how you should start debugging your errors.

5. Handle your gemspec with care

Your gemspec is not just some random file you can treat as an afterthought (as is often done unfortunately with README files, for example), just because it doesn’t live in the ‘all-important’ lib or bin directories. Your gemspec file is literally the roadmap TO YOUR GEM, or a blueprint, rather. It contains tons of valuable information such as your gem’s version number, executables, and run-time dependencies, to name a few.

Furthermore, as you can see, when you actually run the gem build <yourgemname.gemspec> command to BUILD your gem, you have to build it off of the gemspec! It’s just convention. Needless to say, your gemspec file is important. Update it, make sure it contains accurate information, and treat it with care.

================================================================

All of that said, I had a lot of fun building my first gem. Special shoutout to our TA Spencer Rogers for helping me de-bug and ultimately realize that my 3-hour roadblock had, in fact, stemmed 100% from git troubles.

My gem is called billboard_chart.

You can install it by running gem install billboard_chart.

Then simply type billboard_chart to see what happens!

(NOTE: 24 hours after I published my gem, Billboard.com decided to switch up some of their CSS selectors so now the scraper generating my data — I used Nokogiri — is acting funny, which means the data also looks a bit funny now. Looks like I now have some ACTUAL de-bugging to do!!!)

[flatironschool]

The Art of a Question

The following is a guest post by Ian Miller and originally appeared on his blog. Ian is currently in the Ruby-003 class at The Flatiron School. You can follow him on Twitter here.

For once, I want to take a break from exploring programming topics, and explore the rationale behind asking the right question. Or rather, is it insomuch asking the right question, or guiding yourself in the right direction? I think it is a little bit of both, with more emphasis on guidance. It’s not so much the silver-bullet question that will find you the answers, but it’s a series of small, seemingly mundane questions that will ultimately lead to the epiphany where all of the previously perplexing concepts suddenly click together into a cohesive abstraction.

From time to time, I have found myself unable to ask the right questions. It’s as if there is a mental block in my head, unable to move forward beyond an obstacle, a metaphorical wrench stuck in the gears of my neocortex. When that process unfolds, it is easy to doubt yourself and your ability to lead yourself out of the quagmire. There is a reason why many individuals are daunted by the prospect of learning how to program. It is a seemingly endless black hole that manifests itself as a negative feedback loop. Despite the organized chaos that arises, most often overlook the key decision making component of programming: knowing where to start.

Knowing where to start isn’t even the be-all-end-all solution to asking the right question. It is simply a starting place. Just because a runner starts at mile 0.0 after a starter pistol has fired, it does not mean that failure is imminent. It is a mindset. A runner tackles a marathon one step at a time. Step by step, two feet turns into 10. 10 feet turns into a mile. A mile turns into 10 miles. 10 miles turn into 20. It’s about breaking down the obstacle into manageable parts, and moving on from segment to segment. This works because of two reasons: one, it keeps the logical decision making process flowing, and two, it pushes doubt and negative reinforcement to the periphery of human consciousness.

I read an article the other day discussing a concept called cognitive load. According to George Miller, an eminent psychologist who taught at Princeton in the 1950’s, our brain can realistically hold about seven pieces of information simultaneously at one point in time. That number has since been revised to three or four. This is a significant problem in the information age. WIth historically recent advents such as the Internet, we are overly stimulated by the free flow of information, made possible largely by electronic devices. Information enters and exits our cognitive consciousness that we don’t grasp a basic level of understanding. We are unable to transfer it from our short-term to long-term memory stores, and consequently our ability to think critically and conceptually weakens. Ergo, one step at a time.

So it seems that there are valid reasons, backed by science, psychology, and other studies focused on the human brain, that call for the fragmentation of a complete problem into parts. If someone were to ask the question, “how does a route ultimately render a view in a Rails project?” This is my thinking process below.

First off, there are multiple steps that between the route path and the view being rendered. In MVC, there are typically eight steps in the chain of events.

This is an oversimplified example, but the question went from a generic question about a route and respective view rendering to a multifaceted question made up of smaller implicit questions about the 8 steps of MVC progression. Each step can also be broken down further to better understand the process. Each little question is a foundation that can be built upon, and will ultimately to a deep understanding of how a simple Rails application works.

So perhaps the best question is not simply a targeted question that seeks a one-dimensional answer, but rather a series of smaller, more basic questions derived from a larger question. Simplicity is the best form of policy. Start somewhere. Plant your flag in the sand. Let your initial question, and the subsequent chain of questions, be your guide.

[flatironschool]

How Web Apps Work (Part 1)

The following is a guest post by Saron Yitbarek and originally appeared on her blog. Saron is currently in the Ruby-003 class at The Flatiron School. You can follow her on Twitter here.

[flatironschool]

Naming the World

The following is a guest post by Rosie Hoyem and originally appeared on her blog. Rosie is currently in the Ruby-003 class at The Flatiron School. You can follow her on Twitter here.

We’re coming up on National write a Novel Month. In November you can join thousands of aspiring writers from around the country for National Novel Writing Month (NaNoWriMo). This event involves writing 1,800 words a day, which after 30 days, will equal the length as a respectable novel. My attempt to log 50,000 words last year was thwarted by a nasty flu in week two that set me hopelessly behind on my word count and work (that’s my excuse and I’m sticking to it), but NaNoWriMo and I will meet again!

NaNoWri was founded to make a point about writing; the difference between a good writer and a great writer that you may have heard of is largely the number of words a person logs everyday. Sucessful writers write a lot.

The next measure of a great writer vs. a good writer is the quality of the details—are they true and believeable? Do they contribute to building the story? Language is a dynamic thing, and the art of picking the right word for the right ocassion and expressing ideas clearly is both a skill that comes with experience and an art. I’ve been thinking a lot about writing as I learn how to program.

I’ve been reading a lot because I spend nearly two hours a day on the train. My current train reading is Jeff Atwood’s Effective Programming: More Than Writing Code (Get your copy here!). He is the master mind behind the Coding Horror Blog and also co-founded Stack Overflow. In the chapter entitled ‘How to Write Without Writing’ he emphasizes how important it is for programmers to have good writing and communication skills. I found it quite amusing that he claims part of the reason he founded Stack Overflow was to “trick” programers into writing. Writing about code forces programmers to communicate more clearly about what they are doing and why.

Writing, clear communication skills, picking works appropriatelly are all important skills for programmers. I want to hit a few relevant topics for programmers that demonstrate this point.

Naming Things Well

“There are two hard things in computer science: cache invalidation, naming things, and off-by-one errors.” Martin Fowler

Part of clear communcation in code is naming things well. The names programmers give to variables and methods in their programs matter. Naming things is often sited as one of the most difficult aspects of programing to do well. Jeff’s Attwood’s post I Shall Call It.. SomethingManagerlays out several very important reasons why naming things well really matters in programming.

For one, carfully paying attention to naming things clearly and concisely simply makes your program better. It helps to provide better understanding in the future should your code reused after a long interval of time. It not only makes it easier to read through code and understanding what is going on, it is perhaps low hanging fruit for newbies to attain the professional appearance of work product. It may seem funny in the moment to use cute or comical names. Don’t do it. Overly long names or single letter variables also make code more difficult to understand. Don’t do it.

Naming things well can also act as a form of metadata for your program. If you feel like you need comments, reconsider how you’ve used names for your variables and methods in your code. Comments shouldn’t need to explain what a piece of code is doing, they should be used sparingly only to describe why you’ve constructed your code the way you have. Use clear and consistent names to promote consistency within a project and on a development team. Everyone’s productivity will benefit.

Naming Conventions that Make Rack Frameworks Work

Beyond the somewhat subjective benefits of making our code more clear and concise, the most popular Ruby Frameworks, including Sinatra and Rails, are constructed around a variety of naming conventions. Not abiding by these conventions will make your programs not work. When working with ruby frameworks

Just last week my group found this out the hard way. When naming a Sequel table we used candyfor our class Candy rather than candies. This didn’t seem like a major difference… ? Low and behold, Sequal would not recognize our table because it was looking for the exact plural of our class.

These strict naming conventions will only become more important as we move into Active Record and Rails. Attention to these seemingly small details will be the difference between a things working and not.

Namespaces

The idea of namespaces is a broader topic in programming that also amplifies the importance of choosing names well and communicating clearly to the surrounding context. A namespace is a defined area or domain for a set of identifiers or names. Namespaces allows the disambiguation of homonym identifiers residing in different namespaces. Namespaces usually group names based on their functionality, so it is possible to do an endless number of things with limited sets of words.

In Summary.

Programming has more in common with creative writing than most would guess. The careful use of words to describe the world around us and inside our programs will make both work better.