Building Chatter: Part 6 - Welcome to the Realtime
So, we have the barebones of our application up and running, but it wouldn't be much good if you had to constantly refresh the page to see what people are talking about. When I create a post, I want anybody who I mention to know that I've mentioned them immediately, and if I am searching for a topic, then I want any new messages about that topic to be delivered to me immediately.
We will set up the ability to do this in two stages - first we will look at mentions "@username" in a post, and then we will look at searches.
Improving the application
Ok, so we need a system for detecting the tags in the posts as they are submitted. To do this, we are going to open up our Post model again, and make some changes. Processing notifications isn't really something that a post needs to be aware of, that is something that happens in the background, and because of this, Rails provides us with a type of "listener", called an Observer. Observers monitor a model, and carry out actions whenever a callback would have been fired. This means that we can separate our concerns - the Post model describes the posts, and a PostObserver will monitor this and start the communication system.
If you need a copy of the application so far, remember it is on Github, once you've cloned it then "git checkout part5" to bring you up to date.
The first thing we need to do is generate our Observer. In a terminal at the root of your application, run:
Now, go to "app/models/post_observer.rb" and we are going to implement some of the functionality that we already had in our Post model:
class PostObserver < ActiveRecord::Observer observe Post def after_commit(post) TAG_PROCESSOR.push(:tag_class => Tag, :post_id => post.id) end end
We can now remove some lines from "app/models/post.rb":
delete - after_commit :process_tags delete - def process_tags delete - TAG_PROCESSOR.push(:tag_class => Tag, :post_id => self.id) delete - end
Finally, to get this observer observing, we need to update a line in "config/application.rb":
# Activate observers that should always be running. config.active_record.observers = :post_observer
If we were to restart the server now, then we would see that our previous functionality still works, but it now located in a more sensible and logical place. The task at hand now is to work out how we want users mentioned in posts to interact with them. The most simple method, and the one we will use, is to create another relationship between users and posts, however you could look at serializing a field, or a number of other tricks. We are going to use another "has_many, :through =>" type relationship, this time called Mention:
$ rails g model mention post_id:integer user_id:integer
And then in the migration file for this, (in "db/migrate") we need to add a few things to create the indexes for the table. Add the following below the change method block:
add_index :mentions, :post_id add_index :mentions, :user_id
Now we can migrate the database ("rake db:migrate") again, and then set up the relationship in our models. In "app/models/user.rb" we need to add the following, probably near your other relationship declarations ("belongs_to", "has_many" etc).
has_many :mentions has_many :mentioned_ins, :through => :mentions, :source => :post
We also need to add something similar to our Post model in "app/models/post.rb":
has_many :mentions has_many :mentionings, :through => :mentions, :source => :user
Finally we need to add a couple of lines to the Mention model in "app/models/mention.rb":
belongs_to :user belongs_to :post
Perfect. We've got the join model set up properly, so we can go ahead and detect usernames in the posts as they come in now.
To detect the mentions, it's actually going to be very easy, as we have done most of the hard work already! The tool that we need we made earlier, so open up "app/models/post_observer.rb" and make the following changes:
class PostObserver < ActiveRecord::Observer observe Post def after_commit(post) TAG_PROCESSOR.push(:post_id => post.id) users = User.find_all_by_username(detect_usernames(post.body)) if users users.each do |u| u.mentioned_ins << post end end end private def detect_usernames(post) post.scan(/B@(w*[A-Za-z0-9_]+w*)/).flatten end end
This now will find all the usernames in the body of the post, and then for each user it finds that corresponds, it will add a mention for them. The reason we aren't delaying this and using Girl Friday again is that we want these notifications to be absolutely realtime - if someone says my name I don't want to wait to hear it, whereas if the notifications about topics arrive a second or two later when the service is busy, that isn't the end of the world (although obviously that isn't desirable).
Let's spin up the server and create a new post that mentions another (existing) user. To do this, we use "foreman start", if you've forgotten since last time.
So, we have created a new post with (in my case) "@billgates" in there, which means that our observer should have picked this up and created a new record in the mention model for us. We can check this by going to the terminal and typing:
Which fires up a command line IRB session, with our Rails environment fully loaded in it. If you aren't familiar with the rails console - spend some time playing around, you can do an amazing amount of stuff, and it serves as a phenomenal test-bed for experimentation.
You'll see the console perform a database query, and come up with a Mention, linking your last post with the user you tagged in it! Perfect. We can just double check that it worked in both directions, by typing (in my case):
> User.find_by_username("billgates").mentioned_ins
And you should see the post that you mentioned the user in. Type "exit" to get out of the console, and we'll get on with the next steps. Now we have a system for detecting when we have tagged a person, we need to start implementing the "realtime" part of the application.
Rails, and Ruby in general, isn't particularly good at "realtime" web applications. This is due to something called the GIL or Global Interpreter Lock, which massively hampers the amount of concurrency (things done at once) that Ruby can cope with. However, this situation is improving, and often being bound by a constraint like the GIL forces the developer to come up with a cleverer solution to the problem.
There are a lot of "push api" providers - companies that offer a product to abstract this functionality away from the application, and to be honest in most cases using them is a good idea. In a later tutorial we will revisit this next section, and show you how to implement your own push server using something called Faye, but this is a little more complex that we are going to deal with yet. We're going to use a really solid service - PubNub that offers unlimited free testing whilst in development mode. A caveat here - to do this, you have to use a shared "demo" key, which means that anyone could potentially listen in to your messages, so don't transmit anything too important!
To use PubNub, we're going to add a gem to our Gemfile:
And then we need to run "bundle" to get that installed. We also will create a new initializer file - "config/initializers/pubnub.rb":
PUBNUB = Pubnub.new( "demo", ## PUBLISH_KEY "demo", ## SUBSCRIBE_KEY "", ## SECRET_KEY false ## SSL_ON? )
We will also need another observer, so run "rails g observer Mention" and then open the file it created ("app/models/mention_observer.rb").
class MentionObserver < ActiveRecord::Observer observe Mention def after_commit(mention) end end
Before we go any further, we need to know how PubNub (and almost all push notification clients) work, so we can make a design decision for our application. The PubNub client that runs in the browser listens on what is called a "channel" - we need to know what channel we are sending our messages out on, in order to be sure that the right message is going to the right people. Rather than simply using the username, we will do something slightly more advanced.
We aren't going to use a proper message queuing system such as AMQP or RabbitMQ - those are far beyond our needs, but we will need a system for managing our "live" channels. This will become more important when we bring in push functionality for tags as well.
We are going to create another model to hold this data, and we will call it Channel. It is going to need a number of fields, to allow us to access it and to provide some control over how long it keeps broadcasting for. This Channel model is going to be polymorphic - which means we can attach it to a number of other models by declare two fields an _id and a _type, prefixed with the polymorphic identity, which in our case will be broadcastable:
broadcastable_id - the id of the related model
broadcastable_type - the class of the related model
channel_ident - a string to identify the channel
last_validated - a timestamp for when the channel was last declared to be needed
So, lets generate this model:
$ rails g model channel broadcastable_id:integer broadcastable_type:string channel_ident:string last_validated:datetime
Add this line underneath the create_table block in the migration that produced:
add_index :channels, [:broadcastable_id, :broadcastable_type]
You can now run "rake db:migrate" to add this table. Then we need to add some code to a few models:
# in 'app/models/channel.rb' belongs_to :broadcastable, :polymorphic => true # in 'app/models/user.rb' has_one :channel, :as => :broadcastable # in 'app/models/tag.rb' has_one :channel, :as => :broadcastable
Ok, so now we have a mechanism for controlling our channels, lets go back to MentionObserver ("app/models/mention_observer.rb") and finish creating it. Change the after_commit method to the following:
def after_commit(mention) NOTIFY_QUEUE.push(:post_id => mention.post_id, :user_id => mention.user_id) end
You might recognise this as another Girl Friday method, and earlier I said that we didn't want to wait for this to happen. There is a good reason though why we need to at this stage. The call to send the post to PubNub to send to the user requires a call to an external server, and we don't know how long this will take to respond. We don't want to block up our whole application if PubNub is a bit slow in getting back to us, so at this point we need to call on this web service behind the scenes. Let's add another processing action into Girl Friday - edit "config/initializers/girl_friday.rb" and change as so (note the subtle change to the TAG_PROCESSOR as well):
TAG_PROCESSOR = GirlFriday::WorkQueue.new(:post_process, :size => 2) do |msg| Tag.process_tags(msg[:post_id]) end NOTIFY_QUEUE = GirlFriday::WorkQueue.new(:post_process, :size => 2) do |msg| Notify.deliver_message_to_user(msg) end
You'll note here that we are calling a method on a class called Notify. We haven't written that yet, so lets create a new file in the models folder called "notify.rb", and fill it with the following:
class Notify def self.deliver_message_to_user(params) post = Post.find(params[:post_id]) user = User.find(params[:user_id]) user.channel ||= Channel.new( :channel_ident => Digest::SHA1.hexdigest(user.username, user.created_at.to_s)) PUBNUB.publish({ 'channel' => user.channel.channel_ident, 'message' => post.to_json(:include => :user) }) end end
Now, we are nearly ready to test this out. We just need to edit a couple of client side files, and we are good to go. First, we need to add the following to "app/views/dashboard/index.html.erb", just at the bottom of the file.
<script> // ------------------- // LISTEN FOR MESSAGES // ------------------- PUBNUB.subscribe({ channel : "<%= Digest::SHA1.hexdigest(current_user.username, current_user.created_at) %>", callback : function(message) { alert(message) } }) </script>
Then we need to add a couple of bits to "app/views/layouts/application.js", insert these directly before the "<%= yield %>" line:
<div id=pubnub pub-key=demo sub-key=demo ssl=off origin=pubsub.pubnub.com ></div> <script src=http://cdn.pubnub.com/pubnub-3.1.min.js ></script>
We also need to add the MentionObserver to "config/application.rb" - so change that line so that it reads:
config.active_record.observers = :post_observer, :mention_observer
Ok, we are going to need two terminal windows for this bit, and a web browser. Fire up the webserver in one terminal, and a console in the other ("foreman start", and "rails c" respectively). Then, go to the home page in a web browser, and - leaving the browser visible in front of you, type the following into the console, replacing "@USERNAME" with the username of the user you are signed in as.
Post.create(:body => "hello there @USERNAME")
You should be able to see a Javascript alert window pop up with the content of the post that you just created, fired to you in Realtime. Next time we'll go over how to make this a bit smarter and display the post, and we'll be using something called Handlebars.js to make the whole of the interface a lot snappier.
If anyone has any questions, please feel free to ask away, and as ever, the source code is available on Github, this time in the "part6" branch.
