#depressionbot

[dbotlog]

Users Module and Undefined Primary Keys

In the last couple of days I've been rewriting the users module to use the databank library.

The users module is one which automatically tracks users on the servers through which channels they frequent and their various aliases - while it doesn't provide much functionality on its own, its rich API allows other modules to be much more effective in their operations (for example storing stats for a user using several aliases under one account).

The rewrite has gone smoothly for the most part, as it's been relatively easy to move the code over and I've been impressed by the number of efficiency improvements I've been able to make along the way.

However, for the last hour or so I've been hitting my head against the wall over an intensely peculiar bug which came about as soon as I started testing with multiple users.

-!- batman62 is now known as batman64 ~alias batman64 batman64 is an alias of batman62 -!- lslater is now known as lslatest ~alias batman64 batman64 does not currently exist as an alias or known user.

So for some reason, a nick change was processed perfectly until a completely different user changed their nickname, whereupon the stored alias for the original user would disappear. This made no sense, given the two nick changes were entirely separate events operating on entirely different data records.

After a lot of anger and several nearly-executed threats to quit computing and become a watermelon farmer, I decided to print out the UUID of the records.

User records in the new system are stored with a randomly generated unique ID, and we then retrieve the record we want by searching for 'server' and 'username' to uniquely identify a record.

As it turns out, in the event of a nick change I was saving a copy of the user record with the additional alias in the database with a primary key of 'undefined' - this being because I was attempting to use user.id, while the actual ID was being stored in user.uuid. When this was looked up by 'server' and 'username,' the record with the primary key of 'undefined' was being returned (with the correct list of aliases).

However, when another nick change took place from another user, the old 'undefined' record was being replaced with the new one. This meant that a search on the original nick would turn up the original record with the correct UUID, unmodified and therefore without aliases. It was fixed by simply changing my use of 'id' to 'uuid,' as it was stored in the record.

~reality

[dbotlog]

Admin Access Security Issues

For quite a while depressionbot has had the concept of both moderators and administrators, which have different levels of access to the system - for example moderators can have the bot join/part channels while only admins have the ability to reload the bot or run ~ajs commands. The most common case for setting levels of access is by setting an access level on a particular dbot command. This is done by simply setting the 'access' parameter on a command object to either 'moderator' or 'admin' (the administrators and moderators for the bot being set directly in the bot configuration). Then, as part of the command execution flow in the command module, there is a hasAccess check to see if there are special access considerations for a particular command, and return true only if the user attempting to run the command is of the correct access level. This has proven to be a pretty useful model for securing commands, however it is rather easy to make a mistake and leave a command open for use. For example, it was discovered last night that among all of the commands in the admin command, I had forgotten to set 'admin' access on the 'pushconfig' command, which allows you to modify configuration while the bot is running - this meaning any user could run this command and modify the bot's config. It turns out this is just about the worst admin command that could have been exposed, as a simple 'pushconfig admins myusername' would give you access to all of the other admin commands! This was patched as soon as it was discovered, and since then I have been thinking about curtailing possible future incidences of such oversights. To stop further issues with this in the admin module, in which all commands should require elevated access, I have changed the access setting code so that it first sets all of its commands to require administrator access, and then goes on to redefine those which should require only moderator access. This way, the worst case scenario with forgetting to set an access property in the admin module will mean that a moderator will not be able to run a command they otherwise should be able to. ~reality

[dbotlog]

Introduction to DepressionBot (also, we love DataBank)

I suppose after having been asked repeatedly to write some kind of textual introduction to depressionbot and what exactly we're trying to do with it I'd best relent. We've created this dblog so we can post semi-regular (read: when we can be bothered) updates on how development is going as well as writing up some post-mortems on some hilarious and/or enraging bugs or issues we've had to work around. So depressionbot is an IRC bot built entirely in Node JS, running on JSBot which is a library I developed to interface with the IRC protocol. It's been in development for a little over a year, though true interest only having set things moving at a quick pace in the last couple of months. DBot proper is developed primarily by me, while the excellent Stats module is developed by Sam. Additionally we've received substantial patches from many other contributors (I'd list them all, but I'd have to go and find all their websites to link), By now it includes most features expected of a standard IRC bot, however the aim of the project is to use the versatile nature of Node to provide additional powerful features. The bot has a web interface which, while only being a regular module in the eyes of the bot itself, uses the expressjs library to host a web server which allows any of the other modules in DBot to add functionality to - for example, the quotes module includes an addition to the web interface which allows you to view the quotes on the web. This has already allowed us to create rather rich interfaces, such as the web interface for viewing channel statistics (which is currently missing a few days due to a bug). From this point, we thought why not implement some kind of social network using IRC as the backend, IRC is a nice and federated real-time text-based protocol which is pretty good (when you ignore the fact that no one pays any attention to the RFC anymore). The value of this kind of social network is that because it's just people talking in a room, a large part of the social aspect is already complete and data can be quite naturally gained - the bot is augmenting the already-existing experience. To start this off, we created profiles for users, which include links to relevant statistics and we plan to expand on this with additional fields, 'badges' and all sorts. In the further-off future, we're planning on implementing the ability to use the DBot web interface to be able to chat and perhaps retain a chat session on the IRC server. This means that the whole thing would essentially be a social network in a box which uses plain old IRC channels as a communication backend. Things like games with graphical front-ends could be conducted on the web interface, while all that's actually going on is regular text communication in an IRC channel. Before we can continue with a lot of this stuff, we're working on improving the backend. We're currently implementing the databank library, developed by Canadian Internet Superstar Evan Prodromou - it is a database abstraction library which allows a program to use one of a numerous methods of data storage (nosql, disk, memory etc) while providing a common interface for interaction with the data throughout the software. The intended use for this kind of library is that one might run their entire application with any one of a set of data storage solutions, via an option set in the config file - this being useful for when many instances of a piece of software will be run and there will be differing requirements in terms of data throughput; for example for a personal blog one might choose to store their data in a flat file on the disk without any additional overhead, while a large news website running the same software would probably opt for a NoSQL database. DBot is using Databank in a slightly alternative manner, in providing individual modules the option as to which method of data storage they wish to use. Up until now, we were storing all data in a big object and occasionally serialising this to a 'db.json' file (lazy, yes, yes) - this actually wasn't a problem for a long time, until Sam started developing his audacious statistics plugin which brings with it an obscene amount of data throughput in the application. Obviously this meant that we needed something like Redis to conduct the statistics module's data storage with. The problem was that it seemed wasteful to set up a Redis server within the module - perhaps some other module in the future would need this kind of support. So Databank allows us to retain centralised data storage in DBot, while allowing for a number of different storage options as well as providing a common interface to data storage within depressionbot. This means we can use Redis for Stats, while the module which tracks who has ignores set on which modules can continue using disk-based storage without additional overhead or any other such problems. This also has the benefit of not having to add dependencies for the entire application if only certain modules use that functionality - i.e. Redis will be a dependency of Stats, rather than DBot. We hope to finish the Databank implementation very soon, release 0.4 and continue development on everything we have in plan for the future! ~reality