Tommy, can you <T> me?

Rambles on crazy, beautiful, interesting, and amazing aspects of science, programming, and human psychology. Plus, doodles and comics!

[tommycanyoutme]

My wife went to Oberlin and, from what I understand, pretty much everybody does a stint of organic farming after they graduate. If they don't do it already, I hope they start selling Oberlin-branded farm equipment in the school's bookstore.

[tommycanyoutme]

It's topical. Well, it woulda been topical like 9 years ago.

[tommycanyoutme]

I'd watch it

[tommycanyoutme]

I didn't want to leave the last doodle on such a sad note.

[tommycanyoutme]

I got the strongest urge to draw a turtle being held up by a pregnant lady. It would be best not to delve too deeply into where these urges come from.

[tommycanyoutme]

Stop Using Inheritance For "Code Reuse"

Let's talk about class design.

A Typical Approach

My computer science education took place in the 2000s, which means it was fostered in a world heavily focused on certain aspects of Object Oriented Programming (OOP). Introductory and even advanced courses at universities across the country were (and maybe still are?) tailored toward teaching students to think of their program designs in terms of real-world objects. Many a professor could be heard saying things like:

"Today we're writing a poker simulator, so what are some things that you have when you play poker? Yes, cards! So let's have a Card class. Uh huh, and a dealer, so we'll add a Dealer class. Hohoho, no, no Mr. Murphy, we won't need a Beer class!"

There's nothing wrong with this. Encapsulating functionality into classes is a great way to isolate components from each other. The problem that started arising was that the conversation would continue as:

Professor: "We're going to have the Dealer be in charge of shuffling the deck of cards. We're going to want different dealers with different shuffling algorithms. So how can we do that?"

Students: "A base class Dealer with subclasses that override shuffle!"

Professor: "Huzzah! A+'s for everyone and tenure for me!"

Students: "Huzzah!"

Let's make this concrete and see what their class design might look like:

https://gist.github.com/anonymous/8e227243aa9d85f0ba38

Inheritance As A Default Approach

Why use inheritance? When I ask interview candidates this question, the answer almost invariably comes back as "for code reuse". This is a fine thing to desire (reusing code is a nobel and often difficult goal), but there are lots of ways to reuse code, and inheritance is a very heavy-handed (and often very wrong) approach to this goal. Jumping to using inheritance to model specialized versions of an object is a dangerous habit, and unfortunately one that I am seeing more and more of lately.

Let's see what happens when we tack another duty onto our currently useless dealer. He's been sitting at the table shuffling for the last 6 hours, why doesn't he deal the freakin' cards already??

Depending on the game, our dealer might deal in different ways. For instance, in the Midwest of the US is obsessed with a game called Euchre where the dealer deals 5 cards to each player, but instead of dealing cards 1-by-1, she must choose to deal either 2 or 3 cards to each player. Weird, I know -- this game is best played while tipsy on pure corn alcohol. Let's implement the different dealing functions.

https://gist.github.com/anonymous/d160911d517a80919e16

So far so good. Just put the deal() function in our base-class Dealer and we can define it on a per-dealer basis.

https://gist.github.com/anonymous/b40ff062314285674383

On paper, this seems so straightforward -- a DirtyRottenScoundrelDealer is-a dealer, so why not just inherit from AbstractDealer and override the functions we want to do differently?

Well, as demonstrated above, the issue is that dealing and shuffling are orthoganal decisions and jumping to using inheritance to model our specialized dealer has comingled those two decisions. Any given dealer might be rotten and dealing Euchre, or they might be shitty at shuffling and dealing a regular game of poker. Or 7-card poker. Or that game where you use your junk-food-fueled forehead sweat to stick a card to your face. Do we need to create a DirtyRottenScoundrelDealerDealingEuchre subclass? Is it going to have a copy-paste of the Euchre dealing algorithm? Is this code reuse?

In the inheritance-heavy curricula of my formative years, the answer might have been to throw more inheritance at this. It might end up looking more like:

https://gist.github.com/anonymous/8aeb8657de3622628eb8

C++ lets us do this and get away with it for awhile longer. But in time the deadly diamond of death will consume us all.

Besides. Right when we finish creating our CrappyEuchreDealer, our Crappy7CardStudDealer, our GoodEuchreDealer, and our GoodForeheadGameDealer, the professor chimes in again:

"New function time! Add a cut() function to cut the deck!"

HOW MANY SUBCLASSES DO YOU WANT FROM ME?!

Policy-Based Design

Let's try a different approach. Policy-based Design!

The core idea behind Policy-based Design is to identify orthogonal concerns (algorithms in your program that don't affect one another) and just pass whatever implementation you want for each of those algorithms into your object. Instead of overrding functions using subclasses, you just plug in the functionality you want.

There are different ways to accomplish this. For very simple cases, you can get away with just passing functors or function objects into your class's constructor. Then it's just a matter of calling the appropriate functor:

https://gist.github.com/anonymous/6578aa5054a74ee44fa8

Let's look at some of the benefits of this:

We can choose between any shuffling strategy and any dealing style when we create our dealer

Any function that matches the expected signature can be used for our policy

We can store our algorithms somewhere common and reuse them (when sensible)

With orthogonal concerns separated, unit testing each implementation is easy

Test versions of each policy can be leveraged to aid in testing other aspects (for instance, a shuffle policy could put the cards in predefined order)

In more complicated situations, we may need to store some state, or a given policy may consist of multiple functions (think: serialize() and deserialize()). In cases like this, whipping out a little template metaprogramming works like a charm:

https://gist.github.com/anonymous/bd31ec81653cb7b117be

Wrapping Up

I don't want to preach about this topic as though it's the end-all be-all of class design. However, the approach that I do believe is an end-all be-all of class design is carefully considering your domain and weighing your options.

The more tools in your aresenal, the better equipped you'll be to tackle complicated problems. So just stick policy-based design principles next to composition and inheritance in your toolset and may the best tool win!

[tommycanyoutme]

Another comic from the old drawnatwork website. I'm gonna start creating new ones soon!

[tommycanyoutme]

2 Great Features in Python and How to Create A Beautiful, Unholy Union Between Them

If, like me, you have been using Python for awhile now and are starting to really fall in love with some of the nice features it has to offer (not an unhealthy, sexual sort of love, but more like the type of love between a man and his pet goldfish... assuming the man doesn't have sex with the goldfish), then you will likely already be familiar with the concepts of decorators and context managers -- two of Python's most awesome and useful features.

If you are not already familiar with one or both of these concepts, here's a brief overview:

Decorators

Decorators are a great way to reduce boilerplate code and DRY up any functions that need to do similar things throughout your project. Think of decorators like sauces. Once you have a jar of mustard, a bottle of ketchup, or tub o' mayo, you can add one or more of those condiments to any function you want in the future. You define a decorator function (jar o' sauce) which contains that shared functionality (ketchup). Then, when you want some new function (sandwich) to include your decorator's functionality, instead of resorting to a copy-paste of the functionality (weird homemade squished tomato mush), you can just pop an @ketchup above your new function and it will be "decorated" (sauced) with the decorator's functionality. You can add as many decorators as you like, though they aren't guaranteed to mesh well together.

Other writers have done a better job than I could of describing the details behind how decorators work, so I would suggest reading up on them if the idea is new to you. I'll also be providing a concrete example below, but for now the most important things to know for this post is:

1. Decorators extend functionality, 2. Any callable can be a decorator, and 3. Any class becomes a callable if it defines __call__()

Context Managers

Context Managers are another great way to DRY your code and start applying that tasty RAII that you know you're always supposed to use. You have likely encountered Python code that uses the with-statement, for instance when opening a file you might do

with open("StarWarsEpisodeI.txt") as blacklist_corpus:

What this is doing under the covers is running a bit of code at before entering the with-statement's scope, and then running a bit more when we exit the with-statement's scope. A context manager is a class that can be used with the with-statement. By setting up your class properly, you get quick and painless RAII.

For context managers, the key takeaway for this article is that:

Any class that defines __enter__() and __exit__() can be used as a context manager

Decorators In Action

Let's look at a concrete example of a decorator.

Imagine you are in charge of making a change to one of your functions to use a more efficient approach to solving the problem. You want to measure how much faster your version of the function is compared to the original, so you decide to add some timing code to capture how long each function takes to run. (And maybe you implement some A-B Testing by choosing between the old and the new function at random...)

Here's your first pass:

https://gist.github.com/anonymous/469c901781753833ca1f

You could avoid the copy-paste code by creating a decorator out of the timing functionality. It would look more like this:

https://gist.github.com/anonymous/36809b9e4691c9b04caa

Here, you can see that the function we're decorating gets passed into the decorator as an argument. We execute the passed in function from within the decorator, but not before doing some setup before.

Now we have a time_this decorator that we can use as a sauce on top of any function we want to time! IT'S SO COOL THAT I WANT TO CRY.

But now what about those situations where

Context Managers In Action

Let's look at a similar situation, but now instead of wanting to time how long a function takes, we want to time how long a block of code within a function takes. Let's again look at how we might do this naively:

https://gist.github.com/anonymous/cf1c40211d8fca9e720b

Let's hope this is the only place you ever want to do this (and keep in mind, this timing example is pretty trivial), because otherwise we're going to be seeing the same code all over the place. How could we do this using a context manager?

https://gist.github.com/anonymous/ca157a91f964a38567ce

As you might expect from the names of the functions, __enter__ and __exit__ are the methods that get called before entering the with-statement's context, and upon exiting the context. Don't get too hung up on the arguments to __exit__, they're a more advanced detail than we need to worry about now.

Again, I should stress that this timing example is pretty low-key and trivial, but when you start getting into cases where you want to do something like validate a user's credentials, or hit a database, or just do a lot of work, the time and maintenance savings from not copying and pasting all over the place becomes a huge plus.

Strange Bedfellows

So I mentioned something about an unholy but beautiful union. Well, remember what I said earlier about how a decorator can be anything that defines __call__? And how a context manager is anything that defines __enter__ and __exit__? Well, there's no reason a single class can't do both! Let's see what it would look like if we wanted to create a context manager that is ALSO a decorator:

https://gist.github.com/anonymous/2273c1f9a43c4618ca77

There is a lot going on this, and it's really not meant to be consumed by the faint of heart or people new to decorators or context managers. I present it here as an example of the power duck-typing, the power of first-class functions, and the power of dark, horrible magic.

[tommycanyoutme]

Coo-dini the Pidgician!

[tommycanyoutme]

Elegantly Executing a Randomly Chosen Function

Occasionally while developing a system, you'll come across a situation where you want to randomly choose a function to execute from some candidate pool of functions. For instance, maybe you're writing test code and want to simulate random user inputs. Or perhaps you're just a sneaky sunofabitch person who wants to hide some Heisenbugs into your code. Let's say we have a couple of equally viable functions, do_foo() and do_bar():

https://gist.github.com/anonymous/5041313

How can we randomly choose which of these two functions to call?

Here's a naive approach that one might be tempted to try:

https://gist.github.com/anonymous/5041452

This is obviously very brittle. What happens when we add a new function to choose from? Remembering what we know about Python's first-class functions, the next logical step might be to throw the candidate functions into a list. Maybe something like:

https://gist.github.com/anonymous/5041516

This is much better already. Now, if a new function comes along that we care about, we can just add it to our list of options and it will automatically be chosen by our randomizer. But we can do even better. Python's random library gives us a nice way to stop worrying about the length of the list, by using random.choice()

https://gist.github.com/anonymous/5041584

Now we're getting somewhere! We've arrived at a clean, simple, and elegant way to choose from a list of candidate functions and execute the chosen one.

The next question that comes up is what if we want to bias our choice toward a certain function? Maybe we only want our Heisenbug to show up once in a thousand times. Taking advantage of Python's list construction syntax, we can do this elegantly by building our list with more references to the function we want to bias ourselves toward:

https://gist.github.com/anonymous/5041651

biased_options in the snippet above will contain [do_foo, do_bar, do_bar, do_bar, ..., do_bar], so random.choice() will choose do_bar 999 times out of 1000. Replace those hard-coded magic numbers multipliers with configuration values and you're well on your way to a tuneable, maintenance-friendly random function choice.