Notes from the field: programmers' memos

[perfectforloop]

Skills Profiles as an alternative to Performance Reviews

Performance reviews have always been a sore subject for me…

Reviews are often a pain in the neck. If you write one, they are very time consuming and with questionable output. I have participated and heard of those reviews where people were not honest because the corporate structure does not allow you to be fully honest (eg. bonuses highly depend on reviews so people have every incentive to game the system). If reviews are not anonymous, one may be afraid/hesitant to provide honest feedback. If they are anonymous, people may not be committed to the reviews because of the overhead that it provides and if there is no way for anyone to check that you're giving it attention, one can get sloppy.

As a result, when you get a review from your peers/managers they may be too vague or not accurate. Add to the fact that many companies do a once a year review makes them not valuable since there is a very large gap between these discussions.

Another alternative to once a year performance reviews are 1-on-1's. Those small meetings, done every few weeks, or monthly or whatever other interval fits in your situation. Those are better because they are in person and happen more regularly so both parties are committed. My problem with those is that

It's pretty hard to give negative feedback so both parties have to be completely comfortable with each other (I'm guilty of this for sure). Depending on how comfortable you are with this, it can be a pretty awkward situation

You only get one opinion form your manager/supervisor. So you only get one opinion and it's likely that they do not know all aspects of your work

If the feedback is negative, there is nowhere to hide - both of you are right here in the same room or on a video call.

In my current role, I'm in charge of holding 1-on-1's with a group of developers and I found myself thinking about this problem a lot. We had both yearly performance reviews and 1-on-1's but neither of them solved the problems.

So I found myself talking outloud to the CTO and a few experienced developers in the summer of 2017 and what came out of this discussion was one of the best implementation that I've seen (subjectively of course).

Let's talk about Developer Profiles

Prerequisites

Team setup:

At the time of this system, we had 11 developers of varying skill levels in different skills,

All developers work on the same overall system/team,

There are different projects that developers work on, however,

Most projects live within the same (or similar) overarching ecosystem (eg. similar stack),

There are projects that are drastically different, but not enough to make a significant impact in overall evaluation,

From management point of view, all developers report to the CTO,

However, many day to day activities are handing by the Team Lead (happens to be me)

Developer Profiles - the setup

Inspired by a blog post from my co-worker, 4 developers and CTO came up with a gamified version of performance reviews. Rather than summarizing things into a board performance for a person, we decided that we needed to break things down further into specific strengths and weakness for various skills that were important for someone to be successful on our team.

We need to provide feedback to developers that was:

Easy to understand - we needed a way to provide an easy way for the team members to know where they stand in different skills

Actionable - provide concrete examples of leveling up

Informed - or less biases, feedback cannot come from one "authority", people providing feedback should have checks and balances

Well intentioned - the goal of the feedback is not to criticizes but to provide opportunity to level up

Well thought-out - it cannot be something put together in an allnighter

Well rounded - being a developer involves doing many things so feedback must cover different aspects of being a developer, technical and non-technical

Accountable - people giving the reviews must be accountable for their feedback

What we came up was a concept of skill profiles which is loosely based on game profiles in Dungeons and Dragons.

Implementation

Once we decided on the strategy, we got to work on implementation. Here is how we approached all aforementioned topics

Well rounded - We decided on different categories that are most important for our team. These range from hard technical skills (ex.server-side development and client-side development) and go all the way up to soft skills (ex.time management and influencing others)

Well thought-out - yeah, that was a tough one. We spent a lot of time thinking about what categories fit the bill. We started with a brain dump of all the categories that we could think of. And then one by one eliminated things that were too broad, too confusing, or not fully relevant at the time.

Well intentioned - For each categories we decided on how we would rate a given reviewee. We decided to go with a few steps

Novice - someone who is completely new to the the skill

Apprentice - person who has some experience with the skill, they can work independently in many cases but may require help

Journeyperson - someone who can get things done but may or may not be able to teach others

Master - someone who can use a skill for new set of problems and is able to teach/mentor others

Grand Master - People-will-remember-you-after-you-die kind of level

Idea of these levels to see everyone move up from one level to the next one in the next iteration of developer profiles

Informed - We have a total of five people in a room (in our case, virtual) duke it out for every single person being reviewed. Person leading the discussion would announce a category and everyone else types in one letter in Slack at the same time - N, A, J, M. We never had any GM's. If there is agreement, great. If there was a disagreement, people would voice their opinions on why they disagreed. We don't move on until everyone agrees.

This is especially important for the cases where a developer work on specialized projects that are not visible to the whole team so one reviewer may tip the scale by explaining a trait about reviewee that others do no know about

This is by far the most time consuming part of the process, but there is a good news. More on this later

In my opinion, this is also one of the most controversial parts of the whole system as 5 people in a room will inadvertently create bias so it’s important to have people who are not afraid to speak out and have variety of opinions.

Easy to Understand - the end result is a matrix with all the core skills for a team with grades and explaining comments.

So in theory this is easy to process.

In practice, it's not always easy to understand all the categories right away so it does require some team preparedness

Actionable - this is by the most important part. What makes the results of this exercise actionable is that the committee needs to come up with concrete suggestions on what a person needs to do to level up. This can include

Taking a course in particular topics

Pairing with someone when they work on a certain skill

Asking for projects that exercise a given weakness

Etc

Another important part is that after the developer profile is complete, it is easier (although not easy) to make parallels to the particular skill in everyday life

Accountable - given that all decisions are done within a group of 5 people performed live, people cannot be distracted and cannot slack off (you know, for the most part). However, second part is that in order to make these decisions more trustworthy, we made all developer profiles for senior developers (those involved in the grading process) public to  the whole team. This has a few implications

This makes it easier to understand what the ratings mean by setting an example

Creates a bit of vulnerability since everyone making decisions is exposed and is on display, hence, hopefully adding more trust to the process

Exposes that one doesn't need to be an expert in all aspects. Many of the senior developers in the group had several ratings that were either a Novice or an Apprentice. After all, we can't be good at everything.

I think this a very important parts of this process and I believe that if we haven't done this, it would jeopardize legitimacy of the whole process. All other developers have a choice of whether or not to make their skills profiles public. In my mind, ideally all developer profiles are public. But that won't happen any time soon. In our case, just one developer made their profile public.

The Skill Sheet

In the end we came up with a matrix like this

Note that there are three columns about how a reviewer rates themselves, how their manager rates them and how other senior developers rate them. It is very possible that the columns do not match and that is ok. It will provide for some surprises (positive or negative, so be ready for both)

Roll out

In the beginning, going through each profile was a bit difficult.

We had to discuss all details

Discard categories that were too broad/confusing

Argue over the meaning of each ratings

Document all the categories (this will be important for the future as there needs to be a way for anyone come and review meanings of different categories later on)

And just in general, we had to find the flow -  although we worked together for a while, this is a brand new type of exercises that we haven't done together

However, after we ironed a lot of details, things were much quicker. In took about an hour per reviewee. This is probably a bit more than one would have spent writing individual reviews but given the quality of output, well worth the time.

By far the most crucial part of rolling out was getting buy in from the rest of the team. This needs to be done before starting actively work on developer profiles.

We explained the intentions, which was to improve current review process. It helped that most of developers didn't actually like previous way of performance reviews

We made it clear that it is the first time we're doing this and it may not work but we will do everything we can to make it work.

Next hardest part was explaining the meaning of each rating. In our case, Journeyperson was a very broad category and we reiterated over and over that this is a very good place to be at. It's important to note that Master is a very high level of achievement, we should all strive to get there but it will undoubtedly take a long time/effort to get there.

After all developer profiles were done, our CTO met with everyone individually and discussed the results of the profiles.

Results

Developing this new way of doing reviews certainly took a long time. But was it worth it?

Initial feedback from developers was very positive. It seems that most people liked the idea. Which was great. But did they like the implementation. Answer to this is a bit more complicated.

One of the main elements of developer profiles was to provide constructive criticism. We needed a way to tell people where they stand. If they were doing well, they should continue the great job but the expectation is that they will help others to get there. If they are not doing that well, expectation is that they will catch up… or

It is a very viable possibility that a person who is not doing well will start re-evaluating their role in the team. This may ultimately lead to them making a hard choice of leaving. I don't think this is a bad thing. I don't view people leaving as an unnatural process. After all we should all reconsider what we're doing once in a while.

I believe that for most of the developers, this was a very positive experience, hopefully because we did a good job preparing them and making sure that this was done not out of bureaucratic need of reviews but out of genuine desire to improve.

For me as a developer, it was very valuable. A few times it was humbling but a few times it showed that I underestimated my skills.

What's next

Now that we went through all this work, does it stop there? I don't think so. We use these developer profiles skills in every day work. When there is an opportunity to point out how one can improve in a particular skill, use that. If someone did a good job in a particular job, point it out.

There are a few other changes that we'd like to do

Perform re-evaluation 6 months from the original experiment

Included non-developers (eg. QA, Product), this may need more work since the same categories do not apply.

Provide a way for 2-way feedback. Developer profiles took away the ability for developers to provide feedback back to the senior developers. And this is a big problem in my book. So we will be adding this back.

Challenges

Although for the most part, this worked out pretty well. I noticed a few things

Most obvious, is the time that it takes to create these profiles. Everyone needs to agree that it's a worthy exercise

As a team-lead, when I had 1-on-1's with developers after CTO met with them to discuss developer profiles, I needed to do some clarifications. My guess is that some of the categories were too deep that high level manager didn't have full visibility into.

Not everyone understood all the categories the same way. This is an ongoing problem and we're becoming better by communicating more.

Whether this system ties into compensation is still an unanswered question.

A few people were surprised by their results - some were negative, some were positive. It's not necessarily a problem from implementation point of view but it may cause strong emotions, so be ready.

We have a pretty small team, so it worked out ok. However, I don't know what changes would need to happen for larger team.

Half a year in, I do see some people are taking these profiles more seriously than others.

Was it worth it?

Yes, there are problems with using developer profiles. Yes, there is more to be done. But is it worth trying it out. In our team, absolutely! Only time will tell if this method of providing feedback is better than traditional performance reviews.

I'd be curious to know your stories of how you handle performance reviews. And if you decide to  give this method a try, it would be great to find out how it worked out for you. Please shoot me an email at ilyakatz _ at _ gmail

[perfectforloop]

Testing React Router

One of the major struggles that I’ve had since introduction of React to our ecosystem here at ExecOnline has been lack of testing routing on the client side. So this weekend I decide to sit down and solve this problem once and for all. 

Goal:

I have links sprinkled throughout the application created with react-router. User clicking on a link is transitioned to a different url and a different component is rendered. This works great in the browser, now I want to be able to test this behavior in a test

Solution:

Turns out that there are bit and pieces of information that I would need to extract from different places and it took me a few hours to figure out what those pieces are so I decided to collect it all in one place

https://github.com/ilyakatz/react-router-test/blob/master/src/BasicExample/__tests__/test.js

[perfectforloop]

Put your metrics where your tests are

TL;DR

Experiment showed that I spend about 20% of time on coding tests, but experience tells me that I actually save time on testing as compared to doing it manually

Extra time is well worth it in the environment I work in but are there ways to use the metrics to improve?

There is a way to determine time spent on coding automated tests vs features

Sometimes tests can be a sore subject. I am a very devoted advocate of tests in development life cycle. Others not so much. To me, a good testing suite means reliability of code, safety net during refactoring, less errors in production. To others it is time slowing down development, frustration with failing code, increased complexity of development of life cycle that comes from adding one more step to the development flow.

In the summer of 2015 after a discussion with a coworker on this topic, a question popped up in my head - how much does my devotion to tests really take away from coding product features? I wanted numbers. So I decided to take a journey which would show me just how much extra time I spend writing feature code vs. time that I spend on writing and maintaining tests. I had never done this so this will be a good self analysis.

Methodology:

Tools:

I modified Vim plugin BufTimer to save a report of time spent in each file (buffer really, but it doesn’t matter for this discussion) periodically.

Assumptions:

To make it more manageable, I only considered Ruby code and ignored other things that I often work on (configuration files, JavaScript, CSS)

I only counted the time spent editing code - actually running tests, testing the product in the browser, etc cannot be measured reliably (yet). I’m open to ideas how to do that since that would be a real way to measure how tests affect productivity.

Code updates made to accommodate for testing could not be distinguished. I considered this minor but this may be a more significant piece for those new to testing.

Precision

BufTimer is not perfect in terms of getting the amount of time spent editing files but it doesn’t discriminate against any type of file

Calculations

I configured BufTimer to spit out files with reports based on vim pid and date.

I created a simple ruby script that would tally up all the numbers https://github.com/ilyakatz/perfectforloop/blob/master/test_time.rb and save into the a csv file.

Number are grouped by vim session which could last a few minutes or days

I started collecting data on Sept 4, 2015 and finished the first experiment on Oct 8, 2015. So I have a full month worth of data. I plan to continue running to refine the numbers. So, stay tuned!

Results:

My experiments produced the following graph:

Full table with numbers is available here

https://docs.google.com/spreadsheets/d/1MPbcMozRhtANmHx04SSlVSb0g6JNfatG8kbJHEngS7E/edit?usp=sharing

When I tallied up all the numbers it showed that I spend roughly 20% of my coding time writing tests on average. So this means that one fifth of my time is spent writing code that is never seen by our users and is a liability that doesn’t bring in any revenue. I believe this number is well worth the investment but ...

We got the numbers, now what?

While numbers themselves are interesting the next step is a bit more difficult. What is the lesson. There will be a sister blog post that talks about pros and cons of testing, some of which are more obvious than others. But I think there are a few takeaways for this

20% spent on tests is well worth the investment (stay tuned for next post)

I will continue running this experiment to look at trends.

There may come a time of diminishing returns. So far, I have not come up with a way to determine of what number will make me reconsider the necessity of tests.

It is important to note that not all applications or part of the same application deserve the same amount of testing. This experiment did not distinguish this

I would love to get your feedback on what the cost of your tests are so I can compare and maybe come up with some more concrete goals, whether it be to write more tests or less.

I consider this experiment very much a beginning of a brand new journey. This is the first time I’ve seen, much less done it myself, any quantitative analysis of testing. I would love to see this integrated more into developers’ toolkits as continue to find better and more efficient ways to do what we enjoy!

Acknowledgements:

Thanks to

gh:chrisbra for being very helpful in adding additional functionality to BufTimer that I needed for this project.

Naoum Naoum for editing and keeping me honest

[perfectforloop]

A short list of things you can do right now with Bitcoin

A short list of things you can do right now with Bitcoin

Shop online at a variety of online retailers, big and small, such as Overstock, Newegg, Microsoft (digital goods only), TigerDIrect, Expedia

Check out a list of merchants here http://www.coindesk.com/information/what-can-you-buy-with-bitcoins/

Buy gift cards to your favorite stores with 3% back if you pay with Bitcoin at gyft.com

Buy a discounted gift card to Amazon, BestBuy, Starbucks, itunes and more

Give a tip to a someone on the Internet who created content that you liked.

Using https://www.changetip.com/ you can send micropayments in a fun message over most of the social networks. Send 1 cent, $100, 1 beer or 2 muffins to your friends and strangers.

Example: https://twitter.com/Katzmopolitan/status/546143733583212544

Pay your friend for that beer

Choose a wallet that tickles your fancy https://bitcoin.org/en/choose-your-wallet

Pay your friend for that beer, even if they are on a different continent

With Bitcoin there is no difference where the recipient is, this is a truly global transaction

While some countries are hostile to Bitcoin, most of the developed ones are friendly to it’s usage https://en.wikipedia.org/wiki/Legality_of_Bitcoin_by_country

Give to charity - with Bitcoin, charities get near 100% of your donations whereas if you pay with a credit card there is always a fee (~3-3.5%). Here are some example of charities that accept Bitcoin

http://bitcoin100.org/charities/

https://bitcointalk.org/index.php?topic=758674.0

Loan Bitcoin and earn interest

https://btcjam.com/listings

Give a gift that is more versatile than store gift cards but more thoughtful than cash - Bitcoin paperwallet.

With Bitcoin paper wallet you can give a personalized gift to your friends without tying them into shopping at a specific store or paying extra fees for prepaid debit cards

Here is an example of Christmas themed paper wallet https://bitcoinpaperwallet.com/holiday-design/index.php

Tutorial on how to create nice looking Bitcoin paperwallet https://www.youtube.com/watch?v=yo9lWqp_ZA0

[perfectforloop]

Metrics without Acetaminophen

How do you make use of metrics without an overwhelming headache every time you look at it?[0]

There is a huge amount of information about usefulness of metrics in businesses in general and startup’s specifically. However, in this post I would like to tackle something different - with all the information that a business can track, how do you make sure that you don’t track too much for your own good?

It is pretty easy to fall into the metrics frenzy and start collecting anything and everything. Which sounds great. But when it comes down to actually using those numbers, I’ve noticed that things become a little fuzzy. So I encourage you, the reader, to ask yourself and have a few honest discussion. Here are some thing that I’ve heard in the past that made me think really hard about metrics for the real world

I don’t have a use for this now but I will in the future. If I don’t collect them now it will be difficult to do it retroactively.

That is a very valid concern. The practical answer though is not that obvious. In my experience, anytime I heard that claim I’ve never actually seen anyone follow up on their threat. There were a few different reason such as

Predicting future is hard - I’m sure there are people out there who know exactly what they want to concentrate on 6 months from now. Is your business really that stable that you can predict future?

Change is inevitable - depending on the stage of your business, things will change. Product, marketing strategy, company goals and others are all subject to change [1]. Depending on how fine grained your metric is, it may become obsolete in 2 months

You delete unused features, don’t you?

If you don’t, please skip this section and read something that will make you want to go and remove code [2]. I am a huge proponent of reevaluating usefulness of features as product matures (with supporting metrics of course). My colleagues have become accustomed to me asking them whether we still need a certain part of the product and if we have sufficient proof that a given feature is not used enough, we schedule it for removal from the code base.

I believe that the same applies to metrics. It becomes daunting when you log in to a metrics dashboard and there are tons of metrics to choose from. Even worse is when some of these metrics are not used any more. This is when you should ask yourself

Do we still have actionable items for these metrics?

Can we parse through all of this data effectively without being bogged down by the weight of the information?

Do we even know what they metrics means?

This is the point where one may interrupt with a response does it really hurt to have more? Which leads nicely to the next question

How many things can you concentrate on?

If your team is small and you don’t have anyone dedicated to metrics, chances are that no one on the team is a real expert in metrics and you all do the best you can do with this influx of information. I believe that this is the case where less is more. Or rather, less is more manageable. Just like in the infamous study on selection of jams [3], you are more likely to

make a decision when there is a more manageable number of options.

Not everyone like metrics. Can you make it easier for people who don’t enjoy metrics?

Confession: I like metrics. I like looking at metrics, analyzing metrics, I even like making mistakes about metrics. But not everyone enjoys the numbers and graphs the same way. So if you’re implementing metrics that other people will use, it is your responsibility to make metrics more accessible to those people who do not enjoy them. Thankfully, there are a number of solutions that make numbers and graphs more understandable such as

Pick the easiest one for your business and stick to it if it works.

Create a simple dashboard with the fewest number of metrics required to answer given user’s questions

Do you remember to use the metrics you have?

Something amazing has happened while I was typing this - a colleague popped over to my neighbor developer to ask how hard it would be to implement a certain feature that a client was asking for. There was a bit of a discussion about usefulness of the feature, how much time it would take, things like that. All very valid discussions. Except for one thing - we already have metrics about a similar feature and it showed that it wasn’t a particularly very performing feature. So why build a clone? I brought it up, I’m hoping this will be the end of that discussion.  

Are you committed?

Now we come to the most difficult part of this whole extravaganza. Metrics requires commitment. It’s not enough to decide that your company needs metrics.

It’s not enough to add metrics to a fancy dashboard. No, you need to make a commitment to look at metrics. It’s up to you to decide when, but if you don’t use them, they are a waste.

Metrics is a lot of work, there is no question about that and there are always trade offs, so please do yourself and your coworkers a favor - ask the tough questions.

What other questions do you ask when thinking about metrics? I would be curious to know how you deal with metrics within your organizations, especially if you have to be your own metrics “expert”.

[0] Warning: you will probably still have headache, maybe not as much

[1] Didn’t you read the startup fine print - everything changes :)

[2] “Do more faster” may help with that

[3] When Choice is Demotivating, or if you want a more presentable form, here is Freakonomic episode on the topic

[perfectforloop]

Where can I code in NYC?

A few years ago, someone asked on the NYC.rb mailing list a very simple question - where can a geek with a laptop go in New York City to find a decent place to code if he/she doesn't want to stay in the office. What came out of that was a product of a great collaboration of like minded people. Someone in the community created a map where other contributed by adding cafes with WiFi, outlets and other necessities for a productive day.

Here is the latest in the collaboration effort. It was updated a few days ago when I sent a reminder to the mailing list.

Please use it and keep it fresh. Happy coding!

View Where can I code? in a larger map

[perfectforloop]

Reading between the lines of slow tests

This is a reincarnation of a post I wrote for ChallengePost blog

Let's get to the chase - slow test are annoying, depressing, frustrating, scary, or a million other variations of these. However, I will make a bold statement that slow tests are not just they are an indication that you have a bigger problem

  I will make a few obvious statements

Tests are good

Slow tests lead to (s)lower productivity

There are tons of posts about speeding up tests, such as [1][2][3][4] and others

  Now I will make some less obvious statements

Slow tests are a symptom of a bigger problem

No matter how fast your tests seem, they can be faster.

Speeding up tests is great fun!

Your CEO loves it when you announce that you spent the last 3 days speeding up tests

  I'm a bit impatient when it comes to running tests. I have learned that tests are a lifesaver and that any reasonable project must have them. But there comes a point when running your test is becoming painful.

This is the situation that we faced at ChallengePost when I joined just over a year ago. In the worst of cases, our tests were taking close to an hour. It was time for a change. Or in our speak, it was time for a challenge. While working on speeding up the test we learned a few very valuable but lesser advertized lessons:

Build awareness within the business that slow tests have a direct relationship to negative productivity. Believe it or not, but most developers have a boss, investors, board or other form of pressure that is expecting deliverables that make money. As a developer, it’s my responsibility to make sure that my teammates understand how slow tests hurt our business and why fixing them should be a priority. Nothing less than that. Here’s how ChallengePost’s tech team went about this

We made sure to mention the pain points of slow tests during daily standup meetings, our weekly retro meeting and at other relevant times by using specific examples:

- less frequent deploys to staging and production

- more time for completion of each deliverable

Because we made our pain points known and clear, we were able to get dedicated time to work on tests improvements:

50% time decrease the first time around and significant decreases thereafter

We even got kudos from our board!

Set a goal for your team that you think sounds reasonable, then lower it. After we set our goal, my team set up a new task list in Trello. That’s right — get 45-50 minute builds down to 10 minutes. When we started it sounded crazy, but a few months ago our tests ran in 11 minutes — just one minute away from the goal. Maybe it wasn’t crazy enough :) …

Identify your slow or superfluous tests as a sign of a bloated application. Your tests don’t just come out of nowhere — they are a reflection of your business because they test your business (your application!). Scrutinize a test not only as an independent unit but also as a reflection of your features. I would sometimes read the tests and be surprised that it was testing a feature that I didn’t know existed. Sure enough, after looking into various metrics, I wasn’t the only one who didn’t know about this feature. Here comes a new Trello task list “Feature Removals”

Don’t use things that you don’t need. Oi vey! Our application uses solr and redis a lot. But that doesn’t need all of our tests need it. So one of the things we did was just shutdown Solr and let the tests run. How many things failed? Better question: How many things still worked? We shaved double digit percentage points from running the tests.

Evaluate non-traditional ways to speed up your tests. Yes, there are many intersting way to speed things up and you may already know about some of these:

                 - Tools like spork, zeus, spring,

                 - Blog posts about moving code into observers [5],

                 - Notes on tweaking GC [6]

                 - Casts about separating Rails from your tests [7]

All of these resources are excellent and our team has implemented each and every one of them. But we also realized that we could be making more progress by looking outside of the codebase. Here are some way that helped us (as always, YRMV):

Can you parallelize your suite? If you can use parallel_test out of the box, more power to you. In our case, it wasn’t that simple, but we had other ways to split running tests based on functionality (eg. admin area versus user registration/login)

Don’t forget about simple things like keeping on top of development of your dependencies. Running perftools.rb on the test suite this number jumped out

               12.7% Psych::Parser#parse

 Upgrading psych improved some tests’ speeds by about 10%

Don’t let simple solutions stop you. After parallelizing our tests, we hit a bottleneck running things in parallel. To fix that, we got a cheap virtual server from DigitalOcean to help offload Jenkins. Is side effects:

          - Two minute spin up of new servers when we are in a crunch time and need more workers

          - Dead simple backup strategy in case our main build server is offline/unavailable/broken/whatever

Is $10-$20/month worth all the extra goodness? Absolutely!

Build the case for splitting up your app or building an API. It’s the most long term (but best) project that ChallengePost has going. We are actively working on moving the complexity of one application into a few applications. It’s a tough challenge but an exciting one. I am looking forward to removing all those tests...

I'm sure this is not a complete list and there are many other considerations to keep in mind when working on your test suite. But I hope this post provided some additional viewpoints on how to think about correlation between slow tests that health of your application both from technical and business point of view.

What have you learned about your application from your test? Feel free to leave a comment.

[1] http://blog.carbonfive.com/2011/02/02/crank-your-specs/

[2] https://gist.github.com/2063777

[3] http://www.rubyinside.com/careful-cutting-to-get-faster-rspec-runs-with-rails-5207.html

[4] http://collectiveidea.com/blog/archives/2012/11/12/tests-oddly-slow-might-be-bcrypt/

[5] http://ablogaboutcode.com/2011/10/24/using-rails-observers-to-write-faster-tests-and-simpler-models/

[6] http://ariejan.net/2011/09/24/rspec-speed-up-by-tweaking-ruby-garbage-collection

[7] https://www.destroyallsoftware.com/screencasts/catalog/fast-tests-with-and-without-rails

[perfectforloop]

Badge of honor: a word from your local security officer

This post is not about how to make your code more resilient to hacker attacks. This post is not about how to protect your network from intruders. But it is about how to make your business more secure.

Sometimes when thinking about security it is tempting to dive right into the code and server configuration looking for vulnerabilities, run simulations and scanners, set up brakeman and  CodeClimate scanners (or whatever your favorite scanners are) with regular notifications. Yes, all of these are correct and make sense. There is an elephant in the room though - yourself and your colleagues.

A few days ago I saw a posting on my Facebook wall by a friend of mine who received a text message, supposedly from T-mobile asking him to verify his login credentials. Two weeks ago another friend told me how easy it was for someone to get a replacement credit card by a crook from a different state just by calling the bank and answering a few questions. That's right, I believe that one of biggest problems in computer security for many businesses is still the human. And it is important that everyone you work with understands how important it is that they be vigilant about what they do during their regular work day.

I made it a goal of mine to make everyone in the office thinks about security. This included

having a consultant come in and talk to (i.e. scare) everyone about what every employee, tech or not, should and should not do on a daily basis

send out periodic reminders about improvements, such as setting up 2-step authentication for any and all services (eg. gmail, facebook, twitter, etc, etc)

praising individual efforts that improve security

asking everyone to raise alarms if they see suspicious emails

Security is not something that most people want to think about on daily basis. So I wanted to reward people for their efforts and what better way than to publicly praise them. A year or so ago, our QA Engineer followed up on an internal joke and she gave me  a Security Officer badge. I decided that this badge would be rotated among anyone who made efforts to either improve their own security practices or those of the company. If someone in the marketing team sets up 2-step authentication for our twitter account, they get the badge. If someone in the support team warned about phishing email, they get the badge. Developers working on a code improvement to improve security, yup, they get a badge.

At first I thought it would be geeky and no one would be interested. Pretty soon it became apparent that it was not the case - every time I presented someone with the badge I would get a very positive reaction. When I presented the badge to the CEO, he attached it to his breast pocket during the daily standup. I felt like I knighted him. If only for a day.

Yes, security is hard and a process that never ends. Yes, it is annoying and often not recognized until it hits you in the behind. So it is the responsibility of of every company - IT or not - to make sure that employees are empowered and rewarded for taking steps to improve security.

I'm always looking for new ways to remind people, and myself, to be more secure at work and at home. So if you have any suggestions, please drop me a line and/or comment below.

Please, no links to verify my log-in credentials

[perfectforloop]

If you wanted things to work out of the box, you wouldn't be reading this

About 6 months ago I decided to get rid of my iPad and old Windows computer, both of which found very little use for me, and experiment with a Google Chromebook. I won't talk about why I didn't like ChromeOS, you can do your own reading on the topic. But I want to share what I did to finally make my Chromebook usable running Ubuntu 12.04

Getting down to business, code speaks lounder than words:

[perfectforloop]

404 = Error Not Found

How often does it happen to you that you come across a piece of you code that is rewriting the functionality that is shipped with the framework or libraries that you're using. Perhaps the more important question, is how often do you catch yourself reinventing what the framework should do for you? I've done it. I admit, I am a repeataholic. That's my first step toward recovery. As with any decision, choice to implement your own solution will have consequences. Sometimes, these things are so small we don't take a moment to stop and think about it. Here is a story that made me think about this. I recently came across a little piece of code that does something like this What this does in Rails is renders is "Not Found" page for the user in case of certain exceptions raised by the code (here, record not found in the database) I found this while trying to figure out why my specs were not passing. The problem was that specs were failing very mysteriously - I saw an expectation failure where I wanted to see a certain string in the content of a page. Looking at the logs showed this "Rendered partial1" "Rendered partial2" "Rendered view 404" Spec is green. Go figure. What is a developer to do but start digging in the code. When I saw snippet above I almost exploded. What was happening was the code silently handled an exception and showed a 404 page to the user. From the user point of view everything works as expected - they asked for something they were not supposed to ask for, they got a "Sorry, not found" page. From developer point of view everything was exactly the opposite of what he expected. Running a spec that triggers a RecordNotFound should have blown up and not quietly rendered a page. So are there other specs that are in the code base that are green but aren't actually working? Possibly. (Hint: there was one, not bad out of thousands, but still one more than there should have been) So why was it there in the first place. Well, I think it was there for the right reasons - we do want to show 404 page to the users in case they ask for something that doesn't exist. The problem, or the solution really, is that it is no longer necessary. Rails handles it properly - if we ask for a record that doesn't exist, it shows a 404 page in production but throws an exception in development/test. Thinking about it rationally made me realize that I've done this myself in the past. Scanning the code that I wrote 2 years ago for a different project revealed I'm quiet guilty of the same mistake. Must have used the same stackoverflow answer. Next step, correct the wrong but removing the extra exception handling. And there is nothing I like more than removing code. This round of hide and seek rspec failure is conquered. Now, let's run the build with just a little bit more confidence that green is actually green.

[perfectforloop]

Programming is for squirrels

Who doesn't love a cute little squirrel! Those critters are adorable and they always seem to keep busy, getting ready for the winter, gnawing on something and bringing nuts back to their secret stashes. Some people love them, and others despise them and call them rats with fluffy tails. But all that aside, they have one quality that every programmer should be jealous of -- spatial memory. Let me explain.

Writing software is a pretty complex task. Compared to a simple task such as carrying stones around or typing on a computer - it requires a lot more mental capacity. Even more sophisticated tasks such as welding or even doing simple algebra -- they do require critical thinking and logic but are usually limited in the number of elements to be considered. But programming, and especially when using more abstract concepts, like inheritance and recursion, requires a much more involved level of thinking. It requires that the programmer be able to build a mental picture of all the structures and variables that come into play, and not just once on a flat level, but on multiple nested levels with adjusted parameters. It is usually quite a big task to do and requires that the programmer “gets in the zone” so that he or she is able to focus deeply enough and build it all from scratch in his or her head. And how does one do it? To be honest I don’t know if there is a universal answer - is there a trick that’s taught in school? Maybe, but I didn't learn it. Does everyone understand it in their own way? Probably. But I will share my secret and why I am so jealous of squirrels.

To me, solving a recursive problem uses up the exact same brain powers as does navigating or trying to backtrack my steps without a printed map. In these cases, I build a 3D mental picture of the problem, visualize it, and rotate it until it fits in the right spot. It is much like playing a game of 3D Tetris where you see the pieces and you visualize how each rotation would affect the field. For example, if there is an array that I am trying to process by recursively solving the simpler cases, I picture the array as a row of boxes sitting up in the air. Then I imagine them moving into an arrangement that would be produced by my first recursive call and applying any additional work that might be done there (e.g. pre-order traversal might take up the first box and paint it a different color compared to the other ones). After that, they move, like in an animation, further along to the second configuration with more color changes, then the third one, and so on, and the pattern forms. And it goes into a blur, much like looking up the middle of a really long square staircase, until at the very end you see the base case that solves it all. And this is just a simple array example -- the more the parameters and involved components, the more complex the picture becomes. Building that picture is both difficult and exhausting as it requires a lot of mental effort. But once done there is a certain moment of epiphany, when you can “see” in your imagination all the different pieces moving along, like little pieces in a well oiled mechanical machine.

Am I completely crazy for thinking of it that way? You tell me. It works for me, and if you haven’t tried it go ahead - it might work for you too. And in the meantime let’s hope that squirrels don’t suddenly develop human level intellect. If in addition to their amazing spatial capacity they learned how to program they sure would do a better job than quite a few programmers!

(Photo source: Flickr)

[perfectforloop]

Remote team made local

Ask anyone who has worked with me long enough and they will tell you about my hatred of the word “resource” when it comes to describing human capital, which in itself is pretty questionable in my mind. To me a resource is a laptop, a CPU core, a well written blog post. A person who works with me is not a resource, he/she is a colleague, manager, teammate, mentor, friend or any other word that describes a human. See the difference? So, inherently, the term outsourcing also holds a pretty questionable position in my mind. Outsourcing. What does it mean anyway? Well, let me rephrase it. I’m sure in some situations it makes perfect sense. I am talking about a company that has a software development component and the company made a decision to seek help from outside by hiring a different company or individuals. In most cases that I’ve heard the world “outsourcing” used in the context of such companies referred to hiring someone in a different country because it was cheaper. Let’s face it, most companies do that because it is cheaper. It’s just the way the economics work out. Don’t ask me, ask Steve Levitt. I don’t see anything wrong with it per-say. There are plenty of good/smart people in different counties, I have worked with many of them and I’m glad I had. In fact, I developed very personal relationships with them. And that is what my thesis is - if you made a decision to hire developers (or anyone) in a different location, to make it work you need to build a personal relationship with them. Here are a few things that I noticed about hiring someone in a different location to do some or all work for a company

Mentality - I’ve noticed that it is a huge plus, if not an absolute must, I’m still trying to figure it out, that the people you hear in a different location have the same mentality as the people they report to. I have worked with people that had different mentality and it was such a terrible experience for me. We were never on the same page, that caused for lots of frustration, and as a result, things moved slowly and development suffered.

Culture - if you have a remote team, build the culture as if they were right there with you in the same cubicle or coffee shop. It works wonders when people have a sense of belonging. Yes, it is difficult to go out for drinks if you live 3,000 miles (4,828 km) away. But it doesn’t mean you can’t have regular talks, asking for input and ideas, discussions about the state of the company/new technologies/concerns, regular code reviews/refactoring, etc, etc

Developer is a developer - once someone told me that my time was more expensive than that of a developer in a different country so it was better to answer my questions first. That made me furious. My answer was “a developer is a developer”. Once we are both hired we deserve equal treatment because we’re both just as valuable to the company. Yes, there is a question of expertise and knowledge, but there is no question of location.

Timing - there is no denying that timezones are a pain in the ass. There I said it. And I’ll say it again. If a manager tells me that timezones do not make a difference as long as the work gets done, I will know he/she is not a good manager. No way! The only way to do good development is to have real-time conversations so if there no time overlap between developers you will see crappy code pretty darn fast. There has to be sufficient time of overlap. Yes, that means someone will have to shift schedule. Sorry. That’s global economy of you.

Personal relationship - “it’s not personal, just business” is a bunch of bologna. If you work with someone for 8/10/12/whatever hours a day you develop a personal relationship. It is up to you if the relationship is good or sour, but it is there. Same with remote team members, you need to build a personal relationship. Facetime (video), chats, heck, even facebook connection. The absolute ideal way would be to visit them. I’ve done it. It worked wonders. Just one visit of a couple of day improved relationship and productivity by 500% (not a scientifically calculated number)   

Trust - if you do not find a way to trust your developers, well, I’m sorry for you, you’ve picked a wrong team. Find someone you can trust. Sure, it is more difficult to determine trust when you don’t see someone in person but if trust doesn’t develop overtime, you’re screwed.

[perfectforloop]

Open-source blogging

Yes, I know, why does the internet need another blog. Maybe it doesn’t. But we have things to share so perhaps someone will find them useful.

Who is “We”? Many of the blogs out there in the interweb are maintained by one person, express the view of one person, and, frankly, require the dedication of one person to create useful content. The collective “we” decided to create one blog where a few of us would share our thoughts without anyone of us needed the spotlight. We really believe the cliche “The whole is greater than the sum of its parts”

We are a group of professional in software development field who collective have experience working in all environments (corporate to startup to freelance), using a variety programming languages and multitude of tools, contributing to proprietary and open-source projects. Most importantly, “We” always learn from others and this is our attempt to share what we learned.“We” are not static. If you want to become a part of “we”, just drop us a line. This is open-source blogging.