Cryptological

Thoughts on programming and security.

[bk2204]

This is a great article on how to enable TRIM to keep your flash tidy.

One thing I hadn’t realized until now is that it isn’t automatic; you actually have to run fstrim on a periodic basis.

[bk2204]

Benchmarking OpenSSH

At work, I do a lot of development and code review and consequently I push very frequently. Since our pushes are done over SSH, I want them to be as fast as possible so I can get on with what I'm doing. So I wondered, what's the fastest configuration I can get running?

Our Git server at work is a Bitbucket Server instance, so it supports aes128-ctr plus RC4 and a lot of CBC ciphers. It also supports hmac-sha2-256, hmac-sha2-512, and hmac-sha1. For security reasons, I'm only going to ignore CBC ciphers, RC4, and SHA-1. I'm also ignoring key exchange and public key authentication for the purposes of this test.

When I came home, I tried both aes128-ctr with both hmac-sha2 options. I created a 512 MiB file and uploaded it via scp, with compression off. Both maxed out at about 22 MiB/s over Wi-Fi.

Knowing that Wi-Fi was probably the issue, I determined that I'd need to plug in. I have a gigabit network at home, like at work. When I connected Ethernet I found that now both maxed out around 74 MiB/s. There were some minor differences, but they were not statistically significant. To verify that, I also tried with [email protected] and [email protected]. Both also provided around the same speed.

Consequently, I've had to come to the conclusion that it doesn't really matter which algorithms I use, because the network is the bottleneck. Anything reasonably secure is also absurdly fast.

[bk2204]

I had a dream last night that the Git testsuite, as part of one of its tests, created (and then destroyed) pancakes and maple syrup. I only wish it were so.

[bk2204]

Performance of SHA-256 and SHA-512

The popular hash function to use these days for security is SHA-256, and for good reason: it's fast, secure, well-supported, and has a reasonable length. But what if I told you there were a better alternative for the first three.

Turns out, there is: SHA-512.

“But wait,” you're probably asking me, “how is SHA-512 better than SHA-256?” Well, it's not only secure and well-supported, on most messages exceeding 55 bytes (which is almost all the data you care about), SHA-512 is faster than SHA-256.

That's right, faster.

The reason is that on 64-bit systems, SHA-512 hashes twice as much data as SHA-256 (128 bytes per block instead of 64), but performs only 25% more work (80n 64-bit operations instead of 64n 32-bit operations). Now, granted, the processor will have to do a bit more work loading all the data from memory, but for most messages, it's a bunch faster.

Furthermore, everything I said here about SHA-512 applies to SHA-384, and the same goes for SHA-256 and SHA-224. In fact, this is the reason that NIST created SHA-512/t, so you could get the performance of SHA-512 on 64-bit systems when you need a 256-bit output size.

So why would you still want to use SHA-256? Well, if you want better performance on 32-bit systems, or if you need output that is shorter for human reasons (because, say, you want it to fit on an 80-column terminal or humans are going to have to manually read it), then you might want SHA-256. Also, if you're using a 256-bit elliptic curve or some other place with a fixed 256-bit size (or where size is a concern), it only makes sense to use SHA-256.

Overall, though, if you're doing things like setting up SSH parameters or signing data with GnuPG and an RSA key, you might as well use SHA-512.

[maptitude1]

A commentary on the electoral system and proportional representation in government

[bk2204]

Every time I see a map like this, I think about how you find the set of red states that lines up best in population with the blue states, and think about how it’s an instance of the knapsack problem.

[bk2204]

The bacteria report when they have solved the problem by becoming antibiotic resistant.

Wikipedia, on a technique using E. coli bacteria as a computer to sort data

[bk2204]

The EDITOR and VISUAL environment variables

On most Unix systems, there are two environment variables that control the user’s choice of text editor: EDITOR and VISUAL. They have different historical purposes, but are generally interchangeable today.

EDITOR is the older variable, and the one you should generally set if you only set one. It lists the name of your text editor, optionally with flags.

VISUAL is a bit different. If it's set, it overrides EDITOR, unless you're using a dumb terminal. When vi was originally created, people would set their EDITOR value to ex and VISUAL to vi. If they were on a dumb terminal, they'd get ex, but if they were on a terminal with graphics support, they'd get vi.

Nowadays, just setting EDITOR is fine, since you're practically never going to run into a dumb terminal (unless you use gvim's :shell command). However, your editor should be a value that stays in the foreground until you quit it; otherwise, most tools (including Git) will assume you haven't edited the file. If you use gvim, like me, then you need to set EDITOR to gvim -f.

If the user hasn't specified EDITOR or VISUAL, the default is vi. That's a particularly cruel and heartless default for newbies, but it's what people expect when working on Unix systems. Defaulting to something else means that people who need to run an occasional editor-invoking command on an infrequently-accessed server will be surprised. Sysadmins don't like surprises.

There's frequently confusion as to whether the variables should be interpreted by the shell. The answer is emphatically yes, for several reasons. First of all, if your program doesn't interpret it, people have to use shell scripts to get things like gvim -f to work. Secondly, on some platforms, like Cygwin, using commands with spaces in the path is very common, and using shell interpretation allows a standard way to make that work.

Finally, Debian's sensible-editor helper and Git pass the command to the shell. Since Debian recommends the sensible-editor script for packages by default, other packages should emulate its behavior. Similarly, Git is a tool that's widely deployed, and it also invokes the shell.

If you're interested in how Git implements this, you can look at its editor.c file, which provides the (relatively simple) logic. If you have a Debian or Ubuntu system, /usr/bin/sensible-editor demonstrates how to do it in shell.

[bk2204]

As I’m learning more about Rails, I’ve noticed that one things the Rails community is great at is documentation. I can find documentation on pretty much anything I can want to do and the pros and cons of doing it a particular way.

[bk2204]

I think I may have found a way to make vi Turing-complete.  This could end poorly.

[bk2204]

I’ve been interviewed in the latest version of Git Rev News.

[bk2204]

Setting up Two-Factor Auth

After reading about several high-profile open source contributors getting their accounts hacked, I decided it was time to set up two-factor authentication. I’ve used TOTP (Google Authenticator) for GitHub, Google, and Rage4, and additionally phone verification for Tumblr (since they don’t have recovery codes).

The set-up is really easy and it makes it yet more difficult to compromise my accounts or domains. Choosing PRNG-generated passwords with 104 bits of entropy apiece helps, too.

[bk2204]

Legacy Code and Unit Testing

I was reading this article on debugging legacy code and it stuck out to me. At cPanel, we have a function that loads a text file with key/value pairs. It is highly optimized and handles different separators, values, and other parameters. Each of these cases generates a custom, optimized parser and then caches it for later.

Needless to say, the function is pretty unreadable. It is also, like much legacy code, completely undocumented. In order to figure out how it even worked, I had to read the existing unit tests (fortunately there were some) and determine what the various parameters meant.

In the end, the change I wanted to make—ignoring carriage returns so that files copied from Windows would work correctly—could be made with a two-line change plus copious unit tests. I could also be confident I hadn’t broken any of the other code.

[bk2204]

ECDSA

I’m in the middle of rolling out my new 384-bit ECDSA certificate to my servers. One of the benefits of this is that forward security is no longer optional, as the certificate is not capable of RSA key exchange and practically nobody implements the non-ephemeral ECDH cipher suites (nor do I have them enabled).

This does have the downside that people using RHEL or CentOS before 6.5, Windows XP, or Android 2.3 won’t be able to contact me. I don’t feel this is a huge problem though.

I bought a one-year certificate from Comodo since my previous certificate was going to expire just before Let’s Encrypt starts issuing certificates and they aren’t going to be issuing wildcards (or ECDSA certificates) at first.

[bk2204]

Refactoring Git

After taking a break, the struct object_id work is continuing. The patches to for_each_ref and friends have been merged to master, and I'm finishing up part 2, which converts struct object as well. That particular patch has ended up being too large for vger, the Git mailing list server, to handle.

Along with the actual conversions to struct object_id, I'm changing hard-coded 20s and 40s in the codebase to symbolic constants. That allows a quick search-and-replace when we get to the point of adding an additional hash function.

As for what that additional hash function will be, I'm kind of torn between SHA-256 and SHA-512/256. For those that aren't familiar with the latter, it's SHA-512 with different starting constants and truncated to 256 bits. SHA-512/256 runs significantly faster on 64-bit machines (40% faster, to be exact), which is beneficial. However, SHA-256 is more common, and most objects being hashed are not that large.

The reason I'm not looking at SHA-512 itself is that each instance of struct object_id will be as large as the largest hash function supported. Therefore, using SHA-512 means each of those instances goes from 20 bytes with SHA-1 to 64 bytes with SHA-512. SHA-256 is only 32 bytes, which is much better.

Regardless of the addition of the new hash function, though, I think the code will end up being more robust with the opaque struct instead of unsigned char arrays everywhere.

[bk2204]

Right now, my website is built entirely in Java using Ant. This is only because all the working XML tools are written entirely in Java, and forking Java five times per page is slow.

I'm going to spend some time working on the REXML Relax NG validator so that it can handle DocBook and XHTML 5. There's really no reason I should have to use Ant as a build system.

[bk2204]

So I'm now an official contributor to the Asciidoctor project.  AsciiDoc is a text format designed for easy writing and conversion, and Asciidoctor is a faster, more featureful implementation of the original AsciiDoc program in Ruby.

My most recent patches have been to improve the conversion to DocBook manpages. The output was missing some important elements, and as a result, xsltproc would always output to a file called ".1".

I've also got a set of patches that allow building Git with Asciidoctor. This results in a pleasant 42% speedup of the documentation build time. Since the documentation on the Git website is built using Asciidoctor already, it's useful to be able to test it as part of the Git build process.

[bk2204]

BE5192808F: refused data due to SpamAssassin score of 58.955

I can only assume this was intentional, since this is eleven times the normal threshold.