another one of my brains @anotheroneofmybrains - Tumblr Blog

Posts

Ostensibly about Magic: the Gathering, this applies to just about anything.

Generate a file filled with ASCII/junk

I needed to create some junk files to test out a procedure for sharding a large file into smaller files.

The solution: combine dd, /dev/urandom, and base64.

Example:

dd if=/dev/urandom bs=1M count=2 | base64 >random.txt

bs=1M and count determine how much data dd will read out. This example would be 2M of base64.

For smaller amounts:

head -c10 /dev/urandom | base64

Some people like to use this to generate random passwords.

#shell scripting

http GET, POST in Python 2.x

Use urllib and urllib2 for making HTTP requests. Don't use httplib.

urllib2

urllib

httplib is no fun

There are some dumb limitations:

HTTPConnection can only be used once

You can only call getresponse() on an HTTPResponse once

There's no HTTPRequest object as such; you throw all your args into a single HTTPConnection.request() call.

There are one or two little gotchas:

### NO NO NO conn = httplib.HTTPConnection("www.example.com/foo") ### yes conn = httplib.HTTPConnection("www.example.com") resp = conn.request("GET", "/foo")

All in all httplib was cumbersome for interactive development.

A friend speculated that this was a thin wrapper around the C APIs. I don't have anything against C, but this is Python.

urllib and urllib2 instead

urllib and urllib2 have what you need. urllib has quote(), which you want if you want to URL encode a string. More convenient is urlencode(), which takes a map or sequence, and outputs the usual foo=bar%20baz&quux=frotz.

urllib2 supplies a Request object. The constructor is a bit of a bear, but most of it is optional; you can call add_data() or add_header() for to add stuff later.

After that, call urllib2.urlopen() with your Request object. The Response object is a file-like. The body is the file, and methods like info() give you headers, et al.

caveat

It seems like on such as a 403 you'll get an exception (HTTPError) and your response will be empty. IMHO this is dumb. An exception should be in the case of a malformed, incomplete, or otherwise bogus response. A 403 is a legit response. It may have headers, etc. I should be able to look at that through the response object, not forced to paw through exception info trying to find the right fields.

#python #http

rsync

If you can't remember the command line arguments to rsync, take heart. Start with something like rsync -av src dest and work your way from there.

#*nix #shell

Re-indent in vim

You can re-auto-indent (?) some text in Vim with `=`. (It works with ST2, as well.) This can save you a bunch of manual `>` and/or `<`. It's seems close to useless with Python, but otherwise it can be very useful.

#vim

ssh aliases

I always forget this, so here is the most basic of basics. Right now, you're typing this: ssh [email protected] You'd rather type this: ssh home Add this to `~/.ssh/config`: Host home HostName home.bar.com User foo `ssh home` should now work as expected.

#ssh #shell

This is a ruby gem which allows you to monitor files and perform some action when they change. You limit which files you want to monitor with a regex. You can configure which actions `guard` performs yourself, using their DSL, or by installing [one of the many plugins][guardplugins]. If all you want to do is run a shell command, `guard-shell` will suffice. An example `Guardfile`, roughly: guard :shell do watch /(.*)\.mkd/ do |m| `markdown #{m[1]}.mkd >#{mkd[1]}.html` end end That's about all you need to get started. Whenever you alter `foo.mkd`, `guard` will invoke the command line above, which will output a corresponding `foo.html`. Obviously you could use this for compiling, running unit tests, or whatever you like. [guardplugins]: https://github.com/guard/guard/wiki/List-of-available-Guards

#shell #guard #automation

git, ssh, and push

It's actually super easy to use git and ssh to pull repos between your machines. Pull *is* the operative word, though. More on that in a bit. Create a repo on machine A, whose hostname for this example is `desktop.local`. We'll clone from this repo at first. mkdir -p ~/src/my-repo cd ~/src/my-repo git init On machine B, hostname `laptop.local`, we clone the repo: mkdir -p ~/src cd ~/src git clone ssh://desktop.local/~src/my-repo Now your laptop has a copy of `my-repo` and pulls from `desktop.local`. If we want to be able to pull from `laptop.local`, we'll need to tell `desktop.local` about it. git remote add origin ssh://laptop.local/~src/my-repo git branch --set-upstream master origin/master That should do you. Now, you *can* set up a scenario where you push to a repo, but [it will work differently][bare]. You won't be able to work in that repo; it's purely for pushing and pulling, not committing. That's a bit like github except your stuff doesn't have to be public, and is instead secured behind ssh on your private machine. [bare]: http://gitready.com/advanced/2009/02/01/push-to-only-bare-repositories.html

#ssh #git

Holy mother of crap. I forget this one all the time and it drives me up a wall.

This works in bash and zsh:

declare -f

typeset -f

This will list all the functions defined in your shell, including the source.

#shell #bash #zsh

watch

Sometimes you want to monitor what changes are happening to a file or the output of a command. The poor man's way to do it is via something like while true; do echo foo; sleep 1; done Fortunately, there exists a simpler way: watch -n 1 'echo foo' It appears to ship with Linux, but not Mac OS 10.7. [Homebrew](http://mxcl.github.com/homebrew/) has it, however.

#shell

Tunneling VNC over ssh

Assumptions: * You have a remote machine configured to listen for VNC connections on such as :5900. * You can ssh into your remote machine. * You (wisely) have not opened port 5900 in your firewall. * You want to access your machine remotely over VNC. Let's tunnel. ### Rationale VNC is basically plain text, including any keystrokes you send. Authentication is weak compared to such as ssh. And to access VNC remotely you'd ostensibly have to open that port to the Internet. With ssh tunneling, you don't have to open a special port in your firewall for VNC. All authentication and subsequent traffic happens over ssh. ### Set up the tunnel Try this: ssh -C [email protected] -L 5900:127.0.0.1:5900 Feel free to add `-fNq` to background ssh (so you can't accidentally close a terminal window and lose it), `-N` so that it doesn't run any commands, and `-q` so that it's quiet. This connects you to `myhost.com`, forwarding your local port (hence `-L`) 5900 to the remote port 5900. ### Connect Use your favorite VNC client, connecting to `localhost`, port 5900, and with whatever credentials you set up. ### Anecdata: slow? I've only tested this once or twice. It seems pretty slow! I can't tell if it's the quality of this connection or what. So maybe prepare yourself.

#vnc #ssh

split lines in bash/shell

Example: I have a big ol' PATH variable, necessarily `:` separated. I want to see each entry on a separate line. Here you go: echo $PATH | tr ':' '\n' Or if you're feeling even more terse: tr ':' '\n' <<<$PATH `tr` is one of my blind spots, I'll admit. I'm more inclined to reach for `sed` or `awk`, which are overkill for little jobs like these.

#shell

Well, at long last, I've switched from Vim to Sublime Text 2. If I had to summarize my motivation, it would be two parts:

I get 80% - 90% of Vim keystrokes, et al, through ST2's Vintage plugin.

Most of the plugins I installed to Vim already work in ST2 out of the box, with extras I never quite got working.

I highly recommend taking it for a spin.

A license will set you back $59. In the final consideration, it was worth it to me for a few reasons.

The first is that it's cross-platform, Mac and Linux. (Maybe there's a Windows version? I don't know.)

The second is that you can install it on as many machines as you want. I have it on all of my work and home machines.

Lastly, ST2 does nag you if you have no license. However, it's very, very easy to ignore. In other words, you can evaluate it as long as you like. And it's one person writing it, as far as I can tell. This is something I want to support with my money.

#st2

With the `:TOhtml` command in Vim, you can take a file or arbitrary set of lines and convert them into syntax-highlighted HTML, suitable for copy-pasting into a blog.

#vim

The real gem in this blog post is this: cd /usr/bin whatis * That'll enumerate everything in the directory, `apropos`-style. Wow. Apparently it works for some subset of files in `/etc`, as well.

#shell

ls -l `which foo`

Sometimes I want to know when a file in my path has been updated. You can find out where a file is with which foo. Pass that right into ls -l like so:

ls -l `which foo`

#shell #backticks

gobrain

I've been using the Go language in my spare time. I like it a lot so far.

Rather than clutter up this blog with a bunch of Go-specific stuff, though, I've created a new one: gobrain.tumblr.com.

#golang

Trending Blogs

Recently Viewed Blogs

another one of my brains