coding and what-have-you @mrmekon - Tumblr Blog

My new job is Virtual Reality.

Virtual Reality is virtually ridiculous.

I celebrated the new job with Semmeldagen.

And another Semmeldagen, because why not.

And who says Semlor can’t be Våfflor? Semwaffles!

And The Book of Mormon, in Swedish!

Then we started baking.

And baking.

Sigma watched.

Sourdough all the things!

Microchippin'

If I were to describe MPLAB X, the NetBeans-based IDE that Microchip is pushing as the primary development suite for all new PIC32 projects, I believe I would leave it at simply: insufferable.

It is swell that Microchip is kind enough to provide a free IDE to sit on top of free compilers and interface with quite inexpensive debuggers. But the experience of using said tools isn't exactly pleasurable. And, as I recently found out, shelling out lots and lots of cold hard cash for their premium products... doesn't really improve things much on the tools front.

I have one concern, in particular, that seems totally inexplicable. Who ships a development suite with NO FUNCTIONAL COMMAND-LINE DEBUGGER? Forcing me to use your unstable IDE is already bad enough, but when your IDE isn't even functional, how am I expected to debug my code?

#embedded #microchip #PIC #MPLAB #MPLAB X #debugger

This Week In Hobbies

Brewed a Braggot (half-beer, half-mead): Cymreig Bragawd Ddwbwl

Made a Farmhouse Cheddar:

Made Belgian Candi Sugar:

Planted 6 hop bines: Chinook, Nugget, Willamette, Santiam, Cascade, Galena

#hobbies #hops #homebrew #beer #candi sugar #cheese

Tempest-cljs in Mozilla DemoStudio

I submitted the current form of Tempest-cljs to Mozilla DemoStudio:

Tempest-cljs @ Mozilla DemoStudio

Of course, it's always still available on GitHub.

It has been under casual, constant development for about the last month. Since the last post on the subject, it really changed from a ClojureScript demo to an actual game. Just changing the colors to match the original did a lot, psychologically, but it's also full of new features.

Over the past day I have been focusing on performance a little bit. Performance has been all over the place, depending on the browser and the level and what enemies are currently alive. I ended up writing a few benchmarking demos to use along with Chrome's Javascript profiler to get a sense for what's slowing it down.

I've mostly been replacing frequently-calculated values, like the cartesian coordinates of all the points on a level, with cached copies. Since the game started off as just a demo of line drawing, there are still a lot of values in there that are calculated every frame, despite never changing.

The biggest performance surprise, so far, has been the dramatic increase in performance gained by using Clojure's map destructuring in the argument list, instead of manually pulling out the keys I want in a let block.

There was also a dramatic speed increase, for some things, by turning on Google Closure's advanced compilation flag. I believe it does code-inlining, and my guess is that's where the increase is coming from. All told, in the past 12 hours the projectile benchmark has gone from 6 FPS average to 26 FPS average on my MacBook Pro.

#tempest #clojurescript #clojure #html5 #games #benchmarks

Tempest in ClojureScript -- Vector game are the best

I've been learning Clojure/ClojureScript/HTML5 by diving head first into writing a game:

Tempest in ClojureScript (GitHub)

Tempest is, of course, the best vector game ever made. I'm implementing the original arcade version, not the (much improved) Tempest 2000, and sticking with wireframe vector graphics.

ClojureScript has been pretty nice to work with, despite never having really used a Lisp for anything substantial before.

At this point, the game has multiple levels and one enemy, but no actual game logic. The player is drawn, and can be controlled with the left/right arrow keys. Space bar fires a bullet, and bullets destroy an enemy... but nothing is finished, it's all just simple building blocks at the moment.

Google Closure's keyboard handler doesn't seem to be sufficient, and will probably have to be replaced. You can't press one key while holding down another.

#tempest #clojurescript #clojure #html5 #games

Dill dip was not optimized for my phone, but it still worked okay.

#iphone #apple

OS X: Current CPU temperature on command line

Apple doesn't provide any easy way to query the temperature and fan speed on an Intel Mac running OS X 10.5+.

You can access the SMC information with the IOKit API, by asking the kernel for a list of registered 'AppleSMC' devices. It's a pain in the ass, but a nice fellow has made a GPL'd program that handles it for you:

Fan Control (smc utility)

It also comes in the bundle with smcFanControl.

It returns raw sensor readings, so here's a huge one-liner to get the current CPU temperature:

smc -k TC0D -r | sed 's/.*bytes $.*$)/\1/' |sed 's/$[0-9a-fA-F]*$/0x\1/g' | perl -ne 'chomp; ($low,$high) = split(/ /); print (((hex($low)*256)+hex($high))/4/64); print "C\n";'

#os x #mac #one-liner #smc

RF transmission: Whitening

The last in the series on alternating bits. See Bits Gotta Alternate and Manchester Coding.

Another type of line coding, Data Whitening or Scrambling, addresses the issue that radio transmission requires the data bits to alternate often.

What It Does

Whitening addresses all three major issues:

DC-bias is less likely. Unlike Manchester Coding, a DC-bias is not guaranteed to be removed, but it is statistically improbable.

Long bit runs are less likely. Some long runs can still occur with whitening, but the average run length is reduced. More importantly, bit transitions on the input and output are decoupled from each other, so long bit runs in the input data do not cause long bit runs in the output.

Power spectral density is more likely to be evenly distributed. Whitening tends to distribute the data evenly across the radio channel's frequency bandwidth, allowing the transmitter to run at a higher power without violating FCC regulations.

No extra data bandwidth is used. Manchester encoding doubles the data bandwidth: every byte in is two bytes out. Whitening does not: every byte in corresponds to one byte out.

How It Does It

Whitening is so named because it introduces a pseudo-random element that makes the output appear more like white noise -- uniformly distributed.

Whitening is typically implemented by using a maximal Linear-Feedback Shift Register (LFSR) to generate a repeating, pseudo-random pattern of bits that are XORed with the input data. Since XOR is reversible, the receiver can remove the 'noise' from the data with its own LFSR that is synced to the transmitter's.

An LFSR is a shift-register: a sequence of bits that are shifted down on each cycle, and the 'output' bit (the one that would be shifted out of the register) is XORed with some of the higher bits in the register and shifted back into the top as the 'input' bit.

An example LFSR used for whitening 8-bit data is a 9-bit shift register with XOR taps on the 0th and 5th bits. That is, the bit being shifted off (index 0) is XORed with the bit at index 5 and fed back into the top, index 8.

That particular configuration implements a 'maximal LFSR', which means it cycles through every possible combination of a 9-bit number, except 0, with no repeats. The example LFSR starts over every 511 cycles. It must be seeded with some non-zero number. An LFSR filled with all 0s gets 'stuck' and never recovers.

Maximal LFSRs have some neat mathematical properties regarding the length of bit runs. For an N-bit maximal LFSR, there will be one run of N bits, one run of N-1 bits, ..., 1/4th of all runs will be 2 bits, and 1/2 of all runs will be 1 bit.

Input bytes are XORed with the least-significant 8-bits of current value of the LFSR register, and then the LFSR register is shifted 8 times. The pseudo-random distribution from the LFSR mixes up the bits of the input data across the 8-bit range, so big blocks of identical bytes (i.e. 0x00 or 0xFF padding) gets spread out.

Disadvantages

An LFSR improves the resulting radio transmission on average, but does not guarantee anything. Two specific cases should be considered:

If your input data is exactly the output of the LFSR register, each data byte will be XORed with itself. The output stream will be 100% zeroes. This kills the radio.

If your input data is exactly the complement of the output of the LFSR register, each data byte will be XORed with its complement. The output stream will be 100% ones. This kills the radio.

Python Experiment

This script is not intended to be fast, readable, or perfect, but does some statistical analysis of whitening data with the 9-bit LFSR described above. Generated images are shown at the bottom of this post.

whitening.py

Images

The following images show random bytes generated with various probability distributions before and after whitening. Note that, in all cases, whitening results in more uniform distribution of bytes.

#communication #radio #rf #scrambling #whitening #embedded

RF transmission: Manchester coding

In follow up to the post about why Bits Gotta Alternate, let's talk about a common way of forcing them to do so: Manchester coding.

What It Does

Manchester coding is a very simple line coding technique that solves some of the problems from the previous post:

It is DC-balanced -- that is, for every 1 bit there is a matching 0 bit. The average DC level is 0.

It guarantees a maximum bit run of two bits. That is, you can never have three 1's or 0's in a row in the transmitted data.

Manchester coding does not solve the issue of power spectral density -- that is, the transmitted data can still causes spikes in the frequency domain, instead of spreading data evenly across the channel's frequency bandwidth.

How It Does It

Implementing Manchester coding in hardware is extremely easy: XOR the data with its clock. It's basically free.

When you XOR data with the clock (which, recall, has an up cycle and a down cycle for each bit), you end up with twice the output bits. Every 1 from the input becomes '10' on the output, and every 0 from the input becomes '01' on the output.

Example: 11100010 -> 1010100101011001

Since one bit transition on the output is guaranteed for every bit on the input, the longest possible run of bits is 2. The signal is also DC-balanced (since there is a 0/1 pair for each bit).

This is easy enough to do in software, too. You can do it on a bit-by-bit basis, or on a per-byte basis with a small look-up table.

Disadvantages

There's one extremely obvious disadvantage to Manchester coding: your data size doubles. A single transmitted symbol (a 0 or 1 in the air) only represents half of a bit. Manchester coding solves a lot of problems caused by transmitting arbitrary data via radio, but it effectively halves your maximum throughput.

#communication #manchester #radio #rf #embedded

RF transmission: Bits Gotta Alternate

When one converts digital data into bouncing particles of air, using "electromagnetic radiation" and the voodoo magic associated with it, one discovers some necessary tricks to make that magic work. One very important trick:

Bits Gotta Alternate.

There are a bunch of reasons why Bits Gotta Alternate:

Reason 1: Clock Recovery

A radio transmission is serialized data, but, unlike the serial data that often runs between ICs, there's no room for a clock line. That means the transmitter and receiver need to synchronize clocks using the data itself. This is clock recovery. Basic stuff.

The receiver has a free-running clock, and adjusts it to match the bit transitions in the data it receives. The more bit transitions it sees, the tighter the clock sync. The fewer transitions, the more chance for clocks falling out of sync.

If the transmitter sends a really long string of identical bits, say 16 identical bits in a row, the receiver's clock can (and likely will) drift too far from the transmitter's clock, and lose track of how many bits have come in.

An example of something that probably won't work: a 256-byte fixed packet, padded with 0xFF or 0x00 at the end if the data is shorter than 256 bytes. This is not a terribly uncommon situation for non-radio situations, but if you send a packet with only a few non-padding bytes in it over a radio, the receiver's clock is likely to give up the ghost when it sees that huge block of identical bits at the end.

Alternatin' bits are good for clock recovery.

Reason 2: Bandwidth utilization

Ask an RF engineer.

The basics: when you transmit data at some frequency, you also hit nearby harmonic frequencies. You transmit on some center frequency, and also hit a bunch of nearby frequencies. The range of frequencies around the center frequency make up your frequency bandwidth.

So, and here's the voodoo magic, the bytes you transmit affect the individual frequencies you hit around the center frequency. If you send the same values a lot, such as a big ol' block of 0xFFs, you will have a big power spike on some frequency, and the rest in your channel will be under-utilized. And that's bad. Ask an RF engineer.

Alternating bits, alone, doesn't necessarily shift your frequency spikes, but alternating bits at varying rates does. If your data is uniformly distributed across all the possible values, your frequency utilization will be, too.

Reason 3: DC Bias

DC Bias is the mean DC level of the output signal. And it's bad in an radio transmission: a maintained DC voltage can cause filter capacitors (that normally pass AC) to block the signal if the caps saturate. And other things, ask an RF engineer. But, the important part is, most radios don't like transmitting DC.

A big block of 0's or 1's is DC.

So alternate your bits.

Reason 4...N!

There are a lot more reasons. Like Automatic Gain Control, for similar reasons as clock recovery. If the AGC circuit sees all 1's for long enough, it may lose its reference, forget what it's doing and start calling them 0's. Or vice versa.

All of this was just an unnecessarily long lead in to the next posts on how to unlink the transmitted data itself from the periodicity of its bit runs by using certain line coding techniques.

#communication #radio #rf #embedded

OS X: Find source URL of downloaded file

Safari, Chrome, and Firefox add extended attributes to files downloaded in OS X, so the OS can stupidly warn you not to open things you downloaded. The xattr property is 'com.apple.quarantine'.

Safari and Chrome (and maybe Firefox) also add the optional 'com.apple.metadata:kMDItemWhereFroms' property, which stores the URL that the file was downloaded from. If you can get past Item Where Froms without banging your head, you'll be pleased to know that it is stored as a Binary Property List, a binary version of the regular Property List, a long-winded XML document that Apple uses for every damn thing.

Anyway, one-liner mixture of bash and perl to display the property in a readable form:

$ xattr -p com.apple.metadata:kMDItemWhereFroms $$YOUR_FILENAME_HERE$$ |tr -d ' \n' | perl -ne 's/([0-9a-f]{2})/print chr hex $1/gie' |plutil -convert xml1 - -o -

UPDATE: Again, without perl. I didn't know about xxd.

$ xattr -p com.apple.metadata:kMDItemWhereFroms $$YOUR_FILENAME_HERE$$ |xxd -r -p |plutil -convert xml1 - -o -

UPDATE 2: Err, yeah. There it is.

$ mdls -name kMDItemWhereFroms $$YOUR_FILENAME_HERE$$

#shell #os x #apple #perl #one-liner

IEEE 1722, AVB, and PCs

We're playing around with Audio/Video Bridging (AVB) at work, the new, unfinished IEEE standards for sending high-priority, high-fidelity audio over an ethernet network.

There isn't much info about it yet, and we've had to pick through demos and unfinished specs and all sorts of wrong information.

We have demo boards from XMOS and DSP4YOU that stream audio to each other over the network. One of the questions we were investigating is: what can a PC do with AVB? The spec basically doesn't allow for them, and it's generally assumed that a PC cannot be an endpoint. There's just no way a non-real-time OS can handle the strict timing requirements (IEEE 1722AS), or get the latency low enough (2ms end-to-end for Class A AVB), without hardware assistance.

However, I have been testing out 'sniffing' the AVB traffic with a PC. I originally tried writing an OS X kernel extension (kext), but ran into a strange problem. AVB packets are a special case of a VLAN (802.1Q), and the frickin' OS X kernel doesn't pass VLAN packets to Kernel Interface Filters. In a bizarre twist, though, VLAN packets are accessible in user space through the Berkeley Packet Filter API... but they aren't available in kernel space. Go figure.

Anyway, I wrote a sniffer in user-space that uses BPF to filter and process AVB packets. I can play them in near-real-time with PortAudio, and record them to a file with libsndfile. Hooking PortAudio up to SoundFlower lets me pass them off to GarageBand, where I can use AU plugins to modify the sound. The delay probably blows the AVB spec, but my untrained ear can't hear it.

Lookin' good:

$ ./avbsniffer Device 0: Built-in Microphone Device 1: Built-in Input Device 2: Built-in Output Device 3: Soundflower (2ch) Device 4: Soundflower (16ch) Using device 2 Buflen: 524288 Filled AVB buffer! Recieved: 2006388 Dropped: 0 Drop rate: 0.00% Total packets: 1000000 Skipped packets: 0 (0.000000%)

No source code for this one, but I'll give you my BPF filter to extract an AVB packet:

struct bpf_insn insns[] = { BPF_STMT(BPF_LD+BPF_H+BPF_ABS, 12), // load short from 12-bytes into packet (vlan tag) BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, 0x8100, 0, 3), // Exit if not VLAN packet (pkt[12] == 0x8100) BPF_STMT(BPF_LD+BPF_H+BPF_ABS, 16), // load short from 16-bytes into packet (ethertype) BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, 0x22F0, 0, 1), // Exit if not AVB packet (pkt[16] == 0x22F0) BPF_STMT(BPF_RET+BPF_K, AVB_PKT_SIZE), // Accept AVB packet! BPF_STMT(BPF_RET+BPF_K, 0), // Refuse packet ("accept 0 bytes") };

#avb #audio #work #networking

Just an update of what Chess of the Round Table looks like now. Moats and creeks are implemented, the pieces have proper icons and colors. The single-player version is actually fully functional, but still working on multiplayer.

#chess #chessoftheroundtable #screenshot

Wardrobe Rights Management

There's a serious problem in the American clothing industry today: shoplifting and counterfeiting. Cheeky bastards stealing or copying clothing designs without any contribution to the designers who worked so hard to develop and market the original product.

But, thanks to modern technology, there is a solution in sight! I call it Wardrobe Rights Management, and it is the future.

Wardrobe Rights Management, or WRM, refers to the licensed software and hardware technology that can be embedded into clothing and appliances that interact with clothing to verify that its owners are following legitimate usage protocols as defined by the individual garments' licensing agreements.

The idea is simple: a small, biometric RFID tag embedded into each article of clothing. The tag, combined with scanners in washing machines, dryers, entryways, and even public checkpoints and way-stations, can be used to identify the garment, the wearer, the relationship between them, and whether any illegitimate usage has been detected.

Clothing designers will now be open to offer a range of consumer-beneficial licensing agreements, with models such as:

Unlimited use** (full cost)

Home use only (reduced cost)

One-time use (low cost)

Pay-per-wash (micropayments)

Loan to friends (small fee)

Transfer license to a new owner (small fee)

** some exceptions apply, see contract for details

Clothing designers and manufacturers will simply need to register each model of garment with the Clothing Producers Association of America for a small fee, and obtain a unique Garment Unique Identifier for each article sold.

If we all band together, and tell our Congressmen that we want their help, I'm sure we can achieve these great advancements for society, and protect the poor clothing industry professionals from collapsing markets and rampant crime.

#piracy #SOPA #DRM

Emacs regexp replace -- Python print to logging

Handy replace-regex emacs command to convert print statements to logging.debug statements in Python 2.X.

match: print\s-+$".*$ replace: logging.debug(\1)

#emacs #regexp #python

Python, Quartz, GTK, and window focus.

One of my company's products involves a small GUI application for controlling a USB peripheral that we designed. Although the customer only requires it to target Windows at the moment, we are trying to keep it (mostly) working on Linux and OS X as well, since those may need to be supported in the future.

For easy of cross-platform-GUIness, and because I was already somewhat familiar with it, we decided to do windowing with PyGTK. Alternatively, we could have used Tkinter, PyQT, wxpython, or pyglet... but we didn't. Mostly because I already knew some GTK.

So, anyway, cross-platform GTK has been a pain. Maybe that would have been true with any of them.

For OS X, you have the option of building your GTK+ libraries with your windowing backend of choice: X11 or Quartz.

We originally went with X11 because it's the default of Macports, but discovered that StatusIcons ("system tray" or "notification area" icons), don't work in X11 (because X11 isn't running a notification area).

So, I rebuilt everything with the Macports variant trio: "-x11 +no_x11 +quartz", and eventually found myself with a more native-looking GTK. And StatusIcons! StatusIcons that don't work worth a damn!

With the new, native, pretty Quartz windows, GTK has focus issues. New windows pop up to the top, but do not have focus. This is true for the main application, and dialog boxes. And the StatusIcon menu pops up when you click on the icon, but it isn't selectable because -- get this -- it doesn't get focus when you click on it.

I spent hours trying different combinations of GTK *show*() and *activate*() and *present*() and *focus*() functions, all to no avail. I also found this open bug report on the same issue.

Eventually I stumbled upon an unrelated Objective-C post about window focus, and solved my problem with PyObjC:

import platform if platform.system() == "Darwin": from AppKit import NSApp def callback(target, button, time, *args): if platform.system() == "Darwin": NSApp.activateIgnoringOtherApps_(True) menu.popup(None,None, None, button, time) ... icon.connect("popup-menu", callback) window.show_all() if platform.system() == "Darwin": NSApp.activateIgnoringOtherApps_(True)

Windows actually get the system's focus when activateIgnoringOtherApps: is called, and the StatusIcon menu works (mostly) as expected now.

#python #gtk #pygtk #quartz #pyobjc

Building emacs-chess for Aquamacs

I use Aquamacs for OS X, and just installed emacs-chess. It wasn't immediately obvious how to build it for Aquamacs, so here is the run-down:

# git clone git://github.com/jwiegley/emacs-chess.git # cd emacs-chess/ # git submodule init # git submodule update # autoreconf -i # EMACS=/Applications/_Development/Aquamacs.app/Contents/MacOS/Aquamacs EMACSLOADPATH=/Applications/_Development/Aquamacs.app/Contents/Resources/lisp/ ./configure # make

I moved the emacs-chess directory into ~/.emacs.d/, and then added to my ~/.emacs file:

(add-to-list 'load-path "~/.emacs.d/emacs-chess") (load "~/.emacs.d/emacs-chess/chess.el")

Install crafty:

# sudo port install crafty

And finally, run Aquamacs and play chess with M-x chess.

#chess #emacs

Trending Blogs

Recently Viewed Blogs

coding and what-have-you