TJ Holowaychuk

A blog about server side technologies, nodejs, language design and more.

[nodejsreactions-blog]

Spelunking into a module’s source due to bad docs, only to discover CoffeeScript

Submitted by @jxson

[tjholowaychuk-blog]

Modular CSS preprocessing with rework

Several months ago I started a project named Rework, a very fast, simple, flexible, and modular CSS preprocessor. The biggest and most obvious question I get is how this tool compares to something like Stylus, LESS, or Sass, and why would you want to use it.

The simple answer is that Rework caters to a different audience, and provide you the flexibility to craft the preprocessor you want, not the choices the author(s) have force on you.

Our goal with Rework contrasts the others, as it does not provide a higher level language within the CSS documents, Rework's goal is to make those constructs unnecessary, being the most transparent CSS pre-processor out there.

Speed

Rework was designed to be simple and fast out of the box, without implementing complex caching models allowing everyone to hack on Rework itself or associate plugins easily.

Powering rework is a css parser written in JavaScript that may be used with node.js or in the browser. On the other end there is the css compiler which accepts the AST from css-parse, outputting CSS.

So where does Rework come in? Rework is the plugin and transformation engine that sits in-between these two tools. The AST is manipulated by Rework and its plugins (or user-defined plugins), which is then passed to the compiler.

Because these transformations are performed in JavaScript and not in a specialized preprocessor language they are simple to implement and efficient.

Let's check out what it can do!

Modularity & Flexibility

Before diving into the plugins bundled with Rework, let's take a look at how they can be applied. Rework's API is very simple, you can specify vendor prefixes with .vendors(), which all plugins have access to, or you may pass specific vendor prefixes to most plugins themselves. Plugins are passed to a .use() method, and then finally the CSS output is returned with .toString(). In practice this looks something like:

var rework = require('rework'); var css = rework('string of css here') .vendors(['-webkit-', '-moz-']) .use(rework.prefixValue('linear-gradient')) .use(rework.prefixValue('radial-gradient')) .use(rework.prefixValue('transform')) .use(rework.vars()) .use(rework.colors()) .toString()

Out of the box rework provides plugins for the following, which may be mixed and matched as you wish:

media macros — define your own @media queries

ease — several additional easing functions

at2x — serve high resolution images

prefix — add vendor prefixes to properties

prefixValue — add vendor prefixes to values

prefixSelectors — add prefixes to selectors

opacity — add IE opacity support

url — rewrite url()s with a callback function

vars — add css variable support

keyframes — add @keyframe vendor prefixing

colors — add colour helpers like rgba(#fc0, .5)

references — add property references support height: @width etc

mixin — add custom property logic with mixing

extend — add extend: selector support

Let's look at some examples!

Vendor prefixes

The prefix, prefixValue, and keyframes plugins make vendor prefixing extremely simple. Here the prefix plugin is applied to only two properties for illustration:

var rework = require('..') var read = require('fs').readFileSync var css = rework(read('examples/prefix.css', 'utf8')) .vendors(['-webkit-', '-moz-']) .use(rework.prefix('border-radius')) .use(rework.prefix('box-shadow')) .toString() console.log(css)

With the prefix.css file containing the following CSS:

.button { border-radius: 5px; box-shadow: inset 0 0 1px white; }

Rework yields the following prefixed output:

.button { -webkit-border-radius: 5px; -moz-border-radius: 5px; border-radius: 5px; -webkit-box-shadow: inset 0 0 1px white; -moz-box-shadow: inset 0 0 1px white; box-shadow: inset 0 0 1px white }

Extending

The extend plugin adds support for a special property named extend, which propagates the selector back up the AST so that the styles are applied without duplication. For example here's a contrived green join button implementation:

button { padding: 5px 10px; border: 1px solid #eee; border-bottom-color: #ddd; } .green { background: green; padding: 10px 15px } a.join { extend: button; extend: .green; padding: 20px; }

With this plugin enabled both of the extend properties are gone, and you're left with the CSS you might expect:

button, a.join { padding: 5px 10px; border: 1px solid #eee; border-bottom-color: #ddd } .green, a.join { background: green; padding: 10px 15px } a.join { padding: 20px }

Better rgba()

Ever had a color hex and then wanted to alter the alpha channel? Then you probably know how annoying this is with CSS. The colors plugin allows you to pass hex values to rgba(), for example rgba(#c00, .5) which is then compiled to rgba(204,0,0,0.5).

Writing your own plugins

Writing plugins for Rework is simple, suppose for example you want the ability to create your own properties, here expanding single-line position properties:

#logo { absolute: top left; } #logo { relative: top 5px left; } #logo { fixed: top 5px left 10px; }

Into the valid CSS shown below:

#logo { position: absolute; top: 0; left: 0 } #logo { position: relative; top: 5px; left: 0 } #logo { position: fixed; top: 5px; left: 10px }

The AST css-parse provides you is made up of simple objects and arrays, making it extremely easy to manipulate without extra knowledge of how the nodes are constructed or strange internal APIs. In the following plugin function we walk the declarations, check if the property is one of the positions, parse the value by splitting on whitespace and then inject the new css properties.

function positions() { var positions = ['absolute', 'relative', 'fixed']; return function(style){ rework.visit.declarations(style, function(declarations){ declarations.forEach(function(decl, i){ if (!~positions.indexOf(decl.property)) return; var args = decl.value.split(/\s+/); var arg, n; // remove original declarations.splice(i, 1); // position prop declarations.push({ property: 'position', value: decl.property }); // position while (args.length) { arg = args.shift(); n = parseFloat(args[0]) ? args.shift() : 0; declarations.push({ property: arg, value: n }); } }); }); } }

To use this plugin you would simply pass it to .use():

var css = rework('#logo { absolute: top left; }') .use(positions()) .toString()

Significant whitespace

Since CSS has a relatively simple grammar significant whitespace can help improve your productivity without turning the whole thing into an unreadable mess, this is especially true since Rework does not provide the higher level language constructs, which can get lost in the indentation. This is where css-whitespace comes in! It lets you write things like:

ul li background: #eee &:hover background: white &:active background: #ddd

Which yields:

ul li:hover { background: white; } ul li:active { background: #ddd; } ul li { background: #eee; }

Future

In the near future we'll be providing a more intuitive way to apply all or large chunks of the functionality Rework provides without manually adding all of the plugins, as well as improving the rework(1) executable.

[tjholowaychuk-blog]

Mocha JavaScript test framework matrix reporter: https://github.com/visionmedia/mocha-matrix

[tjholowaychuk-blog]

Ark - a work in progress digital painting with Pixelmator

[tjholowaychuk-blog]

First 30 minute digital painting with Corel Painter

[tjholowaychuk-blog]

This short screencast describes how you can use Express' app mounting feature to create modular self-contained applications to construct your product.

[tjholowaychuk-blog]

This screencast walks through creation of web components and some fundamentals driving the movement. Improve your client-side workflow and help us build a strong foundation for the web!

[tjholowaychuk-blog]

An introduction into using the javascript implementation of components, walking through a quick example of how you might integrate them into your application. Note that this is the node implementation only, in the future hopefully we'll have python, ruby, java, etcetera tools as well, all consuming the same things.

[tjholowaychuk-blog]

Building a date picker component

First install `component(1)` with `npm`: $ npm install -g component Or if you don't have `node` installed at all, on OSX execute: $ (cd /usr/local && \ curl -L# http://nodejs.org/dist/v0.8.15/node-v0.8.15-darwin-x86.tar.gz \ | tar -zx --strip 1) \ && npm install -g component \ && printf "installed component(1) %s\n" $(component --version) Next you'll need to bootstrap a new component, you can do this manually or use the quick `component-create(1)` command: $ component create datepicker repo (username/project): component/datepicker description: Datepicker ui component built on component/calendar does this component have js? y does this component have css? y does this component have html? create : datepicker create : datepicker/index.js create : datepicker/datepicker.css create : datepicker/Makefile create : datepicker/Readme.md create : datepicker/History.md create : datepicker/.gitignore create : datepicker/component.json ## Adding dependencies Next `cd` into the new directory and install the [component/calendar](https://github.com/component/calendar) component as a dependency: $ component install component/calendar install : component/calendar@master dep : component/range@master install : component/range@master dep : component/jquery@master install : component/jquery@master dep : component/emitter@master install : component/emitter@master sep : component/in-groups-of@master install : component/in-groups-of@master fetch : component/calendar:index.js fetch : component/calendar:lib/utils.js fetch : component/calendar:lib/template.js fetch : component/calendar:lib/calendar.js fetch : component/calendar:lib/days.js fetch : component/calendar:lib/calendar.css fetch : component/range:index.js fetch : component/jquery:index.js fetch : component/in-groups-of:index.js fetch : component/emitter:index.js complete : component/range complete : component/in-groups-of complete : component/emitter complete : component/jquery complete : component/calendar If you `cat` the _component.json_ file you'll see that `component-install(1)` has added the dependency for us: cat component.json { "name": "datepicker", "repo": "component/datepicker", "description": "Datepicker ui component built on component/calendar", "version": "0.0.1", "keywords": [], "dependencies": { "component/calendar": "*" }, "development": {}, "license": "MIT", "scripts": [ "index.js" ], "styles": [ "datepicker.css" ] ## Creating the component Like any other software it's good to have a suite of tests, or at very least a functioning demo for developers and end-users. In this case we'll just create _example.html_ to test drive the date picker, create the file and add the following HTML:

Datepicker var picker = require('datepicker'); var el = document.querySelector('[name=date]'); picker(el); Our date picker is going to be pretty simple, we're going to handle click events for the `element` passed to `picker()`, and then display the `Calendar` in a `Popover` instance, populating the input when a selection is made. Before getting started you'll want to install the popover component, event utility, and the optional "aurora" theme: $ component install component/popover component/aurora component/event Or with brace expansion: $ component install component/{popover,aurora,event} The following JavaScript snippet is what will go into `./index.js`, the main entry-point script of the component. As you can see here the dependencies are explicitly required at the top of the file via the commonjs module system, making it very obvious what this module is interacting with. Next we export a single object, the `Datepicker` constructor itself which sets up a bit of event handling boilerplate. var Calendar = require('calendar') , Popover = require('popover') , event = require('event') module.exports = Datepicker; function Datepicker(el) { if (!(this instanceof Datepicker)) return new Datepicker(el); this.el = el; event.bind(el, 'click', this.onclick.bind(this)); } Datepicker.prototype.onclick = function(e){ }; __NOTE__: The use of "event" may seem low-level here, and that's because it is, the event module is nothing more than a cross-browser event binding utility. These APIs are not designed to be cute, they're designed to drastically reduce the dependency bloat, by only using what your component needs. "Cute" APIs are provided by higher level components such as the jQuery-like [dom](github.com/component/dom) component, and are typically recommended for app-level use only To finish up the JavaScript portion of the `Datepicker` instantiate a new `Calendar` in the constructor, and add a classname for styling purposes. Next in the `.onclick` handler a new `Popover` is passed the `cal` element for its content, and a `datepicker-popover` classname is given here for styling purposes as well, then finally the call to `popover.show(this.el)` shows the popover relative to the `` element. var Calendar = require('calendar') , Popover = require('popover') , event = require('event') module.exports = Datepicker; function Datepicker(el) { if (!(this instanceof Datepicker)) return new Datepicker(el); this.el = el; this.cal = new Calendar; this.cal.el.addClass('datepicker-calendar'); event.bind(el, 'click', this.onclick.bind(this)); } Datepicker.prototype.onclick = function(e){ this.cal.on('change', this.onchange.bind(this)); this.popover = new Popover(this.cal.el); this.popover.classname = 'datepicker-popover popover'; this.popover.show(this.el); }; Datepicker.prototype.onchange = function(date){ el.value = date.getFullYear() + '/' + date.getMonth() + '/' + date.getDate(); this.popover.hide(); }; ## Building the component To build the component invoke the following `component-build(1)` command: $ component build Or simply use `make`, as the targets are already set up in the `Makefile` for you: $ make A new directory named `./build` will now have been created for you with the resulting `build.css` and `build.js` files. To make this process transparent in development I recommend using the `watch` command that some environments already have, for those that do not have it there's [watch(1)](https://github.com/visionmedia/watch). Before working on a component you may now simply watch the `make` command and chuck that process in the background to get out of your way: $ watch make & When you're done invoke `fg` to foreground the job again, or kill it with: $ kill %1 ## Styling the component Without custom styling the Aurora calendar / popover themes will look something like the following, not exactly great for a date picker since the calendar already looks self-contained.  Add the following CSS to `./datepicker.css` to cancel out the popover border for the datepicker popover only: .datepicker-calendar { font: 10px "Helvetica Neue", Helvetica, Arial, sans-serif; } .datepicker-popover .tip-arrow { top: auto; } .datepicker-popover .tip-inner { border: none; } Now re-build and you should have the following completed datepicker:  ## Publishing To "publish" the component all you need to do is push it to GitHub and then install with `component install myname/datepicker`! No fighting over names, you can call them whatever you like. To increase discoverability of your component you can then add it to the [Wiki component listing](https://github.com/component/component/wiki/Components) and the `component-search(1)` command will index it. Get the code for this component at [component/datepicker](https://github.com/component/datepicker). For more on components check out the [component repo](https://github.com/component/component) and the [Wiki](https://github.com/component/component/wiki).

[tjholowaychuk-blog]

Monitoring processes with mon

[mon](https://github.com/visionmedia/mon) is an extremely small, simple, and light-weight alternative to [monit](http://mmonit.com/monit/) for monitoring processes. Monit's approach of using a [DSL](http://en.wikipedia.org/wiki/Domain-specific_language) is at times inflexible and often annoying. Mon's approach is to monitor only a _single_ process, monitoring of several require a mon instance per process, however reducing single points of failure. This may sound like it's a lot of work to manage, but I promise you it's not! There's another tool for this called `mongroup(1)` to tie them together. ## Monitoring a single process Mon instances weigh in at about 400kb per process, so you don't have to worry about spawning a bunch of them. The simplest use of mon accepts a command to execute, and keep alive: $ mon "echo hello; sleep 5" Mon will execute the command with `/bin/sh -c`, so brace expansion and other shell goodies are completely fine. Once the sub process exits, mon will bring it back to life. By default mon logs to stdout: mon : child 94136 mon : sh -c "echo hey; sleep 5" hey mon : last restart less than one second ago mon : 10 attempts remaining mon : child 94138 mon : sh -c "echo hey; sleep 5" hey mon : last restart 5 seconds ago mon : 9 attempts remaining mon : child 94140 mon : sh -c "echo hey; sleep 5" hey Mon will continue to execute the command until mon itself is signalled with __SIGQUIT__ or __SIGTERM__ at which point it will signal the child using the same signal, and gracefully exit. When a process continues to fail upon restart, cyclic spawning will be detected by mon, it will then exit and log a warning. By default `10` restart attempts within 60 seconds are allowed, however you may tweak this value with `--attempts `. ## Failure alerts Mon provides two facilities for notification. The first is the `--on-restart ` flag, which mon will invoke upon _any_ restart. This is useful for emailing administrators the tail of a log file, notifying your team via IRC, etcetera. The second, and more crucial facility is the `--on-error ` flag, which is invoked only when a cyclic restart is detected, and mon has bailed out. Administrators should be notified immediately as the process is completely down. ## Daemonization Mon's `--daemonize` flag may be used to background the process and disassociate with the terminal, at this point stdio will be redirected to the log file specified by `--log`, otherwise defaulting as "./mon.log", typically this looks something like: $ mon -d -l /var/log/app.log "node app.js" ## Monitoring multiple processes with mongroup Using mon with several flags is obviously not something you'd want to be typing all the time, but part of the benefit of using mon is that shell scripts become your DSL. If you're not interested in doing this yourself but have several processes to manage I recommend checking out a project by [jgallen23](https://github.com/jgallen23) called [mongroup(1)](https://github.com/jgallen23/mongroup). I've forked the project to add some goodies, until merged you can find them [here](https://github.com/visionmedia/mongroup). Mongroup uses a simple configuration file, defaulting to the name `./mongroup.conf`, which simply lists out the `pids` and `files` directories, as well as one or more processes to spawn: pids = /var/run logs = /var/log web1 = node server 3000 web2 = node server 3001 web3 = node server 3002 web4 = node server 3003 redis = redis-server Mongroup ships with typical init-style commands `start`, `stop`, `restart`, `status` and so on, without a lot of the boilerplate associated with writing init scripts. ### Launching processes To check the status of your monitoring group, simply invoke `mongroup status`, or `mongroup`, as it's the default sub-command. You'll see a list of process names, pids, and the status itself:  Firing up the entire suite of processes takes only a single command, `mongroup start`:  Checking the status again will show the process uptime:  You may also start, stop, or restart single processes at a time via: $ mongroup stop [name] $ mongroup start [name] $ mongroup restart [name]  Since mongroup is working-directory sensitive unless the `--config ` flag is specified, I recommend adding a small snippet to your shell profile similar to the following: procs() { (cd ~ && mongroup $@) } If you're super paranoid you can monitor mon with mon: $ mon "mon 'node app'" That's all for now! __EDIT__: changes to mongroup(1) have been merged! https://github.com/jgallen23/mongroup

[tjholowaychuk-blog]

Express 3.0

Express 3.0 is here (finally) and while it is mostly a refinement release, between it and Connect 2.x there are some helpful new features. Sorry for the massive delay! Been busy and wanted to get a reasonable amount of documentation up on [expressjs.com](http://expressjs.com) before releasing, more docs will be coming soon. Keep in mind that the goal of Express is to aid you with HTTP, not to become a framework super-power like Rails, so this update may be underwhelming to some since it's merely a refinement. ## Connect 2.x Changes introduced by Connect 2.x: - added `err.status` support to Connect's default end-point - added `session()` "proxy" setting to trust "X-Forwarded-Proto" - added `cookieSession()` middleware - added `compress()` middleware for gzipped responses - added `multipart()` middleware - added `json()` middleware - added `urlencoded()` middleware - added `limit` option to the three above middleware - added `defer` option to `multipart()` to listen on formidable's events - added `debug()` instrumentation to aid in debugging - changed `basicAuth()`'s `req.remoteUser` to `req.user` - changed `session()` to only set-cookie on modification (hashed session json) - changed `bodyParser()` to be an aggregate of `json()`, `multipart()` and `urlencoded()` - moved many cookie-related utils into npm - moved `static()`'s logic into a separate npm module named "send" - increase perf ~%20 by memoizing url parsing - removed `router()` middleware - fixed default encoding for `logger()`, now "utf8" instead of "ascii" - fixed mount-path case-sensitivity Connect docs are available at [http://www.senchalabs.org/connect/](http://www.senchalabs.org/connect/), and will eventually be mirrored on expressjs.com as well for convenience, along with usage guides. ## Express 3.x Changes introduced by Express 3.x: - added several new examples in `./examples` - added unit testing for the examples (most of them at least) - added `res.jsonp()` to explicitly opt-in to JSONP support - added ETags and conditional-GET handling to `res.send()` responses - added "jsonp callback name" setting - added support for status code as first or second arg to `res.send()` and `res.redirect()` - added `req.range(size)` to parse `Range` header fields - added `req.auth` for basic auth - added `res.links(obj)` to set response the `Link` header field for pagination - added `res.format(obj)` for content-negotiation - added `req.fresh` for conditional-GETs - added `req.stale` for conditional-GETs - added mount-point relative redirection support to `res.redirect()` - added `req.ip` for the remote address (supporting reverse proxies) - added `req.ips` for remote address(es) (supporting reverse proxies) - added `[]` support in jsonp callback - added `app.get(name)` to compliment `app.set(name, val)` - added `app.engine()` to register template engines (replaces `app.register()`) - added `req.subdomains` to return an array of subdomains - added `req.protocol` to return the request protocol string ("http" or "https") - added `req.secure` to assert that `req.protocol` is "https" - added `req.path` to return the parsed url's pathname - added `req.host` to return hostname (`Host` void of port) - added `debug()` instrumentation to aid debugging - added `req.accepts()` - added `req.acceptsLanguage()` - added `req.acceptsCharset()` - added `req.accepted` - added `req.acceptedLanguages` - added `req.acceptedCharsets` - added "json replacer" setting to manipulate json responses (remove private keys etc) - added "json spaces" setting to compress or expand json as you like (defaults to `2` in dev) - added `express.application` prototype - added `express.request` prototype - added `express.response` prototype - added `app.render()` for app-level templates - added `res.type()` to replace old `res.contentType()` - added `{ signed: true }` option to `res.cookie()` - added async signature to `res.render()`, engines in [consolidate.js](github.com/visionmedia/consolidate.js) work OOTB - removed `partial()` - removed express-level layout support (engines provide similar) - renamed "case sensitive routes" to "case sensitive routing" - removed `res.signedCookie()` - removed "root" setting - removed `res.redirect('home')` support - removed `req.notify()` - removed `app.register()` - removed `app.redirect()` - removed `app.is()` - removed `app.helpers()` - removed `app.dynamicHelpers()` Head over to the [New Features in 3.x](https://github.com/visionmedia/express/wiki/New-features-in-3.x) wiki page for a more comprehensive list of additions, or to [3.x migration](https://github.com/visionmedia/express/wiki/Migrating-from-2.x-to-3.x) to help you upgrade if you wish to do so.

[tjholowaychuk-blog]

"path" - 45 minute digial painting with pixelmator

[tjholowaychuk-blog]

more game WIP

[tjholowaychuk-blog]

Mocha 1.4.0

This is another relatively small release but still has some goodies! ## .only() By appending `.only()` Mocha will generate an internal `.grep()` call for you. This is very useful if you have a regression and a test-case fails. Instead of moving the test-case and associated code out in order to debug you can simply invoke `it.only(title, callback)` (for the BDD UI) and Mocha will ignore all others. Here's an example that will execute "bar" and its test-cases only: describe('foo', function(){ it('should foo', function(){ }) }) describe.only('bar', function(){ it('should bar', function(){ }) }) ## .skip() By appending `.skip()` you can tell Mocha to ignore these tests. This was previously called `xdescribe()`, `xit()` etc to match other frameworks (Jasmine I believe?) but we decided to go with `.skip()` which is more readable. The jasmine-style functions will remain for now but they will be removed in the future. This is effectively the same as commenting out cases, however they remain in a pending state so that you do not forget to re-enable them. describe('foo', function(){ it.skip('should foo', function(){ }) }) describe('bar', function(){ it('should bar', function(){ }) }) ## Filtering HTML passes and failures The "passes" and "failures" labels are now clickable links that will filter respectively.  ## Retina The canvas progress reporter is now retina-enabled via the [autoscale-canvas](https://github.com/component/autoscale-canvas) component. No more ugly pixelated progress! ## Changelog Here's the full changelog between 1.3 and 1.4: * add mkdir -p to `mocha init`. Closes #539 * add `.only()`. Closes #524 * add `.skip()`. Closes #524 * add passes/failures toggling to HTML reporter * add pending state to `xit()` and `xdescribe()` [Brian Moore] * add the @charset "UTF-8"; to fix #522 with FireFox. [Jonathan Creamer] * add border-bottom to #stats links * add check for runnable in `Runner#uncaught()`. Closes #494 * add 0.4 and 0.6 back to travis.yml * add `-E, --growl-errors` to growl on failures only * add prefixes to debug() names. Closes #497 * add `Mocha#invert()` to js api * change str.trim() to use utils.trim(). Closes #533 * change dot reporter to use sexy unicode dots * fix HTML progress indicator retina display * fix url-encoding of click-to-grep HTML functionality * fix exports double-execution regression. Closes #531 * fix error when clicking pending test in HTML reporter * fix `make tm`

[tjholowaychuk-blog]

A ~2.5 hour work in progress digital painting

[tjholowaychuk-blog]

"Edge" - a ~3 hour digital painting

[tjholowaychuk-blog]

Components

With the advent of numerous client-side JavaScript package managers, I wanted to write up some of my thoughts about the fragmentation that we have today, and some ways that I think we could really improve delivering packages a community. Keep in mind that these are only my opinions, everyone has done a great job and there's a lot of cool work going on out there.

I think there are a few problems with the current environment, though some are subjective. The first of these and the most important in my opinion is the lack of unification around package consumption and definitions. The second issue is package delivery, what's going to transport them to your machine? The final issue is the use of the components, how are they exposed to the browser for your application.

Once I'm done rambling I'll show you how we've been using components to build our application at LearnBoost, and how they really fare. Before getting into all that let's dig into what I mean by "component".

Building components

There have been many attempts to come up with some sort of client-side package manager, and some of them do it very well, however I think they are missing the big picture, a "component" is much more than just JavaScript.

Create components, not JavaScript packages

A component can be not just JavaScript, but CSS, images, fonts or other resources, or a component may be any combination of these. This is the main idea that I want to sell, we need to broaden modularity beyond just JavaScript source. This is not a new concept at all, Drupal has been employing it for over 7 years but it seems like something that hasn't really caught on in most communities, at least not beyond the private application level.

A great example of a component would be a popover from the Twitter Bootstrap library (not picking on you guys, it's just a good example):

This thing looks so sexy that I just want to install it with my package manager, and have it automatically available to me in the client. For this to happen we need to admit that a component is much more than JavaScript, and should be packaged accordingly. Like the "dialog" in UIKit for example:

UIKit even at this state is not a good example of a solution for this problem, it uses its own custom build script to produce a single ui.js and ui.css file from the components in the repo, so it's still not something the community can consume directly without the custom build step. I've started moving UIKit components over to github.com/component.

Another problem in the community is the size and scope of projects.

Size and scope

The classic battle between DOM manipulation libraries such as jQuery and MooTools serve as an obvious example of fragmentation. Even if one is more popular than the other this doesn't mean we don't have a problem. Have you ever seen a great jQuery plugin and thought to yourself "damn! I'm using MooTools!" or perhaps the other way around? That highlights the problem right there, we should have no "jQuery plugins", no "Dojo modules", just simply "components" that we can all consume.

Components could then utilize these smaller, more modular dependencies to perform tasks. Instead of requiring jQuery as a dependency to convert a string of HTML into elements, one could simply add domify as a dependency and invoke domify(html), similar components could facilitate event handling etcetera. My point here is that ubiquitous libraries like jQuery will eventually be a thing of the past and fragmentation will hopefully decrease.

Another thing I think we really need to avoid, is the use of pre-processed assets within components, this includes things like Stylus, LESS, Sass, Jade, CoffeeScript among others. These are all great and can increase productivity in your application, my opinion is that they do not belong in public components, they fragment the community and reduce contributions, chances are if you're depending on these tools for UI packages your component's scope is too large.

A package such as "UIKit" could then simply aggregate all its parts into one convenient component for those who wish to consume the entire thing, but there's no reason these need to live together, we're just lacking the tools to make this convenient.

This leads to the next issue, modernization!

Modernization

There are a lot great tools being produced to deal with modernization, within reason we should not have to deal with these kinds of issues at the component level. Stylesheets are particularly bad for this, libraries are either faced with using CSS preprocessors to provide vendor prefixed versions of their styles, or of course need to manually declare them. This is a huge pain, and forever changing. What do we do? Nothing!

I strongly suggest that we write our public component stylesheets using regular old CSS. Other tools can still be utilized at the application level if you really need them. With explicit vendor prefixing out of the way we can enjoy building components since we're not focused on vendor details (when possible), they will remain light-weight, and customizable via consumer manipulation via a library such as CSSOM or node-css.

On to the next topic! Structural styling.

Structural styling

Structural styling is another thing I frequently have issues with. JavaScript libraries, jQuery plugins and others often ship with stylesheets, this is great but these libs should only define structural styles.

What do I mean by "structural styling"? Take for example the Dialog from UIKit, it contains only styles necessary to make the component presentable, but does not force a ton of extra styling on you, and serves as a good base to build on. I even took this a little far, box-shadows etc should be omitted by default.

The basic idea here is let stylesheets be stylesheets, application should define the look and feel of things, if you must provide "themed" versions simply have a component named "dialog-dark-theme" with only a stylesheet in it, and so on.

package.json

Components could have a "component.json" much like the commonjs package.json, or we can simply extend package.json. This would of course act as the package manifest, letting the world know if it has stylesheets, templates, scripts, images and so on. I believe we should avoid the minimal gains of magical auto-globbing of files, we can just simply list these and avoid unnecessary complexity and I/O.

A Dialog component would simply look like the following, nothing fancy here, just an explicit manifest.

{ "name": "dialog", "version": "1.0.0", "scripts": ["index.js"], "styles": ["dialog.css"], "templates": ["dialog.html"] }

The reason I would name these styles, and templates instead of css and html would be that when using private components within your application, the build tool could simply sniff out things like ["login.jade"], realize it's a Jade template and compile it appropriately. Like I've mentioned though I dont think these tools belong in public code.

It may also be useful in the future to define optional dependencies for legacy browser support. Suppose I dont care anything below IE 9, I should be able to tell the build system that I'm fine with omitting legacy functionality, and the packages that handle this sort of things for events, the DOM etcetera would simply not load those in, the APIs that those modules provide would remain identical.

Require fragmentation

Another huge issue in the community right now is fragmentation regarding javascript loader definitions, the most common probably being AMD. Personally I'm not a fan of AMD for one reason, it's ugly.

Granted that's not a very good reason to dislike something. AMD provides the nice benefit of working without a build step, however when you need a library to test out a script anyway I feel like a quick build step to convert the more convenient commonjs require() style wins here, at least for me.

The real problem today is that if you want to share your public client-side code, you end up with something like the following:

// AMD support if (typeof define === 'function' && define.amd) { define(function () { return Cookies; }); // CommonJS and Node.js module support. } else if (typeof exports !== 'undefined') { // Support Node.js specific `module.exports` (which can be a function) if (typeof module != 'undefined' && module.exports) { exports = module.exports = Cookies; } // But always support CommonJS module 1.1.1 spec (`exports` cannot be a function) exports.Cookies = Cookies; } else { window.Cookies = Cookies; }

This sucks. I hope to never look at this, nor support this sort of concept ever again! This is really painful. I completely understand that perhaps AMD nor sync-style requires are perfect, but if we pick one we can easily translate scripts to an async style for production builds if necessary.

Delivering components

We'll obviously need a way to store and deliver components, existing tools are close but I think we can still do a little better and think a little larger.

Package managers

Tools like node's npm(1) package manager are again very great, but there are several reasons I think these packages should not live in the npm registry.

The first reason being that not everyone uses node, only us as node developers can be convenienced by using it, other communities should be able to easily access and consume these components with whichever tools they prefer, or if they really want to use one written in another language then sure why not!

Client-side packages in the npm registry are ambiguous, without tagging them as such it's unclear which are intended for which environment. Many packages also end up disambiguating with a suffix, for example debug works both on the client and in node, however without augmenting package.json one would need to publish "debug" and "debug-component" or similar to the registry.

Another reason is that the package manager simply should not matter, and it usually does not matter. You just want some packages on your local machine, these should absolutely be as decoupled as possible.

Component registries

Arguably the best registry would simply be Github, or git repositories in general. There are a few downsides to this of course:

- even shallow clones can be slow - coupled with git both at the consumer and producer levels - large dependency urls (unless we default github to "username/project" etc...)

Some nice benefits as well:

- clear & explicit dependency origins - github is awesome

Utilizing components

Most "real" applications require build steps these days, however even with this being the case, build scripts are extremely implementation specific and may vary greatly for your private application(s) depending on the scope of the project.

Here I would propose a free-for-all, use whatever you like from whichever community you're a part of. As long as the "package manager" can transfer these to your machine, you do whatever you need to integrate them into your application, the details here are largely irrelevant.

Components in practice

Traditionally most applications (that I've experienced) are built in more of a "vertical" approach, spreading a single concept into multiple sections of an application. Making the application, and the components of a specific feature very difficult to reason with. This is akin to how unix libraries splatter themselves all over your system into various directories and files.

At LearnBoost we've been using components for a while now, but like I've mentioned not only for abstract UI components, but for everything in our application, even the build system and application bootstrap are implemented as components.

What does look like? Nothing more than a simple list of directories as shown in the following screenshot. Each component here is comprised of any combination of server-side logic, styles, scripts, images, fonts and so on.

Beyond obvious organizational benefits this helps our team focus on specific tasks without stepping all over each other. Testing is also easier, as each component providing server functionality simply exports a fully self-contained express application, which has its own set of tests, then our root make test iterates those.

Our build system simply produces the output scripts and styles that we need into our root ./public directory, wrapping scripts with commonjs require()s, images, fonts and other assets are served from the self-contained servers.

My point here is that this concept goes beyond public components. Obviously it would be next to impossible, and ill-advised to start sharing components with server portions, but for your own application sticking to whatever language / platform you're using it works very well.

What next?!

Keep building! The JavaScript community has been moving in great directions lately, take what I've said with a grain of salt but I hope to see some attempts at "component" management, not just JavaScript ;)

Until then you may want to keep your eyes on https://github.com/component for future work from us at LearnBoost in regards to this topic. EDIT: I've started a component wiki where we can toss around ideas.

EDIT: Work on the component(1) executable have begun, as well as a JavaScript implementation of a builder for components, aptly named component/builder.js. For now component-build(1) is shipped with component(1) to ease use, otherwise the tools are not coupled. For example you could use `component(1) to perform installation and then use builder.js directly in code and customize your build, use the command, or use an entirely custom solution if you wish. I've started documenting some frequently asked questions, and we're starting a Wiki page listing some of the components available so far, with well over 70 within the last month or so we're off to a good start!