l1x

nothing interesting happens

[lazyengineerchills]

Analysis: strace was used to show that mongoperf was executing read/write/fsync in a loop, resulting in synchronous disk I/O.

Brendan's blog » Another 10 Performance Wins Why am I not surprised? Mongo pls. :) Brendan Gregg is one of the best performance experts ever existed on this planet.

[lazyengineerchills]

A misplaced where clause

The other day I was investigating how could our data scientist (a statistician who lives in SF) take down the Hadoop cluster again. Well Hadoop is a fragile thing, I guess this was a design item of the original Google white paper. I found the query and at the first look it was correct but somehow it took down the cluster so we (ops team) had a closer look.

The original query:

SELECT *

FROM big_table big

JOIN small_table small

ON trim(big.userip) = trim(small.ip_address)

WHERE eventdate = '2013-09-01'

LIMIT 10;

Everything looks right, and than boom the cluster falls on its knees. Hive does not offer a great way to understand why this is happening but running and EXPLAIN DEPENDENCY finally showed the root cause.

I have truncated the JSON:

{ "input_partitions": [ { "partitionName": "default@big_table@eventdate=2010-01-25" }, { "partitionName": "default@big_table@eventdate=2010-06-08" }, { "partitionName": "default@big_table@eventdate=2010-06-09" }, { "partitionName": "default@big_table@eventdate=2010-06-10" }, { "partitionName": "default@views@eventdate=2010-06-11" } ] }

It seems the where clause does not get applied. After running circles and eventually asking the stackoverflow community we re-wrote the query:

SELECT *

FROM

  (SELECT *

  FROM big_table

  WHERE eventdate = '2013-09-01') big

JOIN small_table small 

ON trim(big.userip) = trim(small.ip_address)

LIMIT 5 ;

This runs fine, reading only one partition. Happy days!

[lazyengineerchills]

Erlang gets maps. What is next??

[monopole]

[lazyengineerchills]

http://offworld.gizmodo.com/spaces-on-earth-where-no-one-can-hear-you-scream-472629162

[lazyengineerchills]

Software design techniques used to write the software for a high security functional equivalent of the commercial open source Apache server in 300 lines of state machine code.

You say it can't be done?!

[lazyengineerchills]

good writeup

[tarters3]

「 -by -Yuumei | -編集- 」

[lazyengineerchills]

Node.JS meets reality

I was never a huge fan of node.js and the surrounding hype but this post actually eliminates all the remaining interest to have a closer look. 

http://www.quora.com/Node-js/Why-did-Koding-switch-from-Node-js-to-Go

On the other hand, I like the raising interest in Go. It is a very well designed language and environment. 

[iamafuckingsysadmin]

Two words. Package versions.

Lately at my office, we’ve been iterating to a new version of our internal web hosting infrastructure, moving from, amongst other things, Ubuntu 10.04 LTS to Ubuntu 12.04 LTS. We use Ubuntu because we prefer Debian-style Linux, but with a commercial backing.

...

[lazyengineerchills]

this might be one of the reasons why we don't use Ubuntu at all

[unflujodefotones]

[lazyengineerchills]

Very nice writeup by Joe. 

[lazyengineerchills]

Chicago Boss

This framework is getting ready for prime times. 

http://www.chicagoboss.org/about.htm

Very well designed, amazing solutions and very comprehensive feature set. I especially like how they reason about everything they use.

https://github.com/ChicagoBoss/ChicagoBoss#philosophy-and-features

Joe Armstrong did a good job on the Erlang concurrency model and that is extremely well suited for high traffic websites with long polling clients. This environment scales better with the number of clients than Ruby (Rails more specifically). More on this topic is here:

http://www.youtube.com/watch?feature=player_embedded&v=LGGo6bIuj8w#t=790 

The only drawback is that the Erlang syntax is peculiar and not easy to deal with, but it is worth the effort to learn it. For those who are coming from a Lisp like environment there is Elixir, that gets slowly supported in Chicago Boss.

[poedel-blog]

[lazyengineerchills]

Rendering Markdown to HTML

Sometimes it is worth to see your README.md rendered to HTML before your push it to the repo, or simply you want to store the HTML version with the markdown one in the same repo. I bumped into this Python tool that does the job.

https://github.com/joeyespo/grip

Comes handy.

[lazyengineerchills]

Why

I am have no doubt that most of the stuff coming from Pentaho is a joke, but today they could surprise me. Who on this earth would write a software that couples database to an application like this? What sort of moron you have to be for that? What reasoning could you provide to defend this decision? I would be ashamed to offer such a product even for free to anybody.

Installing the Pentaho Business Analytics Enterprise Edition.

Installation in Progress

0% ______________ 50% ______________ 100%

##############

Error: There has been an error.

bash: /home/pentaho/pentaho/servers/postgresql/bin/initdb: Permission denied

Press [Enter] to continue :

[kav-k]

[lost-philosopher]

McIntosh C2300, Ayre 5xce, Krell 350MCX, B&W 803d by ZombieFish92 on Flickr.