r0b0tz @r0b0tz2013 - Tumblr Blog

Using wkhtmltopdf and an Xvfb daemon to render HTML to PDF

I have a lot of projects that consist of collecting information and then rendering it to PDF, initially I used tcpdf, but the rendering is VERY finicky. I was doing some additional research a few weeks ago for a personal project and found wkhtmltopdf. I was scraping my Harvest time sheets to gather my hours for the previous two weeks, rendering to a PDF, and automatically creating a RightSignature document to be signed by my employers. If you've ever had to do this, it can be somewhat of a pain to do repeatedly, especially when you're really busy. Plus, automating things is awesome. For that process, I used some python (with Requests), wkhtmltopdf, and Xvfb's (X virtual framebuffer) xvfb-run command. This is fine, because I run the process manually, at will. I have some other projects that require repeated rendering on a live site, for multiple users. Running individual xvfb-run processes for each request is just a dumb idea. Your server is going to explode if you do this. Don't do it. In order to use a single Xvfb process, I needed to set it up as a daemon that is run by the www-data user, so that the PHP wrapper I am using (PHPWkHtmlToPDF) can access the virtual frame buffer, and use the webkit rendering engine (Chrome) to save it to PDF. I could (and probably will) setup a processing queue using rabbitmq or the likes, but for now (and for this post), this is fine. Let's get started, shall we? These instructions are centered around Ubuntu, so your milage may vary depending on the setup you are using.

Step 1: Install wkhtmltopdf and xvfb

[bash] # apt-get install wkhtmltopdf xvfb [/bash]

Step 2: Create and enable the init script for the xvfb daemon

Note ":0" in XVFBARGS which sets to display 0, and the --set-guid parameter, which runs the daemon as the www-data user. If you are not running a headless server, make sure to change all references to ":0" to ":#" (ie: :1, :2 or :22) to avoid conflicts with your existing X session(s) Put the following in "/etc/init.d/xvfb": [bash] XVFB=/usr/bin/Xvfb XVFBARGS=":0 -screen 0 1024x768x24 -ac +extension GLX +render -noreset" PIDFILE=/var/run/xvfb.pid case "$1" in start) echo -n "Starting virtual X frame buffer: Xvfb" start-stop-daemon --chuid www-data --start --quiet --pidfile $PIDFILE --make-pidfile --background --exec $XVFB -- $XVFBARGS echo "." ;; stop) echo -n "Stopping virtual X frame buffer: Xvfb" start-stop-daemon --chuid www-data --stop --quiet --pidfile $PIDFILE echo "." ;; restart) $0 stop $0 start ;; *) echo "Usage: /etc/init.d/xvfb {start|stop|restart}" exit 1 esac exit 0 [/bash] Enable your init script: [bash] # update-rc.d xvfb defaults 10 [/bash] Run your init script: [bash] # /etc/init.d/xvfb start Starting virtual X frame buffer: Xvfb. [/bash] Check to confirm your Xvfb is running: [bash] # ps auxU www-data | grep [X]vfb www-data 17852 0.0 0.2 54960 7904 ? S 04:31 0:00 /usr/bin/Xvfb ... [/bash] Great! The daemon is up and running (hopefully).

Step 3: Modify the PHPWkHtmlToPdf Wrapper

We need to modify the wrapper's call to wkhtmltopdf so it knows we are using a specific X session (see the notes about ":0" above) Open up the WkHtmlToPdf.php class/file and find the "getCommand" function (around line #252 at time of writing this post) and add a line to the beginning of the command variable that exports the DISPLAY variable to match the display number you used. In my case it's ":0", since I am running a headless server. [php] /** * @param string $filename the filename of the output file * @return string the wkhtmltopdf command string */ public function getCommand($filename) { $command = 'export DISPLAY=":0";'; $command .= $this->enableEscaping ? escapeshellarg($this->bin) : $this->bin; $command .= $this->renderOptions($this->options); foreach ($this->objects as $object) { $command .= ' ' . $object['input']; unset($object['input']); $command .= $this->renderOptions($object); } return $command . ' ' . $filename; } [/php] Save, and you're ready to....

Step 4: RENDER!

I use twig to fill up some html templates, but for the sake of an example, we'll use PHP to grab the Google homepage and render it to a PDF. [php] include "classes/WkHtmlToPdf.php"; // instantiate the wrapper $pdf = new WkHtmlToPdf; // add the content, addPage can take HTML strings, URLs, or filenames. $pdf->addPage("http://google.com"); // for this example, we'll just use send, which sends the PDF directly to the browser // if (!$pdf->saveAs('/tmp/google.pdf')): // this saves to a file if (!$pdf->send()): throw new Exception('Could not create PDF: ' . $pdf->getError()); endif; [/php] Save this, run it, and you should be presented with well rendered PDF of the Google homepage. There are a few options to get the render a bit more accurate, but you can see those at http://mikehaertl.github.io/phpwkhtmltopdf/. Enjoy and happy rendering!

Using Trackpads to Sign a Digital Document

If you are trying to sign a digital document (a la DocuSign, RightSignature, etc) and are using a MacBook. I've found a method to use that will make sure your signature actually represents your signature, and not a doodle of a three year old. It's really easy.

With your left thumb (or index finger), press and hold on the lower left-hand corner of your trackpad.

While holding, use your right index finger to draw your signature.

Tada!

Normal method:

This method:

Zinnia (django blog) fix for "Continue reading" persistance

The template for zinnia in the version installed by pip always sets continue_reading to "1", so the "Continue reading..." link always pops up on every article. Even articles with less than 100 words. To remedy this, you need modify the content-loop block in entry_search.html and entry_list.html like so: [python] {% if object.html_content|truncatewords_html:100|length == object.html_content|length %} {% include object.content_template with object_content=object.html_content|safe continue_reading=0 %} {% else %} {% include object.content_template with object_content=object.html_content|truncatewords_html:100|safe continue_reading=1 %} {% endif %} [/python] Obviously, you will want to copy over the HTML files from the zinnia package into your project for modification before doing the above :)

Verizon and the HTC One: A match made in heaven

I've been keeping up with the latest updates on Verizon getting the new HTC One, and found a rather puzzling comment from an article on AllThingsD: "Meanwhile, the number people who care about this phone is falling through the floor." You must mean "falling through the floor" because the floor is not capable of supporting that many people. This phone is the best out there right now, you combine that with Verizon's killer LTE network and you have a deadly combo. A combo that is so deadly, I have a creeping feeling that there may be some sort of deal in place by big-red's competitors to keep things hushed and drive more carrier switches to providers that are getting the device right away. More money in the bank initially for HTC, more new contracts for other providers, and then even more money for HTC once Verizon gets it... what do you think? Also, remember the Droid DNA? Me neither. Either way, I want the HTC One and I want it on Verizon. Take my money, please, take my money.

Dropbox Linux CLI snippet -- Exclude all root folders

I was trying to exclude all the root folders on my Dropbox sync to my development machine and found there was only a way to do it one by one: ./dropbox.py exclude add ~/Dropbox/abc ~/Dropbox/etc So, I decided to use some bash / awk magic and make this little guy, which gets all the root directories in ~/Dropbox, escapes spaces and parenthesis in directory names, prepends ~/Dropbox/, and then spits them all out appended to the end of the "exlude add" command: [bash] ./dropbox.py exclude add `ls Dropbox/ -1D \ | awk '{ { gsub(" ","\ ") }; \ {gsub("$","\\(")};\ {gsub("$","\\)")};\ print "~/Dropbox/" $_ }' \ | tr '\n' ' '` [/bash] Enjoy.

Symfony + Doctrine: Single mapping file for multiple entities (global mapping)

Doctrine lets you define database ORM mapping using YAML, XML or PHP Annotations. I wanted to put all my mapping information in a single file, in order to streamline updating for one or two of my own reasons. I found this snippet: [php] <?php $namespaces = array( '/path/to/files1' => 'MyProject\Entities', '/path/to/files2' => 'OtherProject\Entities' ); $driver = new \Doctrine\ORM\Mapping\Driver\SimplifiedYamlDriver($namespaces); $driver->setGlobalBasename('global'); // global.orm.yml [/php] Which is nice but we are not implementing the Doctrine driver ourselves when using Symfony 2.1, so I did a quick search for setGlobalBaseName in the Symfony Doctrine ODM bundle and found that it was setting the global basename to mapping and not global. So, if you wish to have a global YAML mapping definition file, you can create one like: /www/src/Acme/TestBundle/Resources/config/doctrine/mapping.orm.yml The mapping.orm.yml file should look something like this: [code] Acme\TestBundle\Entity\Employee: type: entity tableName: employees fields: ref: { type: char(6), primary: true, notnull: true } first_name: { type: string(50), notnull: true } last_name: { type: string(50), notnull: true } birthdate: { type: timestamp, notnull: true } hiredate: { type: timestamp, notnull: true } email: { type: string(100), notnull: true } oneToOne: company: targetEntity: Company joinColumn: name: company_id referencedColumnName: id Acme\TestBundle\Entity\Company: type: entity tableName: companies fields: ref: type: char(6) primary: true notnull: true name: { type: string(50), notnull: true } [/code] Then, we can generate the model classes (or entities) using: $ php app/console doctrine:generate:entities TestBundle This makes the Employee.php and Company.php in: /www/src/Acme/TestBundle/Entity/ So there you have it, setting up a global YAML mapping file, and generating entity classes from it! Hope this helps someone out there.

Wildcard host resolution for Windows/OSX

In a previous post, I detailed an Acrylic DNS setup for Windows in which you can have a regular expression match for host resolution. This is great for local testing, and vhosts (virtual hosts), but there was no simple way to replicate this setup on OSX. I've known about the Automatic Proxy Configuration ability for a while but never really had a use for it, until now! This new solution works in both Windows and OSX, and probably Linux -- for all modern browsers that take advantage of automatic proxy configuration scripts. For example, I have a folder, /Volumes/I/_WORK/, where I put my projects. I had setup Acrylic and my vhosts config so that I could access the project folders dynamically via my Apache server; *.stage adjusts the document root to the /Volumes/I/_WORK/*/www folder automatically, and I don't have to add a line in my hosts file every time. Examples: tests.stage => /Volumes/I/_WORK/tests/www/ www2.tests.stage => /Volumes/I/_WORK/tests/www2/ project1.random.stage => /Volumes/I/_WORK/random/project1/ We're going to set up resolution of *.stage to 127.0.0.1 with an Automatic Proxy Configuration script. Without any further delay, to match *.stage -- test.stage, websitename.stage etc -- you can use the following setup:

For OSX

1) Create ~/.proxy.pac... [javascript] function FindProxyForURL(url, host) { if (shExpMatch(host,"*.stage")) { return "PROXY 127.0.0.1"; } return "DIRECT"; } [/javascript] 2) Setup ~/.proxy.pac in your network preferences:

Open Network under System Preferences

Select your active network connection and click "Advanced..."

Open the "Proxies" tab

Check "Automatic Proxy Configuration" under "Select a protocol to configure"

For the Proxy Configuration file URL, enter the following -- replacing USERNAME with your actual username: file:///Users/USERNAME/.proxy.pac

Hit OK, and Apply.

For Windows

1) Create proxy.pac (same as above)... 2) Setup proxy.pac in your network preferences:

Open "Internet Options" under Control Panel

Click the "Connections" tab, and click "LAN Settings"

Check "Use automatic configuration script" under "Automatic Configuration"

For "Address" enter the path to your proxy.pac, ex: file:///C:/Dropbox/proxy.pac

Hit OK, and Apply.

If you have setup your regular expression virtual hosts for Apache or nginx already, you should now be able to access your local webserver using *.stage hostnames. If you need an example for nginx virtual hosts, here is my relative nginx config block: [javascript] server { listen 127.0.0.1:80; index index.html index.htm index.php; # regex magic to detect AAA.BBB.stage or AAA.stage hostnames, server_name ~^((?<subdomain>[^\.]+)\.(?<domain2>.+)\.stage|(?<domain>.+)\.stage)$; # my projects stay in I:/_WORK/AAA/ and have www folders or subfolders (I:/_WORK/AAA/BBB) set $new_root I:/_WORK/$domain/www/; if ($subdomain != '') { set $new_root I:/_WORK/$domain2/$subdomain/; } if (!-e $new_root$document_uri){return 404;} log_not_found off; charset utf-8; access_log logs/access.log main; location ~ /\. {deny all;} location ~ \.php$ { root $new_root; fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; include fastcgi_params; fastcgi_param SERVER_NAME $http_host; } location / { root $new_root; } } [/javascript] Now my browsers hit the proxy script, see *.stage as 127.0.0.1, and hit my local nginx setup which recognizes the *.stage vhosts and routes the request to the correct project directory. No more manually adding entries to the hosts file! (for website dev, anyways) Update 12/28/2012: Added fastcgi_param for SERVER_NAME Disclaimer: this is probably not a great configuration for production performance, so modify and test accordingly if you are going to use the nginx config w/ RegEx in a production environment.

Windows 8 tip: Fix for freeze-ups trying to access external drives

Believe it or not, try uninstalling all Flash player plugins. Also make sure that your permissions were migrated correctly. My install showed my account was an administrator, but the property got mismatched somehow and I couldn't access certain things. I had to go in to user accounts in the control panel, set my account to a normal user, then back to administrator. Now everything is working perfectly again.

FlashDevelop Script for CodeFormatter fixes

I've created a fix for single line if, while, and for statements, and adding a space between ){ for bracketed statement blocks. There's a Macro to run codeformatter, then execute the script, and relocate you back to the original cursor position. If you find a bug post the test case and I'll get around to it when I can :) Macro Setup -- CTRL+SHIFT+(~ or `) AutoRun: False Shortcut: Ctrl+Shift+Oemtilde Entries: InvokeMenuItem|RefactorMenu.CodeFormatter InvokeMenuItem|RefactorMenu.OrganizeImports ExecuteScript|Internal;C://pathto//fixifstatements.cs ** Make sure to change the path (C://pathto//fixifstatements.cs) to the correct location of the script! ** Script code is here: http://pastebin.com/Qt5W73mR Hope this eases some minds :D

Modern Warfare 3: PS3 Multiplayer Issues

I found a solution to my own personal issues with trying to connect to multiplayer games, I would always get the "Unable to connect to host" or "Waiting for players" messages and get dropped back to the menu. My solution was to disable blocking of anonymous internet requests on my router. The setting was under "Security", and "Firewall" tab. (For my Linksys) Hope that helps someone :)

Local staging configuration for using Apache 2.0 and DNS wildcards

As a web developer / designer, I do a lot of things that make my workflow easier, one of them is using a local staging server for sites I am working on in the initial build phases. The first step is modifying the httpd-vhosts.config to point to a local folder for a specific domain. This is better than using subfolders in a single public folder, because you can use absolute relative paths (ie: /css/, and in CSS files /images/logo.jpg) Example insert into the vhosts configuration file: [code] <VirtualHost *:80> ServerAdmin [email protected] DocumentRoot "Q:\_WORK\projectname\www" ServerName projectname.stage </VirtualHost> <Directory "Q:/_WORK/projectname/www"> AllowOverride All Order allow,deny Allow from all </Directory> [/code] The *:80 string tells Apache to listen on any IP address for the connection (ie: your internet IP address), you can change this to 127.0.0.1 if you don't want the website to be available outside of your computer (you would need a real internet based DNS entry for that, and access to the DNS Zone for the domain). The DocumentRoot directive should match the Directory path exactly, aside from the use of back/forwardslashes as used above. So now to make the projectname.stage address actually work, we need to add a DNS entry pointing it to our IP address (127.0.0.1)... there's a few ways to do this. 1) You can edit your hosts file (c:\windows\system32\drivers\etc\hosts in windows, /etc/hosts in most *nix based systems) to include a line: 127.0.0.1 projectname.stage or 2) Use a local DNS proxy server (with wildcards!!)... I use Acrylic, which allows wildcards, so I don't need to configure a new stage server name in my hosts every time. You can edit the Acrylic hosts file (mine is at C:\Program Files (x86)\Acrylic DNS Proxy\AcrylicHosts.txt), adding the following line and restart the proxy: 127.0.0.1 *.stage This will make it so any usage of the .stage TLD will point to your local machine. So all you need to do is have the virtual host configured and Bob's your uncle! Now whenever you hit projectname.stage in your browser it will point towards your loopback IP address. Hitting http://projectname.stage would connect to Apache, sending it the hostname in the requests, and Apache will render the files in the location specified in your vhost config! *UPDATE* As soon as I wrote this, I thought to myself, how AWESOME would it be to not even have to configure each virtual host??? With Apache 2 it's totally possible (can I get a hell yes?) As stated here you can use Regular Expressions to dynamically point to directories. You need to uncomment / add the following to httpd.conf: LoadModule vhost_alias_module modules/mod_vhost_alias.so And then add the following to your vhosts config: [code] <VirtualHost *:80> UseCanonicalName Off ServerAdmin [email protected] VirtualDocumentRoot "Q:\_WORK\%1.0\www" ServerAlias *.stage ServerName *.stage LogLevel Debug </VirtualHost> <Directory "Q:/_WORK/"> Options -Indexes FollowSymLinks AllowOverride AuthConfig FileInfo Order allow,deny Allow from all </Directory> [/code] This, along with your wilcard DNS setup, will point *.stage to the appropriate folder; blah.stage => q:\_WORK\blah\www abc1234.stage => q:\_WORK\abc1234\www Without ever needing to touch your config files again.... YEAAAAAAAAAAAAA! Sorry. got super excited.

gucci louie fendi prada - kreayshawn + remix

[yframe url='http://www.youtube.com/watch?v=6WJFjXtHcy4'] dang ya'll. and then...

gucci gucci - kreayshawn (lawrencealan totally unauthorized remix) from la on Vimeo.

Windows 7: Audio / Video playback issue solution

Are you experiencing what seems like thrashing or pauses / breaks in audio or video playback? Audio pausing for a second and then resuming somewhat randomly? I may have the solution. I run quite a bit of new technology on my computer, so I have strange applications and drivers installed, and even strange IP routing configuration and a hosts file that helps me when developing websites. I'm what most would call a "power user" Not too long ago I started experiencing audio issues with both my on-board and external Native Instruments audio2dj. The fact that it occurred when using either meant it was either software or hardware related and not specific to the audio output. First I reset the BIOS, changed the HPET, and set the USB 3.0 transport to xHCI. This seemed to help a bit, but I still had some issues. I loaded up dpclat.exe and sure enough, I was getting large spikes every few seconds. I did some additional searching and found that there was an issue with the BIOS version I was running on my GIGABYTE GA-X58A-UD3R. Using @BIOS, I updated to the "latest" BIOS version from the USA server (which seems to have gotten much much more reliable by the way) This helped some, but I was still getting issues. I closed out the Gigabyte EasyTune application and my spikes all but disappeared. I was still getting spikes, but now it was less than two or three an hour... I tried RATT3, which doesn't seem to report anything in Windows 7 64bit (my reports were all empty)... and then, somehow, I found this little golden nugget: Resplendence Software's LatencyMon LatencyMon allows you to see max execution time and DPC count per driver. [gallery link="file" columns="2"] Using all of the above tools, I was able to eliminate / update drivers (see: nVidia, Himachi) and services that were causing DPC latency spikes. I can finally listen to music and watch flash videos without annoying pauses! Hope this helps someone out there.

Tonight's full "Supermoon" is the closest it will be until 2016

This evening the moon is going to be the closest it's been to Earth since 1992, giving a great view for photographers and astronomers. Another Supermoon will not occur until 2016... you know, if we are still around. Be safe tonight, who knows what will happen! Carpe noctum! http://en.wikipedia.org/wiki/Supermoon

DisplayDuino: LEDMatrix components

This is just a note for the people using the DisplayDuino LEDMatrix board, and looking for connectors. I was able to locate, order and use successfully the a 2.54mm 4-pin polarized connectors found here: http://www.protostack.com/connectors/polarized-headers/polarized-header-4-pin-female For the wiring I am using lightweight and thin Dollhouse Miniature Electrical wire. Just thought I'd post this to save people the trouble of narrowing the connectors down!

BAD DECISIONS @ CAMEO

Make better BAD DECISIONS next weekend in Brooklyn, with a soundtrack by some of the best: J PHLIP (Dirtybird, Liason) BERLIN ATOMLY (Wolf+Lamb, Atomiq Records) MEANS and ENDS (Detek, Black Nation Records) JD HARRINGTON (fat camp, den of thieves) CAMEO 96 N Sixth Street Brooklyn, NY 11211 $5 before 12:30am with RSVP RSVP for the event HERE

Android application PowerAMP is getting an API!

[caption id="" align="alignright" width="164" caption="PowerAMP QR"][/caption] Exciting. Android's most popular third-party audio player is getting it's own API. It is scheduled for the release after build-328. There will be earlier pre-release (non-market install) version of PowerAMP available with the API. APIs will be completely intent based (no service), events, notifications, etc. All documentation and details about the API will be published on the forum at powerampapp.com/forum/. Get PowerAMP:

Trending Blogs

Recently Viewed Blogs

r0b0tz