The Facebook Scraper, Debugger and why pages don't share correctly
When you post a link to Facebook, Facebook makes a call to the URL you post and pulls details such as the title of the page and the image for the post.
This Facebook crawler can be fickle and not pull images or title text correctly from your website. Facebook provide the Facebook Debugger (previously Facebook Linter), but it will often return "Error Parsing URL Error parsing input URL, no data was scraped.", an exceptionally useful error message which doesn't even make grammatical sense and gives no clue as to how to fix the error. They also allow you to see exactly what the scraper sees for your URL by visiting http://developers.facebook.com/tools/debug/og/echo?q=https%3A%2F%2Fmydomain.com%2Fmypage.htm, but this will return "Document returned no data" if there is a problem. Another useful message.
How's it work?
When the Facebook debugger attempts to access your page, your Apache access.log record will look like this:
66.220.153.248 - - [11/Apr/2012:23:06:53 +0000] "GET /path/to/page.htm HTTP/1.1" 206 1893 "-" "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)"
And the HTTP headers passed are:
GET /path/to/page.htm HTTP/1.1 host mydomain.com Accept */* Accept-Encoding deflate, gzip Range bytes=0-40960 User-Agent facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
An unusual point is that it requests only the first 40KB of the page (Range bytes=0-40960), so Apache will respond with a 206 HTTP status code (Partial Content), rather than a 200 (OK). This isn't an error, and the Facebook parser will expect and handle it with no problems. If your page is more than 40KB then you may hit problems. My page was under 40KB and I haven't tested, but I'd guess if your Open-Graph, Title, Meta or Image tags are after those 40KB they will be ignored. As an aside, 40KB is rather large for a HTML page, so you might want to look into shrinking that to speed up page loading.
Invalid SSL Certificates
An invalid SSL certificate may stop Facebook crawling your URL. It is possible for the SSL certificate to be found to be trusted by your browser, but for Facebook to fail to authenicate it. In my case, this happened because Apache wasn't sending the intermediate certificate. A fresh install of Firefox or wget also failed to authenticate this. Wget gave this error:
tom@ubuntu:/mnt/hgfs/workspace/SCRATCH$ wget https://mydomain.com --2012-04-11 19:52:54-- https://mydomain.com/ Resolving mydomain.com... 123.123.123.123 Connecting to mydomain.com|123.123.123.123|:443... connected. ERROR: cannot verify mydomain.com's certificate, issued by `/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=PositiveSSL CA 2': Unable to locally verify the issuer's authority. To connect to mydomain.com insecurely, use `--no-check-certificate'.
Firefox gave me an error "sec_error_unknown_issuer". What's happening here is that Firefox and Wget are verifying that someone (in this case COMODO CA Limited from Salford, Greater Manchester) has certified that I own mydomain.com, but it doesn't know who COMODO is, so doesn't know if it should trust them or not.
To fix this, your server needs to add an additional certificate, certifying that COMODO is a trusted key signer, and that they can be trusted to certify that I own mydomain.com. This additional certificate is called an "Intermediate Certificate" and contains the trust links from an organisation that the browser knows is trusted (such as Verisign) to COMODO. In my case the intermediate certificate was provided in the email with the original issuing of my mydomain.com certificate, and availiable from the Dreamhost "Secure Certificates" section (I ordered the secure certificate through Dreamhost).
On Apache, find your intermediate certificate and add the following line to your httpd.conf or virtualhost file:
SSLCertificateChainFile /path/to/comodo-intermediate-ca.crt
You can then retry with your fresh install of firefox or wget which should now work without any warnings.
Even an image on a site with an invalid SSL certificate can stop the page from loading in the Facebook Debugger. I created a page with only a <img> tag pointing at an image on the same domain, and the image tag was what caused it to fail. Once the chain certificate was added to the domain, the page loaded in the debugged with no problems.
Canonical HTML tag
The second issue that I encountered with Facebook's scraper was that Wordpress (or Wordpress' framework Genesis) was adding a "canonical" HTML tag to the page for SEO reasons:
<link rel="canonical" href="https://mydomain.com/mypage" />
This meant that although I was trying to share https://mydomain.com/mypage?userid=12345, the Facebook crawler was hitting that page, reading the "link rel=canonical" tag, and then attempting to share the "canonical" page. This resulted in a second call by the crawler to https://mydomain.com/mypage and it pulled the title and image from that page, rather than the page which was meant to be shared.
I'm guessing that the reason Facebook does this is so that people sharing several similar links (like http://mydomain.com/somearticle?referrer=twitter, http://mydomain.com/somearticle?referrer=facebook and http://mydomain.com/somearticle?cachebreaker=1334186904 ) can all be grouped together and shown as a single story in the news feed (http://mydomain.com/somearticle).
The fix is to remove that tag from your page by editing the page templates to add the following lines. If you're using Genesis, you need to first disable Genesis' tag, which exposes the basic wordpress one, then remove that one too.
/** Remove canonical URL tag */ remove_action('wp_head', 'genesis_canonical', 5); remove_action('wp_head', 'rel_canonical');
Once your page is loading correctly in the debugger, you can follow it's feedback to add open-graph tags to your pages to customize the Facebook story that gets posted when a user shares your page.