#casperjs

[myfreecourses]

Scrape Websites Using PhantomJS And CasperJS

Scrape Websites Using PhantomJS And CasperJS

Become a better JavaScript Developer and Learn Front-End Testing. We’ll use javascript, lodash and jquery for scraping.

What you’ll learn

Know how to use JavaScript for Data Mining

Be able to Capture, Download and Save Website Data

Understand how to use CasperJS and PhantomJS

Apply What You’ve Learned to Front-end Testing

Create Your Own Scripts for Scraping Data

Have a Better…

View On WordPress

[developer-tv]

Scrape Websites using PhantomJS and CasperJS ☞ http://on.codek.tv/4JmyX9vne Become a Better JavaScript Developer and Learn to Mine Data from Websites. Use NodeJS, jQuery and Functional Programming Price: $19 Original: $199 Save Off: 90% off Offer expires: Limited Time ✽✽✽✽✽

[nodejs-fan]

Scrape Websites using PhantomJS and CasperJS ☞ http://on.codek.tv/4JmyX9vne Become a Better JavaScript Developer and Learn to Mine Data from Websites. Use NodeJS, jQuery and Functional Programming Price: $19 Original: $199 Save Off: 90% off Offer expires: Limited Time ✽✽✽✽✽

[javascriptplanets-blog]

Scrape Websites using PhantomJS and CasperJS ☞ http://on.codek.tv/4JmyX9vne Become a Better JavaScript Developer and Learn to Mine Data from Websites. Use NodeJS, jQuery and Functional Programming Price: $19 Original: $199 Save Off: 90% off Offer expires: Limited Time ✽✽✽✽✽

[sarabandejp]

CasperJS: ユニットテスト

搭載されているテストフレームワークを試してみる。サブコマンドの test で実行する。

casperjs test test.js

function Cow() { this.mowed = false; this.moo = function moo() { this.mowed = true; // mootable state: don't do that at home return 'moo!'; }; } casper.test.begin('Cow can moo', 2, function suite(test) { var cow = new Cow(); test.assertEquals(cow.moo(), 'moo!'); test.assert(cow.mowed); test.done(); });

[sarabandejp]

CasperJS: POST リクエストを送信する

open メソッドの第2引数で HTTP メソッドの種類や送信データを指定できる。

var casper = require('casper').create(); var url = 'http://httpbin.org/post'; casper.start(); casper.open(url, { method: 'post', headers: { 'Accept-Language': 'ja' }, data: { 'foo': 'bar' } }); casper.then(function(response) { this.echo(this.getPageContent()); }); casper.run();

[sarabandejp]

CasperJS: セレクターを使う

PhantomJS の evaluate を使わなくてすむように、セレクターのためのヘルパーメソッドが用意されている。

var casper = require('casper').create(); var url = 'http://localhost:3000/'; casper.start(url, function() { var selector = 'h1.page-title'; if (this.exists(selector)) { this.echo(this.fetchText(selector)); } }); casper.run();

[sarabandejp]

CasperJS: Cookie を扱う

PhantomJS で試したコードを流用。

var casper = require('casper').create(); var fs = require('fs'); var url = 'http://localhost:3000/'; var file = 'cookies.txt'; if (fs.isFile(file)) { var data = fs.read(file); phantom.cookies = JSON.parse(data); } casper.start(url, function() { this.echo(this.getPageContent()); var cookies = JSON.stringify(phantom.cookies); fs.write(file, cookies, 644); }); casper.run();