标签:
使用javascript编写的爬虫源码,用于爬取沪商财富网上的商品信息。
代码粘贴到神箭手云爬虫平台(http://www.shenjianshou.cn/)上就可以直接跑了,
不需要安装编译环境。要爬取其他网站,可以更改源码即可。
var configs = { domains: ["www.hushangcaifu.com"], scanUrls: ["http://www.hushangcaifu.com/invest/index1.html"], contentUrlRegexes: ["http://www\\.hushangcaifu\\.com/invest/a\\d{4}\\.html"], helperUrlRegexes: ["http://www\\.hushangcaifu\\.com/invest/index\\d+\\.html"], fields: [ { name: "title", selector: "//div[contains(@class,‘product-content-top-left-top‘)]/h3/text()", required: true }, { name: "user_name", selector: "//div[contains(@class,‘product-content-top-left-top‘)]/p/span/text()" }, { name: "total_money", selector:"//div[contains(@class,‘product-content-top-left-middle‘)]/div[1]/h4/text()" }, { name: "project_time", selector:"//div[contains(@class,‘product-content-top-left-middle‘)]/div[2]/h4/text()" }, { name: "annual_return", selector:"//div[contains(@class,‘product-content-top-left-middle‘)]/div[3]/h4/text()" }, { name: "return_method", selector:"//div[contains(@class,‘product-content-top-left-middle‘)]/div[4]/h4/text()" } ] }; var crawler = new Crawler(configs); crawler.start();
标签:
原文地址:http://www.cnblogs.com/datafactory/p/5569818.html