scrapy的helloworld

http://scrapy.org/

安装pythonvirlualenv

yuminstalllibxslt-devel

yuminstallpython-devellibffi-devel

cp/usr/lib64/python2.6/lib-dynload/bz2.soScrapy/lib/python2.7/

否则可能汇报bz2的错误

pipinstallscrapy

from scrapy import Spider, Item, Field

class Post(Item):
    title = Field()

class BlogSpider(Spider):
    name, start_urls = 'blogspider', ['http://blog.scrapinghub.com']

    def parse(self, response):
        return [Post(title=e.extract()) for e in response.css("h2 a::text")]

EOF
 scrapy runspider myspider.py

参考

http://scrapy-chs.readthedocs.org/zh_CN/0.24/intro/tutorial.html

scrapyshell"http://app.shafa.com/apk/baobaoanquanjiaoyu.html"

response.xpath('//title')

response.xpath('//title').extract()

response.xpath('//title/text()')

response.xpath('//title/text()').re('(\w+):')

response.xpath('//section[@class="app-info"]//div[contains(@class,"meta-info")][2]//div[@class="info-title"]/following-sibling::span[1]/text()').extract()

http://www.shumeipai.net/forum.php?mod=viewthread&tid=20894&highlight=%D0%A1%B3%B5