码迷,mamicode.com
首页 > 其他好文 > 详细

scrapy错误汇总

时间:2019-05-02 11:47:33      阅读:230      评论:0      收藏:0      [点我收藏+]

标签:爬虫   war   net   awl   ini   object   raw   web   def   

爬虫开始
<pymysql.connections.Connection object at 0x0000012D63A1EFD0>
2019-05-02 09:56:21 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2019-05-02 09:56:21 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2019-05-02 09:56:24 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (301) to <GET http://m.kxzhijia.com/en/pg.jsp?entry=mallNav&pgs=1> from <GET http://m.kxzhijia.com/pg.jsp?entry=mallNav&pgs=1>
2019-05-02 09:56:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET http://m.kxzhijia.com/en/pg.jsp?entry=mallNav&pgs=1> (referer: None)
finished
2019-05-02 09:56:30 [scrapy.core.engine] INFO: Closing spider (finished)
2019-05-02 09:56:30 [chyp_tj] INFO: 结束(正常):chyp_tj

  没有请求头headers时就会这样报错,例如:

‘DEFAULT_REQUEST_HEADERS‘ : {
‘User-Agent‘: ‘Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36‘,
}

scrapy错误汇总

标签:爬虫   war   net   awl   ini   object   raw   web   def   

原文地址:https://www.cnblogs.com/qiaoer1993/p/10801693.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!