python 爬虫--糗事百科段子

标签：pytho   ade   pat   pre   addheader   header   uil   爬虫   user   

import re
import urllib.request
from docx import Document

header=("User-Agent",‘User-Agent:Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36‘)
opn=urllib.request.build_opener()
opn.addheaders=[header]

document = Document()
document.add_heading(‘糗事百科‘,1)

for i in range(0,12):
    data = opn.open(‘https://www.qiushibaike.com/8hr/page/‘+str(i+1)+‘/‘).read().decode()
    pat=‘<div class="content">.*?<span>(.*?)</span>.*?</div>‘

    rst=re.compile(pat,re.S).findall(data)

    for d in rst:
        try:
            document.add_paragraph(d)
            document.add_paragraph(‘- - ‘ * 10)
        except ValueError as err:
            pass
document.save("D:\\data\\糗事百科段子1.docx")

python 爬虫--糗事百科段子

标签：pytho   ade   pat   pre   addheader   header   uil   爬虫   user   

原文地址：http://www.cnblogs.com/xiesongyou/p/7875663.html