标签：user header status json param request com 阶段 sts
Python爬虫目前是基于requests包，下面是该包的文档，查一些资料还是比较方便。
http://docs.python-requests.org/en/master/
爬取某旅游网站的产品评论，通过分析，获取json文件需要POST指令。简单来说：
那么通过POST发送的内容可以大概有三种，即form、json和multipart，目前先介绍前两种
Content-Type: application/x-www-form-urlencoded
将内容放入dict，然后传递给参数data即可。
payload = {‘key1‘: ‘value1‘, ‘key2‘: ‘value2‘}
r = requests.post(url, data=payload)
Content-Type: application/json
将dict转换为json，传递给data参数。
payload = {‘some‘: ‘data‘}
r = requests.post(url, data=json.dumps(payload))
或者将dict传递给json参数。
payload = {‘some‘: ‘data‘}
r = requests.post(url, json=payload)
然后贴一下简单的代码供参考。
import requests import json def getCommentStr(): url = r"https://package.com/user/comment/product/queryComments.json" header = { ‘User-Agent‘: r‘Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:62.0) Gecko/20100101 Firefox/62.0‘, ‘Accept‘: r‘application/json, text/javascript, */*; q=0.01‘, ‘Accept-Language‘: r‘en-US,en;q=0.5‘, ‘Accept-Encoding‘: r‘gzip, deflate, br‘, ‘Content-Type‘: r‘application/x-www-form-urlencoded; charset=UTF-8‘, ‘X-Requested-With‘: r‘XMLHttpRequest‘, ‘Content-Length‘: ‘65‘, ‘DNT‘: ‘1‘, ‘Connection‘: r‘keep-alive‘, ‘TE‘: r‘Trailers‘ } params = { ‘pageNo‘: ‘2‘, ‘pageSize‘: ‘10‘, ‘productId‘: ‘2590732030‘, ‘rateStatus‘: ‘ALL‘, ‘type‘: ‘all‘ } r = requests.post(url, headers = header, data = params) print(r.text) getCommentStr()
标签：user header status json param request com 阶段 sts
原文地址：https://www.cnblogs.com/kaimobile/p/9874088.html
