标签：失败 ade txt 北京 orm original requests tde java
后来在用JAVA爬取北京信件内容过程中出现好多问题。
我该用python爬取。
这个是我爬取出来的各个信件网址的后缀即（http://www.beijing.gov.cn/hudong/hdjl/com.web.suggest.suggesDetail.flow?originalId=AH20021200370）
然后编写代码：
import requests
import re
import xlwt
# #https://flightaware.com/live/flight/CCA101/history/80
url = ‘http://www.beijing.gov.cn/hudong/hdjl/com.web.consult.consultDetail.flow?originalId=AH20021300174‘
headers = {
"user-agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36"
}
def get_page(url):
try:
response = requests.get(url, headers=headers)
if response.status_code == 200:
#print(‘获取网页成功‘)
return response.text
else:
print(‘获取网页失败‘)
except Exception as e:
print(e)
fopen = open(‘C:\\Users\\hp\\Desktop\\list.txt‘, ‘r‘)//这个是存取信件内容网址后缀
lines = fopen.readlines()
urls = [‘http://www.beijing.gov.cn/hudong/hdjl/com.web.consult.consultDetail.flow?originalId={}‘.format(line) for line in lines]
for url in urls:
print(url)
page = get_page(url)
items = re.findall(‘‘,page,re.S)
print(items)
print(len(items))
但是在用正则法爬取内容的时候出现了一些问题。之后问题解决再次编辑。
标签：失败 ade txt 北京 orm original requests tde java
原文地址：https://www.cnblogs.com/jccjcc/p/12309714.html
