网易财经爬取

标签：tree   headers   use   gecko   mozilla   webkit   lis   list   gen   

import requests
from lxml import etree

url = ‘http://quotes.money.163.com/old/‘
headers = {
‘User-Agent‘: ‘Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.79 Safari/537.36‘
}

html = requests.get(url=url,headers=headers).text

tree = etree.HTML(html)

content = tree.xpath(‘//li[@qid="HS"]//li[@id="f0-f7"]/ul/li‘)
for con in content:
one = con.xpath(‘./a/text()‘)[0]
print(one)
two_list = con.xpath(‘./ul/li‘)
for t in two_list:
qid = t.xpath(‘./@qid‘)[0]
print(qid)
two = t.xpath(‘./a/text()‘)[0]
print(two)

网易财经爬取

标签：tree   headers   use   gecko   mozilla   webkit   lis   list   gen   

原文地址：https://www.cnblogs.com/Iceredtea/p/12069065.html