标签:lan ack done 异步编程 ext 一个 return 阻塞 架构
异步协程
实现异步的原理:
重要点:
关键字await,修饰特殊函数内部阻塞的操作,
支持异步网络请求的模块:aiohttp
pip install aiohttp
使用:
1.写出大致架构
async with aiohttp.ClientSession() as s:
# get post方法的使用和requests几乎一致
# 处理代理:proxy=‘http://ip:port’
async with s.get(url) as response:
# text()字符串形式的响应数据
# read()bytes类型的响应数据
page_text = response.text()
return page_text
补充细节:
在每一个with前加async关键字
需要在阻塞操作前加上await关键字
完整代码
async def get_request(url):
print(‘正在请求:‘,url)
async with aiohttp.ClientSession() as s:
# get post方法的使用和requests几乎一致
# 处理代理:proxy=‘http://ip:port’
async with await s.get(url) as response:
# text()字符串形式的响应数据
# read()bytes类型的响应数据
page_text = await response.text()
return page_text
多任务异步协程的特点:
完整代码异步请求
import asyncio
import requests
import time
import aiohttp
start = time.time()
urls = [
‘http://127.0.0.1:5000/bobo‘,
‘http://127.0.0.1:5000/jay‘,
‘http://127.0.0.1:5000/tom‘,
]
# async def get_request(url):
#
# print(‘正在请求:‘,url)
# # requests是不支持异步,不可以出现在特殊函数内部
# page_text = requests.get(url)
# return page_text
async def get_request(url):
print(‘正在请求:‘,url)
async with aiohttp.ClientSession() as s:
# get post方法的使用和requests几乎一致
# 处理代理:proxy=‘http://ip:port’
async with await s.get(url) as response:
# text()字符串形式的响应数据
# read()bytes类型的响应数据
page_text = await response.text()
return page_text
tasks = []
for url in urls:
c = get_request(url)
task = asyncio.ensure_future(c)
tasks.append(task)
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.wait(tasks))
print(‘总耗时:‘,time.time()-start)
>>>
正在请求: http://127.0.0.1:5000/bobo
正在请求: http://127.0.0.1:5000/jay
正在请求: http://127.0.0.1:5000/tom
总耗时: 2.007049560546875
完整代码进行异步解析
import asyncio
import aiohttp
import time
from lxml import etree
start = time.time()
urls = [
‘http://127.0.0.1:5000/bobo‘,
‘http://127.0.0.1:5000/jay‘,
‘http://127.0.0.1:5000/tom‘,
]
def parse(task):
page_text = task.result()
tree = etree.HTML(page_text)
data = tree.xpath(‘//a[@id="feng"]/text()‘)
print(data)
async def get_request(url):
async with aiohttp.ClientSession() as s:
async with await s.get(url) as response:
page_text = await response.text()
return page_text
tasks = []
for url in urls:
c = get_request(url)
task = asyncio.ensure_future(c)
task.add_done_callback(parse)
tasks.append(task)
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.wait(tasks))
print(‘总耗时:‘,time.time()-start)
>>>
[‘凤凰台上凤凰游,凤去台空江自流,吴宫花草埋幽径,晋代衣冠成古丘‘]
[‘凤凰台上凤凰游,凤去台空江自流,吴宫花草埋幽径,晋代衣冠成古丘‘]
[‘凤凰台上凤凰游,凤去台空江自流,吴宫花草埋幽径,晋代衣冠成古丘‘]
总耗时: 2.007239580154419
9、Python Asyncio异步编程-事件循环详解【2】
标签:lan ack done 异步编程 ext 一个 return 阻塞 架构
原文地址:https://www.cnblogs.com/remixnameless/p/13171341.html