码迷,mamicode.com
首页 > Web开发 > 详细

协程demo,1异步爬网页 2异步socket请求

时间:2018-09-25 23:29:19      阅读:410      评论:0      收藏:0      [点我收藏+]

标签:一个   异步处理   eve   oca   accept   tps   cep   sock   coding   

一、异步爬网页

‘‘‘
协程并发爬网页
‘‘‘
from urllib import request
import gevent,time
from gevent import monkey  # 让gevent知道urllib里的哪些操作属于IO操作

monkey.patch_all()   # 标记当前程序所有的IO操作

def f(url):
    print("GET:{0}".format(url))
    resp = request.urlopen(url)
    data = resp.read()
    print({0} bytes received from {1}.format(len(data), url))

time_start = time.time()
f("https://www.python.org")
f("https://www.yahoo.com")
f("https://github.com")
print("同步耗时:{0}".format(time.time()-time_start))

async_time_start = time.time()
gevent.joinall([gevent.spawn(f,"https://www.python.org"),
                gevent.spawn(f, "https://www.yahoo.com"),
                gevent.spawn(f, "https://github.com")])
print("异步耗时:{0}".format(time.time()-async_time_start))

‘‘‘
GET:https://www.python.org
49060 bytes received from https://www.python.org
GET:https://www.yahoo.com
498196 bytes received from https://www.yahoo.com
GET:https://github.com
64978 bytes received from https://github.com
同步耗时:10.61960744857788
GET:https://www.python.org
GET:https://www.yahoo.com
GET:https://github.com
86167 bytes received from https://github.com
49060 bytes received from https://www.python.org
503102 bytes received from https://www.yahoo.com
异步耗时:3.7582149505615234
‘‘‘

二、异步处理socket请求

服务端:

import socket
import gevent
from gevent import monkey

monkey.patch_all()

def server(port):
    s = socket.socket()
    s.bind((0.0.0.0, port))
    s.listen(500)
    while True:
        cli, addr = s.accept()  # 之前多并发是启动一个线程
        gevent.spawn(handle_request, cli)

def handle_request(conn):
    try:
        while True:
            data = conn.recv(1024)
            print("recv:", data)
            conn.send(data)
            if not data:
                conn.shutdown(socket.SHUT_WR)
    except Exception as ex:
        print(ex)
    finally:
        conn.close()

if __name__ == __main__:
    server(8001)

客户端:

import socket

HOST = localhost
PORT = 8001

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
while True:
    msg = bytes(input(">>").strip(), encoding="utf-8")
    s.sendall(msg)
    data = s.recv(1024)
    print("Received:", repr(data))
s.close()

 

协程demo,1异步爬网页 2异步socket请求

标签:一个   异步处理   eve   oca   accept   tps   cep   sock   coding   

原文地址:https://www.cnblogs.com/staff/p/9704166.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!