码迷,mamicode.com
首页 > 其他好文 > 详细

学习笔记 requests + BeautifulSoup

时间:2018-05-15 19:41:26      阅读:160      评论:0      收藏:0      [点我收藏+]

标签:row   str   col   鼠标   keep   ica   lang   shu   div   

第一步:requests

get请求

# -*- coding:utf-8  -*-
# 日期:2018/5/15 17:46
# Author:小鼠标
import requests
url = "http://www.baidu.com"
#res = requests.get(url)  #方法1

res = requests.request(‘get‘,url) #方法2
print(‘响应状态码:‘,res.status_code) print(‘响应内容:‘,res.text)

post请求

# -*- coding:utf-8  -*-
# 日期:2018/5/15 17:46
# Author:小鼠标
import requests
url = "http://www.baidu.com"
data = {
    ‘username‘: ‘xiaoshubiao‘,
    ‘pwd‘: ‘xiaoshubiao‘
}
res = requests.post(url,data)
print(‘响应状态码:‘,res.status_code)
print(‘响应内容:‘,res.text)

第二步:伪装浏览器和伪造cookie

# -*- coding:utf-8  -*-
# 日期:2018/5/15 17:46
# Author:小鼠标
import requests
url = "http://www.baidu.com"
headers = {‘User-Agent‘: ‘Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36‘
                         ‘ (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.39‘
                         ‘64.2 Safari/537.36‘,
            ‘Accept‘: ‘text/html,application/xhtml+xml,application/xml;q=0‘
                      ‘.9,image/webp,*/*;q=0.8‘,
            ‘Accept-Encoding‘: ‘gzip, deflate, sdch‘,
            ‘Accept-Language‘: ‘zh-CN,zh;q=0.8,en;q=0.6‘,
            ‘Cache-Control‘: ‘max-age=0‘,
            ‘Connection‘: ‘keep-alive‘
          }
cookies = dict(name=‘xiaoshubiao‘)
res = requests.get(url,headers = headers,cookies = cookies)
print(‘响应状态码:‘,res.status_code)
print(‘响应内容:‘,res.text)

第三步:使用代理ip

# -*- coding:utf-8  -*-
# 日期:2018/5/15 17:46
# Author:小鼠标
import requests
url = "http://www.baidu.com"
headers = {‘User-Agent‘: ‘Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36‘
                         ‘ (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.39‘
                         ‘64.2 Safari/537.36‘,
            ‘Accept‘: ‘text/html,application/xhtml+xml,application/xml;q=0‘
                      ‘.9,image/webp,*/*;q=0.8‘,
            ‘Accept-Encoding‘: ‘gzip, deflate, sdch‘,
            ‘Accept-Language‘: ‘zh-CN,zh;q=0.8,en;q=0.6‘,
            ‘Cache-Control‘: ‘max-age=0‘,
            ‘Connection‘: ‘keep-alive‘
            }
cookies = dict(name=‘xiaoshubiao‘)
proxies = {‘http‘:‘218.73.134.234:36602‘}
res = requests.get(url,headers = headers,cookies = cookies,proxies = proxies)
print(‘响应状态码:‘,res.status_code)
print(‘响应内容:‘,res.text)

 

 

 

学习笔记 requests + BeautifulSoup

标签:row   str   col   鼠标   keep   ica   lang   shu   div   

原文地址:https://www.cnblogs.com/7749ha/p/9042395.html

踩
(0)
赞
(0)
   
举报
评论 一句话评论(0)
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!