标签：提交数据 static mp4 custom rtb Opens state Once als
以知乎为例
# -*- coding:UTF-8 -*- __autor__ = ‘zhouli‘ __date__ = ‘2018/11/2 21:40‘ # 模拟登陆 import requests headers = { ‘user-agent‘: ‘Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.62 Safari/537.36‘} html = requests.get(‘https://www.zhihu.com‘, headers=headers) print(html.text)
结果为：
<html lang="zh" data-hairline="true" data-theme="light"><head><meta charSet="utf-8"/><title data-react-helmet="true">知乎 - 有问题上知乎</title><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1"/><meta name="renderer" content="webkit"/><meta name="force-rendering" content="webkit"/><meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"/><meta name="google-site-verification" content="FTeR0c8arOPKh8c5DYh_9uu98_zJbaWw53J-Sch9MTg"/><title>知乎 - 有问题上知乎</title><meta name="description" content="有问题，上知乎。知乎是中文互联网知名知识分享平台，以「知识连接一切」为愿景，致力于构建一个人人都可以便捷接入的知识分享网络，让人们便捷地与世界分享知识、经验和见解，发现更大的世界。"/><link rel="shortcut icon" type="image/x-icon" href="https://static.zhihu.com/static/favicon.ico"/><link rel="search" type="application/opensearchdescription+xml" href="https://static.zhihu.com/static/search.xml" title="知乎"/><link rel="dns-prefetch" href="//static.zhimg.com"/><link rel="dns-prefetch" href="//pic1.zhimg.com"/><link rel="dns-prefetch" href="//pic2.zhimg.com"/><link rel="dns-prefetch" href="//pic3.zhimg.com"/><link rel="dns-prefetch" href="//pic4.zhimg.com"/><link href="https://static.zhihu.com/heifetz/main.app.d5097a11ea1e355aef2e.css" rel="stylesheet"/><script defer="" crossOrigin="anonymous" src="https://unpkg.zhimg.com/@cfe/sentry-script@latest/dist/init.js" data-sentry-config="{"dsn":"https://65e244586890460588f00f2987137aa8@crash2.zhihu.com/193","sampleRate":0.1,"release":"253-cbc1f53f","ignoreErrorNames":["NetworkError","SecurityError"],"ignoreErrors":["origin message","Network request failed","Loading chunk","这个系统不支持该功能。","Can't find variable: webkit","Can't find variable: $","内存不足","out of memory","DOM Exception 18","zfeedback sdk 初始化失败！","zfeedback sdk 加载失败！","The operation is insecure","[object Event]","[object FileError]","[object DOMError]","[object Object]","拒绝访问。","Maximum call stack size exceeded","UploadError","无法 fetch","draft-js","缺少 JavaScript 对象","componentWillEnter","componentWillLeave","componentWillAppear","getInlineStyleAt","getCharacterList"],"whitelistUrls":["static.zhihu.com"]}"></script></head><body class="EntrySign-body"><div id="root"><div data-zop-usertoken="{}" data-reactroot=""><div class="LoadingBar"></div><div><header role="banner" class="Sticky AppHeader" data-za-module="TopNavBar"><div class="AppHeader-inner"><a href="//www.zhihu.com" aria-label="知乎"><svg viewBox="0 0 200 91" class="Icon ZhihuLogo ZhihuLogo--blue Icon--logo" style="height:30px;width:64px" width="64" height="30" aria-hidden="true"><title></title><g><path d="M53.29 80.035l7.32.002 2.41 8.24 13.128-8.24h15.477v-67.98H53.29v67.978zm7.79-60.598h22.756v53.22h-8.73l-8.718 5.473-1.587-5.46-3.72-.012v-53.22zM46.818 43.162h-16.35c.545-8.467.687-16.12.687-22.955h15.987s.615-7.05-2.68-6.97H16.807c1.09-4.1 2.46-8.332 4.1-12.708 0 0-7.523 0-10.085 6.74-1.06 2.78-4.128 13.48-9.592 24.41 1.84-.2 7.927-.37 11.512-6.94.66-1.84.785-2.08 1.605-4.54h9.02c0 3.28-.374 20.9-.526 22.95H6.51c-3.67 0-4.863 7.38-4.863 7.38H22.14C20.765 66.11 13.385 79.24 0 89.62c6.403 1.828 12.784-.29 15.937-3.094 0 0 7.182-6.53 11.12-21.64L43.92 85.18s2.473-8.402-.388-12.496c-2.37-2.788-8.768-10.33-11.496-13.064l-4.57 3.627c1.363-4.368 2.183-8.61 2.46-12.71H49.19s-.027-7.38-2.372-7.38zm128.752-.502c6.51-8.013 14.054-18.302 14.054-18.302s-5.827-4.625-8.556-1.27c-1.874 2.548-11.51 15.063-11.51 15.063l6.012 4.51zm-46.903-18.462c-2.814-2.577-8.096.667-8.096.667s12.35 17.2 12.85 17.953l6.08-4.29s-8.02-11.752-10.83-14.33zM199.99 46.5c-6.18 0-40.908.292-40.953.292v-31.56c1.503 0 3.882-.124 7.14-.376 12.773-.753 21.914-1.25 27.427-1.504 0 0 3.817-8.496-.185-10.45-.96-.37-7.24 1.43-7.24 1.43s-51.63 5.153-72.61 5.64c.5 2.756 2.38 5.336 4.93 6.11 4.16 1.087 7.09.53 15.36.277 7.76-.5 13.65-.76 17.66-.76v31.19h-41.71s.88 6.97 7.97 7.14h33.73v22.16c0 4.364-3.498 6.87-7.65 6.6-4.4.034-8.15-.36-13.027-.566.623 1.24 1.977 4.496 6.035 6.824 3.087 1.502 5.054 2.053 8.13 2.053 9.237 0 14.27-5.4 14.027-14.16V53.93h38.235c3.026 0 2.72-7.432 2.72-7.432z" fill-rule="evenodd"/></g></svg></a><nav role="navigation" class="AppHeader-nav"><a class="AppHeader-navItem" href="//www.zhihu.com/" data-za-not-track-link="true">首页</a><a class="AppHeader-navItem" href="//www.zhihu.com/explore" data-za-not-track-link="true">发现</a><a href="//www.zhihu.com/topic" class="AppHeader-navItem" data-za-not-track-link="true">话题</a></nav><div class="SearchBar" role="search" data-za-module="PresetWordItem"><div class="SearchBar-toolWrapper"><form class="SearchBar-tool"><div><div class="Popover"><div class="SearchBar-input Input-wrapper Input-wrapper--grey"><input type="text" maxLength="100" value="" autoComplete="off" role="combobox" aria-expanded="false" aria-autocomplete="list" aria-activedescendant="null--1" id="null-toggle" aria-haspopup="true" aria-owns="null-content" class="Input" placeholder=""/><div class="Input-after"><button aria-label="搜索" type="button" class="Button SearchBar-searchIcon Button--primary"><span style="display:inline-flex;align-items:center">?<svg class="Zi Zi--Search" fill="currentColor" viewBox="0 0 24 24" width="18" height="18"><path d="M17.068 15.58a8.377 8.377 0 0 0 1.774-5.159 8.421 8.421 0 1 0-8.42 8.421 8.38 8.38 0 0 0 5.158-1.774l3.879 3.88c.957.573 2.131-.464 1.488-1.49l-3.879-3.878zm-6.647 1.157a6.323 6.323 0 0 1-6.316-6.316 6.323 6.323 0 0 1 6.316-6.316 6.323 6.323 0 0 1 6.316 6.316 6.323 6.323 0 0 1-6.316 6.316z" fill-rule="evenodd"></path></svg></span></button></div></div></div></div></form></div></div><div class="AppHeader-userInfo"><div class="AppHeader-profile"><div><button type="button" class="Button AppHeader-login Button--blue">登录</button><button type="button" class="Button Button--primary Button--blue">加入知乎</button></div></div></div></div></header></div><main role="main" class="App-main"><div class="SignFlowHomepage"><div class="SignFlowHomepage-content"><div class="Card SignContainer-content"><div class="SignFlowHeader" style="padding-bottom:5px"><svg viewBox="0 0 200 91" class="Icon ZhihuLogo ZhihuLogo--blue Icon--logo" style="height:65.625px;width:140px" width="140" height="65.625" aria-hidden="true"><title></title><g><path d="M53.29 80.035l7.32.002 2.41 8.24 13.128-8.24h15.477v-67.98H53.29v67.978zm7.79-60.598h22.756v53.22h-8.73l-8.718 5.473-1.587-5.46-3.72-.012v-53.22zM46.818 43.162h-16.35c.545-8.467.687-16.12.687-22.955h15.987s.615-7.05-2.68-6.97H16.807c1.09-4.1 2.46-8.332 4.1-12.708 0 0-7.523 0-10.085 6.74-1.06 2.78-4.128 13.48-9.592 24.41 1.84-.2 7.927-.37 11.512-6.94.66-1.84.785-2.08 1.605-4.54h9.02c0 3.28-.374 20.9-.526 22.95H6.51c-3.67 0-4.863 7.38-4.863 7.38H22.14C20.765 66.11 13.385 79.24 0 89.62c6.403 1.828 12.784-.29 15.937-3.094 0 0 7.182-6.53 11.12-21.64L43.92 85.18s2.473-8.402-.388-12.496c-2.37-2.788-8.768-10.33-11.496-13.064l-4.57 3.627c1.363-4.368 2.183-8.61 2.46-12.71H49.19s-.027-7.38-2.372-7.38zm128.752-.502c6.51-8.013 14.054-18.302 14.054-18.302s-5.827-4.625-8.556-1.27c-1.874 2.548-11.51 15.063-11.51 15.063l6.012 4.51zm-46.903-18.462c-2.814-2.577-8.096.667-8.096.667s12.35 17.2 12.85 17.953l6.08-4.29s-8.02-11.752-10.83-14.33zM199.99 46.5c-6.18 0-40.908.292-40.953.292v-31.56c1.503 0 3.882-.124 7.14-.376 12.773-.753 21.914-1.25 27.427-1.504 0 0 3.817-8.496-.185-10.45-.96-.37-7.24 1.43-7.24 1.43s-51.63 5.153-72.61 5.64c.5 2.756 2.38 5.336 4.93 6.11 4.16 1.087 7.09.53 15.36.277 7.76-.5 13.65-.76 17.66-.76v31.19h-41.71s.88 6.97 7.97 7.14h33.73v22.16c0 4.364-3.498 6.87-7.65 6.6-4.4.034-8.15-.36-13.027-.566.623 1.24 1.977 4.496 6.035 6.824 3.087 1.502 5.054 2.053 8.13 2.053 9.237 0 14.27-5.4 14.027-14.16V53.93h38.235c3.026 0 2.72-7.432 2.72-7.432z" fill-rule="evenodd"/></g></svg><div class="SignFlowHeader-slogen">注册<!-- -->知乎，发现更大的世界</div></div><div class="SignContainer-inner"><div class="Register"><div><div class="Register-content"><form novalidate=""><div class="SignFlow-account"><div class="SignFlow-supportedCountriesSelectContainer"></div><div class="SignFlowInput SignFlow-accountInputContainer"><div class="SignFlow-accountInput Input-wrapper"><input type="tel" value="" name="phoneNo" class="Input" placeholder="手机号"/></div><div class="SignFlowInput-errorMask SignFlowInput-requiredErrorMask SignFlowInput-errorMask--hidden"></div></div></div><div class="Captcha SignFlow-captchaContainer Register-captcha" style="width:0;height:0;opacity:0;overflow:hidden;margin:0;padding:0;border:0"><div><div class="SignFlowInput"><div class="Input-wrapper"><input type="text" value="" name="captcha" tabindex="-1" class="Input" placeholder="验证码"/></div><div class="SignFlowInput-errorMask SignFlowInput-requiredErrorMask SignFlowInput-errorMask--hidden"></div></div><span class="Captcha-englishImage"><div class="Captcha-englishContainer"><img data-tooltip="看不清楚？换一张" class="Captcha-englishImg" src="data:image/jpg;base64,null" alt="图形验证码"/></div></span></div></div><div class="Register-SMSInput"><div class="SignFlow SignFlow-smsInputContainer"><div class="SignFlowInput SignFlow-smsInput"><div class="Input-wrapper"><input type="number" value="" name="digits" class="Input" placeholder="输入 6 位短信验证码"/></div><div class="SignFlowInput-errorMask SignFlowInput-requiredErrorMask SignFlowInput-errorMask--hidden"></div></div><button type="button" class="Button CountingDownButton SignFlow-smsInputButton Button--plain">获取短信验证码</button></div><div class="Register-smsBackUp"><span>接收<!-- -->语音<!-- -->验证码</span></div></div><button type="submit" class="Button Register-submitButton Button--primary Button--blue">注册</button></form><div class="Register-footer"><span class="Register-declaration">注册即代表同意<a href="https://www.zhihu.com/terms">《知乎协议》</a><a href="https://www.zhihu.com/terms/privacy">《隐私政策》</a></span><a class="Register-org" href="https://www.zhihu.com/org/signup">注册机构号</a></div></div></div></div><div class="SignContainer-switch">已有帐号？<span>登录</span></div><div class="SignFlowHomepage-qrImage SignFlowHomepage-qrImageHidden"><div></div></div></div></div><button type="button" class="Button SignFlowHomepage-downloadBtn">下载知乎 App</button></div><footer class="SignFlowHomepage-footer"><div class="ZhihuLinks"><a target="_blank" rel="noopener noreferrer" href="https://zhuanlan.zhihu.com">知乎专栏</a><a target="_blank" rel="noopener noreferrer" href="/roundtable">圆桌</a><a target="_blank" rel="noopener noreferrer" href="/explore">发现</a><a target="_blank" rel="noopener noreferrer" href="/app">移动应用</a><a target="_blank" rel="noopener noreferrer" href="/contact">联系我们</a><a target="_blank" rel="noopener noreferrer" href="https://app.mokahr.com/apply/zhihu">来知乎工作</a><a target="_blank" rel="noopener noreferrer" href="/org/signup">注册机构号</a></div><div class="ZhihuRights"><span>? 2018 知乎</span><a target="_blank" rel="noopener noreferrer" href="http://www.miibeian.gov.cn/">京 ICP 证 110745 号</a><span>京公网安备 11010802010035 号<a href="https://zhstatic.zhihu.com/assets/zhihu/publish-license.jpg" target="_blank" rel="noopener noreferrer">出版物经营许可证</a></span></div><div class="ZhihuReports"><a target="_blank" rel="noopener noreferrer" href="https://zhuanlan.zhihu.com/p/28852607">侵权举报</a><a target="_blank" rel="noopener noreferrer" href="http://www.12377.cn">网上有害信息举报专区</a><a target="_blank" rel="noopener noreferrer" href="/jubao">儿童色情信息举报专区</a><span>违法和不良信息举报：010-82716601</span></div><div class="ZhihuIntegrity"><div><img src="https://static.zhihu.com/static/revved/img/index/chengxing_logo@2x.65dc76e8.png" alt="诚信网站示范企业"/><a href="https://credit.szfw.org/CX20170607038331320388.html">诚信网站示范企业</a></div></div></footer></div></main></div></div><div id="data" style="display:none" data-state="{"common":{"ask":{}},"privacy":{"showPrivacy":false},"loading":{"global":{"count":0},"local":{}},"entities":{"users":{},"questions":{},"answers":{},"articles":{},"columns":{},"topics":{},"roundtables":{},"favlists":{},"comments":{},"notifications":{},"ebooks":{},"activities":{},"feeds":{},"pins":{},"promotions":{},"drafts":{}},"currentUser":"","account":{"lockLevel":{},"unlockTicketStatus":false,"unlockTicket":null,"challenge":[],"errorStatus":false,"message":"","isFetching":false,"accountInfo":{},"urlToken":{"loading":false}},"settings":{"socialBind":null,"inboxMsg":null,"notification":{},"email":{},"privacyFlag":null,"blockedUsers":{"isFetching":false,"paging":{"pageNo":1,"pageSize":6},"data":[]},"blockedFollowees":{"isFetching":false,"paging":{"pageNo":1,"pageSize":6},"data":[]},"ignoredTopics":{"isFetching":false,"paging":{"pageNo":1,"pageSize":6},"data":[]},"restrictedTopics":null,"laboratory":{}},"notification":{},"people":{"profileStatus":{},"activitiesByUser":{},"answersByUser":{},"answersSortByVotesByUser":{},"answersIncludedByUser":{},"votedAnswersByUser":{},"thankedAnswersByUser":{},"voteAnswersByUser":{},"thankAnswersByUser":{},"topicAnswersByUser":{},"articlesByUser":{},"articlesSortByVotesByUser":{},"articlesIncludedByUser":{},"pinsByUser":{},"questionsByUser":{},"commercialQuestionsByUser":{},"favlistsByUser":{},"followingByUser":{},"followersByUser":{},"mutualsByUser":{},"followingColumnsByUser":{},"followingQuestionsByUser":{},"followingFavlistsByUser":{},"followingTopicsByUser":{},"publicationsByUser":{},"columnsByUser":{},"allFavlistsByUser":{},"brands":null},"env":{"ab":{"config":{"experiments":[{"expId":"launch-gw_adr_dkts-9","expPrefix":"gw_adr_dkts","isDynamicallyUpdated":true,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-gw_adr_mini-2","expPrefix":"gw_adr_mini","isDynamicallyUpdated":true,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-gw_adr_wbtp-2","expPrefix":"gw_adr_wbtp","isDynamicallyUpdated":true,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-gw_adr_wxfb-2","expPrefix":"gw_adr_wxfb","isDynamicallyUpdated":true,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-gw_anr_wxbk-2","expPrefix":"gw_anr_wxbk","isDynamicallyUpdated":true,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-gw_ios_dkts-8","expPrefix":"gw_ios_dkts","isDynamicallyUpdated":true,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-gw_ios_mini-2","expPrefix":"gw_ios_mini","isDynamicallyUpdated":true,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-gw_ios_wxb-2","expPrefix":"gw_ios_wxb","isDynamicallyUpdated":true,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-gw_ios_wxfb-1","expPrefix":"gw_ios_wxfb","isDynamicallyUpdated":true,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-gw_wbtp-2","expPrefix":"gw_wbtp","isDynamicallyUpdated":true,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-mp_amap_ios-1","expPrefix":"mp_amap_ios","isDynamicallyUpdated":true,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-mp_apm-1","expPrefix":"mp_apm","isDynamicallyUpdated":true,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-mp_hb_si-3","expPrefix":"mp_hb_si","isDynamicallyUpdated":true,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-mp_ios_bvc-2","expPrefix":"mp_ios_bvc","isDynamicallyUpdated":true,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-mp_ios_webp-2","expPrefix":"mp_ios_webp","isDynamicallyUpdated":true,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-mp_video_feed-2","expPrefix":"mp_video_feed","isDynamicallyUpdated":false,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-se_ios_topsearch-2","expPrefix":"se_ios_topsearch","isDynamicallyUpdated":false,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-top_dtmt-10","expPrefix":"top_dtmt","isDynamicallyUpdated":true,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-top_feed_card-1","expPrefix":"top_feed_card","isDynamicallyUpdated":true,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-vd_adrupload_cdn-2","expPrefix":"vd_adrupload_cdn","isDynamicallyUpdated":false,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-vd_ppt_enter_2-2","expPrefix":"vd_ppt_enter_2","isDynamicallyUpdated":true,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-vd_upload_cdn-2","expPrefix":"vd_upload_cdn","isDynamicallyUpdated":false,"isRuntime":false,"includeTriggerInfo":false},{"expId":"launch-vd_zm_core-2","expPrefix":"vd_zm_core","isDynamicallyUpdated":false,"isRuntime":false,"includeTriggerInfo":false}],"params":[{"id":"adr_anr_watch","type":"String","value":"false"},{"id":"adr_grow_guide_login_4","type":"String","value":"3"},{"id":"adr_more_hyb_card","type":"String","value":"0"},{"id":"ios_asp","type":"String","value":"off"},{"id":"ios_video_agent_4_22","type":"String","value":"false"},{"id":"top_ad_slot","type":"String","value":"1","chainId":"_all_"},{"id":"adr_guest_login","type":"String","value":"0"},{"id":"adr_recommend_column","type":"String","value":"0"},{"id":"se_tf","type":"String","value":"1","chainId":"_all_"},{"id":"top_bill","type":"String","value":"0","chainId":"_all_"},{"id":"top_local","type":"String","value":"1","chainId":"_all_"},{"id":"top_nid","type":"String","value":"0","chainId":"_all_"},{"id":"web_stream_render","type":"String","value":"0"},{"id":"top_root_ac","type":"String","value":"1","chainId":"_all_"},{"id":"top_vd_op","type":"String","value":"0","chainId":"_all_"},{"id":"web_new_comment","type":"String","value":"0"},{"id":"ios_ad_web_cache","type":"String","value":"0"},{"id":"ios_hybrid_intercepting","type":"String","value":"1"},{"id":"top_ab_validate","type":"String","value":"0","chainId":"_all_"},{"id":"adr_ps","type":"String","value":"0"},{"id":"ios_wxfb","type":"String","value":"0"},{"id":"se_entity","type":"String","value":"on","chainId":"_all_"},{"id":"adr_traffic_monitor","type":"String","value":"false"},{"id":"top_fqai","type":"String","value":"0","chainId":"_all_"},{"id":"top_mlt_model","type":"String","value":"0","chainId":"_all_"},{"id":"se_consulting_switch","type":"String","value":"off","chainId":"_all_"},{"id":"top_is_gr","type":"String","value":"0","chainId":"_all_"},{"id":"adr_invite","type":"String","value":"false"},{"id":"adr_perm","type":"String","value":"0"},{"id":"ios_real_time_launch_http","type":"String","value":"http_off"},{"id":"top_adpar","type":"String","value":"0","chainId":"_all_"},{"id":"adr_comment","type":"String","value":"false"},{"id":"adr_mbv","type":"String","value":"false"},{"id":"hb_stream_render","type":"String","value":"0"},{"id":"ios_cashier_color","type":"String","value":"0"},{"id":"ios_comment","type":"String","value":"0"},{"id":"ios_next_ans","type":"String","value":"N"},{"id":"se_correct_ab","type":"String","value":"0","chainId":"_all_"},{"id":"se_new_market_search","type":"String","value":"off","chainId":"_all_"},{"id":"top_billpic","type":"String","value":"0","chainId":"_all_"},{"id":"top_sj","type":"String","value":"2","chainId":"_all_"},{"id":"adr_mqtt","type":"String","value":"0"},{"id":"top_nucc","type":"String","value":"0","chainId":"_all_"},{"id":"top_videos_priority","type":"String","value":"-1","chainId":"_all_"},{"id":"ios_video_agent_4_28","type":"String","value":"false"},{"id":"top_billread","type":"String","value":"1","chainId":"_all_"},{"id":"top_user_gift","type":"String","value":"0","chainId":"_all_"},{"id":"adr_comment_new_editor","type":"String","value":"0"},{"id":"adr_refresh_token","type":"String","value":"1"},{"id":"hb_active_answerer","type":"String","value":"0"},{"id":"ios_ad_cta","type":"String","value":"0"},{"id":"se_gi","type":"String","value":"0","chainId":"_all_"},{"id":"top_multi_model","type":"String","value":"0","chainId":"_all_"},{"id":"tp_ios_topic_write_pin_guide","type":"String","value":"1","chainId":"_all_"},{"id":"adr_sdk_data_switch","type":"String","value":"0"},{"id":"adr_sqtc","type":"String","value":"1"},{"id":"adr_video_topic_volume_control","type":"String","value":"0"},{"id":"ios_medal_badge_view","type":"String","value":"false"},{"id":"ls_new_video","type":"String","value":"0","chainId":"_all_"},{"id":"top_roundtable","type":"String","value":"1","chainId":"_all_"},{"id":"top_v_album","type":"String","value":"1","chainId":"_all_"},{"id":"ios_mlssq","type":"String","value":"0"},{"id":"ios_question_new_comment","type":"String","value":"0"},{"id":"ios_spic","type":"String","value":"0"},{"id":"se_rescore","type":"String","value":"0","chainId":"_all_"},{"id":"top_newfollowans","type":"String","value":"0","chainId":"_all_"},{"id":"top_root","type":"String","value":"0","chainId":"_all_"},{"id":"top_yhgc","type":"String","value":"0","chainId":"_all_"},{"id":"adr_anp","type":"String","value":"android_answer_pager_off"},{"id":"adr_pre_load_html","type":"String","value":"0"},{"id":"pin_efs","type":"String","value":"orig","chainId":"_all_"},{"id":"se_dt","type":"String","value":"1","chainId":"_all_"},{"id":"top_followtop","type":"String","value":"0","chainId":"_all_"},{"id":"top_ntr","type":"String","value":"1","chainId":"_all_"},{"id":"top_promo","type":"String","value":"1","chainId":"_all_"},{"id":"adr_topsearch","type":"String","value":"2"},{"id":"hb_major_onebox","type":"String","value":"0"},{"id":"ios_ge4","type":"String","value":"3"},{"id":"ios_qtoc","type":"String","value":"0"},{"id":"ios_roundtable","type":"String","value":"A"},{"id":"top_tag_isolation","type":"String","value":"0","chainId":"_all_"},{"id":"adr_liguangi_test","type":"String","value":"1"},{"id":"adr_unif","type":"String","value":"off"},{"id":"ios_ios_launch_mp4","type":"String","value":"1"},{"id":"ios_pdf","type":"String","value":"n"},{"id":"se_daxuechuisou","type":"String","value":"new","chainId":"_all_"},{"id":"adr_android_medal_badge_view","type":"String","value":"false"},{"id":"ios_ff_cardtype","type":"String","value":"A"},{"id":"ios_more_editcard","type":"String","value":"true"},{"id":"ios_ps","type":"String","value":"0"},{"id":"top_manual_tag","type":"String","value":"1","chainId":"_all_"},{"id":"top_recall_follow_user","type":"String","value":"91","chainId":"_all_"},{"id":"top_yc","type":"String","value":"0","chainId":"_all_"},{"id":"adr_consultation","type":"String","value":"0"},{"id":"adr_profile_label","type":"String","value":"1"},{"id":"ios_magitab","type":"String","value":"0"},{"id":"top_hqt","type":"String","value":"0","chainId":"_all_"},{"id":"adr_ppt_enter","type":"String","value":"1"},{"id":"ios_invite_ans","type":"String","value":"A"},{"id":"ios_search_tab","type":"String","value":"0"},{"id":"top_feedre_cpt","type":"String","value":"101","chainId":"_all_"},{"id":"top_follow_reason","type":"String","value":"0","chainId":"_all_"},{"id":"top_root_web","type":"String","value":"0","chainId":"_all_"},{"id":"ios_answer_preload","type":"String","value":"0"},{"id":"se_consulting_price","type":"String","value":"n","chainId":"_all_"},{"id":"top_ac_merge","type":"String","value":"0","chainId":"_all_"},{"id":"top_ebook","type":"String","value":"0","chainId":"_all_"},{"id":"top_follow_question_hybrid","type":"String","value":"0","chainId":"_all_"},{"id":"top_no_weighing","type":"String","value":"1","chainId":"_all_"},{"id":"ios_ad_skip_pos","type":"String","value":"up"},{"id":"se_minor_onebox","type":"String","value":"d","chainId":"_all_"},{"id":"top_recall_deep_user","type":"String","value":"1","chainId":"_all_"},{"id":"top_slot_ad_pos","type":"String","value":"1","chainId":"_all_"},{"id":"tp_write_pin_guide","type":"String","value":"3","chainId":"_all_"},{"id":"adr_consult","type":"String","value":"0"},{"id":"ios_webp","type":"String","value":"1"},{"id":"top_video_score","type":"String","value":"1","chainId":"_all_"},{"id":"adr_cashier_color","type":"String","value":"0"},{"id":"ios_1752","type":"String","value":"0"},{"id":"ios_db_n_e","type":"String","value":"0"},{"id":"ios_psn","type":"String","value":"n"},{"id":"top_billboard_count","type":"String","value":"1","chainId":"_all_"},{"id":"top_gr_auto_model","type":"String","value":"0","chainId":"_all_"},{"id":"top_lowup","type":"String","value":"1","chainId":"_all_"},{"id":"top_vd_gender","type":"String","value":"0","chainId":"_all_"},{"id":"ios_apm","type":"String","value":"1"},{"id":"ios_hybrid_editor_v3","type":"String","value":"0"},{"id":"adr_spic","type":"String","value":"0"},{"id":"se_product_rank_list","type":"String","value":"0","chainId":"_all_"},{"id":"top_distinction","type":"String","value":"0","chainId":"_all_"},{"id":"top_feedre_rtt","type":"String","value":"41","chainId":"_all_"},{"id":"adr_member_switch","type":"String","value":"0"},{"id":"adr_new_roundtable","type":"String","value":"false"},{"id":"ios_x_z_c0","type":"String","value":"1"},{"id":"top_root_few_topic","type":"String","value":"0","chainId":"_all_"},{"id":"adr_question_invite_v2","type":"String","value":"0"},{"id":"top_quality","type":"String","value":"0","chainId":"_all_"},{"id":"top_rank","type":"String","value":"0","chainId":"_all_"},{"id":"hb_liguangyi_test","type":"String","value":"1"},{"id":"ios_vid_qt","type":"String","value":"0"},{"id":"adr_ans_video","type":"String","value":"N"},{"id":"adr_feed_video_continuous","type":"String","value":"0"},{"id":"ios_searchbox","type":"String","value":"0"},{"id":"pin_ef","type":"String","value":"orig","chainId":"_all_"},{"id":"top_dtmt","type":"String","value":"2","chainId":"_all_"},{"id":"top_free_content","type":"String","value":"-1","chainId":"_all_"},{"id":"top_gif","type":"String","value":"0","chainId":"_all_"},{"id":"top_topic_feedre","type":"String","value":"21","chainId":"_all_"},{"id":"adr_cta","type":"String","value":"0"},{"id":"adr_edit_question","type":"String","value":"0"},{"id":"hb_unfollow_reason","type":"String","value":"0"},{"id":"ios_7324","type":"String","value":"0"},{"id":"ios_consultation","type":"String","value":"0"},{"id":"ios_lans","type":"String","value":"open"},{"id":"top_recall_core_interest","type":"String","value":"81","chainId":"_all_"},{"id":"top_recall_tb_long","type":"String","value":"51","chainId":"_all_"},{"id":"ios_video_upload_cdn","type":"String","value":"1"},{"id":"top_hca","type":"String","value":"0","chainId":"_all_"},{"id":"top_recall_tb_follow","type":"String","value":"71","chainId":"_all_"},{"id":"ios_q_o_b","type":"String","value":"0"},{"id":"se_cm","type":"String","value":"0","chainId":"_all_"},{"id":"top_gr_model","type":"String","value":"0","chainId":"_all_"},{"id":"web_card_style","type":"String","value":"b"},{"id":"web_heifetz_column_api2","type":"String","value":"0"},{"id":"ios_launch_timeout","type":"String","value":"-1"},{"id":"ios_question_invite_v2","type":"String","value":"0"},{"id":"ios_quill_editor","type":"String","value":"0"},{"id":"ios_wbtp","type":"String","value":"1"},{"id":"top_alt","type":"String","value":"0","chainId":"_all_"},{"id":"top_feedre_itemcf","type":"String","value":"31","chainId":"_all_"},{"id":"ios_profile_sig","type":"String","value":"true"},{"id":"zr_ans_rec","type":"String","value":"gbrank","chainId":"_all_"},{"id":"adr_article_new_comment","type":"String","value":"0"},{"id":"adr_use_gd","type":"String","value":"n"},{"id":"hb_live_btn_color","type":"String","value":"default_color"},{"id":"ios_book_is_card","type":"String","value":"1"},{"id":"ios_lssq","type":"String","value":"0"},{"id":"top_gr_topic_reweight","type":"String","value":"0","chainId":"_all_"},{"id":"ios_article_new_comment","type":"String","value":"0"},{"id":"top_hweb","type":"String","value":"0","chainId":"_all_"},{"id":"adr_article_misc","type":"String","value":"0"},{"id":"top_pfq","type":"String","value":"0","chainId":"_all_"},{"id":"adr_wbtp","type":"String","value":"1"},{"id":"ios_video_feed","type":"String","value":"1"},{"id":"top_cc_at","type":"String","value":"1","chainId":"_all_"},{"id":"top_nuc","type":"String","value":"0","chainId":"_all_"},{"id":"top_wonderful","type":"String","value":"1","chainId":"_all_"},{"id":"adr_medal","type":"String","value":"0"},{"id":"adr_new_answer_pager","type":"String","value":"false"},{"id":"adr_ques_comment","type":"String","value":"0"},{"id":"adr_unfollow_reason","type":"String","value":"0"},{"id":"hb_best_answerer","type":"String","value":"0"},{"id":"adr_osen_label","type":"String","value":"old"},{"id":"hb_entity_ui","type":"String","value":"origin"},{"id":"ios_amap","type":"String","value":"y"},{"id":"ios_le_nav","type":"String","value":"0"},{"id":"top_login_card","type":"String","value":"1","chainId":"_all_"},{"id":"top_retag","type":"String","value":"0","chainId":"_all_"},{"id":"ios_profile_badge","type":"String","value":"true"},{"id":"ios_question_answer_preload","type":"String","value":"0"},{"id":"se_wiki_box","type":"String","value":"1","chainId":"_all_"},{"id":"top_new_user_gift","type":"String","value":"0","chainId":"_all_"},{"id":"top_recall_tb","type":"String","value":"1","chainId":"_all_"},{"id":"adr_zmcore","type":"String","value":"1"},{"id":"hb_billboard","type":"String","value":"0"},{"id":"hb_majorob_style","type":"String","value":"0"},{"id":"ios_article_recommend_column","type":"String","value":"0"},{"id":"top_tagore_topic","type":"String","value":"0","chainId":"_all_"},{"id":"adr_use_cashier","type":"String","value":"new"},{"id":"hb_recommend_column","type":"String","value":"0"},{"id":"ios_vid_home","type":"String","value":"0"},{"id":"se_dl","type":"String","value":"0","chainId":"_all_"},{"id":"top_billupdate1","type":"String","value":"2","chainId":"_all_"},{"id":"top_tmt","type":"String","value":"0","chainId":"_all_"},{"id":"ios_km_center","type":"String","value":"0"},{"id":"ios_topsearch","type":"String","value":"1"},{"id":"se_major_onebox","type":"String","value":"major","chainId":"_all_"},{"id":"tp_sft","type":"String","value":"a","chainId":"_all_"},{"id":"adr_bugly","type":"String","value":"n"},{"id":"hb_report","type":"String","value":"0"},{"id":"ios_pay_view","type":"String","value":"new"},{"id":"top_fqa","type":"String","value":"0","chainId":"_all_"},{"id":"top_nad","type":"String","value":"1","chainId":"_all_"},{"id":"adr_test_delete","type":"String","value":"0"},{"id":"adr_wxfb","type":"String","value":"0"},{"id":"ios_wxbk","type":"String","value":"1"},{"id":"se_gemini_service","type":"String","value":"content","chainId":"_all_"},{"id":"top_retagg","type":"String","value":"0","chainId":"_all_"},{"id":"top_sjre","type":"String","value":"0","chainId":"_all_"},{"id":"top_tffrt","type":"String","value":"0","chainId":"_all_"},{"id":"top_uit","type":"String","value":"0","chainId":"_all_"},{"id":"top_vds_alb_pos","type":"String","value":"0","chainId":"_all_"},{"id":"tp_discussion_feed_type_android","type":"String","value":"0","chainId":"_all_"},{"id":"adr_android_launch_ad_mp4","type":"String","value":"open"},{"id":"adr_book_is_card","type":"String","value":"0"},{"id":"web_question_invite","type":"String","value":"N"},{"id":"adr_q_bar","type":"String","value":"NO"},{"id":"ios_notif_new_invite","type":"String","value":"off"},{"id":"ios_q_bar","type":"String","value":"NO"},{"id":"adr_local_server_enable","type":"String","value":"0"},{"id":"adr_prt","type":"String","value":"false"},{"id":"se_ingress","type":"String","value":"on","chainId":"_all_"},{"id":"se_merger","type":"String","value":"1","chainId":"_all_"},{"id":"top_card","type":"String","value":"-1","chainId":"_all_"},{"id":"top_feedtopiccard","type":"String","value":"0","chainId":"_all_"},{"id":"top_keywordab","type":"String","value":"0","chainId":"_all_"},{"id":"top_nmt","type":"String","value":"0","chainId":"_all_"},{"id":"se_auto_syn","type":"String","value":"0","chainId":"_all_"},{"id":"top_recall","type":"String","value":"1","chainId":"_all_"},{"id":"top_tagore","type":"String","value":"1","chainId":"_all_"},{"id":"top_vdio_rew","type":"String","value":"0","chainId":"_all_"},{"id":"web_column_auto_invite","type":"String","value":"0"},{"id":"ios_adr_vid_vol","type":"String","value":"0"},{"id":"ios_q_a_c","type":"String","value":"0"},{"id":"top_30","type":"String","value":"0","chainId":"_all_"},{"id":"top_tr","type":"String","value":"0","chainId":"_all_"},{"id":"adr_mini","type":"String","value":"1"},{"id":"adr_traffic_threshold","type":"String","value":"314572800"},{"id":"se_refactored_search_index","type":"String","value":"0","chainId":"_all_"},{"id":"top_feedre","type":"String","value":"1","chainId":"_all_"},{"id":"adr_android_gdt","type":"String","value":"open"},{"id":"adr_rlp","type":"String","value":"0"},{"id":"hb_consulting_price","type":"String","value":"np"},{"id":"ios_video_continuous","type":"String","value":"0"},{"id":"top_f_r_nb","type":"String","value":"1","chainId":"_all_"},{"id":"top_root_mg","type":"String","value":"1","chainId":"_all_"},{"id":"tp_discussion_feed_card_type","type":"String","value":"0","chainId":"_all_"},{"id":"adr_mqtt_5_24_0","type":"String","value":"0"},{"id":"adr_traffic_leak","type":"String","value":"false"},{"id":"top_keyword","type":"String","value":"0","chainId":"_all_"},{"id":"top_test_4_liguangyi","type":"String","value":"1","chainId":"_all_"},{"id":"adr_android_video_continuous","type":"String","value":"0"},{"id":"adr_editor_version","type":"String","value":"V2"},{"id":"adr_pdf","type":"String","value":"n"},{"id":"adr_task_statistics","type":"String","value":"false"},{"id":"ios_article_misc_panel","type":"String","value":"0"},{"id":"adr_dkts","type":"String","value":"200"},{"id":"adr_next_answer_btn","type":"String","value":"0"},{"id":"ios_answer_hybrid_preload","type":"String","value":"0"},{"id":"ios_input_image","type":"String","value":"0"},{"id":"top_newfollow","type":"String","value":"0","chainId":"_all_"},{"id":"top_raf","type":"String","value":"n","chainId":"_all_"},{"id":"top_recall_tb_short","type":"String","value":"61","chainId":"_all_"},{"id":"top_spec_promo","type":"String","value":"1","chainId":"_all_"},{"id":"top_an","type":"String","value":"0","chainId":"_all_"},{"id":"top_tuner_refactor","type":"String","value":"-1","chainId":"_all_"},{"id":"top_universalebook","type":"String","value":"1","chainId":"_all_"},{"id":"top_video_rew","type":"String","value":"0","chainId":"_all_"},{"id":"adr_launch_ad_new_strategy","type":"String","value":"open"},{"id":"adr_video_upload_cdn","type":"String","value":"1"},{"id":"ios_mini","type":"String","value":"1"},{"id":"se_relevant_query","type":"String","value":"old","chainId":"_all_"},{"id":"top_billab","type":"String","value":"0","chainId":"_all_"},{"id":"top_memberfree","type":"String","value":"1","chainId":"_all_"},{"id":"top_video_fix_position","type":"String","value":"0","chainId":"_all_"},{"id":"adr_profile_medal","type":"String","value":"0"},{"id":"adr_real_time_launch_http","type":"String","value":"http_off"},{"id":"ios_dkts","type":"String","value":"20"},{"id":"top_nszt","type":"String","value":"0","chainId":"_all_"},{"id":"adr_new_hybrid","type":"String","value":"0"},{"id":"adr_wxbk","type":"String","value":"1"},{"id":"ls_play_continuous_order","type":"String","value":"2","chainId":"_all_"},{"id":"top_billvideo","type":"String","value":"0","chainId":"_all_"},{"id":"top_recommend_topic_card","type":"String","value":"0","chainId":"_all_"}],"chains":[{"chainId":"_all_"}]},"triggers":{}},"userAgent":{"Edge":false,"Wechat":false,"Weibo":false,"QQ":false,"Qzone":false,"Mobile":false,"Android":false,"iOS":false,"isAppleDevice":false,"Zhihu":false,"ZhihuHybrid":false,"isBot":false,"Tablet":false,"UC":false,"Sogou":false,"Qihoo":false,"Baidu":false,"isWebView":false,"origin":"Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.62 Safari/537.36"},"trafficSource":"production","edition":{"baidu":false,"sogou":false,"baiduBeijing":false,"yidianzixun":false},"theme":"light","referer":"","conf":{},"ipInfo":{},"logged":false,"tdkInfo":{}},"me":{"accountInfoLoadStatus":{},"organizationProfileStatus":{},"columnContributions":[]},"comments":{"stickers":[],"commentWithPicPermission":{},"notificationsComments":{},"pagination":{},"collapsed":{},"reverse":{},"reviewing":{},"conversation":{},"conversationMore":{},"parent":{}},"pushNotifications":{"default":{"isFetching":false,"isDrained":false,"ids":[]},"follow":{"isFetching":false,"isDrained":false,"ids":[]},"vote_thank":{"isFetching":false,"isDrained":false,"ids":[]},"currentTab":"default","notificationsCount":{"default":0,"follow":0,"vote_thank":0}},"messages":{"data":{},"currentTab":"common","messageCount":0},"register":{"registerValidateSucceeded":null,"registerValidateErrors":{},"registerConfirmError":null,"sendDigitsError":null,"registerConfirmSucceeded":null},"login":{"loginUnregisteredError":false,"loginBindWechatError":false,"loginConfirmError":null,"sendDigitsError":null,"validateDigitsError":false,"loginConfirmSucceeded":null,"qrcodeLoginToken":"","qrcodeLoginScanStatus":0,"qrcodeLoginError":null,"qrcodeLoginReturnNewToken":false},"active":{"sendDigitsError":null,"activeConfirmSucceeded":null,"activeConfirmError":null},"switches":{},"captcha":{"captchaNeeded":false,"captchaValidated":false,"captchaBase64String":null,"captchaValidationMessage":null,"loginCaptchaExpires":false},"sms":{"supportedCountries":[]},"coupon":{"isRedeemingCoupon":false},"question":{"followers":{},"concernedFollowers":{},"answers":{},"hiddenAnswers":{},"createdAnswers":{},"collapsedAnswers":{},"notificationAnswers":{},"invitationCandidates":{},"inviters":{},"invitees":{},"similarQuestions":{},"relatedCommodities":{},"recommendReadings":{},"bio":{},"brand":{},"permission":{},"advancedStyle":{},"commonAnswerCount":0,"hiddenAnswerCount":0,"meta":{},"autoInvitation":{},"simpleConcernedFollowers":{}},"shareTexts":{},"answers":{"voters":{},"copyrightApplicants":{},"favlists":{},"newAnswer":{},"concernedUpvoters":{},"simpleConcernedUpvoters":{}},"banner":{},"topic":{"bios":{},"hot":{},"newest":{},"top":{},"unanswered":{},"questions":{},"followers":{},"contributors":{},"parent":{},"children":{},"bestAnswerers":{},"wikiMeta":{},"index":{},"intro":{},"meta":{},"schema":{}},"explore":{"recommendations":{}},"articles":{"voters":{}},"favlists":{"relations":{}},"pins":{"voters":{}},"topstory":{"topstorys":{"isFetching":false,"isDrained":false,"afterId":0,"items":[],"next":null},"recommend":{"isFetching":false,"isDrained":false,"afterId":0,"items":[],"next":null},"follow":{"isFetching":false,"isDrained":false,"afterId":0,"items":[],"next":null},"followWonderful":{"isFetching":false,"isDrained":false,"afterId":0,"items":[],"next":null},"sidebar":null,"announcement":{},"hotList":[],"guestFeeds":{"isFetching":false,"isDrained":false,"afterId":0,"items":[],"next":null},"followExtra":{"isNewUser":null,"isFetched":false,"followCount":0,"followers":[]}},"upload":{},"video":{"data":{}},"guide":{"guide":{"isFetching":false,"isShowGuide":false}},"reward":{"answer":{},"article":{},"question":{}},"search":{"recommendSearch":[],"topSearch":{},"attachedInfo":{},"nextOffset":{},"generalByQuery":{},"generalByQueryInADay":{},"generalByQueryInAWeek":{},"generalByQueryInThreeMonths":{},"peopleByQuery":{},"topicByQuery":{},"columnByQuery":{},"liveByQuery":{},"albumByQuery":{},"eBookByQuery":{}},"creator":{"tools":{"question":{"invitationCount":{"questionFolloweeCount":0,"questionTotalCount":0},"goodatTopics":[]},"customPromotion":{"itemLists":{}},"recommend":{"recommendTimes":{}}},"explore":{"academy":{"tabs":[],"article":{}}},"rights":[],"rightsStatus":{},"levelUpperLimit":10,"account":{"growthLevel":{}}},"publicEditPermission":{},"readStatus":{}}" data-config="{}" data-useragent="{"os":{"name":"Windows","version":"8.1"},"browser":{"name":"Chrome","version":"67.0.3396.62","major":"67"}}"></div><script src="https://static.zhihu.com/heifetz/vendor.7c9abc3e398528f8abf1.js"></script><script src="https://static.zhihu.com/heifetz/main.app.cea9c613fa1bc3a7bdfa.js"></script><script></script></body></html>
很明显能够看得到这个是未登录状态的。
模拟登陆的实现方法
1，使用selenium登陆模拟
首先理一下流程：
① 初始化ChromeDriver
② 打开知乎登陆界面
③ 找到用户名和密码的输入框
④ 手动单击验证码
⑤ 按下enter键
# -*- coding:UTF-8 -*- __autor__ = ‘zhouli‘ __date__ = ‘2018/11/2 21:40‘ # 模拟登陆 from selenium import webdriver from selenium.webdriver.common.keys import Keys import time drive = webdriver.Chrome() drive.get(‘https://www.zhihu.com/signup‘) drive.find_element_by_xpath(‘//*[@id="root"]/div/main/div/div/div/div[2]/div[2]/span‘).click() name = drive.find_element_by_name(‘username‘) name.clear() name.send_keys(‘17621367901‘) pwd = drive.find_element_by_name(‘password‘) pwd.clear() pwd.send_keys(‘*****‘) name.send_keys(Keys.ENTER) # 模拟键盘的回车键 input(‘输入验证码后回到这里按任意继续‘) drive.find_element_by_xpath(‘//*[@id="root"]/div/main/div/div/div/div[2]/div[1]/form/button‘).click() time.sleep(10) print(drive.page_source) drive.quit()
但是使用selenium来登录实在是太慢了，如果一个网页处理需要十几秒的话，而且如果服务器使用PhantomJS作为WebDriver还会出现内存泄漏的情况，爬虫就会轻轻松松干爆服务器内存，因此不适合大规模爬虫开发。
2，使用cookies登录
为了不让用户每次都进行登陆操作，浏览器会在用户第一次登录成功之后就会放一段加密的信息在cookies中。下一次用户的访问，网站会检查cookies有没有这个加密信息，如果有并且合法的话，就可以跳过登录页面，直接进入登陆后的页面
通过已经登录的cookies，可以让爬虫绕过登录过程，直接进入登录以后的页面。
在已经登录知乎的情况下，打开Chrome，定位到network选项卡，刷新网页，在加载的内容中任意选择一项，查看右侧数据，从request headers中可以找到cookies，不是response headers。
只要把request headers的内容通过requests提交，就能够直接登录登录之后的页面了，
import requests headers = { ‘user-agent‘: ‘Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.62 Safari/537.36‘, ‘accept-encoding‘: ‘deflate, br‘, ‘cookie‘: ‘_xsrf=vsOFdtc3Me0zrt7Md6BBbpt1M5lZRomC; _zap=cacb117d-afa3-4ec7-8f18-15f0a0c7148c; d_c0="AGAoGxo-dQ6PTpzLozGZd_Wg3U95hvGSCE8=|1541166185"; tst=r; q_c1=294aac80f4bd49b390b2df011fdf4fee|1541168123000|1541168123000; capsion_ticket="2|1:0|10:1541169361|14:capsion_ticket|44:MTUxYzY4ZjdkZDNkNGVkMGFiZmI4MzhkZjMzODVhOTk=|676622142717c3f7f31aa9b50e8341ddee2b80087fb52826ce417a9778247807"; z_c0="2|1:0|10:1541169370|4:z_c0|92:Mi4xOTkxekJnQUFBQUFBWUNnYkdqNTFEaVlBQUFCZ0FsVk4ycTdKWEFDY1NSak9wb3pXV3VwTkd4Ukx3UVNJNlRpcHNn|8c6776043bf64c2640d7db50a4290afc2a9152460042b0be512f2a948ac1572c"; tgw_l7_route=69f52e0ac392bb43ffb22fc18a173ee6‘ } session = requests.Session() source = session.get(url=‘https://www.zhihu.com/‘, headers=headers, verify=False) source.encoding=‘utf-8‘ print(source.text)
使用了requests的Session模块。网站会吧每一个会话的ID（Session ID）保存在浏览器的Cookies中标识用户的身份，requests的Session模块会自动的保存网页的一些回复信息，当然也可以通过requests.get()方式去访问，但是每一次调用requests.get（）方法都回去创建一个新的ssession，对于浏览器而言就相当于每一次都是开了一个新的浏览器去访问，。如果直接使用session模块，那么每一次都会用这个session去访问，对于服务器而言，这样相当于在一个浏览器的窗口通过单击进入其他的页面，这个行为更像是人的行为，在官方文档中也建议，如果多次对同一个网站进行请求的话，应该使用session模块，会带来显著的性能提升。对于HTTPS的网站，在使用requests的时候需要带上verify=False这个参数，忽略掉网站证书的警告。
login_url = ‘zzzzzzzzzzzzzzzzzzz‘ login_url_success = ‘xxxxxx‘ data = { ‘username‘: ‘zhouli‘, ‘password‘: ‘zhouli‘, ‘remember‘: ‘YES‘, } session = requests.Session() session.post(login_url, data=data).text
在代码中实例化了一个session的对象，可以看做是新开了一个浏览器，然后data传参提交数据。
标签：提交数据 static mp4 custom rtb Opens state Once als
原文地址：https://www.cnblogs.com/zhoulixiansen/p/9902333.html
