码迷，mamicode.com
首页 > 编程语言 > 详细

Python的Web编程[0] -> Web客户端[1] -> Web 页面解析

时间：2018-01-08 22:30:09      阅读：284      评论：0      收藏：0      [点我收藏+]

标签：UI   family   tin   https   dao   rsh   方法   功能   nts   

 Web页面解析 / Web page parsing

 

1 HTMLParser解析

下面介绍一种基本的Web页面HTML解析的方式，主要是利用Python自带的html.parser模块进行解析。其主要步骤为:

  1. 创建一个新的Parser类，继承HTMLParser类;
  2. 重载handler_starttag等方法，实现指定功能;
  3. 实例化新的Parser并将HTML文本feed给类实例。

 完整代码

 1 from html.parser import HTMLParser
 2 
 3 # An HTMLParser instance is fed HTML data and calls handler methods when start tags, end tags, text, comments, and other markup elements are encountered
 4 # Subclass HTMLParser and override its methods to implement the desired behavior
 5 
 6 class MyHTMLParser(HTMLParser):
 7     # attrs is the attributes set in HTML start tag
 8     def handle_starttag(self, tag, attrs):
 9         print(‘Encountered a start tag:‘, tag)
10         for attr in attrs:
11             print(‘     attr:‘, attr)
12 
13     def handle_endtag(self, tag):
14         print(‘Encountered an end tag :‘, tag)
15 
16     def handle_data(self, data):
17         print(‘Encountered some data  :‘, data)
18 
19 parser = MyHTMLParser()
20 parser.feed(‘<html><head><title>Test</title></head>‘
21             ‘<body><h1>Parse me!</h1></body></html>‘
22             ‘<img src="python-logo.png" alt="The Python logo">‘)

代码中首先对模块进行导入，派生一个新的 Parser 类，随后重载方法，当遇到起始tag时，输出并判断是否有定义属性，有则输出，遇到终止tag与数据时同样输出。

 

Note: handle_starttag()函数的attrs为由该起始tag属性组成的元组元素列表，即列表中包含元组，元组中第一个参数为属性名，第二个参数为属性值。

 

 

输出结果

Encountered a start tag: html  
Encountered a start tag: head  
Encountered a start tag: title  
Encountered some data  : Test  
Encountered an end tag : title  
Encountered an end tag : head  
Encountered a start tag: body  
Encountered a start tag: h1  
Encountered some data  : Parse me!  
Encountered an end tag : h1  
Encountered an end tag : body  
Encountered an end tag : html  
Encountered a start tag: img  
     attr: (‘src‘, ‘python-logo.png‘)  
     attr: (‘alt‘, ‘The Python logo‘)

从输出中可以看到，解析器将HTML文本进行了解析，并且输出了tag中包含的属性。

 

2 BeautifulSoup解析

接下来介绍一种第三方的HTML页面解析包BeautifulSoup，同时与HTMLParser进行对比。

首先需要进行BeautifulSoup的安装，安装方式如下，

pip install beautifulsoup4

完整代码

技术分享图片 
 1 from html.parser import HTMLParser
 2 from io import StringIO
 3 from urllib import request
 4 
 5 from bs4 import BeautifulSoup, SoupStrainer
 6 from html5lib import parse, treebuilders
 7 
 8 
 9 URLs = (‘http://python.org‘,
10         ‘http://www.baidu.com‘)
11 
12 def output(x):
13     print(‘\n‘.join(sorted(set(x))))
14 
15 def simple_beau_soup(url, f):
16     ‘simple_beau_soup() - use BeautifulSoup to parse all tags to get anchors‘
17     # BeautifulSoup returns a BeautifulSoup instance
18     # find_all function returns a bs4.element.ResultSet instance, 
19     # which contains bs4.element.Tag instances,
20     # use tag[‘attr‘] to get attribute of tag
21     output(request.urljoin(url, x[‘href‘]) for x in BeautifulSoup(markup=f, features=‘html5lib‘).find_all(‘a‘))
22 
23 def faster_beau_soup(url, f):
24     ‘faster_beau_soup() - use BeautifulSoup to parse only anchor tags‘
25     # Add find_all(‘a‘) function
26     output(request.urljoin(url, x[‘href‘]) for x in BeautifulSoup(markup=f, features=‘html5lib‘, parse_only=SoupStrainer(‘a‘)).find_all(‘a‘))
27 
28 def htmlparser(url, f):
29     ‘htmlparser() - use HTMLParser to parse anchor tags‘
30     class AnchorParser(HTMLParser):
31         def handle_starttag(self, tag, attrs):
32             if tag != ‘a‘:
33                 return
34             if not hasattr(self, ‘data‘):
35                 self.data = []
36             for attr in attrs:
37                 if attr[0] == ‘href‘:
38                     self.data.append(attr[1])
39     parser = AnchorParser()
40     parser.feed(f.read())
41     output(request.urljoin(url, x) for x in parser.data)
42     print(‘DONE‘)
43     
44 def html5libparse(url, f):
45     ‘html5libparse() - use html5lib to parser anchor tags‘
46     #output(request.urljoin(url, x.attributes[‘href‘]) for x in parse(f) if isinstance(x, treebuilders.etree.Element) and x.name == ‘a‘)
47 
48 def process(url, data):
49     print(‘\n*** simple BeauSoupParser‘)
50     simple_beau_soup(url, data)
51     data.seek(0)
52     print(‘\n*** faster BeauSoupParser‘)
53     faster_beau_soup(url, data)
54     data.seek(0)
55     print(‘\n*** HTMLParser‘)
56     htmlparser(url, data)
57     data.seek(0)
58     print(‘\n*** HTML5lib‘)
59     html5libparse(url, data)
60     data.seek(0)
61 
62 if __name__==‘__main__‘:
63     for url in URLs:
64         f = request.urlopen(url)
65         data = StringIO(f.read().decode())
66         f.close()
67         process(url, data)
View Code

分段解释

首先将所需模块进行导入，其中StringIO模块用来实现字符串缓存容器，

 1 from html.parser import HTMLParser
 2 from io import StringIO
 3 from urllib import request
 4 
 5 from bs4 import BeautifulSoup, SoupStrainer
 6 from html5lib import parse, treebuilders
 7 
 8 
 9 URLs = (‘http://python.org‘,
10         ‘http://www.baidu.com‘)

接着定义一个输出函数，利用集合消除重复参数同时进行换行分离。

1 def output(x):
2     print(‘\n‘.join(sorted(set(x))))

此处定义一个简单的bs解析函数，首先利用BeautifulSoup类传入HTML文本以及features(新版提示使用‘html5lib’)，生成一个BeautifulSoup实例，再利用find_all()函数返回所有tag为‘a’的链接锚集合类(bs4.element.Tag)，通过Tag获取href属性，最后利用urljoin函数生成链接并输出。

1 def simple_beau_soup(url, f):
2     ‘simple_beau_soup() - use BeautifulSoup to parse all tags to get anchors‘
3     # BeautifulSoup returns a BeautifulSoup instance
4     # find_all function returns a bs4.element.ResultSet instance, 
5     # which contains bs4.element.Tag instances,
6     # use tag[‘attr‘] to get attribute of tag
7     output(request.urljoin(url, x[‘href‘]) for x in BeautifulSoup(markup=f, features=‘html5lib‘).find_all(‘a‘))

接着定义一个新的解析函数，这个函数可以通过参数传入parse_only来设置需要解析的锚标签，从而加快解析的速度。

Note: 这部分存在一个问题，当使用‘html5lib’特性时，是不支持parse_only参数的，因此会对整个标签进行搜索。有待解决。

1 def faster_beau_soup(url, f):
2     ‘faster_beau_soup() - use BeautifulSoup to parse only anchor tags‘
3     # Add find_all(‘a‘) function
4     output(request.urljoin(url, x[‘href‘]) for x in BeautifulSoup(markup=f, features=‘html5lib‘, parse_only=SoupStrainer(‘a‘)).find_all(‘a‘))

再定义一个用html方式进行解析的函数，可参见前节使用方式，首先建立一个锚解析的类，在遇到起始标签时，判断是否为‘a’锚，在进入时判断是否有data属性，没有的话初始化属性为空，随后对attrs参数遍历，获取href参数。最后生成实例并feed数据。

 1 def htmlparser(url, f):
 2     ‘htmlparser() - use HTMLParser to parse anchor tags‘
 3     class AnchorParser(HTMLParser):
 4         def handle_starttag(self, tag, attrs):
 5             if tag != ‘a‘:
 6                 return
 7             if not hasattr(self, ‘data‘):
 8                 self.data = []
 9             for attr in attrs:
10                 if attr[0] == ‘href‘:
11                     self.data.append(attr[1])
12     parser = AnchorParser()
13     parser.feed(f.read())
14     output(request.urljoin(url, x) for x in parser.data)
15     print(‘DONE‘)

最后定义一个process函数，对于传入的data，每次使用完后都需要seek(0)将光标移回初始。

 1 def process(url, data):
 2     print(‘\n*** simple BeauSoupParser‘)
 3     simple_beau_soup(url, data)
 4     data.seek(0)
 5     print(‘\n*** faster BeauSoupParser‘)
 6     faster_beau_soup(url, data)
 7     data.seek(0)
 8     print(‘\n*** HTMLParser‘)
 9     htmlparser(url, data)
10     data.seek(0)
11     print(‘\n*** HTML5lib‘)
12     html5libparse(url, data)
13     data.seek(0)

最终解析的结果为网页内所有的链接。

1 if __name__==‘__main__‘:
2     for url in URLs:
3         f = request.urlopen(url)
4         data = StringIO(f.read().decode())
5         f.close()
6         process(url, data)

运行输出结果

技术分享图片 
*** simple BeauSoupParser
http://blog.python.org
http://bottlepy.org
http://brochure.getpython.info/
http://buildbot.net/
http://docs.python.org/3/tutorial/
http://docs.python.org/3/tutorial/controlflow.html
http://docs.python.org/3/tutorial/controlflow.html#defining-functions
http://docs.python.org/3/tutorial/introduction.html#lists
http://docs.python.org/3/tutorial/introduction.html#using-python-as-a-calculator
http://feedproxy.google.com/~r/PythonInsider/~3/TmC0nYZBrz4/python-364rc1-and-370a3-now-available.html
http://feedproxy.google.com/~r/PythonInsider/~3/rMFQQbvrekU/python-364-is-now-available.html
http://feedproxy.google.com/~r/PythonInsider/~3/ubEu3XCqoFM/python-370a2-now-available-for-testing.html
http://feedproxy.google.com/~r/PythonInsider/~3/xUpvN2wKt2s/python-364rc1-and-370a3-now-available.html
http://feedproxy.google.com/~r/PythonInsider/~3/zVC80sq9s00/python-364-is-now-available.html
http://flask.pocoo.org/
http://ipython.org
http://jobs.python.org
http://pandas.pydata.org/
http://planetpython.org/
http://plus.google.com/+Python
http://pycon.blogspot.com/
http://pyfound.blogspot.com/
http://python.org
http://python.org#content
http://python.org#python-network
http://python.org#site-map
http://python.org#top
http://python.org/
http://python.org/about/
http://python.org/about/apps
http://python.org/about/apps/
http://python.org/about/gettingstarted/
http://python.org/about/help/
http://python.org/about/legal/
http://python.org/about/quotes/
http://python.org/about/success/
http://python.org/about/success/#arts
http://python.org/about/success/#business
http://python.org/about/success/#education
http://python.org/about/success/#engineering
http://python.org/about/success/#government
http://python.org/about/success/#scientific
http://python.org/about/success/#software-development
http://python.org/accounts/login/
http://python.org/accounts/signup/
http://python.org/blogs/
http://python.org/community/
http://python.org/community/awards
http://python.org/community/diversity/
http://python.org/community/forums/
http://python.org/community/irc/
http://python.org/community/lists/
http://python.org/community/logos/
http://python.org/community/merchandise/
http://python.org/community/sigs/
http://python.org/community/workshops/
http://python.org/dev/
http://python.org/dev/core-mentorship/
http://python.org/dev/peps/
http://python.org/dev/peps/peps.rss
http://python.org/doc/
http://python.org/doc/av
http://python.org/doc/essays/
http://python.org/download/alternatives
http://python.org/download/other/
http://python.org/downloads/
http://python.org/downloads/mac-osx/
http://python.org/downloads/release/python-2714/
http://python.org/downloads/release/python-364/
http://python.org/downloads/source/
http://python.org/downloads/windows/
http://python.org/events/
http://python.org/events/calendars/
http://python.org/events/python-events
http://python.org/events/python-events/543/
http://python.org/events/python-events/611/
http://python.org/events/python-events/past/
http://python.org/events/python-user-group/
http://python.org/events/python-user-group/605/
http://python.org/events/python-user-group/619/
http://python.org/events/python-user-group/620/
http://python.org/events/python-user-group/past/
http://python.org/jobs/
http://python.org/privacy/
http://python.org/psf-landing/
http://python.org/psf/
http://python.org/psf/donations/
http://python.org/psf/sponsorship/sponsors/
http://python.org/shell/
http://python.org/success-stories/
http://python.org/success-stories/industrial-light-magic-runs-python/
http://python.org/users/membership/
http://roundup.sourceforge.net/
http://tornadoweb.org
http://trac.edgewall.org/
http://twitter.com/ThePSF
http://wiki.python.org/moin/Languages
http://wiki.python.org/moin/TkInter
http://www.ansible.com
http://www.djangoproject.com/
http://www.facebook.com/pythonlang?fref=ts
http://www.pylonsproject.org/
http://www.riverbankcomputing.co.uk/software/pyqt/intro
http://www.saltstack.com
http://www.scipy.org
http://www.web2py.com/
http://www.wxpython.org/
https://bugs.python.org/
https://devguide.python.org/
https://docs.python.org
https://docs.python.org/3/license.html
https://docs.python.org/faq/
https://github.com/python/pythondotorg/issues
https://kivy.org/
https://mail.python.org/mailman/listinfo/python-dev
https://pypi.python.org/
https://status.python.org/
https://wiki.gnome.org/Projects/PyGObject
https://wiki.python.org/moin/
https://wiki.python.org/moin/BeginnersGuide
https://wiki.python.org/moin/Python2orPython3
https://wiki.python.org/moin/PythonBooks
https://wiki.python.org/moin/PythonEventsCalendar#Submitting_an_Event
https://wiki.qt.io/PySide
https://www.openstack.org
https://www.python.org/psf/codeofconduct/
javascript:;

*** faster BeauSoupParser

Warning (from warnings module):
  File "C:\Python35\lib\site-packages\bs4\builder\_html5lib.py", line 63
    warnings.warn("You provided a value for parse_only, but the html5lib tree builder doesn‘t support parse_only. The entire document will be parsed.")
UserWarning: You provided a value for parse_only, but the html5lib tree builder doesn‘t support parse_only. The entire document will be parsed.
http://blog.python.org
http://bottlepy.org
http://brochure.getpython.info/
http://buildbot.net/
http://docs.python.org/3/tutorial/
http://docs.python.org/3/tutorial/controlflow.html
http://docs.python.org/3/tutorial/controlflow.html#defining-functions
http://docs.python.org/3/tutorial/introduction.html#lists
http://docs.python.org/3/tutorial/introduction.html#using-python-as-a-calculator
http://feedproxy.google.com/~r/PythonInsider/~3/TmC0nYZBrz4/python-364rc1-and-370a3-now-available.html
http://feedproxy.google.com/~r/PythonInsider/~3/rMFQQbvrekU/python-364-is-now-available.html
http://feedproxy.google.com/~r/PythonInsider/~3/ubEu3XCqoFM/python-370a2-now-available-for-testing.html
http://feedproxy.google.com/~r/PythonInsider/~3/xUpvN2wKt2s/python-364rc1-and-370a3-now-available.html
http://feedproxy.google.com/~r/PythonInsider/~3/zVC80sq9s00/python-364-is-now-available.html
http://flask.pocoo.org/
http://ipython.org
http://jobs.python.org
http://pandas.pydata.org/
http://planetpython.org/
http://plus.google.com/+Python
http://pycon.blogspot.com/
http://pyfound.blogspot.com/
http://python.org
http://python.org#content
http://python.org#python-network
http://python.org#site-map
http://python.org#top
http://python.org/
http://python.org/about/
http://python.org/about/apps
http://python.org/about/apps/
http://python.org/about/gettingstarted/
http://python.org/about/help/
http://python.org/about/legal/
http://python.org/about/quotes/
http://python.org/about/success/
http://python.org/about/success/#arts
http://python.org/about/success/#business
http://python.org/about/success/#education
http://python.org/about/success/#engineering
http://python.org/about/success/#government
http://python.org/about/success/#scientific
http://python.org/about/success/#software-development
http://python.org/accounts/login/
http://python.org/accounts/signup/
http://python.org/blogs/
http://python.org/community/
http://python.org/community/awards
http://python.org/community/diversity/
http://python.org/community/forums/
http://python.org/community/irc/
http://python.org/community/lists/
http://python.org/community/logos/
http://python.org/community/merchandise/
http://python.org/community/sigs/
http://python.org/community/workshops/
http://python.org/dev/
http://python.org/dev/core-mentorship/
http://python.org/dev/peps/
http://python.org/dev/peps/peps.rss
http://python.org/doc/
http://python.org/doc/av
http://python.org/doc/essays/
http://python.org/download/alternatives
http://python.org/download/other/
http://python.org/downloads/
http://python.org/downloads/mac-osx/
http://python.org/downloads/release/python-2714/
http://python.org/downloads/release/python-364/
http://python.org/downloads/source/
http://python.org/downloads/windows/
http://python.org/events/
http://python.org/events/calendars/
http://python.org/events/python-events
http://python.org/events/python-events/543/
http://python.org/events/python-events/611/
http://python.org/events/python-events/past/
http://python.org/events/python-user-group/
http://python.org/events/python-user-group/605/
http://python.org/events/python-user-group/619/
http://python.org/events/python-user-group/620/
http://python.org/events/python-user-group/past/
http://python.org/jobs/
http://python.org/privacy/
http://python.org/psf-landing/
http://python.org/psf/
http://python.org/psf/donations/
http://python.org/psf/sponsorship/sponsors/
http://python.org/shell/
http://python.org/success-stories/
http://python.org/success-stories/industrial-light-magic-runs-python/
http://python.org/users/membership/
http://roundup.sourceforge.net/
http://tornadoweb.org
http://trac.edgewall.org/
http://twitter.com/ThePSF
http://wiki.python.org/moin/Languages
http://wiki.python.org/moin/TkInter
http://www.ansible.com
http://www.djangoproject.com/
http://www.facebook.com/pythonlang?fref=ts
http://www.pylonsproject.org/
http://www.riverbankcomputing.co.uk/software/pyqt/intro
http://www.saltstack.com
http://www.scipy.org
http://www.web2py.com/
http://www.wxpython.org/
https://bugs.python.org/
https://devguide.python.org/
https://docs.python.org
https://docs.python.org/3/license.html
https://docs.python.org/faq/
https://github.com/python/pythondotorg/issues
https://kivy.org/
https://mail.python.org/mailman/listinfo/python-dev
https://pypi.python.org/
https://status.python.org/
https://wiki.gnome.org/Projects/PyGObject
https://wiki.python.org/moin/
https://wiki.python.org/moin/BeginnersGuide
https://wiki.python.org/moin/Python2orPython3
https://wiki.python.org/moin/PythonBooks
https://wiki.python.org/moin/PythonEventsCalendar#Submitting_an_Event
https://wiki.qt.io/PySide
https://www.openstack.org
https://www.python.org/psf/codeofconduct/
javascript:;

*** HTMLParser
http://blog.python.org
http://bottlepy.org
http://brochure.getpython.info/
http://buildbot.net/
http://docs.python.org/3/tutorial/
http://docs.python.org/3/tutorial/controlflow.html
http://docs.python.org/3/tutorial/controlflow.html#defining-functions
http://docs.python.org/3/tutorial/introduction.html#lists
http://docs.python.org/3/tutorial/introduction.html#using-python-as-a-calculator
http://feedproxy.google.com/~r/PythonInsider/~3/TmC0nYZBrz4/python-364rc1-and-370a3-now-available.html
http://feedproxy.google.com/~r/PythonInsider/~3/rMFQQbvrekU/python-364-is-now-available.html
http://feedproxy.google.com/~r/PythonInsider/~3/ubEu3XCqoFM/python-370a2-now-available-for-testing.html
http://feedproxy.google.com/~r/PythonInsider/~3/xUpvN2wKt2s/python-364rc1-and-370a3-now-available.html
http://feedproxy.google.com/~r/PythonInsider/~3/zVC80sq9s00/python-364-is-now-available.html
http://flask.pocoo.org/
http://ipython.org
http://jobs.python.org
http://pandas.pydata.org/
http://planetpython.org/
http://plus.google.com/+Python
http://pycon.blogspot.com/
http://pyfound.blogspot.com/
http://python.org
http://python.org#content
http://python.org#python-network
http://python.org#site-map
http://python.org#top
http://python.org/
http://python.org/about/
http://python.org/about/apps
http://python.org/about/apps/
http://python.org/about/gettingstarted/
http://python.org/about/help/
http://python.org/about/legal/
http://python.org/about/quotes/
http://python.org/about/success/
http://python.org/about/success/#arts
http://python.org/about/success/#business
http://python.org/about/success/#education
http://python.org/about/success/#engineering
http://python.org/about/success/#government
http://python.org/about/success/#scientific
http://python.org/about/success/#software-development
http://python.org/accounts/login/
http://python.org/accounts/signup/
http://python.org/blogs/
http://python.org/community/
http://python.org/community/awards
http://python.org/community/diversity/
http://python.org/community/forums/
http://python.org/community/irc/
http://python.org/community/lists/
http://python.org/community/logos/
http://python.org/community/merchandise/
http://python.org/community/sigs/
http://python.org/community/workshops/
http://python.org/dev/
http://python.org/dev/core-mentorship/
http://python.org/dev/peps/
http://python.org/dev/peps/peps.rss
http://python.org/doc/
http://python.org/doc/av
http://python.org/doc/essays/
http://python.org/download/alternatives
http://python.org/download/other/
http://python.org/downloads/
http://python.org/downloads/mac-osx/
http://python.org/downloads/release/python-2714/
http://python.org/downloads/release/python-364/
http://python.org/downloads/source/
http://python.org/downloads/windows/
http://python.org/events/
http://python.org/events/calendars/
http://python.org/events/python-events
http://python.org/events/python-events/543/
http://python.org/events/python-events/611/
http://python.org/events/python-events/past/
http://python.org/events/python-user-group/
http://python.org/events/python-user-group/605/
http://python.org/events/python-user-group/619/
http://python.org/events/python-user-group/620/
http://python.org/events/python-user-group/past/
http://python.org/jobs/
http://python.org/privacy/
http://python.org/psf-landing/
http://python.org/psf/
http://python.org/psf/donations/
http://python.org/psf/sponsorship/sponsors/
http://python.org/shell/
http://python.org/success-stories/
http://python.org/success-stories/industrial-light-magic-runs-python/
http://python.org/users/membership/
http://roundup.sourceforge.net/
http://tornadoweb.org
http://trac.edgewall.org/
http://twitter.com/ThePSF
http://wiki.python.org/moin/Languages
http://wiki.python.org/moin/TkInter
http://www.ansible.com
http://www.djangoproject.com/
http://www.facebook.com/pythonlang?fref=ts
http://www.pylonsproject.org/
http://www.riverbankcomputing.co.uk/software/pyqt/intro
http://www.saltstack.com
http://www.scipy.org
http://www.web2py.com/
http://www.wxpython.org/
https://bugs.python.org/
https://devguide.python.org/
https://docs.python.org
https://docs.python.org/3/license.html
https://docs.python.org/faq/
https://github.com/python/pythondotorg/issues
https://kivy.org/
https://mail.python.org/mailman/listinfo/python-dev
https://pypi.python.org/
https://status.python.org/
https://wiki.gnome.org/Projects/PyGObject
https://wiki.python.org/moin/
https://wiki.python.org/moin/BeginnersGuide
https://wiki.python.org/moin/Python2orPython3
https://wiki.python.org/moin/PythonBooks
https://wiki.python.org/moin/PythonEventsCalendar#Submitting_an_Event
https://wiki.qt.io/PySide
https://www.openstack.org
https://www.python.org/psf/codeofconduct/
javascript:;
DONE

*** HTML5lib

*** simple BeauSoupParser
http://e.baidu.com/?refer=888
http://home.baidu.com
http://image.baidu.com/search/index?tn=baiduimage&ps=1&ct=201326592&lm=-1&cl=2&nc=1&ie=utf-8&word=
http://ir.baidu.com
http://jianyi.baidu.com/
http://map.baidu.com
http://map.baidu.com/m?word=&fr=ps01000
http://music.baidu.com/search?fr=ps&ie=utf-8&key=
http://news.baidu.com
http://news.baidu.com/ns?cl=2&rn=20&tn=news&word=
http://tieba.baidu.com
http://tieba.baidu.com/f?kw=&fr=wwwt
http://v.baidu.com
http://v.baidu.com/v?ct=301989888&rn=20&pn=0&db=0&s=25&ie=utf-8&word=
http://wenku.baidu.com/search?word=&lm=0&od=0&ie=utf-8
http://www.baidu.com/
http://www.baidu.com/cache/sethelp/help.html
http://www.baidu.com/duty/
http://www.baidu.com/gaoji/preferences.html
http://www.baidu.com/more/
http://www.beian.gov.cn/portal/registerSystemInfo?recordcode=11000002000001
http://www.hao123.com
http://xueshu.baidu.com
http://zhidao.baidu.com/q?ct=17&pn=0&tn=ikaslist&rn=10&word=&fr=wwwt
https://passport.baidu.com/v2/?login&tpl=mn&u=http%3A%2F%2Fwww.baidu.com%2F
javascript:;

*** faster BeauSoupParser
http://e.baidu.com/?refer=888
http://home.baidu.com
http://image.baidu.com/search/index?tn=baiduimage&ps=1&ct=201326592&lm=-1&cl=2&nc=1&ie=utf-8&word=
http://ir.baidu.com
http://jianyi.baidu.com/
http://map.baidu.com
http://map.baidu.com/m?word=&fr=ps01000
http://music.baidu.com/search?fr=ps&ie=utf-8&key=
http://news.baidu.com
http://news.baidu.com/ns?cl=2&rn=20&tn=news&word=
http://tieba.baidu.com
http://tieba.baidu.com/f?kw=&fr=wwwt
http://v.baidu.com
http://v.baidu.com/v?ct=301989888&rn=20&pn=0&db=0&s=25&ie=utf-8&word=
http://wenku.baidu.com/search?word=&lm=0&od=0&ie=utf-8
http://www.baidu.com/
http://www.baidu.com/cache/sethelp/help.html
http://www.baidu.com/duty/
http://www.baidu.com/gaoji/preferences.html
http://www.baidu.com/more/
http://www.beian.gov.cn/portal/registerSystemInfo?recordcode=11000002000001
http://www.hao123.com
http://xueshu.baidu.com
http://zhidao.baidu.com/q?ct=17&pn=0&tn=ikaslist&rn=10&word=&fr=wwwt
https://passport.baidu.com/v2/?login&tpl=mn&u=http%3A%2F%2Fwww.baidu.com%2F
javascript:;

*** HTMLParser
http://e.baidu.com/?refer=888
http://home.baidu.com
http://image.baidu.com/search/index?tn=baiduimage&ps=1&ct=201326592&lm=-1&cl=2&nc=1&ie=utf-8&word=
http://ir.baidu.com
http://jianyi.baidu.com/
http://map.baidu.com
http://map.baidu.com/m?word=&fr=ps01000
http://music.baidu.com/search?fr=ps&ie=utf-8&key=
http://news.baidu.com
http://news.baidu.com/ns?cl=2&rn=20&tn=news&word=
http://tieba.baidu.com
http://tieba.baidu.com/f?kw=&fr=wwwt
http://v.baidu.com
http://v.baidu.com/v?ct=301989888&rn=20&pn=0&db=0&s=25&ie=utf-8&word=
http://wenku.baidu.com/search?word=&lm=0&od=0&ie=utf-8
http://www.baidu.com/
http://www.baidu.com/cache/sethelp/help.html
http://www.baidu.com/duty/
http://www.baidu.com/gaoji/preferences.html
http://www.baidu.com/more/
http://www.beian.gov.cn/portal/registerSystemInfo?recordcode=11000002000001
http://www.hao123.com
http://xueshu.baidu.com
http://zhidao.baidu.com/q?ct=17&pn=0&tn=ikaslist&rn=10&word=&fr=wwwt
https://passport.baidu.com/v2/?login&tpl=mn&u=http%3A%2F%2Fwww.baidu.com%2F
javascript:;
DONE

*** HTML5lib
View Code

 

参考链接

《Python 核心编程 第3版》

 

Python的Web编程[0] -> Web客户端[1] -> Web 页面解析

标签：UI   family   tin   https   dao   rsh   方法   功能   nts   

原文地址：https://www.cnblogs.com/stacklike/p/8244925.html
踩
(0)
赞
(0)
   
举报
评论 一句话评论（0）
登录后才能评论！
分享档案
更多>
2021年07月29日 (22)
2021年07月28日 (40)
2021年07月27日 (32)
2021年07月26日 (79)
2021年07月23日 (29)
2021年07月22日 (30)
2021年07月21日 (42)
2021年07月20日 (16)
2021年07月19日 (90)
2021年07月16日 (35)
周排行
mamicode.com排行更多图片
更多
友情链接
兰亭集智   国之画   百度统计   站长统计   阿里云   chrome插件   新版天听网
关于我们 - 联系我们 - 留言反馈
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码！