标签：技术 win request 兴趣 max dcl 背景图 alt 编写
1.选一个自己感兴趣的主题（所有人不能雷同）。
我选择了虎扑nba的体育新闻页面，与校园新闻版面类似，爬去50页
2.用python 编写爬虫程序，从网络上爬取相关主题的数据。
利用所学知识，导入要用的类
import requests from bs4 import BeautifulSoup import jieba
审查元素，获取网页内容
3.对爬了的数据进行文本分析，生成词云。
for i in range(2294450,2294500):
pages = i;
nexturl = ‘https://voice.hupu.com/nba/%s.html‘ % (pages)
reslist = requests.get(nexturl)
reslist.encoding = ‘utf-8‘
soup_list = BeautifulSoup(reslist.text, ‘html.parser‘)
for news in soup_list.find_all(‘div‘,class_=‘artical-main-content‘):
print(news.text)
f = open(‘hpnba.txt‘, ‘a‘, encoding=‘utf-8‘)
f.write(news.text)
f.close()
def changeTitleToDict():
f = open("hpnba.txt", "r", encoding=‘utf-8‘)
str = f.read()
stringList = list(jieba.cut(str))
delWord = {"+", "/", "（", "）", "【", "】", " ", "；", "！", "、"}
stringSet = set(stringList) - delWord
title_dict = {}
for i in stringSet:
title_dict[i] = stringList.count(i)
print(title_dict)
return title_dict
4.对文本分析结果进行解释说明。
这是爬取的新闻文字内容，存放在hpnba.txt中
这是我的词云的背景图片，生成的大致样式。
词云如上
5.写一篇完整的博客，描述上述实现过程、遇到的问题及解决办法、数据分析思想及结论。
遇到的问题就是，我原本想生成五角星的词云，找了两张图片，都不行，后来换了一张小猪，就可以了，原因目前不知道。
6.最后提交爬取的全部数据、爬虫及数据分析源代码。
import requests from bs4 import BeautifulSoup import jieba from PIL import Image,ImageSequence import numpy as np import matplotlib.pyplot as plt from wordcloud import WordCloud,ImageColorGenerator for i in range(2294450,2294500): pages = i; nexturl = ‘https://voice.hupu.com/nba/%s.html‘ % (pages) reslist = requests.get(nexturl) reslist.encoding = ‘utf-8‘ soup_list = BeautifulSoup(reslist.text, ‘html.parser‘) for news in soup_list.find_all(‘div‘,class_=‘artical-main-content‘): print(news.text) f = open(‘hpnba.txt‘, ‘a‘, encoding=‘utf-8‘) f.write(news.text) f.close() def changeTitleToDict(): f = open("hpnba.txt", "r", encoding=‘utf-8‘) str = f.read() stringList = list(jieba.cut(str)) delWord = {"+", "/", "（", "）", "【", "】", " ", "；", "！", "、"} stringSet = set(stringList) - delWord title_dict = {} for i in stringSet: title_dict[i] = stringList.count(i) print(title_dict) return title_dict # 获取上面保存的字典 title_dict = changeTitleToDict() graph = np.array(title_dict) font = r‘C:\Windows\Fonts\simhei.ttf‘ image= Image.open(‘./3.jpg‘) graph = np.array(image) font=r‘C:\Windows\Fonts\simhei.TTF‘ wc = WordCloud(font_path=font,background_color=‘White‘,max_words=50,mask=graph) wc.generate_from_frequencies(title_dict) image_color = ImageColorGenerator(graph) plt.imshow(wc) plt.axis("off") plt.show()
标签：技术 win request 兴趣 max dcl 背景图 alt 编写
原文地址：https://www.cnblogs.com/BOXczx/p/8974682.html
