Python爬虫基础知识

1. Python相关库

• requests、re
• BeautifulSoup
• hackhttp

2 BeautifulSoup

2.1 解析内容

from bs4 import BeautifulSoup
soup = BeautifulSoup(html_doc)

2.2 浏览数据

soup.title
soup.title.string

2.3 BeatifulSoup正则使用

soup.find_all(name='x',attrs={'xx':re.compile('xxx')})

3 hackhttp

能发送整个数据包