python web编程

urllib

1. urllib.urlopen()
2. urllib.urlretrieve()
   urlretrieve(url, filename=None,reporthook=None,data=None)

urllib2

1. urlib2.open()
2. urlib2.Requests()

requests

发送网络请求：

r = requests.get('http://www.baidu.com')
r = requests.post ('http://www.baidu.com')
r = requests.put ('http://www.baidu.com')
r = requests.delete('http://www.baidu.com')
r= requests.head('http://www.baidu.com')
r = requests.options('http://www.ichunqiu.com')

为url传递参数

payload = {'key1': 'value1', 'key2':'value2'}
r = requests.get("http://httpbin.org/get", params=payload)
print(r.url)
http://httpbin.org/get?key2=value2&key1=value1

响应内容

r= requests.get('http://www.baidu.com')
r.text
r.encoding 'utf-8'
r.encoding = 'ISO-8859-1'

二进制响应内容

r = requests.get('http://www.baidu.com')
r.content

定制请求

url = 'http://www.baidu.com'
headers = {'content-type':'application/json'}
r= requests.get(url,headers=headers)
注：headers中可以加入cookies

爬虫练习
www.heibanke.com