学习urllib中Post请求

由于get请求方式比较简单，就不写了，贴一段代码作为回顾

get请求

from urllib import parse, request
import random

url = 'http://www.baidu.com/s'

keyword = input('请输入要搜的关键字：')
wd = {'wd': keyword}

encoded_wd = parse.urlencode(wd)
new_url = url + '?' + encoded_wd

print(new_url)
req = request.Request(url)
#为了防止被网站封ip，模仿浏览器访问网站
ua_list = [
    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv2.0.1) Gecko/20100101 Firefox/4.0.1",
    "Mozilla/5.0 (Windows NT 6.1; rv2.0.1) Gecko/20100101 Firefox/4.0.1",
    "Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; en) Presto/2.8.131 Version/11.11",
    "Opera/9.80 (Windows NT 6.1; U; en) Presto/2.8.131 Version/11.11",
    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11"
]

# 在User-Agent列表里随机选择一个User-Agent ;从序列中随机选取一个元素
user_agent = random.choice(ua_list)
req.add_header('User-Agent', user_agent)

response = request.urlopen(req)
print(response.read().decode('utf-8'))

Post请求

说明：post请求是利用有道翻译网页中对单词进行翻译时采用的post方式，这是前提。

1.利用抓包工具获取url

url='http://fanyi.youdao.com/translate_o?smartresult=dict&smartresult=rule '

2.编辑form表单

• 从抓包工具中抓取的部分是：

i=smart%0A&from=AUTO&to=AUTO&smartresult=dict&client=fanyideskweb&salt=1528635335125&sign=df74f2062975d33318777e2cdae0af41&doctype=json&version=2.1&keyfrom=fanyi.web&action=FY_BY_CLICKBUTTION&typoResult=false

拆分整理如下：

formbody = {
    "i": "smart",
    "from": "AUTO",
    "to": "AUTO",
    "smartresult": "dict",
    "client": "fanyideskweb",
    "doctype": "json",
    "version": "2.1",
    "keyfrom": "fanyi.web",
    "action": "FY_BY_CLICKBUTTION",
    "typoResult": "false"
}

---

此处用sublime正则表达式替换比较便捷

内容为    ^(.*)=(.*)$
替换为    "\1":"\2",

• 接下来的代码与get请求并没有什么区别。

data = urllib.parse.urlencode(formbody)
request = urllib.request.Request(url, data=data, headers=headers)

print(urllib.request.urlopen(request).read().decode('utf-8'))

这时候遇见第一个error :
TypeError: POST data should be bytes, an iterable of bytes, or a file object. It cannot be of type str.

• 搜索之，看到这个解决办法 https://blog.csdn.net/peade/article/details/49977515搞定。
  data = urllib.parse.urlencode(formbody).encode(encoding='utf-8')根据错误和解决的办法判断：应该是encode方法实现的功能string转化成bytes？？？（此处存疑，是不是有可能是编码方式变了所以可以了）
• 这个问题还有另一种解决方法：
  data=bytes(urllib.parse.urlencode(formbody),encoding='utf-8') 这个明显是转换成bytes格式。

解决完这个问题运行，返回结果是

{"errorCode":50}

又搜索，居然也有人趟过雷了，http://bbs.fishc.com/thread-96638-1-1.html，按照其中方法修改解决问题。

import urllib.request, urllib.parse

user_agent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv2.0.1) Gecko/20100101 Firefox/4.0.1"


url = 'http://fanyi.youdao.com/translate?smartresult=dict&smartresult=rule'

headers = {'User-Agent': user_agent}
formbody = {
    "i": "smart",
    "from": "AUTO",
    "to": "AUTO",
    "smartresult": "dict",
    "client": "fanyideskweb",
    "doctype": "json",
    "version": "2.1",
    "keyfrom": "fanyi.web",
    "action": "FY_BY_CLICKBUTTION",
    "typoResult": "false"
}
#data = urllib.parse.urlencode(formbody).encode(encoding='utf-8')
data=bytes(urllib.parse.urlencode(formbody),encoding='utf-8')
request = urllib.request.Request(url, data=data, headers=headers)

print(urllib.request.urlopen(request).read().decode('utf-8'))

结果是：

{"type":"EN2ZH_CN","errorCode":0,"elapsedTime":10,"translateResult":[[{"src":"smart","tgt":"聪明的"}]]}

---

对于第二个问题：为啥删除_o就没问题了我也不知道。不过可以思考一下，
http://fanyi.youdao.com/translate_o?smartresult=dict&smartresult=rule直接用鼠标点击在浏览器里显示：

{"errorCode":50}

而http://fanyi.youdao.com/translate?smartresult=dict&smartresult=rule跳转到有道。这给我自己提了个醒以后url一定要先在浏览器里打开试试