Python 爬虫 cookie实战

from urllib import request
from urllib import parse
from http.cookiejar import CookieJar



headers={
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36'
}

cookieer=CookieJar()

handersa=request.HTTPCookieProcessor(cookieer)

opener=request.build_opener(handersa)

post_u = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
post_d = parse.urlencode({
    'username':'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx',
    'password':'XXXXXXXXXXXXXXXXXXXXXXXx'
})
reqa=request.Request(post_u,post_d.encode('utf-8'))
opener.open(reqa)


urls = 'https://XXXXXXXXXXXXX/cook.php?id=XXXXXXXXXXXXXXXXX'
rqw = request.Request(urls,headers=headers)
respa = opener.open(rqw)
print(respa.read().decode('utf-8'))

首先在准备爬取的网站注册用户，然后使用Google浏览器开发者工具查看源码文件

获取 headers  User-Agent

获取 post_u