自动化无聊的东西(无法获取BeautifulSoup来解析HTML)

此脚本使用一个或多个关键字,然后在Google上进行搜索,然后在浏览器中打开结果的标签。该脚本在select方法处返回一个空数组,我对为什么感到困惑。我检查了搜索结果的HTML,CSS选择器似乎应该可以工作。

#! /usr/bin/env python3

import webbrowser,sys,requests,bs4,pyperclip

if len(sys.argv) > 1:
    address = ' '.join(sys.argv[1:])
else:
    address = pyperclip.paste()

res = requests.get('https://www.google.com/search?q=' + address)

soup = bs4.BeautifulSoup(res.text,"lxml")

linkElems = soup.select('.r a')
numOpen = min(5,len(linkElems))
for i in range(numOpen):
    webbrowser.open('http://google.com' + linkElems[i].get('href'))
qq751046302 回答:自动化无聊的东西(无法获取BeautifulSoup来解析HTML)

尝试在标题中设置User-Agent

from bs4 import BeautifulSoup
import requests

url = "https://www.google.com/search?q=python"

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:18.0) Gecko/20100101 Firefox/18.0"
}

response = requests.get(url,headers=headers)
assert response.status_code == 200

soup = BeautifulSoup(response.text,"html.parser")

for element in soup.select(".r a"):
    print(element)
本文链接:https://www.f2er.com/3163493.html

大家都在问