我要抓取的网站是ScienceDirect。点击显示更多按钮后,即可使用该联盟关系。我可以单击它,但是单击“显示更多”按钮后无法抓取已加载的从属关系这是代码。 for循环未打印包含从属关系的dl标签
import time
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium import webdriver
from bs4 import BeautifulSoup
driver = webdriver.Firefox()
driver.get('https://www.sciencedirect.com/science/article/pii/S1571065308000656')
soup = BeautifulSoup(driver.page_source,'html.parser')
time.sleep(7)
try:
element = driver.find_element_by_css_selector('.show-hide-details.u-font-sans')
element.click()
time.sleep(15)
for data in soup.find(id='author-group'):
print(data)
print('---')
except NoSuchElementException:
pass