因此，在下面的代码中，我想打开一个公寓网站的网址并抓取该网页。 唯一的问题是，Beautiful Soup不会等到整个网页都呈现出来。 直到将它们加载到页面上之前，这些公寓才会在html中呈现，这需要几秒钟的时间。我该如何解决？
from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup
my_url = 'https://xxxxx.com/properties/?sort=latest'
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()
page_soup = soup(page_html,"html.parser")
containers = page_soup.findAll("div",{"class":"grid-item"})
#len(containers) is empty since the contents haven't been loaded yet!
如果您要等待页面完全加载其数据，则应考虑使用硒，在您的情况下，它看起来可能像这样：
List[BaseAnimal]我对$query_1 = "some_mysql_code_that_will_insert_data_to_table_a";
$stmt_1 = $this->conn->prepare($query_1);
if ($stmt_1->execute()) {
// run this code under
$p_id = $this->conn->lastInsertId(); // save Jerry's ID here first
$query_2 = "
INSERT INTO table_b
SET
p_id=:p_id,b_title = :b_title,b_genre = :b_genre
";
$stmt_2 = $this->conn->prepare($query);
foreach($p_name AS $key => $value) {
$b_title = $value;
$b_genre = $b_genre[$key];
$stmt_2->bindParam(':p_id',$p_id);
$stmt_2->bindParam(':b_title',$b_title);
$stmr_2->bindParam(':b_genre',$b_genre);
$stmt_2->execute();
}
}
库感到满意。它将为您呈现动态HTML。而且比Selenium更容易实现。
requests_html