我正在将一个旧的PHP项目迁移到一个新的Laravel应用中。我打算分批迁移几百万个记录表user_category_views
。我使用mysqli
获取了旧记录,并使用Laravel DB::Statement
进行了插入。出于某种原因,大约一百万条记录后,该代码将异常掉:
致命错误:允许的内存大小为268435456字节已用尽(试图分配73728字节)
这里的内存溢出了什么?也许$result->free()
不能按照我的方式工作?
$count = 2000000; // actual number will be received from count(*) stmt
$vendors = [561 => '618',784 => '512' /* and so on */ ];
$step = 5000;
for( $i=0; $i<=$count; $i+=$step ){
$q = "SELECT * FROM `user_category_views` LIMIT $i,$step;";
if ($result = $this->mysqli->query($q)) {
$stmt = "INSERT INTO vendor_views (`vendor_id`,`counter`,`created_at`) VALUES";
/* fetch associative array */
while ($row = $result->fetch_assoc()) {
$vendor_id = null;
$id = $row['user_category_id'];
// Here I'm trying to prevent Laravel
// from throwing the exception if the entry
// is not found in $vendors array.
// This habit I've gained from js coding
try{
$vendor_id = $vendors[$id];
} catch (Exception $e) {
continue;
}
if(empty($vendor_id)) continue;
$counter = $row['counter'];
$created = $row['created'] ;
$stmt .= " ($vendor_id,$counter,'{$created}'),";
}
$result->free();
DB::statement( trim($stmt,",") );
$stmt = null;
}
}