要执行与卷发完全相同的操作,必须执行以下操作:
import requests
import shutil
# curl -H "X-Risk-Token: $token" "https://api.nyc3.us.thisismyurl.com/vulnerabilities/download_data_zip" -o file.gz -vv
url ='https://api.thisismyurl.com/vulnerabilities/download_data_zip'
token = 'blahblahblah'
fname = "file.gz"
# 'Content-Type': 'application/json'
headers = {'X-Risk-Token': token } # exactly same header as curl.
# memory friendly way to download big files
with requests.get(url,headers=headers,stream=True) as resp:
print(res.status_code)
with open(fname,'wb') as fout:
shutil.copyfileobj(resp.raw,fout)
原始代码的
json.loads()将消耗大量RAM,并且如果没有足够的可用RAM,可能会使您的进程崩溃。
您到底想对数据做什么?
可视化数据可以通过以下方式完成:
import gzip
import shutil
with gzip.open(fname) as fin:
shutil.copyfileobj(fin,sys.stdout)
我建议仍然没有成功,我也没有得到任何信息,这说明了为什么“ quota”会导致python脚本出现问题,而不会给curl带来问题,我建议进行更多测试。
1。)下载但不存储结果(是网络被阻止/中止了吗?)
url ='https://api.thisismyurl.com/vulnerabilities/download_data_zip'
token = 'blahblahblah'
headers = {'X-Risk-Token': token } # exactly same header as curl.
# memory friendly way to download big files
with requests.get(url,stream=True) as resp:
print(res.status_code)
downloaded = 0
try:
for chunk in resp.iter_content(chunk_size=1024):
downloaded += len(chunk)
except Exception:
print("Downloaded %d bytes and got an exception" % downloaded)
raise
print("Downloaded %d bytes" % downloaded)
请检查结果是否始终相同或获得的数字是否不同
本文链接:https://www.f2er.com/3108129.html