8 KiB后WinHTTP停止下载

2024-04-30 • 问答

我正在使用WinHTTP（通过HTTPS）从.jar和libraries.minecraft.net下载repo1.maven.org文件。功能如下：

// Namespace alias,all stdfs mentions in the function refer to std::filesystem
namespace stdfs = std::filesystem;

bool downloadFile(DLElement element) {
    HINTERNET session = WinHttpOpen(
        // Identify as Firefox
        L"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:10.0) Gecko/20100101 Firefox/10.0",// Don't care about proxies
        WINHTTP_accESS_TYPE_DEFAULT_Proxy,WINHTTP_NO_Proxy_NAME,WINHTTP_NO_Proxy_BYPASS,0);
    if (session == nullptr) return false;
    HINTERNET connection;

    connection = WinHttpConnect(session,element.hostname.c_str(),element.https ? INTERNET_DEFAULT_HTTPS_PORT : INTERNET_DEFAULT_HTTP_PORT,0);

    // accept `.jar` files
    const wchar_t** acceptTypes = new const wchar_t* [2];
    acceptTypes[0] = L"application/java-archive";
    acceptTypes[1] = nullptr;
    HINTERNET request = WinHttpOpenRequest(connection,L"GET",element.object.c_str(),nullptr,WINHTTP_NO_REFERER,acceptTypes,element.https ? WINHTTP_flaG_SECURE : 0);

    bool result = WinHttpSendRequest(request,WINHTTP_NO_ADDITIONAL_HEADERS,WINHTTP_NO_REQUEST_DATA,0);
    if (!result) {
        DWORD hr = GetLastError();
        checkHresult(hr,window,false);
    }
    if (!result) {
    failRet:
        WinHttpCloseHandle(request);
        WinHttpCloseHandle(connection);
        WinHttpCloseHandle(session);
        return false;
    }
    result = WinHttpReceiveResponse(request,nullptr);
    DWORD size = 0;
    DWORD downloaded = 0;
    std::vector<uint8_t> file;
    if (!result) goto failRet;
    do {
        if (!WinHttpQueryDataAvailable(request,&size)) {
            goto failRet;
        }
        uint8_t* buffer;
    alloc:
        try {
            buffer = new uint8_t[size];
        }
        catch (std::bad_alloc&) {
            MessageBox(window,lstr(VCLS_OUT_OF_MEMORY_DESC),lstr(VCLS_OUT_OF_MEMORY_TITLE),MB_ICONERROR);
            goto alloc;
        }
        // Dunno why,do I even need this memset?
        memset(buffer,size);
        if (!WinHttpReadData(request,buffer,size,&downloaded)) {
            delete[] buffer;
            checkHresult(GetLastError(),false);
            goto failRet;
        }
        size_t vectSize = file.size();
        file.reserve(vectSize + size);
        for (size_t i = vectSize; i < size; i++) {
            // Might as well rewrite this and make this more efficient
            file.push_back(buffer[i]);
        }
        delete[] buffer;

    } while (size > 0);
    WinHttpCloseHandle(request);
    WinHttpCloseHandle(connection);
    WinHttpCloseHandle(session);

    element.path.make_preferred();
    stdfs::path dir = stdfs::path(element.path).remove_filename();
    std::wstring fdirStr = dir.wstring();
    fdirStr.pop_back();
    dir = fdirStr;

tryCreateDir:
    try {
        stdfs::create_directories(mcFolderPath/dir);
    }
    catch (const std::bad_alloc&) {
        MessageBox(window,MB_ICONERROR);
        goto tryCreateDir;
    }
    catch (const stdfs::filesystem_error& e) {
        // Error handling removed for brevity

        return false;
    }

    std::basic_ofstream<uint8_t> ofs(mcFolderPath/element.path,std::ios::binary | std::ios::trunc);
    ofs.write(file.data(),file.size());
    ofs.close();

    return true;
}

DLElement的定义如下：

struct DLElement {
    std::wstring hostname; std::wstring object; stdfs::path path; std::string sha1hex;
    bool hasSha = true; bool https = false;
};

问题是，由于某种原因，此功能仅下载完全个实际文件的8 KiB，恰好是WinHTTP的缓冲区大小。这段代码是经过修改的Microsoft WinHTTP example，因此我假设它的正确性和能够读取8 KiB以上的数据的能力。我在做什么错了，为什么WinHttpQueryDataAvailable一旦达到8 KiB就会返回0到size？

操作系统：Windows 10专业版，更新1903
架构：x86-64 CPU，x86-64操作系统
CPU ：Intel Core i5-2300 @ 2.8 GHz
RAM ：8 GiB
交换文件：16000 MB

kissqiao 回答：8 KiB后WinHTTP停止下载

buffer = new uint8_t[size];
size_t vectSize = file.size();
file.reserve(vectSize + size);
...
for (size_t i = vectSize; i < size; i++) {
    // Might as well rewrite this and make this more efficient
    file.push_back(buffer[i]);
}

该代码应将buffer附加到file上，因此for循环应从零开始，而不是vectSize。更改为：

for (size_t i = 0; i < size; i++) 
    file.push_back(buffer[i]);

似乎您使用的是C ++ 11或更高版本，因此可以使用std::vector代替new。考虑将do循环替换为以下内容：

while(true)
{
    if(!WinHttpQueryDataAvailable(request,&size))
        break;
    if(!size) 
        break;
    std::vector<uint8_t> buffer(size); 
    if(!WinHttpReadData(request,buffer.data(),size,&downloaded))
        break;
    if(!downloaded)
        break;
    buffer.resize(downloaded);
    file.insert(file.end(),buffer.begin(),buffer.end());
}

或者直接写入主缓冲区：

while(true)
{
    if(!WinHttpQueryDataAvailable(request,&size))
        break;
    if(!size)
        break;

    size_t current_size = file.size();
    file.resize(current_size + size);
    downloaded = 0;
    result = WinHttpReadData(request,file.data() + current_size,&downloaded);
    file.resize(current_size + downloaded);

    if(!result || !downloaded)
        break;
}

与错误无关：

不需要将缓冲区初始化为零。这是在Microsoft示例中完成的，但是该示例使用了一个以空值结尾的字符串，它只需要将最后一个字节设置为零即可。

考虑使用WinINet而不是WinHTTP。如果您不需要服务器，则WinINet更容易。

Barmak Shemirani's answer为我解决了这个问题，尽管还有另一种方法可以解决此问题，同时仍然使用缓冲区。（尽管我提到的答案仍然更快，更优雅。）下面的单行替换了for循环：

file.insert(file.end(),&buffer[0],&buffer[size]);

将迭代器附加到std::vector上是“ C ++方式”。尽管如此，首先写入vector显然还是更快。如果您可以指定写位置，那就是。

c++http windows winhttp

本文链接：https://www.f2er.com/3166944.html

8 KiB后WinHTTP停止下载

kissqiao 回答：8 KiB后WinHTTP停止下载

大家都在问