S3 PutObject fails on Windows with errors depending on the upload size #2579
Comments
lordgamez
added
bug
|
Hello @lordgamez , Thank you very much for your submission. Could you please provide the trace log associated with the behavior All you need to do is: Best regards, Yasmine |
yasminetalby
added
response-requested
|
Hi @yasminetalby, Thank you for the quick response! I collected trace logs from an 8MB and a 12MB file upload: In the meantime I tried building the library with the BR, |
|
Some additional info regarding the CRT HTTP client: Edit: The same issue appears randomly on Windows 10, additionally when using CRT HTTP lib on Win10 even when the main thread exits, the process keeps hanging and does not exist for some reason. |
yasminetalby
added
investigating
|
Hello @lordgamez , My apologies for the delay of answer. Thank you very much for providing the information requested. First of all let’s go over your first comment. I have gone over the log and we can see two different behavior here: For the attempt to upload a 8MB file, the issue is caused by a difference between the server time and the client time : In this case scenario, the SDK should be adjusting the signer with the skew and attempt the request again. (See. There is a client configuration parameter available enableClockSkewAdjustment which if set to true, adjusts clock skew after each http attempt and default to true. This might also be an issue with your retry strategy. In the case of the 12MB file, the issue is caused by the Date header not being present in the response: due to Encountered network error when sending http request which causes the request to fail. Now looking at the behavior using the CRT client, you are experiencing a similar behavior as the second case with a request timeout. Based on the behavior above, it might be a network issue which would cause latency in the PutObjectRequest and would arise with larger file upload causing the behavior above. There has been issue submission with similar behavior caused by this see. Best regards, Yasmine |
yasminetalby
added
response-requested
|
Hi @yasminetalby , Thanks for investigating the issue. I checked and the While testing it seems that I have found the root cause of the issue and it looks to be related to this issue. In the application code I had a default So it seems that the issue mentioned in #1771 does not only happen with the CRT library but the WinHTTP library as well. When using a build on Linux with CURL for HTTP backend this issues does not appear. After adding this fix I tested uploads with 5MB, 12MB, and 100MB sized files and all seem to work without a hiccup. |
github-actions
bot
removed
the
response-requested
|
I'm glad that you were able to get this working by setting the server side encryption. Please let us know if you have any other questions/potential bugs about this sdk. |
|
This issue is now closed. Comments on closed issues are hard for our team to see. |
Describe the bug
I am building aws-sdk-cpp as a static library linked to my application with the following cmake flags:
-DBUILD_ONLY=s3 -DENABLE_TESTING=OFF -DBUILD_SHARED_LIBS=OFFI am using the SDK to upload files to S3 using the
PutObjectfunction ofS3Client. The application works fine on Linux, but there are some errors reported on Windows depending on the size of the upload.When the file to be uploaded is less than 10MB of size the upload fails with the error
SignatureDoesNotMatch(<bucket>is only a placeholder for the used bucket name):When the file to be uploaded is larger than 10MB it fails with a network error:
First tried with version 1.10.48, but also tried the latest version 1.11.118 with the same result. The issue was tested on AWS S3, but the same issue was reported when using S3 compatible storages: VAST S3 and MinIO S3
Suspected a firewall issue, but disabling the firewall had no effect.
Expected Behavior
PutObject call should succeed.
Current Behavior
PutObject failed.
Reproduction Steps
Aws::Auth::AWSCredentials = aws_credentials_provider_.getAWSCredentials();
auto client_config = Aws::Client::ClientConfiguration();
Aws::S3::Model::PutObjectRequest request;
request.SetBucket("bucket");
request.SetKey("key");
auto stream = std::make_sharedstd::stringstream();
(*stream) << "Hello, World!";
request.SetBody(data_stream);
Aws::S3::S3Client s3_client(credentials, client_config, Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Never, true);
auto outcome = s3_client.PutObject(request);
Possible Solution
No response
Additional Information/Context
No response
AWS CPP SDK version used
1.11.118 and 1.10.48
Compiler and Version used
Visual Studio 2019, MSVC 14.29.30133
Operating System and version
Windows 10
The text was updated successfully, but these errors were encountered: