Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 CRT Path style not working #2500

Closed
FaZeRs opened this issue May 22, 2023 · 2 comments
Closed

S3 CRT Path style not working #2500

FaZeRs opened this issue May 22, 2023 · 2 comments
Labels
bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.

Comments

@FaZeRs
Copy link

FaZeRs commented May 22, 2023

Describe the bug

I have set useVirtualAddressing to false when initializing S3CrtClient, but in logs I see that bucket name is prefixed to endpoint. I'm using Ceph S3 and path looks like https://****/{bucket}/{file_path}

Expected Behavior

Aws::Endpoint::DefaultEndpointProvider [139729996415488] Endpoint rules engine evaluated the endpoint: https://{endpoint}/{bucket}

Current Behavior

Aws::Endpoint::DefaultEndpointProvider [139729996415488] Endpoint rules engine evaluated the endpoint: https:/{bucket}.{endpoint}

Reproduction Steps

#include <iostream>
#include <fstream>
#include <aws/core/Aws.h>
#include <aws/core/utils/memory/stl/AWSStringStream.h>
#include <aws/core/utils/logging/CRTLogSystem.h>
#include <aws/s3-crt/S3CrtClient.h>
#include <aws/s3-crt/model/CreateBucketRequest.h>
#include <aws/s3-crt/model/BucketLocationConstraint.h>
#include <aws/s3-crt/model/DeleteBucketRequest.h>
#include <aws/s3-crt/model/PutObjectRequest.h>
#include <aws/s3-crt/model/GetObjectRequest.h>
#include <aws/s3-crt/model/DeleteObjectRequest.h>
#include <aws/core/utils/UUID.h>

static const char ALLOCATION_TAG[] = "s3-crt-demo";

int main(int argc, char* argv[]) {
  Aws::SDKOptions options;
  options.loggingOptions.logLevel = Aws::Utils::Logging::LogLevel::Trace;
  options.loggingOptions.crt_logger_create_fn = []() {
    return Aws::MakeShared<Aws::Utils::Logging::DefaultCRTLogSystem>(ALLOCATION_TAG,
                                                                     Aws::Utils::Logging::LogLevel::Trace);
  };

  Aws::InitAPI(options);
  {
    Aws::String region = Aws::Region::US_EAST_1;
    Aws::String bucket_name = "****";
    Aws::String object_key = "****";
    const double throughput_target_gbps = 5;
    const uint64_t part_size = 8 * 1024 * 1024; // 8 MB.

    Aws::S3Crt::ClientConfiguration config;
    config.endpointOverride = "****";
    config.throughputTargetGbps = throughput_target_gbps;
    config.partSize = part_size;
    config.enableEndpointDiscovery = true;

    Aws::Auth::AWSCredentials credentials("*", "*");

    Aws::S3Crt::S3CrtClient s3_crt_client(credentials, config,
                                          Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Never, false,
                                          Aws::S3Crt::US_EAST_1_REGIONAL_ENDPOINT_OPTION::LEGACY);

    Aws::S3Crt::Model::GetObjectRequest request;
    request.WithBucket(bucket_name).WithKey(object_key);
    auto outcome = s3_crt_client.GetObject(request);
    if (outcome.IsSuccess()) {
      std::cout << "Object content: " << outcome.GetResult().GetBody().rdbuf() << std::endl << std::endl;
    } else {
      std::cout << "GetObject error:\n" << outcome.GetError() << std::endl << std::endl;
    }
  }
  Aws::ShutdownAPI(options);

  return 0;
}

Possible Solution

No response

Additional Information/Context

No response

AWS CPP SDK version used

1.11.79

Compiler and Version used

GCC 12.1.0

Operating System and version

Ubuntu 22.04.2

@FaZeRs FaZeRs added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels May 22, 2023
@FaZeRs
Copy link
Author

FaZeRs commented May 23, 2023

You need to add that to ClientConfiguration as well for it to work

  Aws::S3Crt::ClientConfiguration config;
  config.useVirtualAddressing = false;

@FaZeRs FaZeRs closed this as completed May 23, 2023
@github-actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. needs-triage This issue or PR still needs to be triaged.
Projects
None yet
Development

No branches or pull requests

1 participant