Breaking News: Grepper is joining You.com. Read the official announcement!
Check it out

aws-sdk-go-v2 - Upload Manager - PutObjectInput Body Field (io.ReadSeeker vs. io.Reader)

CL answered on March 1, 2023 Popularity 1/10 Helpfulness 1/10

Contents


More Related Answers


aws-sdk-go-v2 - Upload Manager - PutObjectInput Body Field (io.ReadSeeker vs. io.Reader)

0

The Body field of the s3.PutObjectInput struct is an io.Reader type. However, this field can be populated with a type that satisfies both the io.ReadSeeker and io.ReaderAt interface to improve application resource utilization of the host environment. The following example creates the type ReadSeekerAt that satisfies both interfaces.

For io.Reader types, the bytes of the reader must be buffered in memory before the part can be uploaded. When you increase the PartSize or Concurrency value, the required memory (RAM) for the Uploader increases significantly. The required memory is approximately PartSize * Concurrency. For example, specifying 100 MB for PartSize and 10 for Concurrency, requires at least 1 GB.

Because an io.Reader type cannot determine its size before reading its bytes, Uploader cannot calculate how many parts will be uploaded. Consequently, Uploader can reach the Amazon S3 upload limit of 10,000 parts for large files if you set the PartSize too low. If you try to upload more than 10,000 parts, the upload stops and returns an error.

For body values that implement the ReadSeekerAt type, the Uploader doesn’t buffer the body contents in memory before sending it to Amazon S3. Uploader calculates the expected number of parts before uploading the file to Amazon S3. If the current value of PartSize requires more than 10,000 parts to upload the file, Uploader increases the part size value so that fewer parts are required. 

Popularity 1/10 Helpfulness 1/10 Language go
Source: aws.github.io
Link to this answer
Share Copy Link
Contributed on Mar 01 2023
CL
0 Answers  Avg Quality 2/10


X

Continue with Google

By continuing, I agree that I have read and agree to Greppers's Terms of Service and Privacy Policy.
X
Grepper Account Login Required

Oops, You will need to install Grepper and log-in to perform this action.