S3 Image Uploads via AWS-SDK with Golang

Uploading files and varying types of media can always be a headache. I was recently tasked with building out a media service; a dedicated Go service that would handle uploads to an s3 bucket. The documentation isn’t completely clear and most of the examples and tutorials are outdated, so I am posting a small snippet of basic working code using the AWS SDK

An Ultra-Basic GO AWS-SDK example for S3 Bucket image uploads

Define Credentials:

awsAccessKey := "Insert Key Here" 
awsSecret := "Insert Secret Here"
token := ""

Next we want to set our credentials into a credentials struct provided by the AWS-SDK. In this case we are using NewStaticCredentials, but only because we have statically set credentials, you might instead use NewEnvCredentials to pull in an environment variable

creds := credentials.NewStaticCredentials(awsAccessKey, awsSecret, token)

Next we want to call the .Get() method on our credentials struct. Get returns the credentials value or an error if the value cannot be retrieved. In this case, we don’t need the credential’s value, we just need to expose an error if there is one

_, err := creds.Get() 
if err != nil {
// handle error
}

Set up Configuration and S3 Instance

cfg := aws.NewConfig().WithRegion("us-west-1").WithCredentials(creds)

Next we need to create a new s3 instance that contains the client and session. s3.New() takes in two parameters, a ConfigProvider and an aws.Config struct

svc := s3.New(session.New(), cfg)

Prepping the file

First we will open the file (for this example to work, you’ll need to pass in a path to an actual file)

file, err := os.Open("test.jpg") 
if err != nil {
// handle error
}
defer file.Close()

next we will determine the size of the file

fileInfo, _ := file.Stat() 
var size int64 = fileInfo.Size()

and create a buffer of the size of the file, set the file content to the buffer and then convert it to Bytes

buffer := make([]byte, size) file.Read(buffer) fileBytes := bytes.NewReader(buffer) 
fileType := http.DetectContentType(buffer)

Finally we make the set the path and put together the s3 request

path := "/media/" + file.Name() 
params := &s3.PutObjectInput{
Bucket: aws.String("nameofBucketHere"),
Key: aws.String(path),
Body: fileBytes,
ContentLength: aws.Int64(size),
ContentType: aws.String(fileType),
}

and submit it to AWS

resp, err := svc.PutObject(params) 
if err != nil {
// handle error
}
fmt.Printf("response %s”, awsutil.StringValue(resp))

Final

import( 
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awsutil"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/service/s3"
"github.com/aws/aws-sdk-go/aws/session"
)
func main() {
aws_access_key_id := "Insert Key ID here"
aws_secret_access_key := "Insert Secret Here"
token := ""
creds := credentials.NewStaticCredentials(aws_access_key_id, aws_secret_access_key, token)
_, err := creds.Get()
if err != nil {
// handle error
}
cfg := aws.NewConfig().WithRegion("us-west-1").WithCredentials(creds) svc := s3.New(session.New(), cfg)

file, err := os.Open("test.jpg")
if err != nil {
// handle error
}
defer file.Close() fileInfo, _ := file.Stat() size := fileInfo.Size() buffer := make([]byte, size) // read file content to buffer

file.Read(buffer)
fileBytes := bytes.NewReader(buffer)
fileType := http.DetectContentType(buffer)
path := "/media/" + file.Name()
params := &s3.PutObjectInput{
Bucket: aws.String("testBucket"),
Key: aws.String(path),
Body: fileBytes,
ContentLength: aws.Int64(size),
ContentType: aws.String(fileType),
}
resp, err := svc.PutObject(params)
if err != nil {
// handle error
}
fmt.Printf("response %s", awsutil.StringValue(resp))
}

Receiving Images from a Form

//r is an *http.Request object
file, h, err := r.FormFile(“data”) // or whatever the form key is
if err != nil {
// handle error
}

From docs: https://golang.org/pkg/net/http/#Request.FormFile returns the first file for the provided form key. FormFile calls ParseMultipartForm and ParseForm if necessary.

the first return value (file) is the buffer, and the second return value (h) consists of several metadata fields (e.g. h.Header[“Content-Type”][0] will return the content-type, h.Filename will return the filename)

Originally published at questhenkart.com on February 3, 2016.

Software Architect

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store