Man Vs. S3

Solution Gist: https://gist.github.com/adrichman/e550885f6011f37d6bd3

For more hours than I like to admit, I struggled to free ourselves from AWS's S3 SDK.

We were trying to facilitate direct uploads from our client to our S3 bucket, but without exposing any credentials that would need to be present on the client side. The answer to doing this requires that you deliver the client a temporary, signed URL or policy that is generated on your own server and returned to the client in order to accompany the payload to S3. Unfortunately this is where the SDK stopped working for us.

I scoured the web for days in search of solutions and found myriad similar stack overflow questions and blog entries on this very challenge. Still, I came up empty handed when I attempted to integrate many of the code examples, I presume because Amazon has since changed their API, or because of some other finicky detail that I wasn't identifying. I suppose that this kind of request just screams CORS and I understand why. I finally found my way to code that worked for us after finding a few posts that shined some meaningful light on the solving this problem. They are here, here and here.

My takeaways upon succeeding:

  • the order of your headers REALLY matters to amazon. This is not evident in their documentation, which at times is contradictory.

  • getting to know HTML5's FormData object was crucial for appending our policy to the payload.

check out my solution on stack overflow

Ultimately, we reverted to using Amazon's SDK and piping uploads from the client through our server on their way to s3. Solving this problem with our web form was not really going to be sufficient because our client is an Angular / Ionic / Cordova mobile app, and this simple upload form was going to require some refactoring and trial-and-error that would cost us precious time that we didn't have. With a little more time, we can change that and the client will retrieve our dynamically generated, signed-policy that will give it temporary credentials for direct upload to our S3 bucket.

Hopefully someone will find this post some day in the future that is soon enough for it to be helpful - before Amazon changes their API.