Microsoft Power Automate (Flow): Amazon S3 bucket storage

Title image of Flow, Couchdrop and Amazon S3 logos and connectivity

All product names, logos, and brands used in this post are property of their respective owners.

When I originally wrote this post in early 2019, there was not a native Amazon S3 connector for Microsoft Power Automate (Flow). As of 2021, I am pleased to announce that Microsoft has released a native Amazon S3 Connector for Power Automate! This is very exciting and promising news. In 2021, I published a quick setup guide for the native AWS S3 Connector for Power Automate here.

The native S3 connector is “read-only” - this means Power Automate can only use it to retrieve data FROM Amazon S3 storage buckets. It cannot send data TO S3 (yet). That being the case, I feel this post is still relevant for those needing to write data to an S3 bucket using Power Automate. At least until Microsoft adds “write” capabilities to the connector.

Back in 2019 (while seeking a way to automate static website updates for sites hosted in S3 buckets), I found a fantastic service called Couchdrop. When used in conjunction with the Microsoft Power Automate (Flow) SFTP - SSH connector, it allows Power Automate to access and manipulate files and folders in AWS S3 buckets. If you have seen some of my other posts, I am sure you know that I am a big fan of SFTP with Power Automate (Flow).

In this case, Couchdrop serves as middleware that translates SFTP commands from Power Automate into S3 API commands. I will describe the configuration required to allow Power Automate (Flow) to interface with S3 buckets, including the Amazon (AWS) S3 setup, Couchdrop setup, and SFTP connector setup.

Amazon S3 (and IAM) Setup

The Amazon AWS setup is relatively simple. I recommend creating 1 API user (IAM) per S3 bucket to keep things compartmentalized and secure.

Assuming you already have a bucket setup, let’s look at setting up the IAM user. This user’s credentials will be added in the storage endpoint settings in Couchdrop.

  1. Browse to the IAM console in AWS

  2. Select Users -> Add User

  3. Enter a username, select Programmatic access, then Next

  4. Select the “Attach existing policies directly” option, then click the Create Policy button

  5. In the new window, select the JSON button

  6. I used one of Amazon’s examples. The name of the bucket this policy grants read/write access to is www.mywebsite.com:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": "arn:aws:s3:::www.mywebsite.com"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::www.mywebsite.com/*"
        }
    ]
}
  1. Click Review policy, name the policy, then click Create policy

  2. Once the policy is created, return to your original IAM window, click the refresh button above the policy list, and use search to find the policy you just created - tick the box next to the policy and click Next: Tags

  3. Add tags as appropriate, click Next: Review and then Next: Finish

  4. Lastly, capture the Access key ID and Secret access key - you will use these in the Couchdrop setup

Couchdrop Setup

The Couchdrop setup is even easier than the AWS setup. We will configure Couchdrop to connect to our S3 bucket (www.mywebsite.com) using the IAM user we created above. Then, we will create a user (key) in Couchdrop that Microsoft Power Automate (Flow) will use to connect with the service. Technically, you could use your default Couchdrop key/account but for security, I recommend creating a second account.

  1. Navigate to your Couchdrop storage area and click Connect Storage if needed

  2. Add your AWS S3 bucket as follows, using the Access key ID, Secret access key, and bucket name - once done, click Save Settings

Couchdrop storage settings for Amazon S3 bucket connectivity

  1. Next, add an SFTP user to the Couchdrop account - navigate to the Couchdrop Users area

  2. Click Add User, then set a password, select Read/Write for Permissions, ensure Allow SFTP/SCP/Rsync is enabled, and click Save Settings

Couchdrop key or account settings for the SFTP interface

  1. Take note of the username and password you set - these are the credentials we will use in the Microsoft Power Automate (Flow) SFTP connector

Microsoft Power Automate (Flow) SFTP Connector Setup

The final step is setting up an SFTP Connector in Power Automate that points to your Couchdrop setup. At this point, the Power Automate (Flow) setup is just like any other SFTP connector setup (thanks to Couchdrop). Once done, Power Automate “speaks” SFTP to Couchdrop and Couchdrop translates that into S3 API commands for Amazon storage buckets.

  1. Add an “SFTP - SSH” Action to your flow (using the Power Automate Editor) and create a new SFTP - SSH Connector

  2. Configure the connector as follows (add the username/password from Couchdrop) and specify sftp.couchdrop.io as the Host server address

Microsoft Power Autoamte or Flow SFTP - SSH connector settings to access Couchdrop

  1. The SFTP - SSH connector and Couchdrop both support additional SFTP security features like key-based authentication and host key finger print validation. For more information on using an SSH private key for authentication, see here and here. For host fingerprint validation, see here.

  2. Add other logic to your flow that leverages the SFTP - SSH connection; you can use actions like “List file in root folder”, “List files in folder”, “Copy file”, “Delete file”, etc.

Side by side comparison of S3 bucket content and “List files in folder” connector content in MS Power Automate or Flow

Closing thoughts

Depending on your needs, Couchdrop is a phenomenal way to interface Microsoft Power Automate or Flow with Amazon S3 (for read and write operations). There is a nominal cost, which starts at $10 per month. In my opinion, this cost is lower than if you:

  • Spin up an Amazon EC2 Linux instance and use something like s3fs-fuse to provide an SFTP interface to S3 storage. The cheapest ec2 instances are ~$4-5/month (if run continuously) and this does not include the setup/configuration (i.e. sftp and s3fs-fuse) and ongoing maintenance required to remain secure and operational.

  • Subscribe to AWS Transfer for SFTP, which is Amazon’s new, native way to connect to S3 buckets via SFTP. The functionality of this service is appealing on the surface, but the cost is very high ($216/month minimum if run continuously) compared to the other options.

If I needed write access to Amazon S3, I would choose Couchdrop every time based on cost and ease of use (no ongoing maintenance and simple upfront configuration). If I only needed read access (pull files/content from S3 using Power Automate), I would opt for the free, native Amazon S3 connector instead. I will also keep my eye on the native connector - I would speculate that write support is in the pipeline.

It is worth noting that Microsoft Azure offers blob storage and CDN service (similarly to Amazon Web Services), and there IS a Power Automate (Flow) connector for Azure blob storage. If you are not bound to Amazon, hosting your static site with Azure and interfacing with it using the native Power Automate (Flow) connector is an option as well.