Microsoft Power Automate (Flow): Amazon S3 bucket storage

All product names, logos, and brands used in this post are property of their respective owners.

When I originally wrote this post in early 2019, there was not a native Amazon S3 connector for Microsoft Power Automate (Flow). As of 2021, I am pleased to announce that Microsoft has released a native Amazon S3 Connector for Power Automate! This is very exciting and promising news. In 2021, I published a quick setup guide for the native AWS S3 Connector for Power Automate here.

The native S3 connector is “read-only” - this means Power Automate can only use it to retrieve data FROM Amazon S3 storage buckets. It cannot send data TO S3 (yet). Additionally, the Amazon S3 Flow Connector became a Premium connector once it was out of preview. It requires a paid (Power Automate License by user or License by Flow plan) license. The connector was briefly available under “Free” plans while it was in Preview. These being the case, I feel this post is still relevant for those needing to write data to an S3 bucket using Power Automate (or make a decision based on license costs).

Back in 2019 (while seeking a way to automate static website updates for sites hosted in S3 buckets), I found a fantastic service called Couchdrop. When used in conjunction with the Microsoft Power Automate (Flow) SFTP - SSH connector, it allows Power Automate to access and manipulate files and folders in AWS S3 buckets. If you have seen some of my other posts, I am sure you know that I am a big fan of SFTP with Power Automate (Flow).

In this case, Couchdrop serves as middleware that translates SFTP commands from Power Automate into S3 API commands. I will describe the configuration required to allow Power Automate (Flow) to interface with S3 buckets, including the Amazon (AWS) S3 setup, Couchdrop setup, and SFTP connector setup.

Amazon S3 (and IAM) Setup

The Amazon AWS setup is relatively simple. I recommend creating 1 API user (IAM) per S3 bucket to keep things compartmentalized and secure.

Assuming you already have a bucket setup, let’s look at setting up the IAM user. This user’s credentials will be added in the storage endpoint settings in Couchdrop.

1. Browse to the IAM console in AWS

2. Select Users -> Add User

3. Enter a username of your choosing, leave the default options, then click Next

4. Select the Attach existing policies directly option, then click the Create Policy button

5. In the new window, select the JSON tab

6. I used one of Amazon’s examples. The name of the bucket this policy grants read/write access to is www.mywebsite.com:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": "arn:aws:s3:::www.mywebsite.com"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::www.mywebsite.com/*"
        }
    ]
}

7. Click Next: Tags, add tags if desired, then click Next: Review, name the policy, then click Create policy

8. Once the policy is created, return to your original IAM window or tab, click the refresh button above the policy list, and use search to find the policy you just created - tick the box next to the policy and click Next

9. Add tags as appropriate, click Next: Review and then Next: Finish

10. View the new user you created using the button near the top of the screen or by clicking the User name in the list

11. On the user screen, click the Security credentials tab, scroll down to the Access keys section, and click Create access key. Select the Application running outside AWS option, click Next, then click Create access key. Capture the Access key and Secret access key values - you will use these in the Couchdrop setup

Couchdrop Setup

The Couchdrop setup is even easier than the AWS setup. We will configure Couchdrop to connect to our S3 bucket (www.mywebsite.com) using the IAM user we created above. Then, we will create a user (key) in Couchdrop that Microsoft Power Automate (Flow) will use to connect with the service. Technically, you could use your default Couchdrop key/account but for security, I recommend creating a second account.

1. Navigate to your Couchdrop storage area, click Add Storage Integration, and select Amazon S3

2. Add your AWS S3 bucket as follows, using the Access key ID, Secret access key, and bucket name - once done, click Save Storage

3. Next, add an SFTP user to the Couchdrop account - navigate to the Couchdrop Users area

4. Click Create New User, specify a username, set a password, ensure Allow SFTP/SCP/Rsync is enabled, and click Save User Settings

6. Take note of the username and password you set - these are the credentials we will use in the Microsoft Power Automate (Flow) SFTP connector

Microsoft Power Automate (Flow) SFTP Connector Setup

The final step is setting up an SFTP Connector in Power Automate that points to your Couchdrop setup. At this point, the Power Automate (Flow) setup is just like any other SFTP connector setup (thanks to Couchdrop). Once done, Power Automate “speaks” SFTP to Couchdrop and Couchdrop translates that into S3 API commands for Amazon storage buckets.

1. Add an “SFTP - SSH” Action to your flow (using the Power Automate Editor) and create a new SFTP - SSH Connector

2. Configure the connector as follows (add the username/password from Couchdrop) and specify sftp.couchdrop.io as the Host server address

3. The SFTP - SSH connector and Couchdrop both support additional SFTP security features like key-based authentication and host key finger print validation. For more information on using an SSH private key for authentication, see here and here. For host fingerprint validation, see here.

4. Add other logic to your flow that leverages the SFTP - SSH connection; you can use actions like “List file in root folder”, “List files in folder”, “Copy file”, “Delete file”, etc.

Closing thoughts

Depending on your requirements, Couchdrop is a great way to interface Microsoft Power Automate or Flow with Amazon S3 (for read and write operations). There is a cost, which starts at $50 per month ($10/user/month x 5 user minimum commitment). In my opinion, this cost is lower than if you:

• Spin up an Amazon EC2 Linux instance and use something like s3fs-fuse to provide an SFTP interface to S3 storage. The cheapest ec2 instances are ~$4-5/month (if run continuously) and this does not include the setup/configuration (i.e. sftp and s3fs-fuse) and ongoing maintenance required to remain secure and operational.

• Subscribe to AWS Transfer for SFTP, which is Amazon’s native service to connect to S3 buckets via SFTP. The functionality is appealing on the surface, but the cost can be high (~$220/month minimum if run continuously) compared to the other options. But, AWS Transfer for SFTP starts to compete with Couchdrop if you have ~20 SFTP users or more.

If I needed write access to Amazon S3 for a small number of users, I would personally choose Couchdrop based on cost and ease of use (no ongoing maintenance and simple upfront configuration).

If I only needed read access (pull files/content from S3 using Power Automate), I would consider the native Amazon S3 connector (depending on the number of users and whether or not I already owned a paid Power Automate license). Currently, the overall cost of Power Automate license for a single user ($15/user/month) is lower than the cost of Couchdrop ($50/month for five users), because of the commitment.

It is worth noting that Microsoft Azure offers blob storage and CDN service (similarly to Amazon Web Services), and there IS a Power Automate (Flow) connector for Azure blob storage. If you are not bound to Amazon, hosting your static site with Azure and interfacing with it using the native Power Automate (Flow) connector is an option as well.