Microsoft Power Automate (Flow): Amazon S3 Bucket Storage

Title image of Flow, Couchdrop and Amazon S3 logos and connectivity

All product names, logos, and brands used in this post are property of their respective owners.

At the time of writing this post, there is not a native Amazon S3 connector for Microsoft Power Automate (Flow). In the course of creating a Flow to automate static website updates, I found a fantastic service called Couchdrop, which when used in conjunction with the Microsoft Power Automate (Flow) SFTP - SSH connector, allows Power Automate to access and manipulate files and folders in AWS S3 buckets. If you have seen some of my other posts, I am sure you know that I am a big fan of SFTP with Power Automate (Flow).

In this case, Couchdrop serves as middleware that translates SFTP commands from Power Automate into S3 API commands. The limits of the “Freebie” version of Couchdrop are as follows, but you can still accomplish quite a bit (depending on the nature of the files you are moving):

  • 10 transactions per month - a “transaction” is synonymous to a connection or session (if you move multiple files in one session, it counts as only a single transaction)
  • 1 user and 1 storage endpoint

If you purchase the service, the transaction limit is removed and the user/storage endpoint limits are relaxed. In my case, the free tier ended up being perfect (my Flow only checks and updates an S3-hosted static site once per week, so 4 transactions per month for checking and 6 left over for manipulating files). I am very impressed with the functionality of the free Couchdrop offering (kudos to the team for making this available and to their awesome support team for promptly answering my question about “transactions”).

In this post, I will describe the configuration required to allow Power Automate (Flow) to interface with S3 buckets, including the Amazon (AWS) S3 setup, Couchdrop setup, and SFTP connector setup.

Amazon S3 (and IAM) Setup

The Amazon AWS setup is relatively simple. I recommend creating 1 API user (IAM) per S3 bucket to keep things compartmentalized and secure. If you have more than 1 bucket, you will end up needing a Couchdrop plan besides the free one (1 endpoint limit).

Assuming you already have a bucket setup, let’s look at setting up the IAM user. This user’s credentials will be added in the storage endpoint settings in Couchdrop.

  1. Browse to the IAM console in AWS

  2. Select Users -> Add User

  3. Enter a username, select Programmatic access, then Next

  4. Select the “Attach existing policies directly” option, then click the Create Policy button

  5. In the new window, select the JSON button

  6. I used one of Amazon’s examples. The name of the bucket this policy grants read/write access to is www.mywebsite.com:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": "arn:aws:s3:::www.mywebsite.com"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::www.mywebsite.com/*"
        }
    ]
}
  1. Click Review policy, name the policy, then click Create policy

  2. Once the policy is created, return to your original IAM window, click the refresh button above the policy list, and use search to find the policy you just created - tick the box next to the policy and click Next: Tags

  3. Add tags as appropriate, click Next: Review and then Next: Finish

  4. Lastly, capture the Access key ID and Secret access key - you will use these in the Couchdrop setup

Couchdrop Setup

The Couchdrop setup is even easier than the AWS setup. We will configure Couchdrop to connect to our S3 bucket (www.mywebsite.com) using the IAM user we created above. Then, we will create a user (key) in Couchdrop that Microsoft Power Automate (Flow) will use to connect with the service. Technically, you could use your default Couchdrop key/account but for security, I recommend creating a second account.

  1. Navigate to your Couchdrop storage area and click Connect Storage if needed

  2. Add your AWS S3 bucket as follows, using the Access key ID, Secret access key, and bucket name - once done, click Save Settings

Couchdrop storage settings for Amazon S3 bucket connectivity

  1. Next, add an SFTP user to the Couchdrop account - navigate to the Couchdrop Users area

  2. Click Add User, then set a password, select Read/Write for Permissions, ensure Allow SFTP/SCP/Rsync is enabled, and click Save Settings

Couchdrop key or account settings for the SFTP interface

  1. Take note of the username and password you set - these are the credentials we will use in the Microsoft Power Automate (Flow) SFTP connector

Microsoft Power Automate (Flow) SFTP Connector Setup

The final step is setting up an SFTP Connector in Power Automate that points to your Couchdrop setup. At this point, the Power Automate (Flow) setup is just like any other SFTP connector setup (thanks to Couchdrop). Once done, Power Automate “speaks” SFTP to Couchdrop and Couchdrop translates that into S3 API commands for Amazon storage buckets.

  1. Add an “SFTP - SSH” Action to your flow (using the Power Automate Editor) and create a new SFTP - SSH Connector

  2. Configure the connector as follows (add the username/password from Couchdrop) and specify sftp.couchdrop.io as the Host server address

Microsoft Power Autoamte or Flow SFTP - SSH connector settings to access Couchdrop

  1. The SFTP - SSH connector and Couchdrop both support additional SFTP security features like key-based authentication and host key finger print validation. For more information on using an SSH private key for authentication, see here and here. For host fingerprint validation, see here.

  2. Add other logic to your flow that leverages the SFTP - SSH connection; you can use actions like “List file in root folder”, “List files in folder”, “Copy file”, “Delete file”, etc.

Side by side comparison of S3 bucket content and “List files in folder” connector content in MS Power Automate or Flow

Closing thoughts

Depending on your needs, Couchdrop is a phenomenal way to interface Microsoft Power Automate or Flow with Amazon S3. If you can get by with the “Freebie” tier, the value is even more appealing. Even if you opt for the paid version ($4 per month at the time of writing), your monthly cost will be lower than if you:

  • Spin up an Amazon EC2 Linux instance and use something like s3fs-fuse to provide an SFTP interface to S3 storage. The cheapest ec2 instances are ~$4-5/month (if run continuously) and this does not include the setup/configuration (i.e. sftp and s3fs-fuse) and ongoing maintenance required to remain secure and operational.

  • Subscribe to AWS Transfer for SFTP which is Amazon’s new, native way to connect to S3 buckets via SFTP. The functionality of this service is appealing on the surface, but the cost is very high ($216/month minimum if run continuously) compared to the other options.

I would personally choose Couchdrop every time based on cost and ease of use (no ongoing maintenance and simple upfront configuration).

It is worth noting that Microsoft Azure offers blob storage and CDN service (similarly to Amazon Web Services), and there IS a Power Automate (Flow) connector for Azure blob storage. If you are not bound to Amazon, hosting your static site with Azure and interfacing with it using the native Power Automate (Flow) connector is an option as well.