The SFTP connector in Microsoft Flow has quickly become my personal favorite. It is quite robust - I am most pleased with its support of SSH key based authentication and SSH/SFTP ports besides 22 (the default). In light of this, I wanted to share some of my findings in hopes this will be of value to someone else.
Read on for more detailed information about these tips
- Allow MS Flow’s source IP addresses (Connectors section under IP address configuration) through your firewall
- Ensure your private key is PuTTY Private Key File (.ppk) format - convert using PuTTYGEN as needed. Ensure you are adding the SFTP connector through the Flow editor and NOT through settings (this is a bug).
- Use the “Disable Resume Capability” option if your SFTP destination is a “drop box” and will not allow you to rename files after they are uploaded
- If using SSH host key fingerprint validation, ensure the finger print is formatted properly
Firewall rules (allow IPs and IP ranges)
If you allow open access to your server via port 22, this may not concern you. However, if you restrict SSH or SFTP access via software (i.e. iptables) or hardware firewall, read on.
Symptoms of Flow being unable to access your server due to a firewall issue include the following error messages:
Please check your account info and/or permissions and try again. Details: Unable to connect to the remote server '...'.
Unable to connect to the remote server '...'. clientRequestId:...
MS Flow’s source IP addresses are documented in a post entitled Limits and configuration in Microsoft Flow. Scroll down to the IP address configuration section and review the addresses and IP ranges under Connectors.
The source addresses definitely change over time; between June 2017 and October 2018, Microsoft added 67 new IPs/ranges across all regions (and added dedicated IPs and ranges for Brazil and the United Kingdom). Ideally, there would be a way to obtain these IP ranges programmatically (via web service) or tap into another type of automated feed. As of October 2018, that is not the case. My only advice is to use a service like visualping or ChangeTower to monitor for changes on the Limits and configuration in Microsoft Flow page and manually update your firewall when Microsoft modifies that page. Or, submit a User Voice idea and recommend that a web service or automatic feed of the current IP ranges be implemented.
SSH Key Authentication
If you are familiar with SSH key authentication, configuring this in MS Flow is relatively easy. The only caveat I discovered is that the private key you paste into the Flow interface must be a PuTTY Private Key File (.ppk). In addition, there are (were?) product issues in Flow itself in Q4 2018 - there is a workaround (more below).
Consider the following if you receive an error like:
Please check your account info and/or permissions and try again. Details: Invalid SSH private key provided.
Invalid SSH private key provided. clientRequestId ...
Use PuTTYGEN to convert the OpenSSH key into PuTTY Private Key File (.ppk) format and paste that into the SSH private key field in your Flow. Even though the SSH private key field is single line text, it will accept the entire PuTTY formatted private key if you paste it in:
The following is an (example) OpenSSH formatted SSH-2 private key which WILL NOT WORK in Microsoft Flow:
-----BEGIN RSA PRIVATE KEY----- MIIEoQIBAAKCAQEApnzCa10mgFV8F6Fe+i7FItcer/OcY/pXUia85s+eOG7a/JoR HLCnwwt4H7hAw3M/xFqiFkQjZEsoRVcIFoILG8uMxn4L+g0cH2fjPsfpeGL668kj i/fszQ/7Nr8RoY3H9JuK3pjJ9yQVaAQDLqzF37W9vf8oozD7VzP09rdw1YN26g3h xQv76xN4Ro1P9CzllRz/HXVl3ek8MATvPKUp2LyTMv7+BtrK0DXcchfJkp9/ED9Q vmPILii2ltNk1rzOaU7tpxNbC9X40MAL0qmGf4TjlpbSDpn0lsk6wmBvtbQm8xEu cP2GpCRKg4WOKEqqQAgET2n6righfsG5lnYl0QIBIwKCAQEAgG7fH5+hW6+o3wd1 J2XoiJdSL/ZxVHCpwwfwz0+0ke8d9hBzmcobBC1rS6tlKQ/DejdJ1qmXpSQJH4xI EV0BQVPhoG/dWn8kUr3bMG5VBRkn97hr4QEVwsMvggEcOsx1rhGXA30uFmxLBxkJ xOuulqIkqIMCF32OsPwyACcch3LMwB0ameSesjP0qmRGor5b7Agk7LoYgZp/7u3k eJvloWcx6i4KyLqGUyJXGt1Mza9/YGQY8cZPJbG0ooqN21MBXqbXDnCd3Eth1WYv lwxqUyGJ4ve1TGC6iPNgNsuFKZxGAZcMs8CcaZHQ5XKnGXn2IDl98ZLkDvwhqkPy clwv+wKBgQDPx1lHuypCJNyjbvPYjaRwC1zYsnMesYgvczVuR6WfxJmcYdG11Ayj MRrpCaaK+I9WH8VouxMq3Yq/y6HJezQN7KAjB89qHrJ987emBXxgV4RGwC7LBcSI 2+kCAlECSdpK5/qPcKCXq+4GTv+V2uDurQHxrpFQ7fq3nPGpuNIJrQKBgQDNIDAP QmQIgGVCWYlBwvn+YUPQoG/bjdskhxZiRbC1vitqQsur/zGyYr8HPCL3fHiD0lb3 FcFTKTxYfTuhGUyPTdaEcBpAvJvWHbzSxwCcU/GUwybsnFgIJcqpELvAzZ2BS2XH bK8auzbgFJKH/PsKvpWexoKLmcohzDWcsbFZNQKBgAvfgXHQLkzsKd10De8eCWV8 /f3BDeR/LFp7nKc3S0r1Sps4yielt5RL8ui+uQ9BZ0a//KbmHlo4i5XuYQQy7Qgc Jmhm2Kb6cJl7pBgdkhQiQhKr5Wqv31hHFKEHclfm9ofv/7Bs1foJ0xZNqDRy6Egn JK65hKWJ8RHOdDWViFhTAoGAF3FklAeWZ18hg+z5vl9tByEPEIddPanP5urd/JpA FMVGylDG0dQFrfyoHhWAVsxli2h+96q3AjCZLq86A8hghWf7QlX0Xyt/hi9Ix6G2 7UtsEQBcOExh1QuiIfNIqFIDXzvRHhsMspkNlfO4+ZlBQw53f9wsLTYlurhApDGQ nHsCgYAiTZ7PecFFiUrQR2QsJH2aYWUcfgEempZLINlcRPi858juLw54+VqSwGej RfCHxj5DRPtAgBMrXZS0irlLUUrTDsxCWVkxQyYuIuTWQn1RMTfiA4dxjCVaGVpV RZJGzieLkrHPDGTjx6fc/9YGAJmczioVLwil0iFaOYcnMsTcHg== -----END RSA PRIVATE KEY-----
The following is an (example) PuTTY formatted private key which WILL work in Flow:
PuTTY-User-Key-File-2: ssh-rsa Encryption: none Comment: imported-openssh-key Public-Lines: 6 AAAAB3NzaC1yc2EAAAABIwAAAQEApnzCa10mgFV8F6Fe+i7FItcer/OcY/pXUia8 5s+eOG7a/JoRHLCnwwt4H7hAw3M/xFqiFkQjZEsoRVcIFoILG8uMxn4L+g0cH2fj PsfpeGL668kji/fszQ/7Nr8RoY3H9JuK3pjJ9yQVaAQDLqzF37W9vf8oozD7VzP0 9rdw1YN26g3hxQv76xN4Ro1P9CzllRz/HXVl3ek8MATvPKUp2LyTMv7+BtrK0DXc chfJkp9/ED9QvmPILii2ltNk1rzOaU7tpxNbC9X40MAL0qmGf4TjlpbSDpn0lsk6 wmBvtbQm8xEucP2GpCRKg4WOKEqqQAgET2n6righfsG5lnYl0Q== Private-Lines: 14 AAABAQCAbt8fn6Fbr6jfB3UnZeiIl1Iv9nFUcKnDB/DPT7SR7x32EHOZyhsELWtL q2UpD8N6N0nWqZelJAkfjEgRXQFBU+Ggb91afyRSvdswblUFGSf3uGvhARXCwy+C ARw6zHWuEZcDfS4WbEsHGQnE666WoiSogwIXfY6w/DIAJxyHcszAHRqZ5J6yM/Sq ZEaivlvsCCTsuhiBmn/u7eR4m+WhZzHqLgrIuoZTIlca3UzNr39gZBjxxk8lsbSi io3bUwFeptcOcJ3cS2HVZi+XDGpTIYni97VMYLqI82A2y4UpnEYBlwyzwJxpkdDl cqcZefYgOX3xkuQO/CGqQ/JyXC/7AAAAgQDPx1lHuypCJNyjbvPYjaRwC1zYsnMe sYgvczVuR6WfxJmcYdG11AyjMRrpCaaK+I9WH8VouxMq3Yq/y6HJezQN7KAjB89q HrJ987emBXxgV4RGwC7LBcSI2+kCAlECSdpK5/qPcKCXq+4GTv+V2uDurQHxrpFQ 7fq3nPGpuNIJrQAAAIEAzSAwD0JkCIBlQlmJQcL5/mFD0KBv243bJIcWYkWwtb4r akLLq/8xsmK/Bzwi93x4g9JW9xXBUyk8WH07oRlMj03WhHAaQLyb1h280scAnFPx lMMm7JxYCCXKqRC7wM2dgUtlx2yvGrs24BSSh/z7Cr6VnsaCi5nKIcw1nLGxWTUA AACAIk2ez3nBRYlK0EdkLCR9mmFlHH4BHpqWSyDZXET4vOfI7i8OePlaksBno0Xw h8Y+Q0T7QIATK12UtIq5S1FK0w7MQllZMUMmLiLk1kJ9UTE34gOHcYwlWhlaVUWS Rs4ni5Kxzwxk48en3P/WBgCZnM4qFS8IpdIhWjmHJzLE3B4= Private-MAC: 9030568136067ba5f327ba6652452f217f19770d
As noted, around October/November 2018, there was a product issue with Flow that prevented creating SFTP connections that use private keys (regardless of the formatting considerations described in this post). In this case, it had to do with Flow’s frontend handling of multi-line text. This was mostly addressed by the Flow team the week of November 12, 2018 (it takes a week or so for changes like this to propagate across Flow); however, you must use the correct method to create an SFTP connection.
Creating an SFTP connection that uses a private key from within the Flow editor (after adding an SFTP action) WORKS
Creating an SFTP connection that uses a private key from Settings -> Connections DOES NOT WORK
In Flow, the following error message was received when creating a new SFTP connection using a valid private key (that works in another application like WinSCP, PuTTY, etc.: Invalid SSH private key provided. clientRequestId:… .
The destination SFTP server log contained this message: error: Received disconnect from X.X.X.X port XXXXX:XX: No supported authentication methods available [preauth], where the X’s are Flow’s source addresses.
Resume Capability and its Functionality
The underlying SFTP engine used by Microsoft Flow is WinSCP. WinSCP has a neat feature called File Transfer Resume which Flow can control via the “Disable Resume Capability” checkbox in the SFTP connector.
In brief, when resume support is enabled, WinSCP “stages” the content you are uploading in a temporary file (with a “.filepart” extension) on the SFTP server until the upload completes successfully; at that point, WinSCP renames the uploaded file to the actual file name (removing the .filepart extension). There are benefits to that functionality, but there can be drawbacks as well. If you can create but not edit files on your SFTP server (or only upload over existing files and not create new ones), you should check the “Disable Resume Capability” box in your SFTP connector in MS Flow.
Here are examples of the upload behavior (as seen from the SFTP server’s perspective) with resume support enabled and disabled:
Uploading the same 10 MB file WITH Resume Capability (enabled, box not checked): (ls -l run intermittently during the transfer)
/scratch/neiltest$ ls -l total 0 -rw-------. 1 neiltest neiltest 0 Dec 23 15:44 testresume10m.bin.filepart /scratch/neiltest$ ls -l total 100 -rw-------. 1 neiltest neiltest 98217 Dec 23 15:44 testresume10m.bin.filepart /scratch/neiltest$ ls -l total 932 -rw-------. 1 neiltest neiltest 949431 Dec 23 15:44 testresume10m.bin.filepart /scratch/neiltest$ ls -l total 2316 -rw-------. 1 neiltest neiltest 2520903 Dec 23 15:44 testresume10m.bin.filepart /scratch/neiltest$ ls -l total 4048 -rw-------. 1 neiltest neiltest 4125114 Dec 23 2018 testresume10m.bin.filepart /scratch/neiltest$ ls -l total 6136 -rw-------. 1 neiltest neiltest 6384105 Dec 23 15:44 testresume10m.bin.filepart /scratch/neiltest$ ls -l total 10148 -rw-------. 1 neiltest neiltest 10411002 Dec 23 15:44 testresume10m.bin.filepart /scratch/neiltest$ ls -l total 10288 -rw-------. 1 neiltest neiltest 10485760 Dec 23 15:44 testresume10m.bin
Note, the file is uploaded (size increases) to filename.filepart until the entire file is uploaded. Then, it is renamed to its final (actual) name.
Uploading a 10 MB file WITHOUT Resume Capability (disabled, box checked): (ls -l run intermittently during the transfer)
/scratch/neiltest$ ls -l total 0 -rw-------. 1 neiltest neiltest 0 Dec 23 15:40 testresume10m.bin /scratch/neiltest$ ls -l total 228 -rw-------. 1 neiltest neiltest 229173 Dec 23 15:40 testresume10m.bin /scratch/neiltest$ ls -l total 2024 -rw-------. 1 neiltest neiltest 2062557 Dec 23 15:40 testresume10m.bin /scratch/neiltest$ ls -l total 3792 -rw-------. 1 neiltest neiltest 3863202 Dec 23 15:40 testresume10m.bin /scratch/neiltest$ ls -l total 5848 -rw-------. 1 neiltest neiltest 5958498 Dec 23 15:40 testresume10m.bin /scratch/neiltest$ ls -l total 10288 -rw-------. 1 neiltest neiltest 10485760 Dec 23 15:40 testresume10m.bin
Note, the file is uploaded (size increases) to filename during the entire transfer (no temporary file or renaming occurs).
SSH Host Key Fingerprint Validation
If you leave SSH host key fingerprint validation enabled (recommended), ensure you provide the SSH Host Key Finger-print in the correct format. If you receive an error like the following, read on:
Please check your account info and/or permissions and try again. Details: SSH host key fingerprint '...' doesn't match the original one '...'.
There are a plethora of ways to obtain your server’s host key fingerprint. Since I use WinSCP anyway, that is my method of choice. Connect to your server using WinSCP, the open the Session menu and select the Server/Protocol Information item.
Copy/paste the value displayed:
ssh-rsa 2048 3f:62:8e:5f:99:ba:e3:e1:59:51:f1:f8:15:24:d5:86