OS X and transferring files (with Transmit)

In my last post I mentioned how one of my server hard drives died and I ended up purchasing a replacement. I had done some thinking on what I wanted to do with regards on how to utilize the new drive, and that ClearOS 7 was now available. I decided to use the new 3TB drive as added storage, and migrate some files. I wanted to merge my Plex media to one source instead of being spread over three drives like it was. To do this was very easy, just time consuming due to the amount of movies I needed to move, somewhere around 450GB worth. I didn’t even think of using the terminal and issuing a “mv” command as I am sure it would have been faster and this issue wouldn’t have occurred. I fired up Transmit FTP and opened two windows. One window was the old drive and the other the new. I simply drag and dropped the files, copying them, and let it run while I went to sleep. I awoke the next day to an error of no free disk space on my Mac Pro. WTF? Upon investigation I found that Transmit was caching the file transfers locally on my Mac. Thats just fucking stupid. So I’m assuming that you cannot transfer more files than you have free disk space, because apparently it will run your drive down and crash your shit.

A reboot will clear the cache (for Transmit) or you can manually delete the cache. Its located at:
~/Library/Cache

There is a ton of other crap in there as well. Most of it should be safe to delete, it is just cache right?

ClearOS and FTP

Getting that same problem? Well apparently you are not alone, seems ProFTP and ClearOS don’t really work well out of the box. FileZilla would constantly timeout on listing the directories, so it would never actually connect. Transmit would timeout too, but it would spit out different errors, usually Error 162 Port Fail. After hours of digging I did find the problem, and the solution. It was actually pretty easy to fix (makes you wonder why the fix is not implemented?). Depending on your ClearOS setup YMMV and other steps may be required. I am running my ClearOS box in gateway mode, it is my server and router.

Start by forwarding all those ports required. Clear uses different ports for different things with FTP.
21 standard ftp port (internet say this is for home folders, but it connects me to flexshares).
60,000-61,000 are used for port 21 PASSIVE connections
2121 standard ftp port (internet says its for flexshares).
65,000-65,100 are used for port 21 PASSIVE connections.
989/990 SFTP ports.

To ditch the unroutable address warning toss this in your proftpd.conf file:
sudo nano /etc/proftpd.conf
MasqueradeAddress ftp.mydomain.com # using a DNS name

This did allow me to finally connect with Transmit with no issues. A friend of mine was able to connect with FileZilla after I made the changes. I had to change my FileZilla settings to use an Active connection versus a Passive or mine would refuse to connect. Odd.

I found these sites to be helpful, and thats where I got my information.
http://www.proftpd.org/docs/howto/NAT.html
https://www.clearos.com/clearfoundation/social/community/remote-ftp-access-through-nat-router-solved
https://www.clearos.com/clearfoundation/social/community/ftp-problem
http://forum.slicehost.com/index.php?p=/discussion/4491/ftp-error-162-port-failed/p1
http://www.theserverpages.com/articles/servers/cpanel/tweaks/Getting_passive_FTP_connections_to_work_through_a_firewall_properly.html
https://forum.filezilla-project.org/viewtopic.php?t=7315