FetchThe sad fact is, FTP and firewalls just weren’t meant to go together.
FTP, the original Internet protocol for transferring files, uses two connections between the client computer and the server. The first connection, called the control connection, is used for commands: the client tells the server what it wants. When the client sends a command to transfer a file, or list all the files in a folder, a second connection, called the data connection, is opened to carry the contents of the file or file list. If calling your local movie theater for showtimes worked like FTP, it’d go something like this:
Theater: Hello.
You: My other phone number is 555-1234, and I’d like to know when The Dark Knight is playing tonight.
Theater: We have that information; we’ll call 555-1234 and tell you.
That pattern — having the server connect back to you — is called active FTP. But what if you’re running a firewall? A firewall blocks incoming connections, turning your computer into the equivalent of a phone without a ringer: it can place calls, but not answer them. So FTP has another pattern, called passive FTP, that works like this:
Theater: Hello.
You: Can you give me another number I can call you at?
Theater: Sure, you can call 555-4321.
You: I’d like to know when The Dark Knight is playing tonight.
Theater: We have that information; call the number we gave you and we’ll tell you.
With passive FTP, the client is originating all the connections. Its ringer doesn’t need to work, because it’s placing all the calls. Way back in 1994 an Internet standards document titled “Firewall-Friendly FTP” recommended that all FTP programs work this way.
And what if the server is behind a firewall? “Firewall-Friendly FTP’ expected server administrators to configure their firewalls appropriately to allow passive FTP; after all, they’re server administrators, configuring servers and firewalls is what they do. Unfortunately that expectation has not been borne out. We increasingly hear from users trying to connect to servers that don’t support passive FTP because the servers are behind firewalls blocking all incoming data connections. It seems to be a particularly common problem with Windows servers, whose administrators have been burned by various security vulnerabilities, leading to a “block connections first, ask questions later’ approach to firewall configuration.
Fetch is clever enough to switch between active and passive FTP when one of them is not working, so if there’s just one firewall in the picture, at either the client or the server end, everything should just work. But if there are firewalls at both ends, it’s like having two phones without ringers — there’s no way for a call to go through.
Before Mac OS X 10.5 Leopard shipped last October, our standard advice for users stuck between two firewalls was to turn off the Mac OS X firewall, at least temporarily. While some security experts recommended having the firewall turned on, it was not turned on by default, and we were not aware of any case where it kept a user’s machine from being compromised. Still, we understood why users would want to be protected from possible future attacks, and knew that in some cases it was office policy to always run a firewall. We wished we had a better answer for those users.
The good news is, Mac OS X 10.5 provides a better answer. While previous Mac OS X firewalls blocked connections based on where the connection was directed — like a phone system that only rings for certain extensions — the Leopard Application Firewall can block or unblock connections based on what application is listening. So instead of turning the firewall off altogether, you can add Fetch to a list of applications that are trusted to receive connections. Now the “phone’ will only ring if it’s Fetch (or another trusted application) waiting for the call, and Fetch can connect successfully even though there are firewalls on both ends.
We continue to encourage hosting companies to configure their server-side firewalls so that passive FTP will work. We also encourage our customers to choose hosting companies that support SFTP (examples include Pair and Dreamhost). SFTP uses a single connection, so it doesn’t have to worry about firewalls blocking a second connection, and it is more secure to boot. But for our customers stuck using FTP with servers behind firewalls, the Leopard Application Firewall is a welcome improvement.