File Transfer, MFT and Beyond: The Story of AS2

This is a work of fiction. The incidents portrayed herein may have taken place under different settings, in a different order, or even not taken place at all; however, it will not refute the origin and evolution of Applicability Statement 2, more commonly known as AS2, the most popular secure B2B document/EDI exchange protocol in the known universe.

This is a work of fiction. The incidents portrayed herein may have taken place under different settings, in a different order, or even not taken place at all; however, it will not refute the origin and evolution of Applicability Statement 2, more commonly known as AS2, the most popular secure B2B document/EDI exchange protocol in the known universe.

The Beginning: File Transfer, the Primordial Soup

Earliest files were paperbacked, and dispatched around via postal and courier services. They took days - or hours at best - to reach their destinations. Needless to say, a major bottleneck in rapidly growing B2B trade ecosystems.

Digital File Transfer

Then came computers and the internet, and things improved dramatically. (There were probably many more intermediates; fax, email, etc.; but we’ll skip them for brevity.)

Now the transfers took just minutes - sometimes even seconds - instead of hours or days. Accuracy also improved greatly - no more deliveries to the wrong doorstep. Protocols like FTP and email soon became the de-factos for digital data exchange and file transfer.

But still, not everything was solid and foolproof:

Acknowledgement and Integrity

There was no receipt or acknowledgement/confirmation; you could send a PO, wait several days for the invoice, and finally phone them to find out they have somehow missed the file - still lying in a corner on their FTP server.

Also, the network did not guarantee integrity (delivery “as a whole”); you could have sent a 1000-row file, and the last 10 - or even 10 randoms from all over the place - could have gotten “lost”. Even worse, a random flip of a few bytes could convert some five-digit price tag into a nine-digit one - I won’t even start on the consequences.

Receipts and Hashes

So people and systems started to acknowledge these transfers with digital receipts; if you sent a file and received an acknowledgement (possibly another file) in return, you had a guarantee (non-repudiation) that the trading partner (or his system) has seen the file and taken it up for processing.

Now that there was two-way communication, there was also a way to verify that the receiver got exactly what the sender sent out (integrity); the receiver would calculate a hash (a finite-length summary; think of finding the “lucky number” from your birthday) of the received file, and send it back in the receipt. The sender could then do the same on the original file she sent, and verify that the two match - which would mean the content on both sides is practically identical (down to just a subatomic mergin of error).

Two problems solved; but still, there were more serious ones lurking around.

Eavesdropping, Tampering and Spoofing

File content was flowing through the (inherently unsecure) internet, in cleartext. This meant, any of your adversaries could eavesdrop on the files you are sending - and worse, even modify (tamper) the data being transmitted, to sabotage or manipulate your transactions.

True, you are already exchanging and validating the hash for the sent/received content; but a decent man-in-the-middle could easily pause the original data flow, do his modification and generate a new hash that would match the modified data.

(Or, if he had complete control over the data flow, he could simply intercept the partner’s receipt as well, replace the hash with the old one, and relay it back to you, the unsuspecting victim - who would naturally assume everything went along just fine.)

On the other hand, when it comes to receiving files or receipts back from your partner, you had to somehow make sure that it was actually your partner sending those data in the first place - not something spoofed by a malicious third party.

Digital Signature: for Tamper-proof Integrity

What we needed here was simple: a way for you (sender) to place a “seal” on the file (very much like the wax seals from the old snail-mail days) and for your partner (recipient) to verify that the seal was not broken while things were in transit.

So the digital signature was born. Basically you take the hash of the file (as before), encrypt it with your private key, and send it alongside; and publish the corresponding public key (certificate) so that other people - usually your partners - can decrypt (verify) that blurb of data upon receipt.

This was a rather smart move! It achieved many goals:

• Since only you had the private key, the partner now had a strong guarantee that the file came from you - and you couldn’t refute that either, even if you wanted to; authentication and non-repudiation.
• The “safely transmitted” hash guaranteed the file’s intactness; integrity.
• Even if there was no anti-tamper encryption (next section) in place, if anyone managed to modify the file and generate a new matching hash, they would not be able to generate a verifiable signature out of it - without breaking into your private space and stealing your very own private key.

Good; now if we could just make the file unreadable to prying eyes, in transit…

Encryption: for Invisibility (Confidentiality)

As the title suggests, encrypting the file content makes it invisible (more correctly, incomprehensible) while it flows through the internet. Which means there is no room for the adversary to read or modify the file content.

Of course there had already been ‘secure’ protocols like SCP, SFTP and FTP/S that would transfer encrypted content (ciphertext) instead of cleartext; however, you still had to make sure that you were connecting and sending to the correct remote partner/system; jumping over traps like DNS spoofing.

But with AS2 encryption, even if you connected to a wrong server, your content was encrypted using a certificate (public key) that was pre-shared with you by the real partner - and not the certificate of the currently connected server. So, even if you did accidentally send the file to a hacker, they could not possibly decrypt it without access to the partner’s own secret private key - which never comes out in the open, throughout the whole process.

AS2: there you are!

What we (or rather, many ingenious people over several decades) invented so far - adds up to the AS2 protocol - which happens to be the most popular secure file transfer protocol today. Especially after Walmart the Giant mandated it for all its trading partnerssince 2002.

You can learn more about the protocol from its official RFC 4130 document, which is very informative and fairly simple to understand.