Email - From, To Who?

One of the most (if not the most) popular means of communication over the Internet is the email messages. According to a study made by Redicati (http://bit.ly/1l2gJ2z), the number of email accounts will hit little over 4.3 billion in 2015 generating 194 billions of emails per day.

Typically, to access the content of an email account, a user makes use of a mail client. These, allow users to compose, send, receive and read emails. They can either be local clients like Outlook Express or Thunderbird or via cloud based clients like gmail.com. But how does it all work?

The email service is built upon two different phases, the send and receive phases. To send an email, the Simple Mail Transfer Protocol (SMTP) is used while, to receive, depending on the email client, it is possible to use the Post Office Protocol (POP) or the Internet Message Access Protocol (IMAP).

Send

To send an email, the SMTP protocol is used. This protocol delivers the message between the sender and the receiver email server. This protocol may in fact be used more than once during the email transfer.

When an user sends an email, the client or mail-user-agent (MUA) connects to the SMTP server on port 587 (port 25 or 465 should no longer be used for this purpose) where a mail transfer agent (MTA) listening for new outgoing email messages. Once connected, a series of messages are exchanged between the client (sends 4byte commands) and the server (replies with a number and a description) in order to establish the email transfer parameters and transfer the email message from the MUA to the MTA.

Once the message is on the MTA, the receiver address is splitted into the user and the domain. After, the MTA consults a DNS in order to convert the domain into a mx record (mail exchanger record) that, is nothing more than the ip address of the SMTP server of the receiver of the email.

With the mx record, the MTA connects to the receiver SMTP server on port 25 and transfer the email message. This process of transferring an email message from one server to another is known as relay.

Once on the receiver server, two things may happen:

1. This SMTP server aggregates more than one email domain and executes the protocol again to deliver the email message to another SMTP server;
2. This is the final email server and in which case, the email message is delivered to the POP/IMAP server where it awaits for the MUA to connect and download the email for reading.

Simplified SMTP flow

Receive

When an email is delivered to the receiver MTA, it goes to a processing queue (that are composed by flat files) where, it awaits for the receiver to retrieve it.

As stated before, there are two different protocol to retrieve email. Each, runs on a separate server from the SMTP server (may be on the same physical machine) and are intended to be used on different situations.

Simplified Receiver Email Server Architecture

POP3

The first protocol is the Post Office Protocol which, currently is on version 3 (POP3). This protocol is intended to be used by remote clients and works on a download-and-delete basis.

When a client wants to read is email, the MUA connects to the POP3 server either on port 110 (clear text connection) or port 995 (encrypted connection). Once the connection is established, the server enters the Authorization phase where the MUA authenticates to the server.

If the MUA has provided valid credentials, the server enters the Transaction phase where it acquires an exclusive lock to the maildrop (where the mail of the client is stored). On this phase, the MUA will download the new emails and mark them to be deleted.

After the connection is closed, the server enters the Update phase where it will delete all marked messages.

Although it is possible to maintain the download messages, the protocol was designed to save server space and, thus, it deletes the messages by omission. The clients may change this behaviour on the MUA by indicating that it wishes to save a copy on the server.

The POP3 is a very simple protocol and as such, it has several limitations like for example:

1. Doesn't know the concept of folders or flags;
2. Only allows for one MUA to be connected to a specific account at any given time;
3. It can't read the different parts of a MIME envelope, enforcing the download of the full email message and not just part of it (example: the header).

IMAP

As POP3, IMAP is also designed to be used by remote clients. However, when compared with POP3 it:

1. Doesn't follow the download-and-delete philosophy, maintaining the messages on the server after retrieving them;
2. Offers a more complete set of functionalities, allowing it to, for example, manipulate folders and flags;
3. Allows for more than one MUA to be connected at the same time, thus creating the possibility for a client to connect more than one device to the mail server at the same time;
4. Can read the different parts of a MIME envelope and download each one as needed

Aside these differences, the protocol works pretty much the same as POP3. The client MUA connects to the server on port 143 (clear text) or 993 (encrypted), authenticates, manipulates the mailbox and logs out.

We have spent the better part of this post giving a brief overview of the send and receive phases, but we haven't explained what an email message is.

So, what exactly is an email message? In simple words its plaintext. In an in more depth analyses, an email message is divided into two different parts, the header and the body.

The header contains structured information in the form of fields. These, are composed by the pair name: value and in them, we find the To, From, CC, BCC, Subject and so on.

The body contains the email content. The initial specification (RFC 822) indicated that it could only contain ASCII characters which, meant that only simple text could be sent. However, and since this imposes a lot of limitations, the Multi-Purpose Internet Mail Extension (MIME for short) was developed.

MIME among other things, has enabled email messages to carry for example multimedia, images, styled text, documents and other types of content. It manages this by adding a multi-part support where, a message is divided into several parts. Each, can have a different format that is specified in a "part" header (remember, more than ASCII).

MIME part example

The above picture displays an example of a raw MIME email part. It starts with a boundary that indicates where the previous part ends. After, we have a part header that describes the content of the MIME part followed by the content and at the end, we will have another boundary.

The interesting thing is the "Content-Transfer-Encoding: base64" which indicates how the attachment is sent with the email message. Remember, that an email can only contain simple text and, as such, an attachment needs to be converted to simple text, and this is the way to do it. The file is converted to base64, written on the email and sent as text. Once it arrives, it is reconverted to a file.

On this post we have described the basic functionality of the email service. Our next post address the security issues related to the email and present a few solutions to enhance it.