Digital Integration Support Forums


Product support for software brought to you by Digital Integration

SMTP and line length limits

As software developers we come across lots of useful snippets of information and ways of using technology. This forum is a repository of tips that we've found helpful that might also be helpful to anyone scouring the web for answers.

SMTP and line length limits

Postby Josh Gibbs » Mon Mar 05, 2012 5:33 pm

This weeks tip is a call to arms for compliance with e-mail standards - specifically the defined in stone limits of both RFC 2821 (4.5.3.1) and RFC 2822 (2.1.1). Those rules have been in place since 2001 and did not change the original specification from 1982.

Over the years we've had to deal with a range of systems that produce e-mail that does not conform to the MIME and SMTP specifications, and in fact have had to fix bugs in iMail that have been at fault. Our stance on compliance is that if we generate faulty MIME messages then it is our responsibility to fix the problem and we'd like to make sure others start holding themselves to these same standards. This post is thanks in part to some frustrating correspondence with this very topic recently ourselves.

Let's start off with the rule. Both of the abovementioned RFC's state quite clearly that a line in an e-mail message MUST NOT exceed 1000 characters (incuding the end of line <crlf> pair).

So why would we want to set a limit at all? This is explained by looking at how e-mail systems receive messages over the SMTP protocol. To begin sending data, a client says to the server 'DATA' and it then receives a reply from the server, '354' stating that the server is now ready to receive. At this point the client starts dumping data at the server until it's done. How does the server know it's done? The end of transmission is indicated by a special line which contains a single period on a line by itself.

To detect the terminating line, the method in which a server must receive a message is on a line-by-line basis so that it can check for the final transmission unit. Prior to that last line being received, the server process must know ahead of time how much data it MAY receive on any given line so that it can prepare its memory to receive each line. It is certainly possible to increase the receive size of each line on any particular server, but this creates a situation where one server may accept 8000 characters from its client, only to find the next server that obeys the 1000 character rule won't accept the message. The result of servers (yes Microsoft Exchange, I'm pointing my finger squarely at you) increasing their default limit, and allowing adjustment of the limit is a mess of servers that allow different sized messages to be created by developers who are ignorant of the rules in their message project they've been assigned to write. This has started creating a frustrating cycle of invalid e-mails requiring ever increasing line lengths to accommodate the next incompetent developer. Ignorance of the rule is an acceptable excuse, but a) not fixing it escalates the write to incompetence and b) server implementations letting this go is utterly indefensible.

On top of the inconvenience at the server level of not knowing how many characters your server may need to receive per line, then there's the mail client itself. Once the final destination server has received an invalid message, there has to be the assumption that the client can understand the message - which it's not obliged to do thanks to RFC 2822 that imposes the exact same limit. As developers we write our software based on the specifications for interoperability and any relaxation of the rules creates havoc for a very wide audience.

"But what if you must send a line that's longer than 1000 characters? Is there no way to send that", I hear you ask. Well as it happens, this limit has nothing to do with what can be written or displayed on a mail client. It's all about the transmission of the data. You can send a million character line in an e-mail if you have a need to, but the rule is that it must be broken down into lines under 1000 characters each and there are a variety of standard ways that this is done. In fact all popular mail clients today will break the lines up at around the 80 character mark anyway so that the message is readable in a simple text editor. All e-mail client programs know how to stitch the lines back together on their end once the message has been received.

This problem would go away promptly if mail server implementers followed the rules and would quickly turn the ignorant mail designer into an informed mail designer.

Apart from extensions to the RFC (which I don't believe are necessary) I don't know the solution to combat this insidious problem, but hopefully this post will help start some kind of ground swell of understanding to reduce the impact and dissemination of mail systems breaking not only this spec, but any they see to make it convenient for them at the time.
Josh Gibbs
Site Admin
 
Posts: 46
Joined: Fri Oct 15, 2010 1:37 pm

Return to Tech tips

Who is online

Users browsing this forum: No registered users and 2 guests

cron