12 Apr 2009

CFMAIL spool bug in ColdFusion 8.01

After applying a patch to fix a cfmail problem in 8.01 I noticed a new problem. When creating more than 3000 or so emails in one go about 100 ended up in the Undelivr directory. There was nothing wrong with those emails, dropping them into the spool directory by hand sent them quickly on their way.

The exception.log was showing “IOException while sending message”. I reported this to an Adobe engineer in July 2008. In October the engineer found that the issue can be “fixed” by increasing the max session size in the Microsoft SMTP service. Microsoft don’t recommend increasing it too much but the more I increased it the less emails were sent to the Undelivr directory.

The bug is with ColdFusion and it appears to not behave correctly if the SMTP server closes a session because it has reached the maximum size. ColdFusion should react by simply opening up a new session and continue to send emails from where it last left off. But it doesn’t do that. This bug has been affecting a major application for nearly a year and Adobe haven’t been able to fix it.

MS SMTP logs “552 4.3.1 Session size exceeds fixed maximum session size” which is fine, it expects the app at the other end (CF) to continue by opening a new session. I’ve now increased the session limit to a very large amount but as the application is used more the limit is reached and emails stop being sent again. CF is misbehaving and is causing operational headaches.

Has anyone else had similar problems? I’m looking for others who can help add some pressure to get this bug fixed. Please do get in touch. Needless to say I’m very disappointed that CF is unable to send a large amount of (legitimate, non-marketing) emails problems.

8 comments:

  1. I have a couple sites (on separate servers) that send in excess of 4k+ (each) digest type emails a night, as well as the several hundreds of emails during the day. I have not experienced this issue as you have, although I don't use the the Microsoft SMTP service for sending emails. Currently we use Alt-N MDaemon Email Server. Maybe try another mail server?

    ReplyDelete
  2. What size are your emails? Ours are at least 30KB each so the total exceeds the max session size set on the SMTP server. I've been able to repeat the problem on our development servers too which run on different hardware.

    Yes, perhaps using a different SMTP server may help but a fix for ColdFusion would be the preferred option. Thanks for recommending MDaemon, I hadn't come across that one before.

    ReplyDelete
  3. I had this exact problem last week. I am running an Intranet outlook like application.

    10% of the mails ends up in the undelvr folder. So i deciede to write a window task, that will move all files in the undelvr folder to spool folder every 5min, and this solved it.

    I think adobe should really look into this issue

    ReplyDelete
  4. That's a good practical fix but because undeliverable emails will constantly get moved into spool you won't be able to see which emails are genuinely undeliverable and may be on an endless journey back and forth.

    Adobe did come up with a hotfix but it wasn't successful on our productions servers. I've not heard from them for months. :-( I wish existing bugs were prioritised over new features.

    ReplyDelete
  5. Are you basically requesting that CF handle the 552 error, and keep processing the request.

    I am assuming cfcatch is not catching 552.

    What if you wrap your cfmail calls in a cfthread, that might provide the insulation you need.

    ReplyDelete
  6. @Mike, yes, CF *should* be able to handle the 552 error. The SMTP service says "you've exceeded the max session size". Therefore CF needs to say "okay, I'll start a new session and start by resending the last email that didn't go through". How difficult is that? This is what applications that talk to SMTP servers are supposed to do.

    I don't see how using cfthread will fix the problem? I'm using an asynchronous gateway event to send emails which is more or less the same principle. I use the "query" attribute of cfmail to send an email to every user identified in a query result. Just to be clear, the cfmail tag isn't looped over for each recipient, it's called once only for effeciency (but internally I guess CF loops over the query and Java does its mail thing for each recipient).

    ReplyDelete
  7. Hi Gary:
    We run CF8. And I have been having this problem with the mail lately, it is hit and miss.
    The error in the mail log says "IOException while sending message". The other thing is if I manually try to put it in the spool. It goes into the undeliverable folder again.
    And the error says
    "The ColdFusion mail spool encountered an invalid spool file in the spool directory"
    There was fix i found on the internet
    http://helpx.adobe.com/coldfusion/kb/patch-cfmail-error-coldfusion-8.html
    Does this fix the problem. Or is there anything else i need to do. COuld you please reply to this. In the other company we had CF7, never had this problem at all. Please can you make the CAPTCHA little bit readable

    ReplyDelete
  8. Have you tried increasing the max session size in the Microsoft SMTP service? I increased mine by a factor of 10 (starting at x2, x5, then x10) to realise this was a good number as it "fixed" the problem. You should also make sure all the CF hotfix updates are applied to your server. I think the cumulative update v4 was the latest (from a few years back). Good luck.

    Unfortunately I have no control over the CAPTCHA used by Blogspot, sorry. :-(

    ReplyDelete