Sending email whether its part of a registration process or notifying users of important events is a pretty common scenario in most applications.Sending an email in an application appears to be more complicated than one would expect it to be, as mentioned in this post there are several things to consider if you expect the message to be delivered every single time. Doing the same from a cloud platform like azure provides additional challenges, considering the lack of control one would have to face (primarily because you don't get a dedicated IP address in the cloud as mentioned in this post) one of the better solutions from a few mentioned here would be to offload all these complexities to a third party email service provider. There are many out there that provide transactional and marketing email delivery capability while ensuring reliability.
The code to send/forward an email message to a third party service also should be a trivial matter. However along with the transient nature of cloud you may also want to consider some other scenarios that could possibly occur. Imagine the code was to send out an e-mail whenever a user registers. If the mail server is not available for that millisecond, the send fails. Why would that happen? Because the application logic expects that it can communicate with a mail server in a synchronous manner.
Now let’s remove that expectation. We can do that if we introduce a queue in between both services, the front-end can keep accepting registrations even when the mail server is down. And when it’s back up, the queue will be processed and e-mails will be sent out. Also, if you experience high loads, simply scale out the front-end and add more servers there. More e-mail messages will end up in the queue, but they are guaranteed to be processed in the future at the pace of the mail server. With synchronous communication, the mail service would probably experience high loads or even go down when a large number of front-end servers is added.
The basic premise for CloudMail client library is the above idea. The code for this library can be downloaded from here, I will provide a brief design description of this component and follow up with how to plug it into an azure application
Design Considerations
Before delving into the design details, lets look at some of the goals
1. Reliability - as mentioned above we would want to ensure that every email message is delivered irrespective of the transient issues with the cloud application or the SMTP service
2. Async - The email sending should be non blocking
3. Scalability - The component should be able to handle high loads in case the front end scales up based on application operating requirements
4. Capability to handle large messages - At times we do send some attachments along with the email messages, the component should cater to these without having a limit on the size, this leads us to an important design consideration which I will discuss later.
Points 1,2 & 3 are easily achieved by using a queue, as discussed earlier the queue decouples the application from the SMTP service thereby enabling to cater to demands of both the application and the SMTP service and ensuring that message is eventually sent to the service.
From a design perspective we could use the storage queue service a.k.a windows azure queue or the service bus queue (basically a more heavy duty queue with a larger message size, infinite time to live & many other interesting messaging features). I have chosen the storage queue service as it should satisfy our component's requirements
Coming to point number 4 we could split this to two other requirements, one is that the object to be used to relay the message should be flexible enough to handle additional functional requirements of an email message such as attachments apart from the standard recipient mail address, sender mail address etc. This lead me to use the standard implementation within the framework which is System.Net.Mail.MailMessage, this type should suffice with the wide range of properties it offers, however here we run into a problem, if we intend to use this type within azure queues then we should be able to serialize it, but the problem is MailMessage type is not serializable.
To work around this I have implemented a serialized MailMessage type CloudMail.MailMessageSerialization.SerializableMailMessage. This type's constructor accepts a MailMessage and converts it into a serializable equivalent,
To deserialize and get back the original type there is the GetMailMessage method, which would return the MailMessage Type
public MailMessage GetMailMessage()Using MailMessage would give us the desired flexibility to handle a large number of email messaging scenarios which leads us to the next sub requirement of infinite/no message size limit. This cannot be achieved if we decide to use the queue as it has a max message size limit of 64 KB (also a message time to live of 7 days which is something I feel we can live with in this context except for emails scheduled for future which again could fit into this design).
How about using another storage service such as blobs for this purpose, but then its not always that we would have 10 MB message being sent out from the application, therefore there should be some mechanism to toggle between the two storage services based on the email message size. There is very good article on the topic of handling large messages here which discusses in detail how to choose the appropriate message store based on the message size the solution discussed in the article is pretty generic in the sense it allows several overflow stores to be configured along with the overflow criteria based on the message size. In this case i have implemented just the blob overflow store, so if the incoming message to the queue is > 64 KB then the message is stored in the blob ,additionally this piece also takes care of compressing the messages before uploading to a blob thereby catering to the cost concerns of storage, also the metadata of this blob goes into the queue so when a worker reads the queue it should be able to figure out where the actual message is stored.
Finally the library has an inbuilt multi-threaded task scheduler-dispatcher which would basically be invoked on the worker and would continuously perform the task of retrieving the messages from the store and forwarding it to the SMTP service provider configured for the application. The scheduler-dispatcher handles the message forwarding part using this piece
/// <summary>
/// The relay mail.
/// </summary>
/// <param name="mailMessage">
/// The mail message.
/// </param>
public void RelayMail(MailMessage mailMessage)
{
if (mailMessage == null)
{
throw new ArgumentNullException("mailMessage");
}
// Invoke the smtp client to send mail
using (var client = new SmtpClient(this.emailServiceInfo.MailHostAddress, this.emailServiceInfo.Port))
{
client.Credentials = (!string.IsNullOrEmpty(this.emailServiceInfo.Domain))? new NetworkCredential(this.emailServiceInfo.SmtpUserName, this.emailServiceInfo.SmtpUserPassword)
: new NetworkCredential( this.emailServiceInfo.SmtpUserName, this.emailServiceInfo.SmtpUserPassword, this.emailServiceInfo.Domain);
mailMessage.Body = mailMessage.Body.Trim();
if (!mailMessage.IsBodyHtml && mailMessage.Body.StartsWith("<html>", true, CultureInfo.CurrentCulture) && mailMessage.Body.EndsWith("</html>", true, CultureInfo.CurrentCulture))
{
mailMessage.IsBodyHtml = true;
}
RetryAction.Execute(() => client.Send(mailMessage));
}
}One could very well define his own email message forwarding mechanism by implementing the IEmailRelay interface and passing the type that implements this interface as an argument to the Task Scheduler-Dispatcher Type's constructor
Implementation
Refer the CloudMail assembly in your worker and other roles as required.The library would have two primary functions a) en queue the message to azure storage b) retrieve the messages as a batch and forward it to the SMTP service provider. Typically the first part would be done closer to the functional part of the application that requires sending emails so, if we revisit the user registration scenario, after adding registration details to your database you would want to do a).
The code would look something similar to the one below
/// <summary>
/// The btnRegister_ click.
/// </summary>
/// <param name="sender">
/// The sender.
/// </param>
/// <param name="e">
/// The e.
/// </param>
protected void btnRegister_Click(object sender, EventArgs e)
{
DoRegister();
var emailClient = new EmailClient(RoleEnvironment.GetConfigurationSettingValue("MyAzureStorage"));
emailClient.SendMail(new MailMessage(from, to, subject, message));
}The part b) of the above is taken care by a worker where as mentioned earlier a task scheduler-dispatcher would be invoked to continuously monitor, retrieve and forward messages to the SMTP service provider, you would have to pass in your storage connection string and the appropriate EmailService instance which is the container for email service related information into the task dispatcher constructor so the worker run method would look somthing like this
/// <summary>
/// The run.
/// </summary>
public override void Run()
{
// This is where the Email Dispatcher starts
Trace.WriteLine("EmailWorker entry point called", "Information");
this.cancellationToken = new CancellationTokenSource();
var emailDispatcher = new EmailDispatcher(RoleEnvironment.GetConfigurationSettingValue("MyAzureStorage"), new EmailService("smtp.sendgrid.net", 25, "MyUserName", "MyPassword"));
emailDispatcher.Start();
this.cancellationToken.Token.WaitHandle.WaitOne();
}