Connected Community Email - Improving Reliability, Scalability, Deliverability, and Transparency

By Ben Goodkind posted Sep 10, 2013 10:05

  
Higher Logic communities generate tens of millions of emails per month.  Ensuring deliverability - the likelihood that an email message arrives at its intended destination -  has been a top priority since Higher Logic started.  This seemingly simple task is complicated by the fact that a large percentage of global email is spam.  Internet Service Providers (ISPs) have implemented strategies - DNS related lookups, content filtering, rate limiting, reputation management, feedback loops, etc. - to combat this global nuisance and protect their subscribers.  This cat and mouse game that has emerged between spammers and ISPs creates challenges for legitimate businesses that send large volumes of email and for members that often wonder why they did not receive an expected email.

Higher Logic expends a great deal of energy making sure your members get their email.  Our goals are straightforward: a ConnectedCommunity member should receive their email with minimal delay and there should be transparency regarding deliverability issues.  To date, we've done well on the first goal but less well on the second goal.  I'm happy to announce some upcoming changes that will be phased in over the next few months that will improve both goals.

Higher Logic will be switching all of our clients’ outbound Connected Community discussion email to Amazon Simple Email Service (SES).  Our new email platform will be running on the same platform that Amazon.com utilizes for their worldwide sites.  Speaking geek for a bit, Amazon manages their IP reputation and works with all the major ISPs to ensure successful delivery of massive volumes of emails.  Amazon allows Higher Logic to utilize DKIM signed emails and provides Higher Logic with a consolidated feedback loop for email delivery failures and complaint handling.  What does that geek-speak mean for you?  Improved reliability, scalability, deliverability, and transparency into delivery issues.  

We have been phasing in changes so that there is no disruption and the platform is well tested.  All of our client email unrelated to community discussions (community invitations, direct messages) has been going through AWS for many months.  All email from the HUG site has been using SES for the last several months.  Our next step will be to transition clients sending discussion email through the egroups.connectedcommunity.org domain to AWS.  Following that, we will transition clients sending through their own domain.  There are no new steps required for you or your members.  Everybody should whitelist the domain connectedcommunity.org.  If your members require specific IP addresses to whitelist then they should refer to this blog:  http://sesblog.amazon.com/post/TxEH4YOF3YJG0L/Amazon-SES-IP-addresses.

Regarding transparency, we have developed new reports that show which of your members’ email addresses have deliverability issues as well as the details of the root cause of the issue.  Access is provided to the administrator for all email addresses and to each member for their own email addresses.  This functionality is currently in testing and will be delivered in an upcoming release.  I will provide more details in a future blog.

This is the first of several enhancements related to email.  Stay tuned.
4 comments
276 views

Permalink

Comments

Sep 17, 2013 10:41

Anne-Margaret - We have built open rate tracking into the new email architecture. Currently we are collecting the data but have not built the user interface to expose the data to our customers. Please email me directly with any specific requirements we should consider when building the interface. It is on our list of enhancements for the email platform but I don't have a timeline to share yet.
Open rates have traditionally been used to measure marketing email campaign effectiveness. Their accuracy is impacted by things such as image blocking, reading emails through the preview pane, and mobile phone email applications. For a community platform there are even more variables to consider regarding a message's reach. Members may read the content through the website, the mobile application/website, or an RSS feed. None of these would contribute towards the open rate.
For those interested in this metric, I suggest reading the Wikipedia entry, http://en.wikipedia.org/wiki/Open_rate, and the references related to "Tracking Concerns". You should be aware of open rate pitfalls if using this as a reporting metric for your community platform.

Sep 16, 2013 10:47

Ben -- will the new email system provide us the opportunity to track open rates?

Sep 11, 2013 10:20

Hi Diana -
There is new functionality related to the "From" email address. After the switch, real time emails will be from "[first name] [last name] via [community site name]" and digests emails will be from "[community site name]" This is configurable. Your comment email to me was received from "Diana Mertz via Higher Logic Users Group". Take a look at HUG real time and daily digest emails for an example.
The actual from email address will be the connectedcommunity.org domain. There are a lot of technical configurations related to delivery that Higher Logic maintains and we need to be able to react quickly should any issues appear.
Our goal is to give our clients control of branding (images, colors, text in the from field, etc.) and allow us to maintain control of the technical aspects impacting delivery.
Ben

Sep 11, 2013 09:00

With this switch to Amazon SES will we have the ability to customize the email address that is delivering the digests (in place of the noreply@connectedcommunity.org email)?