In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
Marketer’s Field Guide
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
Thanks to better ways to filter and block spam, inboxes are nearly free of unwanted mail, but not without a price. According to past Return Path studies, permissioned based email never reaches its intended destination — your customers’ inboxes — about 20% of the time.
Email deliverability today is driven by content and reputation filtering. While it would be great if there was a single algorithm that all of the mailbox providers used, there isn’t. Gmail, Outlook.com, and Yahoo! Mail all have their own unique spam filtering rules and services to help senders. Our field guide aims to help businesses overcome confusion from the different rules of deliverability, or those who are just strapped for time.
Introduction
In
tr
oduc
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
Gmail
Gmail by the Numbers
Launched: 2004Active Users as of 2012: 425 Million
U.S. mobile users using the Gmail app: 44.3%*
How many days prior to Gmail’s launch it took to design the Gmail logo: 1
Number of email addresses that can be created by adding a + symbol in front of the email address: ∞
The year Gmail purchased the domain Gmail.com from Garfield.com: 2005
Unlike many of its large competitors, Gmail doesn’t provide feedback loops, whitelists, disclose use of public blacklists or provide a lot of direction, tools or support for senders. Don’t worry about the lack of information Gmail provides. We’re here to help.
How Google Works - The Story of Send
G M
ail
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
Gmail, like many major mailbox providers, primarily uses its community of users to determine whether email is spam or not. Gmail considers the following user feedback important in their spam filtering decisions:
1. Report spam 2. Not spam 3. Messages Read
4. Messages Read and Deleted 5. Starred messages
6. Messages replied to 7. Inbox Activity Content
As with most spam filters, the content of the email headers, body and attachments are also scoured for keywords, images, HTML, URLs, malware and many other common components. Content is always tied and added to the overall spam filtering algorithm, but how much weight it carries usually depends on the reputation of the sender.
How Gmail Filters Spam
Sending History
Gmail temporarily blocks new IP addresses without a reputation for the first 2 – 24 hours, then delivers small amounts of mail to both the inbox and the spam folders to gauge subscriber perceptions. If complaints are too high, future emails will be delivered mostly to spam. If more people click on “not spam” during this period, this indicates that the mail is safe to be delivered to the inbox.
Engagement
The number of active Gmail users may influence your reputation, and therefore your inbox placement. A look at top brands and the percent of their subscribers that are engaged – showing any activity regardless if its negative or positive – shows a strong correlation.
Chart: Brands with higher percentages of active users are more
likely to receive inbox placement.
G M
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
Blacklists None.
However, our research shows a correlation between spam filtering at Gmail and Spamhaus.
Enterprise Filters
As an aside, it’s important to understand that although Google owns both Gmail and Postini products, each service utilizes its own unique proprietary filtering technology. Google Postini Services is a B2B spam filtering solution used primarily by corporations to manage their enterprise mail. However, Google is switching current subscribers of Google Postini services to Google Apps by the end of 2013.
If you find your mail getting stuck in Google Postini Services’ spam filter, Google does provide a Message Analyzer that allows senders to examine the tags inserted into an email header to help decipher the reasons for the quarantine. You may find a hint of the issues with this tool but the only way to figure out how to get through this filter is to constantly test different email content.
G M
ail
When it comes to having a proper email set up, Gmail’s advises: 1. Consistently send from the same IP address(es)
2. Have valid rDNS for sending IP addresses pointing to your domain
3. Use the same “From:” domain based on the type of email 4. Authenticate sending domains with DKIM and SPF More on Authentication
SPF and DKIM authentication should be implemented. Be sure that your From: domain matches the d= in your DKIM record. Senders using shared IPs at ESPs should sign DKIM with their own unique signing domain. For example, one should be signing mail with something like d=esp.client.com instead of d=esp.com. If for some reason you are not authenticating then Gmail creates a “best-guess” record using a combination of record lookups to match your sending IP with your domain to authenticate your email.
Partners None.
Gmail does have thousands of hosted business domains that are using Google Apps.
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
G M
ail
1. Gmail Bulk Sender Guidelines
Gmail advises the best way to reach the inbox is by complying with their Bulk Sender Guidelines:
• Publish an SPF record and sign with DKIM •Get Permission
• Have an Unsubscribe link prominent
• “Not Spam” clicks – tells Gmail the messages are solicited
• Like with like – use different email address, domain, and IP for
different types of mail, like promotional and transactional Senders can submit messages continually delivered to spam, despite following and exceeding best practices, for review on their Bulk Sender Contact Form.
Postmaster Services
2. Feedback Loop
Gmail does not offer a feedback loop service in the traditional sense. For senders using the list-unsubscribe header, whenever an email is marked as spam, the subscriber will be asked if they want to mark the email as spam and/or also unsubscribe from future mailings. To prevent abuse of this feature, the unsubscribe feature will only work for senders with good reputations in the Gmail network.
In enable this functionality, the following are required: • The email messages have a list-unsubscribe header which points to an email address or an URL that a subscriber can easily unsubscribe
•Sending domains publish an SPF authentication record, sign outgoing email with DKIM, and pass at least one authentication protocol
•The sender has a good reputation within the Gmail system. Go to Gmail’s Bulk Sender Guidelines for more information, or verify your reputation at SenderScore.org, where inbox placement at Gmail and one’s Sender Score are highly correlated.
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
What is list-unsubscribe?
Josh Baer (Chief Innovation Officer and founder of OtherIn-box, a Return Path company), and his friend Grant Neufeld created the list-unsubscribe mechanism in 1998 while still in college. RFC 2369 details the technical specifications of the header, but the concept is simple – add a unsubscribe mechanism in the email header and email providers can offer better unsubscribe functionality than the link buried in the footer of most emails that are required by the United State’s federal law, CAN-SPAM, as well as other internai-tonal laws regulating email and spam. Josh Baer also has a website devoted to list-unsubscribe that includes the specification and the participating mailbox providers that use it.
Unsubscribe Methods
Besides reporting email as spam, Gmail users can access the list-unsubscribe functionality by showing message details.
Show Message Details
To see the list-unsubscribe link, the recipient needs to expand the message details by clicking on the small arrow next to where it says “<<Sender>> to me.” This arrow brings down a message details list and the “Unsubscribe from this sender” hyperlink will appear at the bottom.
After the hyperlink is clicked, the following is displayed:
G M
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
Report Spam and Unsubscribe
This method will trigger a spam complaint, as it’s tied to the Report Spam functionality. The sender will also receive an unsubscribe notification, as shown above.
Regardless of the method selected, Gmail will send the following unsubscribe request:
Whitelists and Prioritized Delivery.
Gmail does not offer their own whitelisting services nor do they subscribe to any third-party whitelists.
Webmail providers have long been offering ways to deal with one’s inbox, and one of the drivers behind this is the emergence of what’s called graymail. Graymail lies somewhere between essential emails and legitimate spam. Classifying emails as spam is in the eye of the beholder most of the time and is rarely black and white, hence the term “gray” mail. Graymail is essentially the overwhelm-ing number of newsletters, coupons and advertisements that have been legitimately opted in for, but are non-essential, and over time can be difficult, or just downright annoying, to manage on a day-to-day basis.
Gmail Priority Inbox
The Gmail Priority Inbox feature was launched in September 2010 and uses predictive analytics, as well as manual feedback, to iden-tify and prioritize important email. Gmail predicts if a message is important by examining how messages were treated in the past, such as read or deleted and not read or manually marked as im-portant, how the message is addressed to the recipient, and many other factors.
Gmail’s Inboxes
G M
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
User Interface Features
Gmail Classic How to “Mark as important”
How to “star” a message Priority Inbox enabled
G M
ail
A starred promotional message will appear in the Primary Inbox.
While the tabbed inbox is now default, Priority Inbox is still an option for Gmail users.
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
Gmail’s Tabbed Inbox and Category Labels
In May of 2013, Gmail further innovated the inbox by releasing a new email productivity feature based on inbox tabs and category labels. Gmail’s tabbed inbox is now the default view for new Gmail accounts, and has been enabled for most existing Gmail users. Although, Gmail users have the option of turning the tabs off.
There are five labels total that are then categorized into their corresponding tab within the inbox. Three of the tabs are on by default (Primary, Social and Promotions), as well as two tabs that can be turned on manually (Forums and Updates). Gmail labels these emails using a special algorithm that looks at
everything from content, metadata and user feedback. Most of the categorizations rules came from Smart Labels, a Gmail Labs project that was released in March of 2011 that aimed to automatically label and sort incoming mail.
G M
ail
• Primary - person-to-person conversations and messages that don’t appear in other tabs
• Social – messages from social networks, media-sharing sites, online dating services, and other social websites
• Promotions – deals, offers and most other marketing emails •Updates – personal, auto-generated updates including
confirmations, bills, receipts, and statements
•Forums – messages from online groups, discussion boards and
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
1. Right-clicking and selecting the desired tab from the “Move to tab” menu
2. Starring a message. The starred message will appear in the pri-mary Inbox, but only that particular message will appear in the primary inbox. Future promotional messages will still be filtered to the Promotions, Social, Updates, or Forums tab.
Gmail users have the ability to influence how messages are labeled by:
3. Dragging and dropping a message into the primary folder. When a user drags and drops a message into a different tab, Gmail prompts users if they would like all future messages from that sender to go to the selected tab. If yes is selected, all mes-sages from that sender will automatically be sent to the primary folder going forward. However, if the Gmail user selects no, messages from that sender will continue to go to the original tab. Below is a campaign from Moosejaw asking their
subscrib-ers to move them to the Primary tab. G M
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
4. Selecting the message and choosing a label from the Label Dropdown. If a Gmail user has Priority Inbox enabled, tabs are not displayed. However, the message labels show up in the category views in the side bar. These messages will still appear in the main inbox, and users can click on the category label to view all messages with that particular label.
If a Gmail user has Priority Inbox enabled, tabs are not displayed. However, the message labels show up in the category views in the side bar. These messages will still appear in the main inbox, and users can click on the category label to view all messages with that particular label.
G M
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
Launched in 1996, Hotmail was one of the first web-based email services. In fact, the name Hotmail is a play off of HTML, or
HoTMaiL. Its popularity now spans the world. Hotmail is one of
the most popular domains in the world, anddominates markets in Brazil, Australia and most European countries. Microsoft has established offices throughout the world to support its email infrastructure, providing them with the insight needed to tailor their tools and services to a diverse set of global users.
In 2013, Microsoft made a bold move and rebranded Hotmail and Windows Live Mail to Outlook.com, a sign that consumer and enterprise offerings are converging into the cloud.
Microsoft by the Numbers
Active Users:400 Million
Emails received per day: 8 Billion
Of those received emails that reach the inbox: 30% - 35%
Domains: Outlook.com, Hotmail.com, Live.com, MSN.com
Hotmail Then (1996)
Hotmail (Outlook.com) Now
Outlook.com
O
utlook
.c
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
SmartScreen
Outlook.com utilizes Microsoft’s patented SmartScreen anti-spam filtering technology. This technology uses a machine-learning approach to help protect users’ inboxes from junk email. SmartScreen technology learns from known spam and phishing threats as well as from Outlook.com’s customers who have chosen to participate in the Feedback Loop Program (FBL).
Sender Reputation Data
Like many ISPs, Outlook.com considers volume, spam complaints and spam traps into their filtering. However, the company places a significant proportion of their filtering decisions from data in their Sender Reputation Data (SRD) network, which Microsoft refers to as the Spamfighter Club. Along with other sources of reputation data such as the Junk eMail Reporting Program (JMRP), SRD helps to train and improve the way SmartScreen technology classifies messages based on email content and sender reputation. Participants in the SRD program are selected from active
Outlook.com users at random from over 200 countries and no one can volunteer for the program. Their votes on whether or not they think your email looks spammy holds a lot of weight in filtering.
Spam Filtering
O
utlook
.c
om
1. Smart Network Data Services (SNDS)
SNDS is a free service that provides data on email volume, subscriber complaints, spam traps, and more.
SNDS uses a color coded system which indicates trustworthiness:
Green indicates a positive sending reputation, and higher inbox placement rates to Outlook.com are likely.
Yellow signals reputation issues starting to surface, and a mix of inbox and junk folder placement is likely.
Red means that there are serious underlying reputation issues, which are preventing most, if not all, emails sent to land in the junk folder.
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
2. Junk Mail Reporting Program (JMRP)
Microsoft offers their Junk eMail Reporting Program (JMRP) as a free feedback loop (FBL) service to anyone who wants to sign up. These FBLs are the result of spam complaints originating from Outlook.com users only, and does not include SRD
participants. To improve one’s sending reputation, complainers should be removed from your lists immediately. The headers of the received FBL complaint will also indicate if the subscriber marked it as junk or phishing. Emails marked as phishing indicates that the sending identity wasn’t known, or perhaps something in the content looked suspicious.
3. Support
Microsoft offers self-help and escalation paths for senders having deliverability issues. Senders are asked to ensure they are following all best practices on the Outlook.com troubleshooting page. If senders are certain they are following best practices, they can enter a ticket to the postmaster team.
Outlook.com Self-help Outlook.com Support Ticket:
Like Gmail, Outlook.com also takes advantage of the
list-unsubscribe header. There are two ways subscribers access the List-Unsubscribe functionality – through the “too many newsletters” footer or the “sweep” feature.
Too many newsletters? Unsubscribe.
As shown below, Outlook.com displays the “Too many newsletters?” verbiage under the email. It is important to note that this verbiage is not part of the email. Rather, it’s part of Outlook.com’s user interface. When the hyperlink is clicked, the following pop-up will be displayed if a List-Unsubscribe header is present.
Unsubscribe
Methods
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
O
utlook
.c
om
If the List-Unsubscribe header is not included, the following pop-up will be displayed:
Sweep Unsubscribe
Using Outlook.com’s Sweep feature, users can choose to unsubscribe from the mail in question.
Either method of unsubscribing, the user will see a screen like this where they can click a button to confirm the unsubscribe.
Microsoft does not maintain an internal proprietary whitelist. They do however participate in Return Path’s Certification Program.
Participants in this program will bypass all spam filters and have their email delivered to Outlook.com users with
images on.
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
O
utlook
.c
om
Microsoft asks that senders’ mail infrastructure comply with the following:
1. Sender ID and SPF – Microsoft asks all senders publish an SPF
or Sender ID record for the PRA domain.
2. DKIM - Microsoft also conducts DomainKeys Identified Mail
(DKIM) checks on inbound email. DKIM will be checked if Sender ID fails. If both authentication methods fail then their filters will make a decision to bulk the email or drop it all together.
3. DNS – Microsoft requires a valid reverse DNS with a Fully
Qualified Domain Name (FQDN). If Microsoft webmail domains are unreachable, Microsoft recommends querying the most current list of MTAs through nslookup: “nslookup –q=mx hotmail.com”
Mail Infrastructure
Microsoft has only one known partner that uses its infrastructure and that is Sympatico (Bell Canada).
Outlook.com Inbox and User Interface
Partners
User Interface
Features
The Outlook.com interface makes it easy to sort through personal mail, social notifications, graymail, and more.
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
Message features
Scheduled Cleanup Categorization
O
utlook
.c
om
Scheduled cleanup allows users to set rules to automatically delete old emails, and only keep the latest from a sender. For most businesses sending email, this isn’t an issue. For senders that send multiple times a day, have different From: addresses for each mail stream may be needed.
Outlook.com’s categorization can be used as Quick Views for users, which is a similar concept to Gmail’s tabbed inbox. The big difference is that Outlook.com allows categories to be customizable.
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
Archive Delete All From
O
utlook
.c
om
Outlook.com’s Archive feature is similar to Gmail’s, but moves all messages to the Archive folder keeping the main inbox clutter free.
The “delete all from” functionality allows users to easily delete all email from a particular sender based on the From: address. This is good news for any business that has had to deal with Hotmail users using the “Report Junk” button in the past.
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
Yahoo M
Yahoo Mail
Yahoo! Mail was one of the first free webmail providers to hit the market and is one of the three largest providers in the world with 281 million users. If smaller providers are included that utilize the Yahoo! infrastructure, such as AT&T and BellSouth, Yahoo!’s mail service covers 320+ million users globally. Yahoo! also re-designed its email client in October of 2013 to make it more responsive and better optimized for mobile devices.
Yahoo by the Numbers
Number of WorldWide users: 281 MillionNumber of WorldWide users including hosted infrastructure: 320+ Million
Number of active mobile users: 300+ Million
How much more Yahoo! users spend on electricity compared to Gmail users: $110
How much Yahoo! spent to purchase Four11 for the Rocketmail webmail service, which soon after became Yahoo! Mail: $96 Million
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
Yahoo M
Yahoo! builds the majority of their spam algorithms in-house. They put a heavy emphasis on content, URL filtering and spam complaints from its users. They are one of the few mailbox providers that filter email by domain versus only IP address.
Reputation
Your reputation is critical in determining inbox or bulk folder placement and is driven by many factors including unknown user rates, spam complaints and known bad addresses or spam traps. You should know that a sender’s reputation is made up of a combination of IP Address +
Domain + From Email Address so it’s important that you maintain consistency in all three of these components since changing any one of these may lower your reputation and degrade your ability to get to the inbox.
Engagement
Engagement has always been an important measure of subscriber interest for senders, but mailbox providers are starting to make significant investments in research, in-house spam filters and third-party software to help measure subscriber engagement to better determine appropriate folder placement. Factors being considered include what messages are opened, what messages are moved between folders and how quickly mail is deleted. So what can you do about this? Get back to basics. Send relevant email that your subscribers want by tracking opens, clicks and conversions. If you have a large number of subscribers that have never opened or engaged with your mail then it’s time to take a second look at those addresses and determine if you can win them back or let them go. Inactive subscribers will ultimately hurt your ability to get delivered at all mailbox providers.
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
Postmaster Services
1. Best practices for bulk mail senders and postmasters for
sending to Yahoo Mail
Yahoo! tells senders the best way to get delivered to the inbox is to avoid looking like a spammer. Start here to troubleshoot Yahoo! delivery issues.
2. Troubleshooting SMTP Errors
If you’re mail isn’t getting delivered to Yahoo!, senders should refer to their SMTP logs. Once the bounce code is discovered, review Yahoo!’s list of common SMTP errors for the reason of the error.
3. Yahoo! Complaint Feedback Loop
Yahoo! offers a complaint feedback loop service that is hosted by Return Path. We recommend that you suppress that user immediately from future mailings to maintain a positive reputation with Yahoo!. Note that DomainKeys or DKIM authentication is a requirement to sign up for FBLs.
Yahoo’s Whitelisting Options
There are 2 kinds of priority listings: Internal whitelisting and Re-turn Path Certification.
1. Yahoo! Internal Whitelist: Yahoo!’s own whitelist is free, and business’ can sign up for Yahoo!’s whitelisting service here. Although it’s a whitelist, brands that are whitelisted are still subject to certain filters, including user level filtering, content filtering, and URL filtering. While the exact criteria to be ac-cepted into Yahoo!’s whitelist isn’t public, they do require a posi-tive sending reputation. Mailing from a new IP address, such as moving to a new Email Service Provider or adding new IP addresses, will require at least 30 days of reputation before Yahoo! will consider whitelisting.
2. Return Path Certification: Yahoo! also honors the Return Path Certification accreditation program. Return Path Certification also acts as a whitelist, but allows accredited senders to bypass certain filters to reach the inbox. Those accepted into the ac-creditation program are required to maintain the standards set by Return Path and its participating partners. More information on the Return Path Certification program can be found here.
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
Mail Infrastructure
1. SPF, DKIM, and DMARC: Authenticating email with SPF and DKIM is recommended. Yahoo! supports DMARC and the standards that are required.
2. Connection and Throughput: Servers should be configured to allow for simultaneous connections with throughput set at 20 emails per connection. Rate limiting will occur for high complaints and poor reputation, so use this as a guideline to determine what the optimal setting for you should be. 3. rDNS: publish a PTR for your sending IPs, and use a Fully
Qualified Domain Name (FQDN) so it doesn’t look like a dynamically assigned IP is sending mail.
4. Use a Consistent and Static “From:” address: Changing domains and sending addresses will benefit senders in which their subscribers whitelist their mailing address. It will also help prevent email campaigns being identified as spam by Yahoo!’s spam filters.
Yahoo! has several partners including AT&T (which includes BellSouth, SBC and Ameritech), Verizon, Rogers, BT, TNZ, ExtraNZ, and Nokia, who utilize Yahoo!’s infrastructure for its customers’ email. Each of these partners has their own filtering at their gateways before it gets handed over to be processed by Yahoo!’s filters, but we have found email sent to these partners should have inbox placements rates that are consistent with direct Yahoo! users.
Partners
Yahoo! User Interface
In
tr
oduc
tion
Yahoo M
O
utlook
.c
om
G M
ail
returnpath.com
About
Return Path
Return Path is the worldwide leader in email intelligence. We analyze more data about email than anyone else in the world and use that data to power products that ensure that only emails people want and expect reach the inbox. Our industry-leading email intelligence solutions utilize the world’s most comprehensive set of data to maximize the performance and accountability of email, build trust across the entire email ecosystem and protect users from spam and other abuse. We help businesses build better relationships with their customers and improve their email ROI; and we help ISPs and other mailbox providers enhance network performance and drive customer retention. Information about Return Path can be found at:
USA (Corporate Headquarters)
rpinfo@returnpath.com
Australia
rpinfo-australia@returnpath.com
Brazil
rpinfo-brazil@returnpath.com
Canada
France
rpinfo-france@returnpath.com
Germany United Kingdom
