This is part of the Semicolon&Sons Code Diary - consisting of lessons learned on the job. You're in the web-development category.
Last Updated: 2024-04-26
Tag each set of email differently (transactional, news, daily new products). This allows good comparisoin
Received: headers show the path that a message takes from the message sender to the final recipient. They are read in reverse order, so the Received: header at the top of the message is the one that delivered the message to you, and the last one shown should be the one where it left the sender.
In case this isn't already clear, there are multiple Received headers per email.
SPF is a way for you to tell the world where email for your domain originates (e.g. with the SendGrid service etc.). You can create an SPF policy for your domain with a DNS TXT record
This way, receiving mail servers will be able to check that your messages originate from a server that's on your list. E.g. spf.mtasv.net is Postmark's SPF record and if it isn't on the DNS record, then the receiving server can consider it fake and act accordingly.
Look up the current SPF value with
nslookup -type=txt oxbridgenotes.co.uk
# oxbridgenotes.co.uk text = "v=spf1 include:sendgrid.net ~all"
Watch out: this can get out of date if you change mail server.
DKIM, cryptographically signs outgoing messages to stop impersonation and make sure messages aren’t altered in transit.
You setup DKIM a bit like SPF, by adding a new TXT record to DNS for your domain. You’ll need to add a unique public key for each source of email for your domain. The server sending the email has the private key. If you are using a third party service like Postmark or SendGrid, they'll probably manage this private key for you.
Email for the domain is digitally signed by the sending mail server using the private key for the domain. Receiving mail servers use the public key from the domain's DKIM record to validate that messages have not been tampered with. Specifically, the sending Mail Transfer Agent (MTA) normalizes the message following the rules in RFC 6376. Then it creates a hash of the normalized email and uses the domain's private key to generate a cryptographic signature of the hash. When email is received, the receiving mail server examines the DKIM-Signature header field and performs a DNS lookup to retrieve the DKIM record for the domain.
Often (but not always) it is store at _.domainkey.{DOMAIN}
nslook -type=text _domainkey.example.com
# _domainkey.example.com. 3600 TXT "v=DKIM1; k=rsa; p={key string omitted}"
Return-path is a hidden email header that indicates where and how bounced emails will be processed. This header, also referred to as a bounce address or reverse path, is an SMTP address that is separate from your original sending address, and is used specifically for collecting and processing bounced messages
Soft bounce happens for a few reasons - email too large - user’s inbox was full (unlikely in this day and age IMO) - their email server was down - email message blocked due to content - email message does not respect the recipient's anti-spam policies - email message does not meet the recipient server's DMARC requirements for authorization
To find out the exact reason, look at the email sent back from the inbox (using the Return-path email) after the attempting delivery.
It will contain headers like
Action: failed
Status: 5.2.2 (mailbox full)
Diagnostic-Code: smtp;552 5.2.2 <rubyramos0303@icloud.com>: user is over quota
and perhaps some content describing this is plain English
Hello, this is the mail server on mta12-ab1.mtasv.net.
I am sending you this message to inform you on the delivery status of a
message you previously sent. Immediately below you will find a list of
the affected recipients; also attached is a Delivery Status Notification
(DSN) report in standard format, as well as the headers of the original
message.
<xyz@icloud.com> delivery failed; will not continue trying
Hard bounce happens when there is a permanent reason for non-delivery
You should unsubscribe these recipient's to prevent damage to your email domain deliverabiltiy
DATA SMTP commandWith the DATA command, the client asks the server for permission to transfer the mail data. The response code 354 grants permission, and the client launches the delivery of the email contents line by line. This includes the date, from header, subject line, to header, attachments, and body text. A final line containing a period (“.”) terminates the mail data transfer. The server responses to the final line.
Example:
DATA
354 (server response code)
Date: Wed, 30 July 2019 06:04:34
From: test@client.net
Subject: How SMTP works
To: user@recipient.net
Body text
.
Timeouts occur during the DATA command. This may occur if you have an antivirus proxy or similar, where it checks the email before sending the acknowledgement to the sender. If the antivirus (or similar) proxy is taking too long to check the message, the sender is timing out and dropping the connection, to retry the message at a later time. But once the antivirus (or similar) proxy finishes processing email 1, it still forwards the email through to the mail server. This problem is usually identified since an email is continually duplicated, not just the once, a new copy keeps arriving, with different Received header, and the problem emails are larger in size than usual.
The standard way to debug duplicate delivery is to compare the complete messages and see where the Received: headers diverge - generally the last host that produced the same Received: header in both messages is the culprit.
In order to see a user's Received headers, they need to download the email somehow. This should output an .eml or .zip file that they send to you in as an attachment. Forwarding the problematic emails will just replace the Received headers with new ones, and is therefore of no value to debugging.
A unique string assigned by the mail system when the message is first created. These can easily be forged.
Their value can be in debugging duplicate emails and connecting inbound emails to the original message should the original message subject etc. be changed by the user.
Message-IDs are required to have a specific format which is a subset of an email address[2] and be globally unique. No two different messages must ever have the same Message-ID. If two messages have the same Message-ID, they are assumed to be the same and one version is discarded.
Some services like PostMark might override your MessageID header if you don't ask them to specifically leave them in.