By Ashutosh GawaliEmail authentication (email validation) is a process used to verify the legitimacy of the sender server and integrity of an email message.
It establishes trust between senders and recipients by ensuring your identity of the sender is verified.
Before discussing the email authentication methods, lets first discuss an important concept ( “MAIL FROM” and “FROM” address) which is the base of few email authentication methods.
Table of contents
- “MAIL FROM” vs “FROM” address.
- Sender Policy Framework (SPF).
- Enhanced Filtering for Connectors (Skip Listing).
- SPF Alignment.
- DomainKeys Identified Mail (DKIM).
- DKIM Alignment.
- Domain-based Message Authentication, Reporting, and Conformance (DMARC).
- Composite Authentication (Comp Auth).
- Authenticated Received Chain (ARC).
- Add Trusted ARC Sealers.
- Conclusion.
“MAIL FROM” vs “FROM” address.
If you view the internet message headers of the email and use the Microsoft Message Header Analyzer tool to analyse the headers, you will get to know that there are 2 types of email addresses shown in the email headers.
- “FROM” address / 5322.From address / Sender address: It shows who is the sender of the e-mail, the mailbox of the person responsible for writing the message. This address is directly visible to the recipient when they view the email in Outlook or OWA or any other email program they use.
- “MAILFROM” address / 5321.MailFrom address / Return-path / Envelope address: It shows the origin of the message (the sending mail server), This address is not directly visible by the recipient. Receiving mail servers use the “MAIL FROM” address to return bounce messages and other error notifications of the email.
There are several methods of verifying the identity of the email sender using EOP.
Sender Policy Framework (SPF).
An SPF record is a DNS TXT record containing a list of the IP addresses that are allowed to send email on behalf of your domain.
The receiving mail server will check if the IP address of the email sending server is available in the TXT record of the domain listed in “MAIL FROM” address.
Example: If the “MAIL FROM” address is sender@gmail.com then the TXT record for gmail.com will be queried using DNS server by the Receiving Mail servers. If the “MAIL FROM” address is sender@live.com then the TXT record for live.com will be queried using DNS server by the Receiving Mail servers.
SPF will check if the IP address of the sender server is present in the TXT record of the domain listed in “MAIL FROM” address then the SPF is passed otherwise the SPF will SOFTFAIL or HARDFAIL (depending on the SPF record setup).
If the SPF check is failed then the spam policy configured on the destination / receiving email server determines what to do with the message.
Example: Consider the below scenario.
"FROM": You@sender***.com "MAIL FROM": You@sender***.com Sender server IP address: 10.0.0.1 (for this demonstration I am using a Private - non routable IP address) "TO": aashu@aashu.co.in
Example format of SPF for sender***.com: v=spf1 ip4:10.0.0.1 -all
When EOP receives the email from the sender server (10.0.0.1) on behalf of the recipient aashu@aashu.co.in, EOP will check the “MAIL FROM” domain (sender.com), the DNS server for sender.com will be contacted and the SPF record of the domain sender.com will be queried, If the SPF record of sender.com contains the IP address (10.0.0.1) then the SPF will be passed.
A spammer can make sure that the SPF record check is passed on the recipient server. Example: Consider the below scenario of spoofed “MAIL FROM” address.
"FROM": You@sender***.com "MAIL FROM": You@SPAMMER***.com Sender server IP address: 192.168.10.1 (for this demonstration I am using a Private - non routable IP address) "TO": aashu@aashu.co.in
Now, when the SPF check will be done at EOP against the domain in the “MAIL FROM” (SPAMMER***.com) the sender server IP address 192.168.10.1 will be available in the SPF – TXT record of the domain (SPAMMER***.com).
This is because the spammer has made sure that the SPF – TXT record with the sender server IP address is already added in the DNS for (SPAMMER***.com).
Example format of SPF for SPAMMER***.com: v=spf1 ip4:192.168.10.1 -all
the above SPF record entry shows to the recipient server that SPAMMER***.com is authorized to send emails from IP address 192.168.10.1
- v=spf1: Shows that the version of SPF is version 1 and it is an SPF TXT record.
- ip4 or ip6 : Shows if an IPv4 address or IPv6 addresses is used to send email on behalf of the domain.
- include : Shows if a domain name is used instead of many IP addresses.
Enforcement rule: It instructs that what action to take when an email fails authentication.
- -all (Hard fail): If the receiving server check the SPF record and the sending server IP address is not present in the SPF record AND the SPF enforcement rule for the domain has -all, this means the SPF is HARD FAIL and the spam policy configured on the destination (receiving) email server determines what to do with the message.
- ~all (soft fail): If the SPF enforcement rule for the domain has ~all, and the sending server IP address is not present in the SPF record, then the SPF is SOFT FAIL, the email is typically delivered in this scenario.
- ?all (Neutral): This enforcement rule means do not mark the message envelope, used for testing SPF.
Please refer the Microsoft article to Set up SPF to help prevent spoofing
Enhanced Filtering for Connectors (Skip Listing).
Consider a scenario, where the MX record for the organization is pointed to a third party spam filter or it is pointed to on premises Exchange Server.
Once the email enters the EOP from third party spam filter or on premises Exchange Server, the SPF record will be checked again on EOP, now the SPF will fail as it would be checked on the domain listed in “MAILFROM” address but the connecting IP address would be the IP address of third party spam filter or IP address of the on premises Exchange Server.
If we use Enhanced Filtering for Connectors feature, then the last connected IP address (in this case 192.168.10.1) will not checked for SPF record of the domain present in “MAILFROM” address, instead the IP address 192.168.10.1 will be skipped and the IP address 10.0.0.1 will be checked for the SPF record check.
This feature is sometimes referred as Skip Listing as we are skipping the IP address check for the connected IP address.
SPF Alignment.
In order for an email to be considered SPF aligned, the domain part of the “MAIL FROM” / Return-Path of the email must match the domain part of the “FROM” address.
Example of SPF Alignment Pass: "FROM": You@sender***.com "MAIL FROM": You@sender***.com Sender server IP address: 10.0.0.1 "TO": aashu@aashu.co.in
Example of SPF Alignment Fail: "FROM": You@sender***.com "MAIL FROM": You@SPAMMER***.com Sender server IP address: 192.168.10.1 "TO": aashu@aashu.co.in
DomainKeys Identified Mail (DKIM).
DKIM is an email authentication mechanism which detects if the emails are altered in between sending and receiving mail servers.
In DKIM, the sending mail server will digital signature to outbound email messages in the message header using private key, the receiving mail server will use the public key to validate the digital signature in the message header of the email message.
The email hosting provider generates 2 keys, a public key and a private key.
They provide the public key to the domain owner, who stores the public key in DNS server in the form of DKIM record.
The sending mail email server uses private key to sign the DKIM header of the email.
Example: The email is sent from user@gmail.com to the receiver aashu@aashu.co.in
The sender gmail.com server will input the below data in the header fields in the message header.
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706595125; x=1707199925; darn=aashu.co.in; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=YxFp2KVXxOM+nJMwWb3oHDX8XBI1pqMyPJl6LzkS7R0=; b=fNJCM0OAQBrs2bOpRH+L5aQXud9ndkjx1fUXvNSB+9assbddOIgDqZva88DAacO7k5 bnw+kJB41MzJkHcJZPF2736R9AFquVqe8EbMzkOHxUznyA8uxNTMl377CiMbMTj7Mvjk aDnDQXSVFdzlJaqs8w6K2sdxWpeE78FOrV9Da2uQWzlifFv4g3KKHcK9a9+hKT4TohJ+ w4QhOfpuU+jPSA/uTnWfYEfaEVYjr+/b0eGFAuxoRNaYyu0JbRleEvcS+OYMuQLpr6AO 5cCp42LaAyDJE5VPZXa/lACWtFxP6kMJ5cTYtNp6zohnHQupVIMpHLjJAGkdoSTWQDIH r0cA==
The sender signs a specific set of headers (the header values defined in the h= attribute above) and the message (bh=).
This is a snapshot in time of the message from sending mail server and is stored in the DKIM signature (b=) of the email headers.
DKIM Singing on sending mail server: Private Signing Key + h (header fields - list of fields in the header that have been signed) + bh (body hash) = b (signature of headers and body)
When the email is received by the receiving mail server, it will first check the value of the field “d=“.
As d= gmail.com, the DKIM record of gmail.com will be queried and it gets the public key ( value of p= ) listed in DKIM record of gmail.com.
After that the receiving mail server takes the algorithm used in the value of a= of the email headers and computes one hash for the values defines in the h= field AND computes another hash of the email body using the algorithm used in the value of “a=“.
DKIM verification on receiving mail server: Public Key + h (header fields - list of fields in the header that have been signed) + bh (body hash) = b (signature of headers and body)
If the created hash on the receiver server matches the value of signature in the value of field b= of the DKIM header, then the DKIM is passed, otherwise the DKIM is failed.
Please refer the Microsoft article to Use DKIM to validate outbound email sent from your custom domain.
DKIM Alignment.
An email is considered to be DKIM aligned when the domain part of “FROM” address must match to the domain set in the “d=” field in the DKIM signature field.
Example of DKIM Alignment Pass: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706595125; x=1707199925; darn=aashu.co.in; "FROM": aashu@gmail.com "TO": aashu@aashu.co.in
Example of DKIM Alignment Fail: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706595125; x=1707199925; darn=aashu.co.in; "FROM": You@SPAMMER***.com "TO": aashu@aashu.co.in
Domain-based Message Authentication, Reporting, and Conformance (DMARC).
It is a set of policies that specify the actions to be taken by the receiving email server when an email fails authentication (SPF and DKIM).
It also provides reports on email being spoofed on behalf of your domain.
The policies are published by the Domain Owner in the form of TXT records, and retrieved by the Receiving mail server during the SMTP session, via the DNS.
The format of DMARC TXT record in dns should be.
_dmarc.domain TTL IN TXT "v=DMARC1; p=policy; pct=100"
- domain part is the domain you want to publish the DMARC record for.
- TTL stands for Time To Live, the valid values of TTL should be Hours, Minutes or Seconds. (will vary depending on the registrar for your domain).
- pct=100 indicates that this rule should be used for 100% of email.
- policy specifies what the receiving mail server should do with the email if the DMARC fails. Valid values of the policy are none, quarantine, or reject.
- Policy “p=none” means that the Domain Owner is not asking the Receiver to take action if a DMARC check fails.
- Policy “p=quarantine” means the recipient server should move the email to Quarantine.
- Policy “p=reject” means the recipient server should reject (delete) the email.
DMARC requires that the domain used in “FROM” email address should match either of the domains used in DKIM (d=) or SPF records (“MAIL FROM”).
Only the emails that are either SPF aligned or DKIM aligned can pass DMARC, otherwise the DMRAC check will fail for that email.
DMARC is said to be fully aligned when the domain part in the “FROM” is matching the domain part in “MAIL FROM” address and the domain part in “d=” of DKIM signature of the email.
If an organization does not want the DMARC sender policy actions to be taken if the DMARC of the email fails, then EOP has few settings in Anti-phishing policies that can change the behaviour of how the email is processed if the DMARC fails and the email is detected as spoof.
If the message is detected as spoof and sender DMARC Policy is set as p=quarantine, then
- Quarantine the message.
- Move the message to the recipients’ Junk Email folders.
If the message is detected as spoof and sender DMARC Policy is set as p=reject, then
- Quarantine the message
- Reject the message
Composite Authentication (Comp Auth).
Composite Authentication is basically a score which is applied to all the incoming emails.
Comp Auth combines different authentication (SPF, DKIM and DMARC) AND sender reputation check, sender history, recipient history, behavioural analysis, and other advanced techniques AND other part of the email to confirm if the email is authenticated.
The value of comp auth is stored in Authentication-Results header in the email headers and it would be in the below format.
Authentication-Results: compauth=<fail | pass | softpass | none> reason=<yyy>
There may be multiple reasons for the compauth fail, pass or softpass, the details of reasons are defined in the Microsoft article Anti-spam message headers in Microsoft 365
Authenticated Received Chain (ARC).
Authenticated Received Chain (ARC) is an email authentication standard that allows receiving mail servers to check the authentication results of an email before the email is forwarded or relayed by an intermediate (email forwarding) server.
When an email is forwarded from one server to another, the SPF of the email breaks or when the email is sent to a mailing list first the DKIM signature of the email breaks (as mailing list will change the email body, which will result in different value of body hash).
As the email authentication fails on EOP, the email will be rejected.
Now, let’s introduce the concept of ARC on the Email Forwarding Server
ARC preserves the original authentication results from the first hop of an email’s journey and verifies the identity of each email forwarding server along the way.
When an email passes through a trusted email forwarding server, it digitally signs the message and adds this ARC signature to the email header.
When the email reaches EOP, It verifies the ARC headers added by email forwarding server.
Once the ARC headers are verified, the EOP overrides the Authentication failure decision and delivers the email to inbox. then overrides the Authentication failure and delivers the email to inbox.
ARC Headers consist of the below fields:
- ARC-Authentication-Results: This field stores authentication headers (SPF, DKIM, DMARC).
- ARC-Message-signature: This is digital signature comprising the message header information and entire message body (except the ARC-Seal header).
- ARC-seal: A digital signature comprising the ARC headers generated by each intermediate server
Add Trusted ARC Sealers.
Trusted ARC sealers is an admin-created list of intermediary domains that use ARC sealing. When an email is routed to Microsoft 365 through an ARC trusted intermediary, Microsoft 365 validates the ARC signature and (based on the ARC results) can honor the provided authentication details.
The administrator can open the ARC sealers section in Microsoft Security portal using the link https://security.microsoft.com/authentication and they can add the domain of the legitimate services that modify email content in transit before the email gets delivered to EOP.
The domain name administrator enters in the above section, must be a match to the domain shown in the domain “d=” tag in ARC-Seal and ARC-Message-Signature headers of the email.
Please check the Microsoft article to Configure trusted ARC sealers
Conclusion.
Exchange Online Protection (EOP) uses different types of authentication mechanism to confirm the authenticity of the sender of the email.
SPF checks if the IP address of the sending mail server is present in the TXT record of the domain in “MAIL FROM” address.
SPF Alignment is when domain listed in MAIL FROM” / Return-Path of the email matches the domain part listed in “FROM” address.
DKIM creates HASH of headers and body of the email using private key on sending mail server and when the email reaches the receiving mail server, the public key of the domain is used to create another HASH of header and body of the email, if both HASH matches then DKIM will pass.
DKIM alignment is when the domain part of “FROM” address matches the domain listed in the “d=” field in the DKIM signature field.
DMARC requires that the domain used in “FROM” email address should match either of the domains used in DKIM (“d=”) or SPF records (“MAIL FROM”).
Comp Auth combines different authentication (SPF, DKIM and DMARC) AND sender reputation check, sender history, recipient history, behavioural analysis, and other advanced techniques AND other part of the email to confirm if the email is authenticated.
ARC is an email authentication standard that allows receiving mail servers to check the authentication results of an email before the email is forwarded or relayed by an intermediate (email forwarding) server.