Understanding Extended Message Trace in Exchange Online.

Sorting the excel file.

Once the Filter option is applied, sort the Column A from A to Z.

After that, select View from the toolbar, drop down Freeze Panes and select Freeze Top Row.

Column Names and Details.

Below are some important column name and explanation of what they do.

• source (Column H): The EOP component that’s responsible for the event mentioned in the event_id (Column I).
  • AGENT: A transport agent in EOP has processed the email, to find out more detail about which specific agent processed the email we have to look at the details of source_context and event_id.
  • MAILBOXRULE: An inbox rule has acted on the email.
  • RESOLVER: The recipient was searched in Active Directory for the tenant.
  • ROUTING: The transport service (Component) tried to route the email to another service (component) in EOP.
  • SMTP: The email is either Sent to external or Received by the SMTP server.
  • STOREDRIVER: A mailbox has sent the email to the local server (on which the mailbox is hosted).

• source_context (Column F): Detailed information provided for the (Column H) source.
  • Advanced Routing – Connector Based Router: This is when the normal routing is changed and the email is sent to the connector to be delivered to the recipient.
  • AgentDefer – Transport Rule Agent: The transport rule agent has postponed processing (checking) of transport rule on the email.
  • AgentFork: The email is sent to another component by transport agent for processing.
  • Forwarding SMTP Address: Shows the email address where the emails needs to be forwarded (Email address is present in the related_recipient_address (Column Q).
  • AutoForwardedRecipients: The email that needs to be autoforwarded is resubmitted to another component in the transport pipeline.
  • Auto-Forwarded recipients: It is used when the transport service tries to resolves the forwarding email address (find out if the forwarding email address is internal or external).
  • CatContentConversion: The transport agent converts the email content to the format specific to the recipient. More details on Content conversion.
  • Mailbox Rules Agent: An inbox rule is applied on the email.
  • Transport Rule Agent: A transport rule is applied on the email.
  • Mailbox: The guid of the mailbox from which the email is composed (sent).
  • MessageClass: The type of the email. IPM.Note for email, IPM.Appointment for calendar events, IPM.Contact for contact, IPM.Task for tasks.
  • CreationTime: The time when the email is sent from the mailbox.
  • ClientType: The client used to send the email. REST for Outlook, OWA for Outlook on the Web, ActiveSync for Mobile devices.

• event_id (Column I): The events happened on the email which is being processed by different components of EOP.
  • AGENTINFO: This is used by the transport agent logs the custom events in source_context (Column F).
  • DELIVER: The email is delivered to the mailbox.
  • RECEIVE: The email is either received by the SMTP server to send the email externally or by the EXO server to send the email to the recipient mailbox in EXO.
  • RECIPIENTINFO: This entry is logged when the RESOLVER service tries to find out the user object associated with recipient mailbox in the Active Directory for the tenant.
  • REDIRECT: The email is redirected to another recipient (Example a forwarding is applied on the mailbox then the email will be redirected to the Forwarding SMTP Address and this entry will be logged in event_id (Column I)).
  • RESUBMIT: The email is resubmitted to another component in Transport pipeline in EOP.
  • SENDEXTERNAL: The email is sent to the SMTP server responsible to receive the email for the external email address.
  • SETROUTE : This action is logged when the normal email path to the recipient mailbox is changed and the email is sent using a specific connector, Advanced Routing – Connector Based Router is logged in source_context (Column F) and connector_id (Column G) will show the connector details.
  • SUBMIT: The email is submitted to the transport service. (this action is logged when the email is created) the source_context (Column F) will show the guid of the mailbox which submitted the email, MessageClass of the email, Creation time of the email, ClientType used to create the email.
  • TRANSFER: The email is transferred to another recipient which is in bcc, cc or to a member of distribution List.

• connector_id (Column G): Connector details using which the email is sent to the recipient.

• message_id (Column K): A globally unique identifier generated by the email server. message_id can be used to generate EMT for a selected email.

• network_message_id (Column L): A unique identifier which is common across all the servers present in EXO.

• recipient_address (Column M): Email address of the recipient of the email. Value can be single email address or multiple email addresses.

• recipient_status (Column N): Status of action taken on Recipient email address.
  • UserMailbox.Forwardable.Resolver.CreateRecipientItems.40: The recipient email address is Internal and found by the RESOLVER during the RECIPIENTINFO in the Active Directory for the tenant.
  • NotFound.OneOff.Resolver.CreateRecipientItems.10: The recipient email address is External and not found by the RESOLVER during the RECIPIENTINFO in the Active Directory for the tenant.
  • NotFound.OneOff.Expansion.AddAddress.10: The recipient email address is forwarding address and the address was not found by the RESOLVER during the RECIPIENTINFO in the Active Directory for the tenant.
  • UserMailbox.Forwardable.Resolver.CreateRecipientItems+SmtpFwd.40;NotFound.OneOff.Expansion.AddAddress.10: This is the pair of internal email address with the external email address (on which the forwarding address is set to). Example if the mailbox aashu@aashu.co.in has email forwarding is set user@gmail.com then the recipient_address (Column M) will show the addresses as aashu@aashu.co.in;user@gmail.com
  • Message Forked: This action is logged when another copy of the email is created, example: email is needed to send to the members of distribution group, or if the email is cc’d or bcc’d to multiple recipients.
  • LED=550 4.3.2 QUEUE.TransportAgent; message deleted by transport agent: Transport rule to delete the email is applied on the email.
  • LED=450 4.4.312 DNS query failed: The DNS server for EOP was unable to get the response from DNS server for the recipient email server.
  • LED=250 2.1.5 RESOLVER.GRP.Expanded; distribution list expanded: This is when the recipient is a distribution group, and EOP forest will query the Active Directory forest of the tenant and gets the membership details of the distribution group.
  • 420 4.7.3 Resubmitting quarantine message: This is when the anti spam quarantine agent sends the email to the quarantine of the tenant.
  • [Stage: DeliverPreDelivery];StoreDriver.Rules; message is deleted by mailbox rules: This is when the email is deleted by the system inbox rule (Like junk email rule or delegate rule) or a user created inbox rule.
  • LED=550 5.2.5 Folder full: we will get this error when the folder in which the email is supposed to be delivered is full and cannot accept any more email. (The mailbox can have space available but the folder limit of email is reached).
  • 250 2.1.605 Spam filter added recipients (redirect/bcc): The email is bcc’d or redirected to another email address as per the organizations anti spam policy.
  • 400 4.7.721 Advanced Threat Protection scanning in progress: The ATP agent is scanning the email for threats and will take the action mentioned in the ATP policy.
  • Detonation Processing Complete: The ATP agent has completed the scanning of the email.

• related_recipient_address (Column Q): This field shows the forwarding email address that is applied on the mailbox. the source_context (Column F) will show the entry Forwarding SMTP Address.

• reference (Column R): This will show message_id of the email which the sender is responding to. Example: if user@gmail.com has sent and email to aashu@aashu.co.in (message_id: <1234@mail.gmail.com>) and then aashu@aashu.co.in responded (replied) to the same email to the recipient user@gmail.com (message_id: <8910@mail.gmail.com>) | if we check the EMT for (message_id: <8910@mail.gmail.com>), we can see the reference (Column R) stamped with <1234@mail.gmail.com> as the email is referenced to the earlier email which was sent.

• sender_address (Column T): FROM address of the email.

• return_path (Column U): MAILFROM address of the email.

• message_info (Column V): Shared additional information about internal components that processed the email. Example: CATRS-DLP Policy Agent=0.626 | CATRS-DC Content Filter Agent=0.278 | CATRS-Spam Filter Agent=0.115 | CATRT-RMS Encryption Agent=0.078.

• Directionality (Column W):
  • Incoming: The email is inbound to your Exchange Online organization.
  • Originating: The email is sent from Exchange Online mailbox for your organization.

• original_client_ip (Column Y): The IP address of the sender SMTP server (For internal emails, this would be IP address of the EXO server and for External emails (received by EOP) this will be IP address of the external SMTP server which was connected to EOP to deliver the email).

• original_server_ip (Column Z): IP address of EOP server which received the email on behalf of the recipient domain.

Reading custom_data field for Inbound email.

Open Notepad++ application –> Paste the data in the Notepad++ –> Open Replace Option –> Replace ; (semicolon) with \n (backslash n – new line) –> make sure the search mode is set to extended search –> Select Relace All.

After the above step, all the data after semicolon will be shifted to a new line.

Custom data for source:SMTP | event_id:RECEIVE.

In this event, the email is received by the the EOP server and the details regarding email format, number of attachments and the servers which received the email is listed in the custom data.

• S:ProxyHop1: The original EOP server which received the email on behalf of the recipient domain.
• S:TenantServiceProvider: The service provider which sent the email to EOP. If the sender organization is hosted in EXO then S:TenantServiceProvider=FOPE (Forefront Online Protection for Exchange) and if the sender organization is hosted in google then S:TenantServiceProvider=Google (I haven’t came across any other value than FOPE and Google, if the email is sent from on premises server then the value might be blank).
• S:MimeParts:
  • Att: Number of attachment in the email.
  • Emb: Number of Embedded file in the email.
  • MPt: MIME Parts
  • MTy: MIME Types
  • MCS: Multi-part Content Specification of the email.
  • NormalRegAtt: Normal Regular Attachment are present in the email.
  • SummTNEF: Attachments are present in TNEF (Transport Neutral Encapsulation Format).
  • Compliant: shows that the email is compliant to the MIME standards.

To know more about MIME. please read the article Sample MIME Message and MIME (Multipurpose Internet Mail Extensions).

• S:InboundTlsDetails: The details of the TLS version, TLS cipher, TLS key length and TLS key exchange algorithm used by the email sending server.

• S:FromEntity: Shows from where the email is received
  • S:FromEntity=Internet: The email is received from Internet
  • S:FromEntity=Hosted: This entry shows up if the email is sent from EXO.
  • S:FromEntity=HybridOnPrem: When the email is sent from Microsoft service accounts (azure-noreply@microsoft.com), we can see the FromEntity as HybridOnPrem.

• S:ProxiedClientIPAddress: The IP address of the email sending server which connected to EOP to deliver the email to EOP.

• S:ProxiedClientHostname: FQDN of the email sending server which connected to EOP to deliver the email to EOP.

• S:AccountForest: Server details where the accounts for the recipient tenant is hosted. This forest is queried by the RESOLVER to confirm if the recipient is a part of the organization.

Custom data for source:RESOLVER | event_id:RECIPIENTINFO | event_id:RESOLVE.

In this event the EOP tries to resolve the recipient email address and checks if there is any other internal or external present in the email.

• S:OriginalFromAddress: Original FROM address present in the email.

• S:AllowAliases: Confirms if RESOLVER will search the recipient email address against the Aliases in the Account forest of the tenant.
  • N: The RESOLVER will not search the recipient email address against the Aliases in the Account forest of the tenant.
  • Y: The RESOLVER will search the recipient email address against the Aliases in the Account forest of the tenant.

• S:ExtRecipCount: Number of external recipients in the email.

Custom data for source:AGENT | event_id:AGENTINFO.

This event will capture all the logs related to different agents processing the email (agents like transport rule agent, anti malware agent, antispam agent).

• S:TRA: (TRA stands for Transport Rule Agent).
  • S:TRA=ETRP: Exchange Transport Rule Performance (The transport rule is processed but not applied on the email).
  • S:TRA=ETR: Exchange Transport Rule (If this value is present then the transport rule is applied on the email).
    • ruleId: guid of the transport rule.
    • st: Last Modified Date of a rule.
    • Conditions: Conditions defined in the transport rule.
    • action: Action taken by a transport rule; could have multiple actions per rule.
    • ExecW: Execute Wall Clock.
    • ExecC: Execute CPU Clock.

• S:DPA: (DPA stands for DLP processing agent).
  • S:DPA=SL: Sensitivity Label.
    • labelId: guid of the sensitivity label.
  • S:DPA=DPR: data loss prevention rule.
    • policyId: guid of the DLP policy.

• S:CFA=AS: (Content Filtering Agent) | AS: AntiSpam Agent.

• sfv: Spam Filter Verdict of the email.

• rsk: Risk associated with the email.

• scl: Spam Confidence Level of the email.

• bcl: Bulk Confidence Level of the email.

• pcl: Phishing Confidence Level of the email.

• score: Internal score associated with the email.

• sfs: Internal spam rules which are matched on the email.

• di: Specifies what action has taken on the email.
  • di=sd: The email is deleted.
  • di=sq: The email is quarantined.

• CIP: The connecting IP address of the server which connected to EOP to send the email.

• IPV: Verdict of the IP address of the server.
  • IPV:NLI: The Connecting IP address is not present on any IP reputation list (DNSBL).
  • IPV:CAL: The Connecting IP address is added in the Connection filter (IP allow list).

• asf: Advanced Spam Filtering applied on the email.

• hctfp: The guid of the policy which processed the email.

The test email EMT shows hctfp=dfaac24c-6b94-4603-9923-04a6131bcf3c. To find out which policy processed the email I have to execute the below command in PowerShell after connecting the PowerShell to EXO module.

PS C:\Users\ashutosh> get-hostedcontentfilterpolicy | where{$_.guid -eq "dfaac24c-6b94-4603-9923-04a6131bcf3c"}

Name    SpamAction HighConfidenceSpamAction IsDefault
----    ---------- ------------------------ ---------
Default MoveToJmf  MoveToJmf                True

Custom data for source:STOREDRIVER | event_id:DELIVER.

This event captures the details of email delivery to the recipient mailbox.

• S:Mailboxes: Shows the guid of the mailbox where the email is delivered.
• S:MsgRecipCount: Number of recipient of the email.
• S:AttachCount: Number of attachment in the email.
• S:IncludeInSla: If the email is delivered within the SLA.

Reading custom_data field for Outbound email.

For outbound email, where the email is sent to any third server, the EMT shows details like, if the email is sent from HDRP or normal email delivery pool and other details.

Custom data for source:AGENT | event_id:AGENTINFO.

• SFV: Spam Filter Verdict
  • SFV=NSPM: Email is not marked as outbound spam.
• SFP: Confirms if the email is routed to normal delivery pool or high risk delivery pool.
  • SFP=1102: Email is sent via normal email delivery pool.
  • SFP=1501: Email is sent via High Risk Delivery Pool (HRDP).
• DIR=OUT: Directional of email is outbound.

Custom data for source:SMTP | event_id:SENDEXTERNAL.

This section will show us what happened when the EOP email sending server tried to connect to the external email recipient server.

• S:OutboundProxyTargetIPAddress: IP address of the recipient email server.
• S:OutboundProxyTargetHostName: FQDN of the recipient email server.
• S:OutboundProxyTargetEhloDomain: FQDN of the recipient email server where EOP tried to connect.
• S:OutboundProxyFrontEndIPAddress: IP address of the EOP server which connected to the recipient email server.
• S:OutboundProxyFrontEndName: FQDN of the EOP server which connected to the recipient email server.
• S:OutboundTlsDetails: TLS details of the connection negotiation between EOP and the recipient email server.
• S:IsSmtpResponseFromExternalServer: Confirms if we got the SMTP response for the connected from the recipient email server.
• S:MsgRecipCount: Number of recipient in the email.
• S:SendingOrg: Sending domain hosted in EXO.
• S:ExpirationTimeV2: Time by which the email is considered as expired (Default 2 days from the time of email is submitted by the submitting server).
• S:IsDsn: shows if the email is an NDR (if IsDsn is True) or normal email (if IsDsn is False).
• S:OutboundIpPool: Shows the outbound delivery pool of the email
• S:OutboundIpPoolName:
  • S:OutboundIpPoolName=RegularOutboundPool: email is sent from normal email delivery pool.
  • S:OutboundIpPoolName=HighRiskDeliveryPool: email is sent from high risk delivery pool.

Differences in terms.

AgentFork vs Message Forked.

message_id vs network_message_id.

Conclusion.