We recently worked on an incident with a client that had Defender for Endpoint with a Microsoft Business Premium license. This version of Defender XDR has the detection capabilities of EDR but lacks endpoint telemetry in Advanced hunting which as a consequence makes it a lot harder to use KQL to identify malicious activity across all endpoints.
However, we had access to endpoint activity through the Device Timeline feature of Defender for Endpoint in the Microsoft Security Portal. What’s even better is that this data goes back 180 days, as opposed to the 30 days maximum retention for Advanced hunting – so this should be useful even for those with E5 Security licensing.
To get around these limitations we created a combination of tools which allowed us to export all data and import it as JSON in Sentinel.
After exporting the full Timeline data from several hundreds of clients and uploading the data to Sentinel, we then used KQL to hunt for specific process names, hashes and network activity across all devices. This helped us determine which machines the threat actor had accessed and which machines were compromised.
In this blog post we’ll share some details, a link to the Github repo where you can find the tools you need to export your own data from Device Timeline and import it to Sentinel. You will also find a useful few KQL queries.
Device Timeline in Defender
The Timeline pane in Defender for Endpoint contains information that is highly useful for us when trying to figure out the sequence of events that led up to a compromise as it allows us to see events like:
-
Processes events
-
Network events
-
Logon events
For a more detailed dive into the features, please read this Microsoft article on Device Timeline.
The problem
While the data that exists in Timeline is valuable, this feature has several significant drawbacks that prevents us from using it in an effective way when working on large scale incidents.
One device at the time
Using Defender’s XDR portal you are only able to work with one machine at a time. For obvious reasons, this doesn’t scale very well when we want to ask questions and investigate events across the entire estate.
Limited search capabilities
The search functionality is very limited and can be summed up as “search and see if there is a match”. It lacks support for if statements, conditions, sorting or any other operator we all have grown accustomed to with query languages like KQL.
No access to raw data
The only way to access the data is through the user interface, which is in general slow and often glitches, resulting in you having to scroll past the same events several times. In addition, it makes it cumbersome to copy events of interest from the Timeline to another master timeline, resulting in screenshots being the only documentation of your investigation.
The solution
Maxime Thiebaut had already developed a working tool for exporting Device Timeline data from devices, so we developed some scripts around this tool that:
-
Identifies all endpoints and exports Timeline data in a Defender XDR tenant
-
Bicep template that creates Azure components required to receive the logs in Microsoft Sentinel
-
Script that ingests all the JSON files in Microsoft Sentinel
-
KQL queries that you can use to hunt for file hashes, processes and network connections
We have created a GitHub repository called “Timeline-to-Sentinel” containing instructions on how to install and use all the scripts you need here.
How to query your data
We have developed some KQL queries with performance in mind, which can help you with:
IOC lookup using SHA256
let SearchSha256 = "Your SHA256 here";
Timeline_CL
| where TimeGenerated >= ago(365d) // Query adjustable - Data is old now
| search SearchSha256 // Search for hash directly as it can be in many nested structures
| extend TypedDetailsParsed = parse_json(TypedDetails)
| mv-apply TypedDetailsParsed on (
summarize Bag = make_bag(pack(tostring(TypedDetailsParsed.key), tostring(TypedDetailsParsed.value)))
)
| evaluate bag_unpack(Bag, columnsConflict="keep_source")
| project ActionTime, InitiatingProcess, InitiatingProcessParent, InitiatingUser, Machine, Content, "Content sha256" // Example columns..
Process activity
let Process = "YouEvilProcessHere.exe"; // Process name you want to find
Timeline_CL
| where TimeGenerated >= ago(365d) // Query adjustable - Data is old now
| where TypedDetails contains "Created process image file" // Hack to filter JSON data faster
| extend MachineParsed = parse_json(Machine)
| extend TypedDetailsParsed = parse_json(TypedDetails) // Contains e.g. "Created process image file", "Created process command line"
| where isnotempty(TypedDetailsParsed)
| mv-apply TypedDetailsParsed on (
extend key = tostring(TypedDetailsParsed.key),
value = tostring(TypedDetailsParsed.value)
| where key == "Created process image file" and isnotempty(value) // We want the image name
| extend ImageName = value
)
| extend MachineName = tostring(MachineParsed.Name)
| project ActionTime, MachineName, ImageName, InitiatingProcessParent, InitiatingUser
| where ImageName contains Process
// For events relating to one endpoint only simply add ""| where MachineName == "YourMachineNameHere""
Remote IP Connections
let SearchIP = "EvilIP"; // Remote IP you want to search for
Timeline_CL
| where TimeGenerated >= ago(365d) // Query adjustable - Data is old now
| where TypedDetails contains "Network connection remote address" // Hack to filter JSON data faster
| extend MachineParsed = parse_json(Machine)
| extend TypedDetailsParsed = parse_json(TypedDetails)
| where isnotempty(TypedDetailsParsed)
| mv-apply TypedDetailsParsed on (
extend key = tostring(TypedDetailsParsed.key),
value = tostring(TypedDetailsParsed.value)
| where key == "Network connection remote address" and isnotempty(value)
| extend RemoteIP = value
)
| extend MachineName = tostring(MachineParsed.Name)
| project ActionTime, MachineName, RemoteIP, InitiatingProcessParent, InitiatingUser
| where RemoteIP contains SearchIP
// For events relating to one endpoint only simply add ""| where MachineName == "YourMachineNameHere""
Pål André : Mar 15, 2023 3:05:02 PM
