This latest update to EventSentry improves your security posture with validation scripts, simplifies IT troubleshooting for both administrators and users, gives you visibility into installed browser extensions along with many other usability improvements in the web reports.
Proactively identifying (potentially) malicious behavior is the cornerstone of any security defense, and a key feature of log management / SIEM solutions. But many security violations are the direct consequence of incorrect or missing settings on endpoints.
Traditional log management solutions may show you when something is happening that shouldn’t be happening, yes. But wouldn’t it be better to assess key OS components and security settings on a regular basis, and identify known weaknesses?
Consider a motion-triggered camera that will let you know when somebody is snooping around your property at 3AM in the morning. That camera is extremely important, and the foundation of any serious property security system – without it, you wouldn’t even know what was going on!
But wouldn’t it be even better if somebody was inspecting your windows, fence and locks on a regular basis, to let you know if a door or window was unlocked, or an insecure lock was being used at one of the entrances? If your overall perimeter was more secure in the first place, there would be fewer potential intrusion attempts.
And that’s exactly what EventSentry’s 60+ validation scripts do. Our managed security & health checks continuously compare critical settings on your monitored hosts with our baseline, immediately indicating potential risks. These checks identify a wide variety of potential risks, such as:
A Windows server/workstation is not on the latest patch
Windows firewall is disabled
No A/V software installed
Insecure TLS protocols are enabled
Microsoft accounts aren’t blocked
EventSentry already includes a number of features that help detect security violations, rogue network devices, unauthorized software, suspicious network activity and more. But by utilizing the new validation scripts, you can fix many problems at the source – before they show symptoms.
The scripts are managed by NETIKUS.NET, updated regularly, and can be downloaded through the management console with a single click. Validation scripts are also tagged with keywords such as #server #compliance #stig-high-server to make sure that only relevant checks are assigned.
Which Browser Extensions are lurking in your network?
While web browser extensions can boost productivity and excite your end users, they also have inherent privacy and security risks. All major web browsers let users install as many extensions as they wish by default – without restrictions!
But do you actually know how many Firefox, Chrome or Edge extensions are installed on browsers across your IT infrastructure?
As an “extension” (no pun intended) of EventSentry’s software monitoring component, all browser extensions of Mozilla Firefox, Google Chrome and Microsoft Edge (Chromium-based) are inventoried with support for:
Alerts (extensions are installed/updated/uninstalled)
With this information at the fingertips, an initial discovery can be performed, a baseline set and reports or alerts can be received on a regular basis showing new extensions being installed.
Troubleshoot, Document & Support End Users with “EventSentray”
Supporting your end users has probably never been more challenging, considering they’re distributed all across the place and not conveniently squeezed into an office building anymore.
With the tray app “EventSentray”, your end users can submit support tickets to many common ticketing systems via email or HTTP requests right from the tray with a customizable link. And the best part? Support tickets created by the app not only include pertinent system information (current CPU %, host name, uptime, …) but can also include a current screenshot.
But we didn’t just design the tray app to give end users a way to submit support tickets right from their desktop, but also to help sysadmins.
Let’s be honest, when we log on to a server then it’s often because something isn’t working the way it should. Wouldn’t it be nice if one had easy access to information like:
CPU, Memory, Disk Usage & Utilization
Top 3 apps consuming CPU and memory
IP address, host name and connection speed
Whether the host needs a reboot
Simply double-clicking the EventSentry icon and the System Information dialog will show all of the above information – and more. Hovering over the charts will reveal additional hardware information as well.
And for those working in teams with shared responsibilities, right-clicking the tray app also lets you add notes (including a screenshot) for the monitored host. Those notes are then visible in the web reports and ensure that everyone on your team is on the same page when you make significant changes to a server or workstation. Documentation is key!
Tracking Administrator Activity
Many compliance frameworks require that you track activity by Administrators (e.g. Domain Admins) on your network. ADMonitor users now have the ability to filter all compliance reports (e.g. Logon Activity, Process Activity) to only show activity from users with domain admin privileges.
Dashboard Import / Export
To make setting up dashboards easier and faster, EventSentry now ships with a number of dashboard templates that you can import. You can also export your own dashboards and import them on another EventSentry installation.
Webcam & Image Dashboard Tiles
The latest edition of the web reports includes a number of dashboard improvements, but the new image / webcam tile type definitely sticks out.
With the new “Image” tile you can point the web reports to a static image or stream to be displayed on any dashboard!
Since Active Directory is the foundation of all Windows networks, monitoring Active Directory needs to be part of any comprehensive security strategy. Up to version 3.5, EventSentry utilized Windows auditing and the security event log to provide reports on:
User Account Changes
Computer Account Changes
While this functionality provides a good basis for monitoring the most relevant changes to Active Directory, we felt that a more comprehensive approach to monitoring Active Directory was needed – without the need to install & maintain yet another product!
ADMonitor is new (optional) component included in EventSentry that vastly improves Active Directory monitoring with these additional features:
Monitors changes to all objects (e.g. OUs) – not just users/groups/computers
Captures every attribute change made to an object, not just high level changes
Provides before & after values for all changes
Monitors Group Policy changes
User status reports (show idle users, users with non-expiring passwords, …)
Monitoring does not require auditing
We’re excited that we can now offer EventSentry ADMonitor to our users who are looking for a more in-depth Active Directory monitoring solution.
Active Directory is essentially a representation of the employees and their roles in your organization. But employees come and go, roles/responsibilities change, contractors get temporary access and so forth. But while adding users and additional access is usually reflected properly in Active Directory (otherwise IT would get a call because somebody presumably can’t do their job), removing access is often forgotten. As a result, users that should have been removed from AD a long time ago continue to exist. With ADMonitor it’s easy to identify orphaned user accounts (and many others) and keep your Active Directory lean and clean.
Since a significant development effort stands behind ADMonitor, it will be offered as an optional component that is licensed on a per-user basis. Pricing is very competitive with other solutions and we also offer bundle discounts to customers who already own or will purchase agent licenses; please request a quote here.
But enough theory, let’s look into the installation, configuration and reporting of ADMonitor.
Since ADMonitor is a component of EventSentry, it’s easily activated as part of the main EventSentry setup. Just like with other components of EventSentry (Heartbeat Monitor, Collector, …), users have the option to enable ADMonitor during the post installation setup procedure.
ADMonitor can be installed on any host that is part of the domain that needs to be monitored, it does not need to be installed on a domain controller.
Immediately following the initial installation, ADMonitor will initialize itself by creating an offline copy of all Active Directory objects. This process can take from a few seconds to a few minutes, depending on the number of objects in AD, connection speed to the domain controller as well as the overall performance of the host running ADMonitor.
The initial configuration of ADMonitor is simple and only requires you to pick a password for the ADMonitor service account. If you’re adding ADMonitor to an existing installation you may also need to select the appropriate EventSentry database action to which ADMonitor reports changes. Otherwise, ADMonitor is ready from the get go and will monitor all Active Directory changes.
ADMonitor provides three types of reports:
Group Policy Changes
Shows any change made to an AD objects. Reports can be filtered on the type of action performed (added, removed, modified), on the object type (user, group, organizationalUnit, …) and on the user who performed the action.
Note that the detailed changes to group policies are available in the “Group Policy Changes” report below. Of course you can expect the same type of summary view you’re already used to from most other EventSentry features and create reports like:
Show all changes to organizational units
Show all new objects created
Show all users that were changed
Group Policy Changes
When a group policy is changed, it is first indicated on the “Object Changes” report, since the versionNumber attribute of the AD object changes. The actual group policy settings themselves are available on the “Group Policy Changes” report however, since group policy settings are not stored in AD.
The screen shot below shows that the Default Domain Policy was changed, with the Specify traps for the public community setting being enabled.
The users report helps you identify potentially problematic user accounts such as idle users, users who haven’t change their passwords in years and others.
This report contains a list of all user objects in Active Directory including the following details:
Name, Full Name, SAM Account Name, Path, UPN
Administrative Account (yes/no)
Password Never Expires (yes/no)
Password Expired (yes/no)
Password must change (yes/no)
Locked Out (yes/no)
Password Last Set
Account Expiration Date
With ADMonitor you can now get detailed user stats with just a few clicks and quickly identify user accounts that need to be reviewed, changed or deleted. Of course you can also schedule all reports directly from the web reports and get daily/weekly AD status reports directly in your inbox, e.g.:
List of all Group Policy changes
List of all idle user accounts
List of all newly created users and/or groups
You can also create your own reports for just about anything that involves a change to an Active Directory object, for example all organizational units created in the last 24 hours.
ADMonitor also includes a number of stand-alone utilities that support advanced features such as filtering and email notifications that I will cover in a future post.
With ADMonitor, EventSentry users can now gain the additional visibility needed to fully audit all Active Directory & Group Policy changes. As a result, EventSentry users can more easily enhance compliance, security and accountability in their network without the need to install additional software – saving both time and money.
EventSentry v3.5 continues to increase visibility into networks with additional vantage points, making it easier for EventSentry users to reduce their attack surface as well as discover anomalies.
Process & Network Activity Tracking
One major focus of this release is process network activity, an important component in any monitoring strategy. Do you know which applications listen for incoming connections on your monitored machines – or when a new process suddenly starts accepting incoming traffic? Do you know which processes perform outgoing network connections, and to where? How much data are they transferring?
To help you (and possibly your overzealous auditors) answer these questions, EventSentry v3.5 takes the existing process tracking functionality to the next level by integrating with Sysmon and showing processes with active or listening network connections. With EventSentry deployed you can now see:
Complete Process Details (start & stop times, duration, caller, PID)
Process command line
All processes listening for incoming connections
All active processes
Network activity initiated by a process (Figure 1, requires Sysmon)
Correlation with EventSentry NetFlow (Figure 2, requires NetFlow component)
This means that you can easily see which network connections a host establishes if you have Sysmon installed, and can even correlate that information with the EventSentry NetFlow component with just a click (see below). This information is invaluable for forensics and troubleshooting alike.
But even without Sysmon, EventSentry can now show you every open TCP port on a monitored machine (optionally all active connections as well), making it easy to discover rogue services on a network – even if they are blocked by the firewall. Figure 3 below shows all active processes which are listening for incoming connections, grouped by host.
A new member in the compliance tracking features family is registry tracking. Similar to file access tracking, it normalizes all registry audit events on a monitored machine, making it much easier to report on the registry activity and changes. Configuring registry tracking to work with existing registry auditing is incredibly easy and can be enabled in 60 seconds if the proper audit settings are already in place. The screen shots below show a list of recent registry activity as well as the details of changes:
Users managing a large number of hosts will appreciate the new “Tags” feature which addresses a shortcoming with the existing flat group structure. Tags allow groups or hosts to be tagged with keywords (e.g. production, staging, development). The created tags can then be used in the web reports (e.g. Show me disk space from all hosts tagged with “development”) and for dynamic package assignments.
The file checksum monitoring component received a few enhancements to help reduce noise while also adding new functionality.
FIM can now verify the digital signature of executable files and optionally suppress alerts if a file is digitally signed – think Windows updates. This can reduce the number of alerts you get significantly and thus make the remaining alerts more meaningful. The digital signature status can also be displayed in the web reports as a new column.
FIM can also calculate the entropy (essentially a measure of randomness) of files with a scale from 0 to 10, with 10 being the maximum of a completely random file. This is useful for Ransomware detection, since encrypted (and compressed files as well) files have a higher entropy than regular files. Combined with a threshold filter this can detect when a large number of encrypted files are suddenly being processed in a given directory and thus indicate a Ransomware infection.
We also replaced the existing SHA-256 checksum algorithm with a faster version in 3.5 which should result in a lower CPU utilization on systems which need to calculate a large number of checksums.
Disk Space Monitoring
A common annoyance with disk space monitoring are large volumes where an otherwise useful limit of, say 5%, is just not useful. For example, 5% of a 2 Tb drive is still 100 Gb, and in most cases there is probably no reason to sound an alarm. Dynamic thresholds (a new feature) addresses this issue by automatically adjusting the limit based on the drive size. The result: Fewer alerts!
EventSentry will log an event to the application event log when dynamic thresholds are enabled AND the current settings warrant a change. An event will look something like this:
The percentage-based threshold on drive F:\ has been dynamically adjusted from 5 percent to 0.5 percent based on the total drive size of 999 GB. A low disk space alert will be triggered when the available space on this volume falls below 19 GB.
The software inventory page (detailed tab) will now show which hosts do NOT have a particular software installed when the search is restricted to a specific software product. Also related to processes, process tracking can now generate the checksum of all execute files, which can then be searched for at Malware databases like virustotal.com. If you utilize the maintenance mode feature in EventSentry then you can now see whether a host is in maintenance mode or not in the web reports. And last but not least, event logs can now be sent to a remote Syslog receiver via TLS.
Under the Hood
We always tweak and improve EventSentry to ensure it runs as efficiently as possible. In this release we replaced the SHA 256 algorithm with a more efficient version, resulting in less CPU usage by the agent when calculating SHA 256 checksums. As we gradually move to a full 64-bit monitoring suite, the Heartbeat Agent is the next component now also available as a 64-bit process so that all EventSentry services are now available in 64-bit. We plan on porting all executables over to 64-bit within the next 6-12 months. At that point you will not be able to run EventSentry on 32-bit platforms anymore; monitoring 32-bit hosts will of course still be supported for the foreseeable future.
To help with the stability of all EventSentry and simply troubleshooting, all server-side components will now automatically generate crash dumps if they encounters a problem. Finally, the management console includes additional context and ribbon buttons.
Windows supports a code-signing feature called Authenticode, which allows a software publisher to digitally sign executable files (e.g. .exe, .msi, …) so that users can verify their autenticity. The digital signature of a file can be viewed in the file properties in Windows explorer on the “Digital Signature” tab.
Digital signature verification has been added to the checksum utility, which already calculates the checksum and entropy of a file. When using the new /s switch, checksum.exe will tell you whether:
the file is digitally signed
a counter signature exists
the digital signature is valid
the algorithm used (e.g. SHA 256)
who signed the file
who issued the certificate
when the file was signed
The utility also sets the ERRORLEVEL variable accordingly; if a signature check is requested with the /s switch but the file is unsigned, then checksum.exe will return %ERRORLEVEL% 2. Below is a sample output of the utility in action:
Digital signature verification will be added to EventSentry’s FIM monitoring component (“File Checksum Monitoring”) in the upcoming v3.4.3 release, which will give you the option to only get notified when unsigned files are changed, thus reducing overall noise.
You can download the latest version from here – enjoy!
Imagine someone getting the seemingly innocent ability to run a couple of commands on a machine on your network WITHOUT installing any new software, but those commands resulting in a reverse shell running on that same machine – giving the intruder a convenient outpost in your network. Now stretch your imagination even further and pretend that all of this happens without leaving any unusual traces in logs – leaving you completely in the dark. It’s like somebody living in your house or apartment yet you have no idea they’re there. Are you getting goose bumps yet?
Not too long ago I talked with Michael, the creator of the popular cheat sheets which cover PowerShell, the Windows Registry, Windows Logging, and more. Michael ran a few scenarios by me that involved exploiting PowerShell and was curious how EventSentry could help detect those. This really sparked my interest in the topic, and after coming up with a few RegEx expressions that could be used in an EventSentry filter I decided to look more into this subject. I really have to take the opportunity to thank Michael here, whose cheat sheets and input helped me come up with this article and the new PowerShell Security event log package in EventSentry.
If you’re not an InfoSec professional then you may not be fully aware that PowerShell – you know, the language you’re supposed to be fluent in by now – is quite commonly used in attacks. In fact, InfoSec already reported back in 2016 that 38% of all attacks utilize PowerShell in one way or another. And let’s be honest – why wouldn’t you utilize a tool that is pretty much guaranteed to be installed while giving you full access to the .NET Framework and all Windows APIs? So if you haven’t already done so, then securing PowerShell in your environment is something you should think about sooner rather than later. This and the follow-up articles will assist you with this effort.
So what’s so potentially bad about PowerShell in particular? Now, Windows has always shipped with VBScript, a scripting language that’s easy to use for both simple and potentially more complex tasks. In fact, most of the things people do in PowerShell can be done with VBScript just the same. A key benefit of PowerShell, however, is the ability to utilize the .NET framework, something VBScript can’t since it can only interact with COM objects. And since PowerShell is, well, a shell, you get to pipe input/output and create powerful one-liners. On top of that, PowerShell contains some nifty features like encoding scripts, making it possible to run fairly complex code without ever having to use an actual .ps1 script file on disk. It’s VBScript on Steroids.
Here are some concrete examples as to what evildoers can do with PowerShell:
Remember when I talked about “without leaving a trace” above? That’s because Microsoft didn’t introduce the ability to log detailed PowerShell activity until version 5, although PowerShell 3 & 4 generate reasonably useful audit logs as well. In order to protect ourselves against PowerShell attacks, we need to first detect it, which we can only do if PowerShell leaves traces. PowerShell’s ability to produce useful audit logs greatly depends on the version, however, which the table below illustrates:
Which version of Windows ships with which version of PowerShell
What is the highest supported version of PowerShell for each version of Windows
Shows the versions of PowerShell that ship with Windows as well as the highest supported version of PowerShell
As you can see from the table above, thankfully most versions of Windows are compatible with PS v5, so unless you’re unfortunate enough to be running Server 2008 (or Vista), you should be able to deploy PowerShell 5.1 to most of your systems. I say most, because some Microsoft applications (e.g. Exchange Server 2010) aren’t compatible with PowerShell v5, so you’ll want to make sure you do some research on those machines that actively use PowerShell to prevent disruption.
Coexistence & Legacy
An important thing to note here is that PowerShell v1/v2 can peacefully coexist with PowerShell v3-v5, while versions 3 and later are always upgraded to the latest version. This means that you could have v2 and v4 installed (and many systems do), but not v3 and v5. What’s also interesting is that PS v2 is installed with every major version of Windows (including Server 2016!) although not usable until the .NET Framework v2.0.50727 is installed.
Starting with EventSentry v22.214.171.124 you can thankfully use EventSentry’s software inventory to determine which versions of PowerShell are installed on your network. If you haven’t manually deployed PS v5 yet and aren’t running Windows Server 2016 widely yet, then you will probably see PowerShell v2 and v4 installed on most hosts on your network. EventSentry’s grouping mechanism comes in real handy here.
Please note that even though PowerShell v2 may be installed on a machine it doesn’t necessarily mean that PowerShell v2 is actually usable. PowerShell relies on the .NET Framework being installed, and PowerShell v2 specifically relies on the .NET Framework 2.0.50727 (which is part of the 3.5 .NET Framework) – something that is usually not installed by default. I will explain later why this is a good thing.
OK, but enough about boring PowerShell versions. If you just remember one thing from the above tables and paragraphs it’s this:
Thankfully you don’t need version 5.x to get useful logging – even PowerShell v3 & v4 can log relevant details in the (Windows PowerShell) event log, e.g. the PowerShell command line or commands executed within the PowerShell shell. In fact, even the (decoded) commands are logged to the event log when obfuscated with the -encoded switch.
Logging can be enabled either through group policy or via registry settings. There are three general areas for logging available:
Script Block Logging
Module Logging Since everything that is executed in PowerShell is essentially located in a module, module logging will at least generate a high-level audit trail of PowerShell activity and potentially malicious activity. At minimum this will show which commands were executed through PowerShell. This logging level should always be enabled and is useful starting with PS version 3.
Important: Module Logging only works if you specify at least one module to be monitored. Since it’s difficult and cumbersome to predict and edit a list of all modules that could potentially cause harm, I recommend just specifying the * wildcard characters as the module list – see screenshots below.
Script Block Logging Script Block Logging is more verbose than module logging and provides additional context and output, especially when functions are called and function output itself is invoked as a command. The amount of noise heavily depends on the type of PowerShell activity, but I’d recommend turning this option on as well. If it ends up producing too much noise / volume it can always be disabled or customized later.
Transcription This provides a full log of all input and output and requires additional considerations in regards to where the transcription files are placed. I’d only recommend this for high-secure environments, you can learn more about it here. Transcript files are stored in the file system, so it’s a little more work than just adding up a couple of registry values. If you enable this feature then you’ll need to make sure that the actual transcript files (which likely contain sensitive data) are protected from unauthorized access.
It’s definitely recommended to configure these options via Group Policy to ensure that all machines in the domain receive the settings. If changing group policy is not an option in the short term then you can at least set the registry options until you have an opportunity to set it via group policy. You can use a tool like the EventSentry Admin Assistant to push registry settings out to multiple hosts with just a few clicks.
Group Policy: Configuring this is unfortunately less straightforward than you’d think or expect, depending on the OS version of your domain controller. You can expect the “Module Logging” option to be available in the group policy editor on 2008 R2 and later, however “Script Block Logging” is only available on server 2016 or after manually updating ADMX files. See this thread on how to update your ADMX files. In my environment I just had to replace the PowerShellExecutionPolicy.admx and en-US\PowerShellExecutionPolicy.adml files in the %SYSTEMROOT%\SYSVOL\sysvol\[DOMAINNAME]\Policies\PolicyDefinitions directory with the newer versions after installing the latest version from here.
Registry: Only the HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\PowerShell key exists by default, the other two sub keys “ModuleLogging” and “ScriptBlockLogging” have to be created before you can add the “EnableModuleLogging” and “EnableScriptblockLogging” DWORD values inside those sub keys.
For Module Logging, as shown in the screenshot below, you’ll also need to create the “ModuleNames” sub key along with a list of modules that will be monitored. I recommend just using the asterisk character which monitors any module.
Configuring PowerShell Event Logging
Key: HKLM\SOFTWARE\Policies\Microsoft\Windows\PowerShell\ModuleLogging Name: EnableModuleLogging Data: 1 (DWORD)Key: HKLM\SOFTWARE\Policies\Microsoft\Windows\PowerShell\ModuleLogging\ModuleNames Name: [ModulePattern] Data: [ModulePattern] (REG_SZ)See the screenshot above for example on module logging.
Policies\Administrative Templates\Windows Components\Windows PowerShell\Turn on Module Logging
You don’t need to restart after setting the registry values, they will become effective immediately. The same applies to group policy – as soon as the target host has applied the group policy settings, PowerShell will enforce the new logging options.
PowerShell logs a lot of different events to two different event logs, and the table below shows the events I have observed on test systems. Even though the table may not be 100% complete, it does list all the events that are relevant for threat detection. If an event is not listed below then it is likely not relevant for forensics. We will update the list if necessary.
What’s interesting to note is that newer versions of PowerShell will often log to both event logs simultaneously.
Security Event Log Auditing
PowerShell logging is great, but given the discrepancies between the different versions and the possibility to evade it (more on that later), I prefer to have as many methods as possible at my disposal that tell me what PowerShell is doing.
Since PowerShell code is usually invoked via powershell.exe (I’m point this out because you technically don’t have to use powershell.exe, and attackers are coming up with creative ways to launch it through other ways – more in part 2 of this series), and because we’re after that processes’ command line, it’s important to monitor Process Start (event id 4688) events from the security event log in addition to events logged by PowerShell itself. This means you’ll need audit the following sub categories from the Detailed Tracking category:
If you are not using EventSentry then I recommend collecting both 4688 and 4689 events so that you can not only determine whether a powershell.exe process was started, but also how long it remained active. If you are an EventSentry user then you just need to verify that Process Tracking (an object for Compliance Tracking) is enabled and configured to capture the command line of a process. EventSentry can automatically parse and correlate 4688 and 4689 events and provide a history of all processes on a monitored system.
EventSentry users can also utilize the Audit Policy Status page to verify that process creations are indeed being audited. You’ll also want to make sure that “Include command line in process creation events” is activated, so that Windows logs the command line of every process as part of 4688 events. After all it doesn’t help us that much just knowing that powershell.exe has been running, we need to know what exactly it has been running.
This can either be enabled via group policy (Administrative Templates\System\Audit Process Creation\Include command line in process creation events) or via the registry (set HKLM\Software\Microsoft\Windows\CurrentVersion\Policies\System\Audit\ProcessCreationIncludeCmdLine_Enabled to 1).
Disclaimer: This option is available starting with Windows 7 / Server 2008 R2, earlier versions of Windows don’t support it. Things are a little easier for EventSentry users, which attempts to obtain the command line of a process if it’s not included in the 4688 event and subsequently makes it available as variable $STR9. But more on that in part 2 when we discuss ways to detect and mitigate attacks.
I hope I was able to convince you of the risks that PowerShell poses, what versions of PowerShell are out there, and what type of logging needs to be enabled in order to detect and stop malicious PowerShell in its tracks. In part 2 I’ll talk about how to actually mitigate PowerShell-based attacks – with specific instructions for EventSentry.
In Mr. Robot‘s episode 9 of season 2 (13:53), Angela Moss needs to obtain the Windows domain password of her superior, Joseph Green, in order to download sensitive documents that would potentially incriminate EvilCorp. Since her attack requires physical access to his computer, she starts with a good old-fashioned social engineering attack to get the only currently present employee in the office to leave.
Once in his office, she uses a USB Rubber Ducky, a fast and automated keyboard emulator, to obtain Joseph’s clear text password using mimikatz. Please note that there are some holes in this scene which I will get into later. For now we’ll assume that she was able to obtain his credentials by having physical access to his computer.
After she gets back to her workstation, she analyzes the capture which reveals Joseph’s password: holidayarmadillo. Not the best password, but for this particular attack the quality of the password wouldn’t have mattered anyways. Mimikatz was (is) able to get the password from memory without utilizing brute force or dictionary techniques. Once she has the password, she logs off and logs back on with Joseph’s user and downloads the documents she needs.
As somebody who helps our users improve the security of their networks, I of course immediately contemplated how this attack could have been detected with EventSentry. Since most users only log on to one workstation on any given day with their user account, Angela logging in with Joseph’s account (resulting in “joseph.green” logging on to two different workstations) would actually be an easy thing to detect.
Introduced with v3.4, collector-side thresholds allow the real-time detection of pretty much any user activity that originates from an event, for example user logons or process launches. You can tell EventSentry that (physical) logons for any user on more than one host (within a given time period – say 9 hours) should trigger an alert (aka as “lateral movement”). Had this been in place at EvilCorp, Angela logging in as Joseph would have immediately triggered an alert. With the right procedures in place, countermeasures could have been taken. Of course most viewers wouldn’t want Angela to be caught, so please consider my analysis strictly technical. Watch a short video on lateral detection with EventSentry here.
So what’s the hole? Well, the rubber ducky (mimikatz, really) requires access to an active logon session, which Angela most likely didn’t have. It looked like Joseph had been out of the office for a while, so his computer was likely either locked or turned off, rendering any attack based on mimikatz useless. Mimikatz – since it obtains passwords from memory – only works if the computer is unlocked. And had the computer been unlocked then she could have just downloaded the files from his computer – although this would have been even more risky with people walking around the office.
Cyber attacks are becoming more potent every year and are often sponsored by powerful criminal gangs and/or governments. It’s important that companies employ multiple layers of defense to protect themselves (and their customers) from these increasingly sophisticated and destructive attacks.
EventSentry is the only monitoring solution that utilizes robust agent-based technology that goes beyond logs, enabling the fusion of real-time log monitoring with in-depth system monitoring to not only detect but also react to attacks and anomalies. See for yourself and download a free evaluation of EventSentry.
Integrate with open source solutions (Graylog, ELK, Nagios Log Server & others)
and more. We’ve also been focusing on making the data EventSentry collects more actionable and subsequently more helpful, and as result you will see additional contextual data provided with some alerts & reports, and one new search page in EventSentry.
All in all, this upcoming release has a lot of improvements in store to help you do your job better by ensuring that your network is as reliable, secure and fast as possible.
While high-profile Ransomware attacks have slowed down somewhat in recent weeks, businesses – especially small businesses – are still hit with Ransomware infections every day. Even though EventSentry is not positioned as a AV or a AntiMalware software, it does include a variety of functionality which can detect Ransomware outbreaks.
What sets EventSentry apart from AV and most AntiMalware solutions is that it looks for pattern exhibited by the Ransomware – regardless of the variant. What’s new in version v3.4 is the ability to detect changes to the MBR and bootsector as well as the ability to calculate the entropy of (changed) files.
MBR/BootSector Monitoring & Backup
Some more recent Ransomware variants modify the MBR and/or boot sector and trigger a reboot to initiate an offline encryption process. EventSentry v3.4 can now monitor the MBR and detect changes in near real-time to alert the admin when suspicious activity is occurring.
By utilizing EventSentry’s advanced filtering engine it is also possible to potentially stop the encryption process, e.g. by hibernating the infected host. EventSentry even backs up the MBR and boot sector, making it possible to repair an infected system (with a boot disk) without having to perform a full restore from backup.
Entropy describes the randomness of a file, essentially a metric that can help detect compressed and encrypted files.
Since most Ransomware encrypts large amounts of files, EventSentry can utilize the entropy of a file, combined with event log thresholds, to make a determination that a Ransomware infection is in progress and take corrective measures.
Lateral Movement Detection with Thresholds
EventSentry has always included agent-side thresholds in order to throttle the alert volume or detect repeated events. Because these thresholds were executed on the agents, event patterns which involved more than one host could not be detected that way.
By adding a threshold component to collector – which analyzes and processes all events in real time – we can leverage this feature to new heights and detect network-wide event-based patterns – in real time!
Collector-side thresholds are configured exactly like agent-side thresholds with one the key difference – the threshold limit applies to all aggregated events sent to the collector. Collector-side thresholds also introduce the “Group By” setting that makes the lateral movement detection possible – the ability to count unique instances of an event property, and not just the total number of events.
Some of the event-based threat patterns you can detect:
The same user logging on to multiple hosts within a specific time frame
A process spreading (trickling) across multiple hosts within a specific time frame
A user running too many processes – either on a single or multiple hosts
Authentication failures of a user on too many hosts
Too many unique logon types used by a user account
Any event property and insertion string can be used to craft thresholds – the sky is the limit.
Software Version Checker
Earlier versions of EventSentry include a substantial hardware, software and patch inventory, making it extremely easy to find out which software packages are installed on your network, but also get alerted when software is installed or removed from a server/workstation.
In v3.4 we are taking this to the next level by providing the latest version available from the publisher for a growing list of 100+ software packages so that you can effortlessly identify outdated software on your network. You can now create reports listing any software on your network which is outdated, a dashboard tile is also available. The Web Reports refresh the version info list every 2 hours to ensure all reports are accurate.
If an application you are using on your network is not currently included then simply email us the name of the software as it is detected by EventSentry (and ideally the URL where we can obtain the current version) and we will add it to our list.
64-bit Web Reports for Windows
The EventSentry web reports are now available as a 64-bit application, and upgrading to v3.4 will automatically upgrade the existing v3.x 32-bit web reports to 64-bit on 64-bit when installed/upgraded on a 64-bit version of Windows. The new 64-bit web reports will allow you to run larger reports that would not run due to limits with the address space associated with 32-bit applications.
UPS & Battery Monitoring
Any UPS directly attached to a server/workstations that is detected by Windows can now be monitored by EventSentry. The status of the UPS will show up on the host inventory page, and alerts will be generated when a host is on battery power and back on AC power. EventSentry can also initiate a shutdown when the remaining run-time or charge level falls below a certain limit.
Batteries in laptops are also detected and listed on the host inventory page (battery capacity and current charge level), but generated alerts are informational only.
User Activity Tracking
While EventSentry provides its users with a wealth of information from multiple angles, it can be tedious to piece together data from multiple reports that is associated with a specific user. Data which can be linked to a user is scattered among process tracking, file access tracking, compliance logons and other pages.
The new “User Activity” page, which is located in the “Dashboard” menu, solves this problem by displaying data from the following pages on a single page:
Active Directory Changes
The user activity page makes seeing all activity by a user as easy as never before!
Integration with third-party log management solutions
A few months ago, one of our users approached with the need to integrate EventSentry into an existing log management system which was already in place at the location where EventSentry was to be deployed. While reviewing the request we recognized that even though we position EventSentry as a one-stop log management solution with a compelling and robust web-based reporting component, an integration with other products can be helpful in some cases.
Supplement EventSentry’s built-in reporting with additional reporting
Integrate EventSentry with an existing log management solution located in a different business unit
Integrate EventSentry’s sophisticated real-time agent and deployment infrastructure with a different log management back-end
In version 3.3 and earlier, EventSentry can be integrated with 3rd party products using the HTTP, process and Syslog action. The HTTP & process action are intended to be used with ticketing systems where only a low volume of alerts are submitted while the Syslog action obviously supports submitting a high volume of events. The Syslog format was however limited to the traditional RFC 3164 format, making an integration with other log management systems difficult.
Starting with version 3.4, EventSentry now supports the following formats in the Syslog action
RFC 3164 (legacy)
Nagios Log Server
Common Event Format (CEF)
If a log management server you need to integrate with is not listed above but supports the JSON format, then you can craft your own JSON packet with the JSON format, also introduced in v3.4.
Disk Space Alerts
Part of the effort to make EventSentry’s alerts more actionable is reflected in our improved disk space alerts which now list the 15 largest files and folders of the volume where disk space is low. The supplemental data will in many cases be enough to immediately identify the culprit so that corrective action can be taken immediately, without the need to run disk space analyzers on the volume.
Audit Policy Status
Since the introduction of the compliance tracking components, EventSentry has been recording all audit (and many other!) changes performed in Windows as part of the “Policy Changes” feature. It wasn’t however possible to see the current status of all audit categories and subcategories at a glance. Reviewing the current audit status of all monitored hosts can be important however, if only to verify that group policies are configured correctly.
In v3.4 we now have the new “Status” page available under “Compliance -> Audit Policy” which delivers information such as:
Compare/review audit settings of a particular sub category (e.g. “Registry”) among all monitored hosts
View all disabled audit settings across all or select hosts
(Re)view audit settings based on computer types (e.g. domain controllers, servers, workstations)
Our NetFlow component can now provide bandwidth visualization based on the collected NetFlow data. The information can either be accessed on the NetFlow page or as a dashboard tile. Even though bandwidth data can already be determined using SNMP, the data gathered by NetFlow should be preferred since it contains additional data not available via SNMP, such as:
Bytes per packet
Bytes per packet as well as packets sent received can be used to detect anomalies, e.g. when a host sends a large amount of network packets, or network packets with large/small content.
Today I have good news and bad news. You’d like to hear the good news first? OK. The bad news is that AutoAdministrator is being retired and will no longer be developed.
OK, now on to the good news. AutoAdministrator is of course not entirely history and is undergoing a similar transformation the NTToolkit did a few years back. AutoAdministrator is joining the “EventSentry” brand as the “EventSentry Admin Assistant”.
This brings the total number of software products under the EventSentry brand to three:
Almost everyone in IT has heard of VNC – which actually stands for “Virtual Network Computing”. The RFB (Remote Framebuffer) protocol which VNC relies on, was developed around 1998 by Olivetti & Oracle Research Labs. Olivetti (unlike Oracle) isn’t much known outside of Italy/Europe, and the ORL was ultimately closed in 2002 after being acquired by AT&T. But enough of the history.
When the need arises to remotely log into a (Windows) host on the network, Microsoft’s Remote Desktop application (which utilizes Microsoft’s RDP protocol – not RFB) is usually the default choice. And why wouldn’t it be? It’s built into Windows, there is no additional cost, and it’s usually quite efficient (=fast) – even over slower connections.
Remote Desktop has a few disadvantages though, especially when it comes to the IT help desk:
You cannot view the remote user’s current desktop
It’s not cross-platform
You can’t use RDP if it’s disabled or misconfigured
Especially when troubleshooting user problems, being able to see exactly what the user is doing is obviously very beneficial. VNC-based applications are a good alternative since they allow you to view the user’s desktop and subsequently interact with the user. This makes VNC viable for help desk as well as troubleshooting. Nevertheless, VNC-based solutions have their own shortcomings:
Free variations of VNC usually offer no deployment assistance
With over 10 variants available, finding the best VNC implementation is a daunting task
VNC is still deemed as somewhat insecure
VNC can be slow
We set out to solve these shortcomings by creating a number of scripts around UltraVNC that integrate with the EventSentry management console (although they’ll work well without EventSentry as well!). Using the QuickTools feature, you can then connect to a remote host via VNC with 2 clicks, even if the remote host doesn’t have VNC installed.
Important: The scripts only work in environments where you have administrative access to the remote hosts. The scripts need to copy files to the remote host’s administrative shares and control the remote VNC service.
Alternatively, you will also be able to start a VNC session by running the following command:
Even better, VNC can be automatically stopped and deactivated (until vnc_start.bat is run again) once the session is completed in order to reduce the attack surface.
As long as you have administrative access to the remote host(s), the script will remotely install VNC and even setup a firewall exclusion rule if necessary – although the UltraVNC installer takes care of this out of the box.
To reduce the attack surface of machines running VNC you can automatically stop the VNC service after you have disconnected from the remote host. Our connection script will automatically start the remote service again when you connect the next time.
For the utmost security you can also completely uninstall VNC when you are done, a script (vnc_uninstall.bat) is included for this purpose.
Even though VNC is generally not as fast as RDP, it’s usually sufficiently fast in LAN environments (especially for shorter trouble-shooting sessions) and the UltraVNC port which we’ll be covering in this post performs reasonably well even over slower WAN connections.
Integration with EventSentry
Monitoring workstations with EventSentry strengthens the capabilities of any IT helpdesk and IT support team with:
Software & Hardware Inventory
Access to process utilization and log consolidation
Enhanced security with security log & service monitoring
User console logon tracking
Pro-active troubleshooting with access to performance and other system health metrics
Remote desktop sharing is an additional benefit with the UltraVNC package which is included with the latest version of EventSentry (v126.96.36.199). Customizing the scripts and integrating them with EventSentry literally shouldn’t take more than 5 minutes, and once setup & configured will allow you to remotely control any monitored host with a couple of clicks. The scripts do not require EventSentry, but are included with the setup and integrate seamlessly into the EventSentry Management Console.
The EventSentry Management Console includes the “QuickTools” feature which allows you to link up to 8 commands to the context menu of a computer item. EventSentry ships with a few default QuickTools commands, for example to reboot a remote machine. Once configured, you simply right-click a computer icon in the EventSentry Management console and select one of the pre-configured applications from the QuickTools sub menu.
How does it work?
When you run the vnc_start.bat script, it will first check to see if UltraVNC is already installed on the remote host. If it is, it will skip the installation routine and bring up the local VNC viewer. If you configured the script to automatically stop the VNC service when not in use, it will start the service beforehand. When you disconnect, it will (optionally) stop the VNC service again so that VNC is not accessible remotely anymore.
If VNC is not installed, the script will remotely install & configure UltraVNC using psexec.
If you do not want to leave the UltraVNC service installed on the remote computer, the vnc_uninstall.bat script can be run when the remote session is done. Automatically stopping the remote VNC service is however sufficient in most cases.
The scripts need to be configured before they can be used in your environment, unless you are an EventSentry user, in which case you only need to download & install the prerequisites.
Super Quick Setup for EventSentry Users It’s no secret, we’re a little biased towards our EventSentry users, and as such setting this up with an existing EventSentry installation is rather easy:
Get psexec.exe and save it in C:\Program Files (x86)\EventSentry\resources.
Download the UltraVNC installers (they have 32-bit and 64-bit – download for the platforms you have on your network) and store them in the C:\Program Files (x86)\EventSentry\scripts\ultravnc folder.
Install UltraVNC on the computer where EventSentry is installed so that the VNC Viewer is available. It’s not necessary to install the whole package, only the viewer component is required.
If “VNC” is not listed in your QuickTools menu, then you will need to add it under Tools->Options->QuickTools. Simply enter “VNC” as the description and specify the path to the vnc_start utility, e.g. “C:\Program Files (x86)\EventSentry\scripts\ultravnc\vnc_start.bat $COMPUTER”. You can optionally check the “Hide” box to prevent the script output from being shown before you connect.
You’ll notice that no password was configured – that’s because you will be logging in with a Windows user and password – only allowing domain admins access by default. This can be configured in the authorized_acl.inf file, if you want to give additional groups and/or users access that are not domain admins.
That’s literally it – easy as pie. Even though we designed this thing to be easy peasy, since things do occasionally go wrong I recommend testing a first connection from the command line. Just open an administrative command prompt, navigate to C:\Program Files (x86)\EventSentry\scripts\ultravnc and type vnc_start somehost.
Now just right-click any host – or use the “Quicktools” button in the ribbon – and select the “VNC” menu option. Keep in mind that first-time connections will take longer since the VNC setup file has to be copied and installed on the remote computer. Subsequent connections should be faster.
Manual Normal-Speed Setup for Non-EventSentry Users
So you are not an EventSentry user but still want to utilize these awesome scripts? No problem – we won’t hold it against you. The setup is still easy – you’ll just need to customize a few variables in the variables.bat file.
In variables.bat, set the VNCSETUP_X86 and VNCSETUP_X64 to the setup file names you just downloaded.
Download the PSTools and extract psexec.exe into the working directory, or a directory of your choice.
In variables.bat, point the PSEXECFILE variable to the location where you just saved psexec.exe.
Optional: Edit the authorized_acl.inf file to specify which Windows group or user will have access to VNC. You can either change the first line, or add additional lines to give additional users and/or groups permission.
Install the respective version of UltraVNC on your workstation so that the VNC Viewer is available.
Open a command line window and navigate to the folder to which VNCSOURCE points to. Test the setup by running vnc_start hostname, replacing “hostname” with an actual host name of a remote host of course.
When presented with the login screen of the VncViewer, log in with a Windows domain admin user.
That wasn’t so bad now, was it? Just remember that you’ll need to initiate any VNC session with the vnc_start.bat file. Just launching the Viewer won’t work – even if VNC is already installed on the remote machine – since the VNC service is stopped by our scripts by default. To use the folder names we created, you’ll just run
Enjoy, and happy RFBing!
Configuration – variables.bat For the sake of completeness the variables.bat file is explained below:
VNCSETUP_X86: The file name of the 32-bit installer. This needs to only be changed whenever UltraVNC comes out with a new version. VNCSETUP_X64: The file name of the 64-bit installer. This needs to only be changed whenever UltraVNC comes out with a new version.
REMOTEINSTALLPATH: The directory where the script files will be copied to on the remote host.
VNCSOURCE: This is the folder where all the vnc-related files, including the setup executables, are located on the source host from where you initiate VNC connections – e.g. C:\Deployment\UltraVNC. VNCINSTALLDIR: The directory in which UltraVNC will be installed in (on the remote hosts).
VNCPASSWORD: This variable is not currently used since UltraVNC is automatically configured to authenticate against Windows, by default giving only Domain Admins access to VNC. This is generally more secure than using a password. You can edit the file authorized_acl.inf to give additional users and/or groups access to VNC. The file supports one ACL entry per line.
PSEXECFILE: Unfortunately we are not allowed to bundle the nifty psexec.exe file for license reasons, so you’ll have to download the PsTools and point this variable to wherever you end up copying the psexec.exe file to. If you already have psexec.exe installed then you can save yourself 2 minutes of time and just specify the path to the existing file here.
SET_VNC_SVC_TO_MANUAL: If you don’t entirely trust the security of VNC, maybe because you know what a brute force attack is, and you only want administrators to access VNC then you can set this variable to 1. As long as you only connect to the remote host(s) using the vnc_install.bat script, the scripts will ensure that the remote VNC service is started before you connect and stopped after you disconnect. Between the two of us, I’d always leave this set to 1 unless you have the desire to launch the VNC Viewer directly, or need non-administrators to be able to connect to the remote host(s).
ADD_FIREWALL_RULE: As the name (almost) implies, this will create a firewall exclusion rule on the remote host(s) if you’ve been doing your homework and enabled the Windows firewall. If you don’t like our boring firewall rule name then you can even change the name below by editing the FW_RULE_NAME variable. Enabling this is usually not necessary since the UltraVNC setup adds firewall exclusion rules by default.
VNCVIEWER: If you find that a different version of the VNC viewer works better than the version which we are shipping, then you can change the file name here.
In my previous post I talked about our new NetFlow component as well as the new agent management capabilities now available in EventSentry v3.3. In this post I’ll cover the remaining new features and improvements we’ve made in v3.3, starting with the web reports.
There are a number of new features and improvements in addition to NetFlow visualization. There are a few new dashboard tiles, including a “Recent Activity” tile which – as the name implies – shows recent relevant changes such as newly detected processes, software (un)installed, ping status or service status changes.
Anybody who works in a team of two or more Sysadmins should find the new notes feature incredibly helpful. It lets any web reports user add comments (=notes) which are subsequently visible to others. Notes can be associated with one or more hosts (ensuring they show up in the “Documentation” tab of the respective host status page) and can include documents as attachments as well! Do you have warranty documents or network diagrams you want to store in a central place – easily accessible? That’s what the notes are for.
The overall look and feel has also been refreshed, and we’ve reorganized the menu to make it faster to access dashboards and easier to find pages.
The visualization of data has been improved, since some chart types work better with certain features of EventSentry. You can now visualize grouped data using either pie charts, tree maps or column charts.
The security of the web reports has also improved with a lockout policy which will locking an account after too many unsuccessful logon events.
As mentioned in part 1, the EventSentry is agent is now available in 64-bit, making it possible to monitor 64-bit counters and easier to monitor files in 64-bit directories. For users upgrading from an earlier version, the EventSentry management console will automatically migrate any existing 32-bit agents on 64-bit versions of Windows.
Application & Services Event Logs
While monitoring Application & Services event logs, often referred to as “custom” event logs was possible, the way this needed to be configured in the management console was a common source of confusion. Some users also needed the ability to monitor more than 30 different logs. Consequently, monitoring additional event logs is now straightforward, and users can monitor as many event logs as they wish.
With thresholds, timers, schedules, insertion strings, EventSentry already offers a sophisticated engine for monitoring events in real time. New in this release is the ability to setup filter chaining. This makes it possible to trigger actions only when 2 or more events occur, and you can even link events together using insertion strings. Chaining is enabled on the package level, and every filter in a “chaining” package is automatically part of the filter chaining rules.
It happens frequently that we get alerts that require us to do additional research based on the information provided in the alert. For example, we may get an alert about an IP address for which we then need to do a reverse lookup or find the geoip location. Audit Success & Failure events from the security event log are another example, and often contain error codes and numbers which are not explained.
We set out to improve upon this, and starting with v3.3 EventSentry will annotate email alerts in a number of ways whenever possible:
IP addresses will include a reverse lookup
IP addresses will include a geoip location
Security events will have various error codes resolved
Please note that (1) and (2) are only supported for emails sent through the collector since it requires access to a local geoip database. (1) and (2) will need to be enabled in the email action “Options”, (3) is automatically enabled for all emails.
Insertion Strings & Regex
By making insertion strings from events accessible in filters and actions (e.g. through the $STR1, $STR2, … variables), it’s possible to create highly granular thresholds, customize emails, easily trigger corrective actions which utilize content from events and more. Based on our own requirements we took this capability a step further however, and you can now apply regex filters to events to define your own insertion strings. This is particularly useful for alerts which don’t use insertion strings or for events which contain log data. For those types of events, you can now parse parts of log strings and assign them to insertion strings. The previous blog article, Detecting Web Server Scan in Real-Time, shows a practical example of how to apply this new feature. It does require you to be a bit familiar with Regular Expressions, but the management console includes a handy dialog where you can test your regular expressions, shown below.
Faster is better! We’ve improved performance in a number of areas:
The database insert performance of the Syslog daemon has been improved for Microsoft SQL Server databases
The delimited log file feature now includes an additional index to increase database insert performance
The heartbeat agent now relies less on RPC-based agent status monitoring and can instead obtain the status of a remote agent either directly from the collector or the database, resulting in less network traffic and faster heartbeat monitoring cycles.
With new features & improvements in a variety of areas, this release should contain improvements for everyone. Remember that you can also submit feature requests here.