Windows supports a code-signing feature called Authenticode, which allows a software publisher to digitally sign executable files (e.g. .exe, .msi, …) so that users can verify their autenticity. The digital signature of a file can be viewed in the file properties in Windows explorer on the “Digital Signature” tab.
Digital signature verification has been added to the checksum utility, which already calculates the checksum and entropy of a file. When using the new /s switch, checksum.exe will tell you whether:
the file is digitally signed
a counter signature exists
the digital signature is valid
the algorithm used (e.g. SHA 256)
who signed the file
who issued the certificate
when the file was signed
The utility also sets the ERRORLEVEL variable accordingly; if a signature check is requested with the /s switch but the file is unsigned, then checksum.exe will return %ERRORLEVEL% 2. Below is a sample output of the utility in action:
Digital signature verification will be added to EventSentry’s FIM monitoring component (“File Checksum Monitoring”) in the upcoming v3.4.3 release, which will give you the option to only get notified when unsigned files are changed, thus reducing overall noise.
You can download the latest version from here – enjoy!
Integrate with open source solutions (Graylog, ELK, Nagios Log Server & others)
and more. We’ve also been focusing on making the data EventSentry collects more actionable and subsequently more helpful, and as result you will see additional contextual data provided with some alerts & reports, and one new search page in EventSentry.
All in all, this upcoming release has a lot of improvements in store to help you do your job better by ensuring that your network is as reliable, secure and fast as possible.
While high-profile Ransomware attacks have slowed down somewhat in recent weeks, businesses – especially small businesses – are still hit with Ransomware infections every day. Even though EventSentry is not positioned as a AV or a AntiMalware software, it does include a variety of functionality which can detect Ransomware outbreaks.
What sets EventSentry apart from AV and most AntiMalware solutions is that it looks for pattern exhibited by the Ransomware – regardless of the variant. What’s new in version v3.4 is the ability to detect changes to the MBR and bootsector as well as the ability to calculate the entropy of (changed) files.
MBR/BootSector Monitoring & Backup
Some more recent Ransomware variants modify the MBR and/or boot sector and trigger a reboot to initiate an offline encryption process. EventSentry v3.4 can now monitor the MBR and detect changes in near real-time to alert the admin when suspicious activity is occurring.
By utilizing EventSentry’s advanced filtering engine it is also possible to potentially stop the encryption process, e.g. by hibernating the infected host. EventSentry even backs up the MBR and boot sector, making it possible to repair an infected system (with a boot disk) without having to perform a full restore from backup.
Entropy describes the randomness of a file, essentially a metric that can help detect compressed and encrypted files.
Since most Ransomware encrypts large amounts of files, EventSentry can utilize the entropy of a file, combined with event log thresholds, to make a determination that a Ransomware infection is in progress and take corrective measures.
Lateral Movement Detection with Thresholds
EventSentry has always included agent-side thresholds in order to throttle the alert volume or detect repeated events. Because these thresholds were executed on the agents, event patterns which involved more than one host could not be detected that way.
By adding a threshold component to collector – which analyzes and processes all events in real time – we can leverage this feature to new heights and detect network-wide event-based patterns – in real time!
Collector-side thresholds are configured exactly like agent-side thresholds with one the key difference – the threshold limit applies to all aggregated events sent to the collector. Collector-side thresholds also introduce the “Group By” setting that makes the lateral movement detection possible – the ability to count unique instances of an event property, and not just the total number of events.
Some of the event-based threat patterns you can detect:
The same user logging on to multiple hosts within a specific time frame
A process spreading (trickling) across multiple hosts within a specific time frame
A user running too many processes – either on a single or multiple hosts
Authentication failures of a user on too many hosts
Too many unique logon types used by a user account
Any event property and insertion string can be used to craft thresholds – the sky is the limit.
Software Version Checker
Earlier versions of EventSentry include a substantial hardware, software and patch inventory, making it extremely easy to find out which software packages are installed on your network, but also get alerted when software is installed or removed from a server/workstation.
In v3.4 we are taking this to the next level by providing the latest version available from the publisher for a growing list of 100+ software packages so that you can effortlessly identify outdated software on your network. You can now create reports listing any software on your network which is outdated, a dashboard tile is also available. The Web Reports refresh the version info list every 2 hours to ensure all reports are accurate.
If an application you are using on your network is not currently included then simply email us the name of the software as it is detected by EventSentry (and ideally the URL where we can obtain the current version) and we will add it to our list.
64-bit Web Reports for Windows
The EventSentry web reports are now available as a 64-bit application, and upgrading to v3.4 will automatically upgrade the existing v3.x 32-bit web reports to 64-bit on 64-bit when installed/upgraded on a 64-bit version of Windows. The new 64-bit web reports will allow you to run larger reports that would not run due to limits with the address space associated with 32-bit applications.
UPS & Battery Monitoring
Any UPS directly attached to a server/workstations that is detected by Windows can now be monitored by EventSentry. The status of the UPS will show up on the host inventory page, and alerts will be generated when a host is on battery power and back on AC power. EventSentry can also initiate a shutdown when the remaining run-time or charge level falls below a certain limit.
Batteries in laptops are also detected and listed on the host inventory page (battery capacity and current charge level), but generated alerts are informational only.
User Activity Tracking
While EventSentry provides its users with a wealth of information from multiple angles, it can be tedious to piece together data from multiple reports that is associated with a specific user. Data which can be linked to a user is scattered among process tracking, file access tracking, compliance logons and other pages.
The new “User Activity” page, which is located in the “Dashboard” menu, solves this problem by displaying data from the following pages on a single page:
Active Directory Changes
The user activity page makes seeing all activity by a user as easy as never before!
Integration with third-party log management solutions
A few months ago, one of our users approached with the need to integrate EventSentry into an existing log management system which was already in place at the location where EventSentry was to be deployed. While reviewing the request we recognized that even though we position EventSentry as a one-stop log management solution with a compelling and robust web-based reporting component, an integration with other products can be helpful in some cases.
Supplement EventSentry’s built-in reporting with additional reporting
Integrate EventSentry with an existing log management solution located in a different business unit
Integrate EventSentry’s sophisticated real-time agent and deployment infrastructure with a different log management back-end
In version 3.3 and earlier, EventSentry can be integrated with 3rd party products using the HTTP, process and Syslog action. The HTTP & process action are intended to be used with ticketing systems where only a low volume of alerts are submitted while the Syslog action obviously supports submitting a high volume of events. The Syslog format was however limited to the traditional RFC 3164 format, making an integration with other log management systems difficult.
Starting with version 3.4, EventSentry now supports the following formats in the Syslog action
RFC 3164 (legacy)
Nagios Log Server
Common Event Format (CEF)
If a log management server you need to integrate with is not listed above but supports the JSON format, then you can craft your own JSON packet with the JSON format, also introduced in v3.4.
Disk Space Alerts
Part of the effort to make EventSentry’s alerts more actionable is reflected in our improved disk space alerts which now list the 15 largest files and folders of the volume where disk space is low. The supplemental data will in many cases be enough to immediately identify the culprit so that corrective action can be taken immediately, without the need to run disk space analyzers on the volume.
Audit Policy Status
Since the introduction of the compliance tracking components, EventSentry has been recording all audit (and many other!) changes performed in Windows as part of the “Policy Changes” feature. It wasn’t however possible to see the current status of all audit categories and subcategories at a glance. Reviewing the current audit status of all monitored hosts can be important however, if only to verify that group policies are configured correctly.
In v3.4 we now have the new “Status” page available under “Compliance -> Audit Policy” which delivers information such as:
Compare/review audit settings of a particular sub category (e.g. “Registry”) among all monitored hosts
View all disabled audit settings across all or select hosts
(Re)view audit settings based on computer types (e.g. domain controllers, servers, workstations)
Our NetFlow component can now provide bandwidth visualization based on the collected NetFlow data. The information can either be accessed on the NetFlow page or as a dashboard tile. Even though bandwidth data can already be determined using SNMP, the data gathered by NetFlow should be preferred since it contains additional data not available via SNMP, such as:
Bytes per packet
Bytes per packet as well as packets sent received can be used to detect anomalies, e.g. when a host sends a large amount of network packets, or network packets with large/small content.
In my previous post I talked about our new NetFlow component as well as the new agent management capabilities now available in EventSentry v3.3. In this post I’ll cover the remaining new features and improvements we’ve made in v3.3, starting with the web reports.
There are a number of new features and improvements in addition to NetFlow visualization. There are a few new dashboard tiles, including a “Recent Activity” tile which – as the name implies – shows recent relevant changes such as newly detected processes, software (un)installed, ping status or service status changes.
Anybody who works in a team of two or more Sysadmins should find the new notes feature incredibly helpful. It lets any web reports user add comments (=notes) which are subsequently visible to others. Notes can be associated with one or more hosts (ensuring they show up in the “Documentation” tab of the respective host status page) and can include documents as attachments as well! Do you have warranty documents or network diagrams you want to store in a central place – easily accessible? That’s what the notes are for.
The overall look and feel has also been refreshed, and we’ve reorganized the menu to make it faster to access dashboards and easier to find pages.
The visualization of data has been improved, since some chart types work better with certain features of EventSentry. You can now visualize grouped data using either pie charts, tree maps or column charts.
The security of the web reports has also improved with a lockout policy which will locking an account after too many unsuccessful logon events.
As mentioned in part 1, the EventSentry is agent is now available in 64-bit, making it possible to monitor 64-bit counters and easier to monitor files in 64-bit directories. For users upgrading from an earlier version, the EventSentry management console will automatically migrate any existing 32-bit agents on 64-bit versions of Windows.
Application & Services Event Logs
While monitoring Application & Services event logs, often referred to as “custom” event logs was possible, the way this needed to be configured in the management console was a common source of confusion. Some users also needed the ability to monitor more than 30 different logs. Consequently, monitoring additional event logs is now straightforward, and users can monitor as many event logs as they wish.
With thresholds, timers, schedules, insertion strings, EventSentry already offers a sophisticated engine for monitoring events in real time. New in this release is the ability to setup filter chaining. This makes it possible to trigger actions only when 2 or more events occur, and you can even link events together using insertion strings. Chaining is enabled on the package level, and every filter in a “chaining” package is automatically part of the filter chaining rules.
It happens frequently that we get alerts that require us to do additional research based on the information provided in the alert. For example, we may get an alert about an IP address for which we then need to do a reverse lookup or find the geoip location. Audit Success & Failure events from the security event log are another example, and often contain error codes and numbers which are not explained.
We set out to improve upon this, and starting with v3.3 EventSentry will annotate email alerts in a number of ways whenever possible:
IP addresses will include a reverse lookup
IP addresses will include a geoip location
Security events will have various error codes resolved
Please note that (1) and (2) are only supported for emails sent through the collector since it requires access to a local geoip database. (1) and (2) will need to be enabled in the email action “Options”, (3) is automatically enabled for all emails.
Insertion Strings & Regex
By making insertion strings from events accessible in filters and actions (e.g. through the $STR1, $STR2, … variables), it’s possible to create highly granular thresholds, customize emails, easily trigger corrective actions which utilize content from events and more. Based on our own requirements we took this capability a step further however, and you can now apply regex filters to events to define your own insertion strings. This is particularly useful for alerts which don’t use insertion strings or for events which contain log data. For those types of events, you can now parse parts of log strings and assign them to insertion strings. The previous blog article, Detecting Web Server Scan in Real-Time, shows a practical example of how to apply this new feature. It does require you to be a bit familiar with Regular Expressions, but the management console includes a handy dialog where you can test your regular expressions, shown below.
Faster is better! We’ve improved performance in a number of areas:
The database insert performance of the Syslog daemon has been improved for Microsoft SQL Server databases
The delimited log file feature now includes an additional index to increase database insert performance
The heartbeat agent now relies less on RPC-based agent status monitoring and can instead obtain the status of a remote agent either directly from the collector or the database, resulting in less network traffic and faster heartbeat monitoring cycles.
With new features & improvements in a variety of areas, this release should contain improvements for everyone. Remember that you can also submit feature requests here.
We are very excited to release EventSentry v3.3, a major update to our award-winning monitoring solution EventSentry, less than 10 months after the release of the previous major version 3.2. Version 3.2 included the collector component which supports secure and reliable communication with remote agents as well as better database throughput, switch port mapping and many improvements to the web reports.
I’d like to also thank everyone who took the time to fill out our annual survey – we read every single response in detail. If you haven’t taken it yet then you can still do so here.
The v3.3 release, which builds upon some of the architectural changes we have made in v3.2, and offers new functionality to help you:
Visualize, measure & investigate network traffic better with the new NetFlow component – with discounted introductory pricing until 12/31 2016!
Spend less time managing agents – the collector can now push configuration as well as agent updates automagically – think laptops!
Deployment via MSI is much easier – MSI file creation now only takes a few seconds
Investigate issues faster with email alerts which have geo location, reverse lookups as well complex security codes included inline
Visualize any data in the web reports more easily with additional dashboard tiles and treemaps throughout
Managing and using custom event logs is now more straightforward and scalable
Database throughput has been improved for Syslog data and delimited log files
Even more advanced filtering is possible with filter chaining and insertion string override via regular expressions
Communicating and documenting your network has just become a lot easier – add notes and/or upload documents in the web reports
Monitor 64-bit operating systems with a native 64-bit agent
With a brand new component and many new features in a variety of areas, v3.3 will have something of interest for everyone. Let’s dive in and look at the new features in more detail.
NetFlow NetFlow is a new component which is part of the “Network Services” service (along with Syslog, SNMP, ARP) and is licensed separately. Pricing is very competitive and an additional introductory discount will be available until the end of this year, 12/31 – including competitive upgrades. You can request a quote here.
Collecting NetFlow data allows you to see all traffic meta data which passes through network devices that support NetFlow, including:
Source IP, destination IP
Source host, destination IP (when resolvable)
Source port, destination port
Geo location (when available)
IP protocol used
Amount of traffic sent and received
Number of packets transmitted
EventSentry v3.3 currently supports the NetFlow v1, v5, v9 as well as sFlow flow protocols. NetFlow is usually supported by most commercial routers and firewalls whereas sFlow is most commonly supported by switches. NetFlow is generally preferable over sFlow – especially for forensic analysis since sFlow samples traffic and only sends every nth flow. sFlow can be preferable when dealing with large amounts of data, but EventSentry’s NetFlow implementation (as well as NetFlow itself) has a way to group flows and therefor condense traffic.
Do you need NetFlow, and is it worth looking into? Without NetFlow there is impossible to know which hosts communicate with each other (unless you capture network traffic). What traffic enters the network, and what traffic leaves it? Broadly speaking, implementing NetFlow lets you:
Visualize all network traffic in a variety of ways and reports
Analyze network data for forensic purposes
Utilize network traffic data for troubleshooting purposes
Map network traffic to geo location
Correlate network traffic with Active Directory users (requires workstation monitoring)
Measure bandwidth utilization
On the EventSentry side, setting up NetFlow should take less than 5 minutes; and setting it up on the network device side is generally just a matter of enabling NetFlow and pointing it to EventSentry.
EventSentry ships with the GeoLite geo database from MaxMind which does a good job of associating IP addresses with physical locations down to the city level. If you are looking for more accuracy however, then you can also purchase the full geo location database from MaxMind here.
Active Directory User Correlation
A unique feature of EventSentry’s NetFlow implementation is the ability to correlate workstation logins with network traffic, making it possible to associate network traffic with individual users. This requires that workstations are monitored with EventSentry and works best when users have a dedicated workstation.
Agent Management & Deployment
If you are utilizing the collector service then you have now a great time-saving feature available. Pushing a configuration update to remote hosts after you made a change or deploying agent updates after a patch installation are a thing of the past once you activate the respective options in the collector dialog.
Managing automatic configuration updates can be done in 2 ways: Either by automatically deploying a configuration update after you click “save”, or by deploying only approved configuration updates (recommended). If you select the latter, then you just have to click the new “Save & Deploy” sub-option on the ribbon and the collector will do the rest. It’s no longer necessary that the EventSentry agent is directly reachable from the management console; it will receive the latest configuration as soon as it connects to the collector.
Please note that you will still need to manually deploy a v3.3 agent once in order for automatic agent updates to work, since the self-update code is embedded in the new agents.
Creating MSI files has also been greatly simplified – a x86 and x64 agent MSI file is created with just a few mouse button clicks. Manually editing MSI files with tools like ORCA is a thing of the past. The only prerequisite is the (free) WiX Toolset which has to be installed only once.
In addition to saving most EventSentry users a lot of time, these new deployment features also make it possible to monitor laptops which aren’t permanently connected to the network. Simply deploy the agent MSI file with your favorite deployment tool (or deploy with the management console) and enable the configuration and agent management options in the collector. From that point on, any agent connecting to the collector will automatically receive the latest configuration AND any new agent updates – completely automatically – no matter where in the world they are located.
EventSentry v3.3 now ships with both a x86 and x64 agent, so that 64-bit editions of Windows can be monitored natively. The key benefit of this change is that 64-bit only performance counters can now be monitored, these counters were off limit with 32-bit agents. Utilizing 64-bit agents also results in the following changes:
Agents will be automatically converted to 64-bit when v3.3 is deployed. It is not possible to use a 32-bit v3.3 agent on a 64-bit version of Windows
File system redirection via “Sysnative” or in the File Checksum Monitoring packages is no longer necessary
Memory consumption will be slightly higher compared with 32-bit agents
Please note that EventSentry has not completely migrated to 64-bit yet, some components (management console, heartbeat agent, web reports) are still shipped as 32 bit executables. We plan on migrating all components to 64-bit by the end of 2017.
There are just too many new features in v3.3 to fit them all into one blog post, so stay tuned for part 2 which will follow shortly.
Our latest patch for EventSentry v3.2 (v18.104.22.168) requires some additional information in addition to the release notes.
Heartbeat Monitoring (Agent Status) By default, the EventSentry Heartbeat Monitor ensures that all remote agents are running by querying the status of the remote “EventSentry” service. While is an accurate way to ensure the remote agent is running, the Microsoft RPC mechanism isn’t very efficient when connecting to remote hosts across a slow (WAN) link, and concurrently checking the service status of 100+ hosts at the same time can on occasion also cause issues. In these situations, the heartbeat agent may not be able to monitor all hosts in the configured monitoring interval. Furthermore, querying the remote status of a service requires that the EventSentry Heartbeat Agent run under a domain account, otherwise the dreaded “Access Denied” error appears on the heartbeat status page in the web reports.
To address these issues for larger EventSentry deployments (500+ hosts) and deployments where the remote agents are connected through a slower WAN link, we have added the ability to query the remote agent status through the EventSentry database where the remote agents periodically check in. This check is enabled by default for new installations, but existing installations will need to make a database permission change in order to give the heartbeat agent permission to query the agent status. More information can be found here.
In the next release of EventSentry (v3.3), this functionality will be configurable, and the heartbeat agent will also be able to determine the current agent status by communicating directly with the collector service (when enabled) for even better accuracy. The Heartbeat Monitor will always attempt to revert back to the legacy method of checking the service status directly if it cannot obtain the status through other means.
Service Monitoring: Configuration Changes EventSentry distinguishes between three types of service changes: Status changes (e.g. Running to Stopped), service configuration changes (e.g. changes to the startup type) and services being added or removed. Up until release 22.214.171.124, all status changes and service configuration changes were logged with the same event severity, which we didn’t think was very fitting since the status change of a service is very different to a change of the service itself. As such, starting with 126.96.36.199, only service status changes will be logged under the severity configured under “Monitor Service Status Changes” category. All other service changes will be logged under the severity configured under “Monitor Service Addition / Removal” category.
Management Console: Quicktools
The EventSentry QuickTools allow you to run an application/script against a server or workstation in your EventSentry configuration. EventSentry includes a few default QuickTools entries, such as “Reboot”, “Remote Desktop” and others. Starting with the latest release we added a new “Hide” option, which will not show the executed application on the desktop. This will be useful for integrating our upcoming VNC wrapper scripts (Blog article coming soon), which will allow you to install & launch a (Tiger)VNC client directly from the EventSentry management console.
EventSentry Light 3.2 Starting with this release, EventSentry Light v3.2 will also be available. We have good news for all EventSentry Light users: We have increased the number of full hosts you can remotely manage to 5, and also increased the number of network devices you can monitor to 5. As such you can now monitor up to 10 hosts with EventSentry Light completely for free.