Category Archives: Tools & Utilities

Managing Windows Services & Service Credentials

Every Windows server runs a seemingly ever increasing number of services which range from built-in services providing core Windows functionality (e.g. Print Spooler, Bitlocker, WMI) to 3rd party services added when installing 3rd party software (e.g. various software update services, MySQL) – all of which run in the context of a specific user account.

For example, Windows Server 2012 includes more than 300 services, about half of which are automatically running (this particular server has SQL Server installed as well):

Services on Windows Server 2012 grouped by user

Services on Windows Server 2012 grouped by user

That user account is either a built-in security principal of Windows (e.g. NetworkService), a user account specifically created for that service, or another user account from the server or domain.

Common Practices
Services should always run under a user account which has the least amount of privileges necessary to do its job. It’s common, and often tempting, to run a service an administrative account like “Administrator”. While this often the easiest way to “get it working”, it’s also the least secure.

When a service runs under the “Administrator” account – especially if it’s the domain Administrator account – the service has almost unrestricted access to all resources on the host or, in case of a domain admin, on the domain. This is not something a service usually needs nor you want. It also means that the service will stop working whenever the password of the Administrator account is changed (the service will continue to work until it is restarted).

Less is Better
Whenever possible, try to use one of the built-in security principals available in Windows to run a service under, or create a specific user account for the service. For example, if you have a file synchronization app which runs as a service, create a “ServiceFileSync” or similar account and configure the service to run under that account. Carefully examine the rights the service requires, and only assign those privileges to the user account which the service actually needs.

When creating the user account, give it a very strong & complex password. Users won’t have to log on with that user account, so the password can be complex and long. You can optionally check the “password does not expire” option if you feel that the password is sufficiently secure and you have a short password expiration policy on your domain which could interfere with the service starting after the password expired.

In domain environments I also recommend giving those user accounts (since you will most likely end up with more than one) either a common prefix or suffix (e.g. svc_mysql) and/or moving the accounts into a specific OU. This makes managing and distinguishing these accounts easier – especially in teams with more than one SysAdmin.

The quick way: Local Services grouped by User Account

Sample output from srvsec

Sample output from srvsec

To view all locally installed services grouped by the user account they are running under, download the EventSentry SysAdmin Tools and just run srvsec.exe. This will show you all locally running services, and group the output by the user account they are running under. Srvsec can also be pointed at a remote host, and can also change the passwords stored in services. Click here for more information on srvsec.

Srvsec is a great tool to quickly see what’s going on a single host, but to manage services on an entire domain effectively a more scalable solution is available: EventSentry + AutoAdministrator – the dynamic duo!

The right way: Making sense of ALL installed services
Even when passwords for service accounts are sufficiently strong, they should still be changed on a regular basis. But which services are installed where and are using which service account?

If this is your first time examining service accounts on your network, you should first identify which services run under which user accounts. EventSentry’s service monitoring feature combined with the web-based reporting really makes this a breeze. Assuming that you have a service monitoring system health package assigned to all of your servers, you can simply open the web reports and navigate to Status – Services and get a birds-eye view of all installed services.

In the Overview view, all installed services are grouped by common attributes, including startup type (automatic startup services vs manual startup services), current status, service name and, most importantly for this post, the service user account.

Service overview of all services installed in a domain / forest.

Overview of all installed services in a domain.

Click the “Show All” link to see all user accounts, or click on a specific user account (e.g. “LocalSystem”) to filter the list and only show services running under this specific user account. In most cases you will want to click on “Detailed” to see a list of all services with more detail.

In addition to filtering and viewing details, you can also click on the header of the

All user accounts used by services

All service user accounts

username (or any other) column to see a chart depicting all user accounts used by services from all monitored servers and workstations.

Any report viewed in the web reports can also be scheduled with a job, e.g. a list of all user accounts used by services could be emailed daily/weekly. Simply click the “Save as Report” link to create a report and setup a job.

Managing Services
The standard way to configure the user account and password used by a service is through the “Services” application in Windows. This works well for one or two servers, but not when you need to update the password for a service on multiple hosts.

Managing services with AutoAdministrator

Managing services with AutoAdministrator

This is where AutoAdministrator comes in: A free graphical tool which lets you do just that (and quite a bit more): Update the username and/or password of a service on multiple servers in a domain or work-group. Since AutoAdministrator is multi-threaded, even tasks affecting a large amount of hosts usually only take a few seconds.

To update the stored password of a service, open AutoAdministrator and select “Services” from the drop-down list on the top left.

Service Key Name

Service Key Name

Next, select the service you wish to update from the “Service key / display name” drop-down. If the service is not listed, simply specify the service key name in the service field. The key name is the internal name used by the service and can be obtained by double-clicking a service name in the “Services” MMC application in Windows.

Updating service credentials

Updating service credentials

Next, click on the “Set logon” tab and specify the new username and/or password. Of course you can also specify other service actions, such as restarting the service or changing the start-up type.

As the next step, select the hosts you wish to apply the selected changes to. You can select hosts from Active Directory, EventSentry, custom groups or work groups (Microsoft Windows Network).

Once the correct hosts are selected, click the “Start” button. The number of hosts which will be affected by any action is always shown on the bottom right of the application.

How the EventSentry SysAdmin Tools Focus on File System Maintenance

EventSentry SysAdmin ToolsOver the past couple months, we’ve taken time to go through the various EventSentry SysAdmin Tools, one by one, and show you how they can benefit your environment in powerful ways. We’ve talked about the security tools, the networking tools, and the “check” monitoring utilities. As you know, the SysAdmin Tools offer a set of graphical and command-line utilities designed to help you with your daily administrative tasks. These tools are always being honed to provide simple yet powerful functionality.

This month, let’s take a look at the extremely beneficial file-system utilities: ADSList, CheckSum, DirMon, DirectorySize, FileReplace, PurgeTemp, and SuperDel. Here’s what they can do.

ADSList
ADSList scans a folder structure to find any alternate data streams (aka “hidden” data streams). Alternate date streams are a feature of the NTFS file system in which you can hide payload (additional files) inside existing files. The jury is still out about whether malware uses these streams, but it’s always a good idea to make sure nobody has hidden something malicious in alternate data streams, because the Windows Explorer and directory listings don’t show them.

ADSList lists any alternate data streams that are associated with a file. When the tool finds an alternate data stream, it displays the name of the stream along with the regular file the stream is associated with. The output will also show a summary that lists the number of files analyzed, the number of files that have an alternate data stream associated with them, the number of alternate data streams that have been found, and the elapsed time.

The main purpose of ADSList is to give you a command-line utility that can be run/scheduled on a regular basis to reveal any hidden streams on a server or workstation. The /s option lets you include subdirectories.

CheckSum
CheckSum generates a one-way checksum (error detection scheme) of a file with a configurable algorithm and displays it onscreen. This capability is useful for ensuring the integrity of a file and making sure that it hasn’t been modified. CheckSum not only supports the SHA set of cryptographic hash functions (e.g., SHA256, SHA512), but also less secure hash functions (e.g., MD5).

To display and create a file’s checksum, simply supply the filename as the first argument. Keep in mind that generating checksums of large files (e.g., greater than 100Mb) can take a significant amount of time and CPU time.

The CheckSum utility is also included in EventSentry as an add-on to the File Monitoring feature, which can automatically generate SHA checksums and detect file modifications based on checksum changes.

DirMon
Directory Monitor (DirMon) is a useful troubleshooting tool that monitors a directory (and optionally subdirectories) and displays all file changes in real-time. You simply run it on the command line, and it displays any file activity occurring on a given folder (or subfolder).

DirMon will show you when files are added, deleted, or modified. DirMon also lets you specifically include or exclude filters, so you can skip files that you aren’t interested in or show only files that you are interested in. The /I (/includefiles) option includes only files that match a wildcard filter, and the /e (/exclude) option does the opposite. The /s (/subdirectories) option includes subdirectories.

DirectorySize
The DirectorySize (dirsize.exe) utility calculates the current size of a directory, including subdirectories, and displays it onscreen. The output shows the number of files and directories searched, and the total size in physical (actual size taken up on the disk) and logical (actual file size) bytes.

DirectorySize will process the current directory if you pass no command-line arguments.

PurgeTemp
PurgeTemp is a new and exciting tool that lets you purge files that are older than a certain number of days. The tool traverses the %TEMP% directory (or a manually specified directory) and deletes files that have not been modified in 120 days (by default). Because it scans the temp folder by default, you can incorporate PurgeTemp into a login script or run it with Task Scheduler to clean up temp files, for example. It’s a great way to keep users’ temp folders small.

You can customize and configure all of PurgeTemp’s parameters, including /t (time in days) and /p (path). When called without arguments, PurgeTemp simply shows the configured temp directory, the number of files in the directory, and their cumulative size.

SuperDelete
SuperDelete (superdel.exe) essentially deletes all instances of a specific file. It parses a directory (including subdirectories) and deletes multiple occurrences of one file.

Suppose you have a thumbs.db file that Windows Explorer creates in every folder containing images, and you want to remove that from every folder on a drive. You can use SuperDelete for that purpose, using the <directory> variable to specify the directory to search (subdirectories are included), and the <fileToDelete> variable to find all occurrences of a file in the directory (wildcards are supported).

FileReplace
FileReplace is a command-line utility that parses a directory (including subdirectories) and replaces multiple occurrences of one template file with a template file of the same name.

Suppose you have 50 instances of various myfile.txt files scattered on your computer. You can quickly replace them all with a new myfile.txt file.

Another useful example is this: You have file C:\WebSite\Default\index.html and want to replace all other index.html files in the directory D:\WWW (including subdirectories) with C:\WebSite\Default\index.html. FileReplace lets you accomplish that with one command.

Streamline Your File System!

This is just another taste of the free, constantly evolving tools available in EventSentry SysAdmin Tools. Give them a try—they’re all free and will help you manage your IT infrastructure more effectively.

The Network Monitoring Focus of the EventSentry SysAdmin Tools

EventSentry SysAdmin ToolsWe’ve already talked about the security-focused and “check” monitoring utilities included in the freeware EventSentry SysAdmin Tools, part of the larger EventSentry network-management solution. The SysAdmin Tools offer a set of graphical and command-line utilities designed to help you with your daily administrative tasks. These tools are always being honed to provide simple yet powerful functionality.

Now let’s take a look at the extremely beneficial network monitoring utilities: Fping, Gethttp, IPMon+, Ntpclient, Pagesnpp, and WakeOnLan. Here’s what they can do.

Fast Ping (Fping)
NETIKUS.NET introduced Fast Ping (fping.exe) years ago as part of the NTToolkit. The tool was developed as a way to offer a faster way to ping remote hosts. Frankly, we were annoyed by the built-in Windows ping, which is far slower than its Linux and Apple OS X counterparts. That’s right, on non-Windows OSs pinging a remote host (especially one that is online) is a lightning-fast prospect—so why not on Windows? Fping solves the problem.

Fping also offers some fun options. For example, you can use the Solaris-style syntax, which shows you only whether a host is up or down. You can check a TCP port instead of doing an Internet Control Message Protocol (ICMP)-based ping. You can play a sound on successful or failed ping—a more useful capability than you might think! You can also see silly comments, and you can save your presets—something you can’t do with any other ping utility.

fast ping utility screenshot

fping.exe in action (fast ping utility)

The parameters of this command-line utility are straightforward: The required <host> variable identifies the host name or IP address to ping; the /brief (/b) parameter performs a quick ping and only indicates whether the host is up or down; the /count (/c) parameter determines the number of packets to send; the /defaultset (/w) parameter sets the current options as the default; the /comment (/u) parameter shows unhelpful comments when performing a brief ping; the /playok (/p) and /playfailure (/f) configure sounds; the /loop (/l) parameter pings indefinitely, allowing an abort with Control + C.

As part of the SysAdmin Tools, this utility is better than it’s ever been. In addition to those fun options, it is a fully customizable tool. You can control the number of packets, the packet size, sound, display mode, and the delay. You can even set your preferences and store them as the default. To check the TCP port, simply append a colon and the port number to the host name (e.g. fping www.eventsentry.com:80).

GetHTTP
GetHTTP (gethttp.exe) is a simple command-line utility to download files from a website through the HTTP protocol. Mostly useful for scripts, it supports HTTPS and proxy servers and shows the progress of the download in the command-line window. If you’re familiar with Curl (curl.exe), you have an idea what Get HTTP does.

GetHTTP in action

GetHTTP in action

The parameters of this command-line utility are straightforward: The /usewininet parameter utilizes the Windows proxy engine; the /proxyport parameter determines the IP port of the proxy server; the /proxyhost parameter determines the host name or IP address of the proxy server; the /quiet (/q) parameter specifies quiet output; among others, include username and password authentication parameters.

IPMon+
An excellent troubleshooting utility, IPMon+ is a GUI tool that shows all TCP, UDP, ICMP, and ARP connection endpoints between the local computer (default) and remote hosts. It’s the graphical version of IPMon, offering functionality that isn’t available in the command-line version.

IPMon+IPMon+ is terrific for troubleshooting network connections and revealing incoming and outgoing network traffic for those situations where you don’t need to see every packet detail. The tool monitors all network traffic on the specified interface and shows which hosts communicate with the local host, how much data is being transferred through the IP connection, the direction of traffic, and which UDP/TCP ports are used in the communication. If IPMon+ runs in promiscuous mode, traffic from non-local hosts is also displayed. IPMon+ and IPMon both require the free WinPcap.

NTP Client
A simple but essential tool, NTP Client (ntpclient.exe) checks the local time against an NTP server, and optionally updates the local time to match that of the server. NTP Client supports the Network Time Protocol (NTP) up to version 3 and takes network latency into consideration when setting the local time. (Note that NTP Client doesn’t run as a service, and as such will have to be called repeatedly if you want to keep the time of a computer synchronized.)

NTPClient in action

NTPClient in action

Network latency is taken into consideration when calculating the clock offset, providing precision down to milliseconds. The primary parameter of this command-line utility does all the work: The /set (/s) parameter sets the time according to the time retrieved from the NTP server.

PageSNPP
PageSNPP (pagesnpp.exe) sends a message to a pager using the internet-based Simple Network Paging Protocol (SNPP). The tool has a message limit of 1500 characters, but you can check with your paging provider to determine the maximum supported message length for your plan and device (usually less than 500). PageSNPP returns an %ERRORLEVEL% of 0 when the message was sent successfully, and an %ERRORLEVEL% greater than 0 when the message could not be sent.

The primary parameters of this command-line utility do all the work: The <SNPP_HOST> variable identifies the host name or IP address of the SNPP host, the <SNPP_PORT> variable identifies the ICP port used, and the <MESSAGE> variable displays the message to send, enclosed in quotes. (The maximum is 2,048 characters.)

WakeOnLan
The WakeOnLAN (WOL) utility sends a “magic” packet to a remote network interface card (NIC), based on the MAC address. If the NIC supports the Wake On LAN feature (and the feature is enabled in the computer BIOS of the computer), the computer will power on automatically after receiving the packet. You can also send the magic packet to a router, if the router supports direct broadcasts.

wakeonlan

The primary parameters of this command-line utility do all the work: The required <MAC Address> variable identifies the MAC address without delimiters, and the /IP Address (/ip) parameter identifies the IP address to send the packet to (usually a router) if the remote host is not in the local subnet.

More to Come!
This is just a taste of the free, constantly evolving tools available in EventSentry SysAdmin Tools. Give them a try—you won’t be able to stop with just one.

The “Check” Monitoring Utilities of the EventSentry SysAdmin Tools

EventSentry SysAdmin ToolsLast week, we talked about the security-focused utilities of the freeware EventSentry SysAdmin Tools, part of the larger EventSentry network-management solution. The SysAdmin Tools offer a set of graphical and command-line utilities designed to help you with your daily administrative tasks. These tools are constantly under development, always being honed to provide simple yet powerful functionality. Three of these tools are vital monitoring utilities: CheckDB, CheckTCP, and CheckURL. Here’s what they can do.

CheckDB verifies a database connection through the Open Database Connectivity (ODBC) interface. With this capability, you can not only verify that a database server is up and running, you can also check that a database is online. checkdbYou can optionally run a SQL statement of your choice. CheckDB is particularly useful because it doesn’t merely verify that a database server is online (e.g. through a port check), it also verifies that a SQL statement was successful. That capability improves the usability of this tool because it verifies that the SQL server is accepting logins and is working correctly (at least as far as that statement is concerned). Also, this tool will work with any database that supplies ODBC drivers, so it will work with MySQL, MSSQL, and so on. You can schedule CheckDB from within EventSentry (“Application Scheduler”), and even time it. The scheduling capability is a bit advanced, and the setup requires a few steps, but after getting it up and running, you can easily schedule a statement and configure it to notify you if it takes more than two seconds, for example. The parameters of this command-line utility are straightforward: The <DSN/Connectionstring> parameter is the DSN or connection strong to connect to; the /q (or /query) parameter is the SQL query you can run upon successful connection; the /u (or /username) parameter is the DSN unsername to connect as; and the /p (or /password) parameter is the password for “username.” CheckDB can log output either to the console or to the event log, making it easy to receive alerts from the utility through EventSentry or any other log monitoring software. The /I (/logToLog) and /c (/logToConsole) parameters take care of this functionality.

CheckTCP is another command-line application, this one letting you quickly determine whether a TCP port on a host is open. Additionally, you can receive initial data sent from the remote host through an open TCP connection, such as when connecting to most SMTP hosts. CheckTCP exists because Windows doesn’t really offer a built-in way to check whether a TCP port is open. Yes, Nmap is a powerful utility, but you probably often just want to know whether a server that you rebooted is available for remote desktop login. checktcpFor that, you can simply run “checktcp server123 3389.” It’s not fancy, but it accomplishes a vital task. If you use the /s switch, you can get only the first line of the response. For example, if you use it against a mail server, you would get this:

checktcp /s mymailserver 25
Data: 220 mx.somedomain.com Microsoft ESMTP MAIL Service ready at Fri, 25 Apr 2014 15:07:33 -0500

The parameters of this command-line utility are straightforward: The /s parameter, as mentioned, gets initial data from the remote port (for example, when connecting to an SMTP port); the <Port> parameter displays the TCP port to connect to; and <Hostname> identifies the IP address of hostname to connect to. Although you can use this utility to display any data sent by the remote host over the established connection, CheckTCP is not intended to be used as a port scanner.

CheckURL is the HTTP version of CheckDB, and it lets you detect changes in web pages (through checksums) and look for text inside web pages. With CheckURL you’ll know when a web page changes or when a particular string is or isn’t included in a page. You might use this tool to monitor your corporate pages (at least those which are static and don’t have dynamic content) and also development pages to ensure that they don’t return a HTTP error. This is beneficial because you can have CheckURL look for specific text on the pages. The checksum feature is cool, too, because it lets you know when a page changes. As with with CheckDB, you can schedule CheckURL from within EventSentry (“Application Scheduler”), and even time it.checkurl The scheduling capability is a bit advanced, and the setup requires a few steps, but after getting it up and running, you can easily schedule a statement and configure it to notify you if it takes more than two seconds, for example. At  NETIKUS.NET, we monitor our online store that way. If the store takes more than three seconds to load, we get an alert. Like CheckDB, CheckURL can log output either to the console or to the event log, making it easy to receive alerts from the utility through EventSentry or any other log monitoring software. CheckURL supports SSL as well as proxy servers.

More to Come!
This is just a taste of the free, constantly evolving tools available in EventSentry SysAdmin Tools. Give them a try—you won’t be able to stop with just one.

The Essential Security Tools of the EventSentry SysAdmin Tools

toolsPart of our larger EventSentry network-management solution, the freeware EventSentry SysAdmin Tools offer a set of graphical and command-line utilities designed to help you with your daily administrative tasks. These tools are constantly under development, always being honed to provide simple yet powerful functionality. Three of these tools are vital security utilities: Password Assistant, Service Secure, and Task Secure. Let’s take a look at what they offer.

Password Assistant
Password Assistant is a simple yet powerful tool that lets you update the passwords of user accounts on multiple Windows machines. You simply enter the username, the old password, and the new password (with confirmation); after doing so, you can select the computer name(s) from a network neighborhood list (with a filter option) or choose the computer(s) from a text file. The update process can also be logged to a text file.
All the parampassword_assistanteters of the command-line utility are provided in clear terms: The /u (or /username) parameter specifies the username whose password needs to be changes, the /pwold parameter takes that account’s old password, and the /pwnew parameter accepts the new password; there are also /n (or /network), /f (or /filepath), and /filter parameters for specifying computers, as mentioned above. The /ignore_rest parameter ignores the rest of the labeled arguments following this flag, and the /version parameter displays version information and exits.
A great sample use of Password Assistant is when you need to update the administrator passwords on all of the workstations in your environment. Password Assistant provides a one-stop interface—or a simple command-line utility—for making the task hassle-free.

ServiceSecure
Service Secure provides a simple command interface that displays all of your system’s services, grouped by service account. The tool also lets you easily reset service passwords by specifying username and password rather than having to manually configure those services through the Microsoft Management Console (MMC).srvsec_1
All the parameters of the command-line utility are provided in clear terms: The /p (or /password) parameter sets a password; the /c (or /changepwd) parameter changes a password; the /r (or /restart) parameter restarts the service after the password has been changed; the /u (or /username) parameter lists only those services running under a certain username; the /ignore_rest parameter ignores the rest of the labeled arguments following this flag; the /version parameter displays version information and exits.
Suppose service security has been compromised, and you need to quickly change the passwords of a user account used by a number of services. That task is no longer a logistical nightmare: You can now simply use ServiceSecure in a batch file and update all affected services in your entire network in a matter of moments.

TaskSecure
Task Secure provides a simple command interface that displays all of your system’s scheduled tasks, grouped by task account. The tool also lets you easily manage the passwords stored in scheduled tasks on your network. Simply specify the username and password, and Task Secure will reset the password stored in all scheduled tasks (using the specified username) on the specified computer (local or remote).

TaskSecureAll the parameters of the command-line utility are provided in clear terms: The /r (or /remote_host) parameter lists all the scheduled tasks on a given host; the /u (or /username) parameter lists only those scheduled tasks running under a certain username; the /p (or /password) parameter sets a password for every scheduled task configured for a certain user account; the /ignore_rest parameter ignores the rest of the labeled arguments following this flag; the /version parameter displays version information and exits.
Suppose you need to quickly change the passwords of all the scheduled tasks used by one user. You can now simply use Task Secure in a batch file and update all scheduled tasks in your entire network in a matter of moments.

More to Come!
This is just a taste of the free, constantly evolving tools available in EventSentry SysAdmin Tools. Give them a try—you won’t be able to stop with just one.

Automatically Restarting a Failed Windows Process

Whether it’s a critical process running on a server or an application on your desktop – sometimes processes terminate and need to be restarted – immediately.

With EventSentry & EventSentry Light you can do just that: Automatically restart processes immediately after they terminate.

In the past, one drawback of EventSentry launching a process was the side effect that any process started by the EventSentry agent would run under the same account as the EventSentry agent itself (usually a privileged domain account or LocalSystem).

In this post I’ll discuss how you can work around that limitation in a secure manner using a scheduled task. When the critical process fails, instead of launching the process directly through a process action, EventSentry will trigger a scheduled task instead. Why? Because scheduled tasks allow you to configure under which user a task will run – and the user’s password is securely stored in Windows.

The recipe for accomplishing this feat is as follows:

  • Process Monitoring monitors the process
  • An event log filter looks for the “failed process” event and triggers a process action
  • The process action starts a scheduled task

Let’s look at this in detail. First, on the host where the critical but unstable task is running, you create a schedule task in the Windows “Task Scheduler”. Under General, give the task a descriptive name (“Start Super Important App”) and change the user under which the program should be running under. In most cases you will also want to make sure that you configure the task to run whether a user is logged on or not. Then, under “Actions”, add a new action “Start a program” which points to the executable that should be launched. After you click “OK” you will be prompted for the password for the user.

Scheduled Task

Creating a scheduled task

The next step is to setup process monitoring in EventSentry. Right-click “System Health” and create a new package and assign it to the computer(s) in question. Right-click the newly added package and select “Add – Processes”. Click the newly added object and add the name of the process which should be monitored. You can configure how many instances of the processes are required, and with which severity the event will be logged when the process is inactive.

process monitoring

Configuring process monitoring

Now we create a new “Process” action. Right-click the “Actions” container, select “Add” and enter a descriptive name (e.g. “Trigger Super Important App”). In the Filename field specify:

%SYSTEMROOT%\system32\schtasks.exe

And for the Command Line Arguments enter:

/Run /TN “Start Super Important App”

This uses the built-in Microsoft utility schtasks.exe to run the task we created in our first step. At this point EventSentry will monitor the specified process and log an event if the process is inactive. And while we do have an action to trigger the scheduled task, we still need to tell EventSentry when to launch that action.

EventSentry Process Action

Configuring a process action to start a scheduled task

For the next step, right-click the “Event Logs” container, select “Add Package” and give that package a descriptive name. Then assign the package to the same host. Right-click the newly added package and add a filter by clicking “Add Filter”. In the filter dialog, add the “ Trigger Super Important App” action to the action list and configure the following fields:

Event Log Include Filter Rule

Setting up a rule to trigger the process action

Event Severity: Information
Event Log: Application
Event Source: EventSentry
Category: Process Monitoring
Event ID: 10401
Content Filter (wildcard): *critical_app.exe*

Important Notes: The event severity will need to match whichever severity you selected when adding the process monitoring object in the system health package. The content filter can also be configured to match insertion string #1, in which case the wildcards are not necessary.

And that’s all there’s to it, simply save the configuration when you are done. If the process is running on a remote host then don’t forget to push the configuration to that host.

EventSentry Mobile v1.3 for iOS Available!

We’re excited to announce EventSentry Mobile v1.3 for the iOS platform. The new version remains free, is optimized for iOS 7, sports a refreshed interface and also includes a brand-new feature for pinging hosts in real-time.

Since the new version of EventSentry Mobile was optimized for the new web reports in EventSentry v3.0, it will not work if you are running EventSentry v2.93 on your network.

The interface of the app remains largely the same, but integrates more seamlessly with the iOS 7 flat look. Icons on the network status dialog are now only shown when a host is at least in a warning state, and the icons on the heartbeat dialog have been changed to new flat icons.

EventSentry Mobile Network Status

Network status dialog

Viewing computer details has been tweaked for hosts which are monitored via SNMP since there is currently less information available when compared with Windows hosts monitored by the EventSentry agent.

But since we didn’t feel that visual and internal changes were enough, we also added a new “Tools” section to the app which can:

  • Ping a host
  • Perform a DNS lookup
  • Provide GEO lookup data (when available)

The tools dialog has a single input field with one button where you can enter either an IP address or host name. After you enter the host name or IP address and click the target button, the web reports will ping the host, perform a DNS (reverse) lookup and try to obtain GEO information.

EventSentry Mobile Tools

Ping, DNS & GEO lookup tools in EventSentry Mobile app

The ping and DNS lookup are performed from the web reports rather than from the phone directly, GEO data is pulled by the iOS device from http://freegeoip.net/. If the IP address is not private (e.g. 192.168.1.x) and coordinates are available then the location can be viewed on a map directly on the iOS device as shown below.

Location of IP address after GEO lookup of www.unvienna.org

Location of IP after GEO lookup of www.unvienna.org

Reminder: If you are using EventSentry v2.93, iOS 7 and have automatic updates enabled on your iOS device, then EventSentry Mobile will stop working.

If you have an iPhone or iPad and haven’t already installed EventSentry Mobile then simply search for “EventSentry” in the App store or click here. Enjoy!

CryptoLocker Defense for Sysadmins

It seems as if CryptoLocker has been making the rounds lately, much to the dismay of users who don’t have working backups of their precious office documents.

While I admire Cryptolocker’s simplicity and effectiveness from a purely technical and entrepreneurial standpoint, what the software is doing does appears to be illegal in most countries and so I’d like to offer some advise on how to tame the beast. If you’re looking for a 5-minute fix then I have bad news: implementing the CryptoLocker defense I have outlined below, while completely free, will take a little more than 5 minutes to implement. But knowing that you have an effective defense against CryptoLocker may very well be worth it. After all, CryptoLocker seems to find its way into a lot private networks these days.

CryptoLocker Screenshot

The ideas set forth in this blog post apply mostly to Windows-networks with file servers, but could be adapted for individual computers as well (though this is not covered here – let me know if you’d like me to include this scenario).

About CryptoLocker
For those who have not heard of CryptoLocker yet, it is a piece of software which encrypts pretty much all common office-type documents, including Microsoft Office, AutoCAD, PDFs, images and more. This blog article from MalwareBytes has a complete list of extensions. Once encrypted, CryptoLocker charges you to decrypt (your own files) again. It’s public key cryptography gone wrong; I wonder if Diffie & Hellman saw this one coming. And to make the whole spiel even more interesting, you only get a limited amount of time to pay before your files will remain encrypted. Forever. Oh – and the longer the wait, the more you have to pay. And with recent bit coin exchange rates in excess of USD 1000, the amount that needs to be paid can be uncomfortably high.

It is pretty difficult to defend against something like CryptoLocker other than through usual means of AntiSpyware software, user eduction and strict policies against opening and downloading files from the Internet, email attachments and such. In most cases CryptoLocker comes in form of a ZIP attachment disguised with a PDF icon.

One reason CryptoLocker is so effective – yet difficult to block – is because it exhibits the same behavior as users would: It “simply” accesses and modifies files like a user would. And infecting a machine isn’t all that difficult since CryptoLocker doesn’t require any elevated permissions to run. On the contrary, it wants to run in the same context the user does, so that it can access and see the same files a user does. As such, security features like UAC are utterly useless against ransomware like CryptoLocker – it’s a whole new type of software.

Backups
The most effective defense against CryptoLocker is to have a working, tested backup. Let me repeat this: A WORKING and TESTED backup. Users have lost all their data because they thought that they had a backup in place when their backup was broken in some way.

We’ve seen posts of users who deleted all the files CryptoLocker encrypted, thinking they had a working backup. They had a backup, but it was apparently not recently tested and as a result the user lost all of their data.

Naturally, CryptoLocker does not like backups. It dislikes them so much that when CryptoLocker runs, it even tries to delete any Windows Shadow Copy backups. Cloud backup services (including Dropbox, Skydrive and Google Drive etc.) which keep versions of your files offer some protection, but restoring older versions of your files may be a tedious process.

The Defense
The most obvious defense against CryptoLocker is AntiSpyware software, e.g. MalwareBytes. Most AntiSpyware & AntiVirus software still uses signatures however, so new versions of the ransom ware often remain undetected at least for a few days.

So instead of detecting CryptoLocker itself, we can sniff its tracks so to speak. CryptoLocker’s predictable behavior can be used against it. CryptoLocker’s objective is of course to encrypt and hold hostage as many files as possible, so to increase the likelihood of the user purchasing the decryption key from the thugs.

And it is that very pattern that we will try to exploit and use as a trigger to detect and take corrective measures. The approach consists of measuring how many files are being changed in a certain time interval, and if a certain threshold is being exceeded (say more than 10 files modified in 1 minute) we assume that CryptoLocker found its way into our castle. Even though users modify their documents on a regular basis, users can usually make only so many changes at a time and most likely at a much slower rate than any sort of script / software would.

Another approach would be to create one or more honeypot or canary files, which we know (or hope) a user would not modify. If a checksum change in one of those files were detected, we could (more or less) safely assume that CryptoLocker was on one of his rampages again and take corrective measures. The honeypot file would have to be modifiable by users (otherwise CryptoLocker would also not be able to modify it), which makes accidental modifications by users possible (although somewhat unlikely).

This 2nd approach isn’t quite as solid in my opinion, since CryptoLocker is most certainly adapting to changes, and may skip files that it may suspect are a trap. For example, it could skip small files or skip directories with a very small number of files and so forth.

A more sophisticated approach, where we detect an unusually large number of files changes in a small time period, is going to be harder to circumvent by CryptoLocker. The good news is that we have a free (it’s really free, not a trial) software tool available which can do just that. It can:

  • detect file changes
  • measure the rate of file changes (through event log alerts)
  • stop/start services or launch processes
  • send out alerts

EventSentry (Light) to the rescue
EventSentry Light is the free version of our full-spectrum monitoring & compliance solution EventSentry. The features we can utilize to come up with a defense are:

  • File Checksum Changes (part of System Health Monitoring)
  • Filter Thresholds (part of Event Log Monitoring)
  • Action (control services, send out emails)

File Checksum Monitoring
Monitors any folder and detects file size changes, checksum changes and file additions and deletions. EventSentry Light will log file checksum changes to the event log (it’s big brother can also log them to a database), which in turn is monitored by the real-time event log monitoring component.

Event Log Monitoring & Thresholds
This component supports a variety of sophisticated features, one of which are thresholds. The thresholds feature lets you essentially detect event log entries that occur at a certain pace. For example, if 10 specific events occur in 1 minute then let me know and/or take corrective action.

Service Action, Email Action
EventSentry supports a variety of action types to be triggered when an event occurs, with email usually being the most commonly used one. You can also control services, use REST APIs, launch processes and much more. We’ll use the former to stop the file sharing services (LanmanServer) when we have determined that CryptoLocker is on the loose.

I will go into step-by-step instructions on how to configure EventSentry at the end of the post.

The Baseline
The most difficult thing to determine is the maximum rate of file changes we deem normal, as we need to have a baseline in order to configure the threshold slightly above that. This number will vary from network to network, with file servers serving lots of users obviously requiring a larger threshold. I’d like to repeat that determining the right threshold is very important. If it is too low, then normal user activity will trigger an alarm; if it is too high then the alarm may never be triggered and CryptoLocker won’t be caught in time.

The best approach is to setup file monitoring and let it do its job for 1-2 days to determine a baseline. Once the baseline is established, we can increase it by a certain factor (say 1.5) and use that as the threshold.

Setting up the trap requires 3 steps. In this case we assume that EventSentry is either installed directly on the file server, or an agent is deployed on the file server (in which case you will need to make sure that configuration updates are pushed to the file server(s) in question).

Step 1: Monitoring the directory/ies
In EventSentry, right-click the system health packages and add a new package. Right-click the package, select “assign” and assign it to all file servers. Right-click the package again and add a “File Monitoring” package. Click the new object. Directories are monitored in real time by default, but EventSentry requires a recurring scan as well – in case Windows doesn’t send real time notifications. This is usually a good thing, but when you are monitoring large directories it’s best to set the interval very high (future versions will allow for this to be unchecked).

File Checksum Monitoring Settings

In the package, add all the folders which should be monitored and only check the “checksum change” check box. Do not check any of the other check boxes in the bottom left section at this time. Since we haven’t established a baseline yet, we’ll set the severity of the event log alerts to “Information”. If the monitored folders contain a lot of non-Office files then it may be a good idea to adjust monitoring so that only office files (e.g. .doc, .xls, etc.) are monitored. If you prefer to monitor all files, simply change the setting to the green PLUS icon and make sure the list of exclusions is empty (or specifies files that should be excluded, e.g. *.tmp). Below is a screenshot of how this can be configured.

File Checksum Monitoring Settings

When you save the configuration, EventSentry will enumerate all files in the folder and create an initial checksum for every file. The agent will log event 12215 when the scan starts, and event 12216 when the scan is complete. When that happens, EventSentry is essentially “armed” and will detect, and log, all checksum changes to any of the files in the monitored directories.

At this point we’ll want to let this run for at least 24 hours during a “normal” work day, as to determine how many file changes occur on average. You are going to be at a bit of an advantage if you are running the full or the trial version with database support, as it will be a lot easier to determine the number of file changes occurring through the web-based reporting.

Step 2: Setting up the trap
Now that we have established a baseline, we’re ready to setup a threshold. This time we’ll create a new event log monitoring package. Right-click “Event Log Packages” and add a new package and call it “CryptoLocker Rules”. Like before, assign it to the file servers we are monitoring. Right-click the package again and add a new event log filter. Configure the filter as shown in the screenshot below. Note that we are triggering an email action for now. The content filter can be used to restrict the filter further, e.g. to only match certain directories if you are monitoring several directories with EventSentry.

Event Log Filter Setup

Now things are getting interesting. The goal is to create an error event in the event log when X amount of file checksum changes occur in a given time period. To get there, we’ll start with the “General” tab where we tell the filter what type of event we are interested in (see below). Once that event is defined, we’ll move on to the “Threshold” tab which is where we specify the threshold parameters. For the purpose of an example, let’s assume that we have established a baseline of 100 file checksum changes per day, with a work day starting at 8am and ending at 7pm. Assuming that activity is somewhat spread throughout the day, this amounts to about 9 file changes per hour. Naturally we’ll have to assume that file changes aren’t always evenly spread out throughout the day, but setting up a “if 20 checksum changes occur in 1 minute shut file sharing down” is probably a reasonable threshold. Configure the threshold as shown in the screenshot below, with whichever threshold you came up with.

Event Log Filter Threshold Setup

Step 3: Triggering corrective action
When our threshold is reached, EventSentry will log an error to the event log with event id 10601 and trigger the specified action(s) from the “General” tab (Default Email) one time per threshold interval.

At this point we would merely receive an alert when we suspect that CryptoLocker is at it again. If you are cautious then you can retain this setup for a little while (e.g. a day or two) to ensure that you are not getting any alerts about the threshold being met (assuming that CryptoLocker is not active on your network in which case you should get the emails).

To go all in and trigger a server service shutdown, we’ll need to create a service action now. On Windows, file sharing services are provided by the “Server” service, which uses the internal name of “LanmanServer”. The service action allows you to control any service (start/stop/restart), and in this case we’ll obviously want to stop the server service, so that clients cannot access the file shares on your server anymore. We’ll trigger an email action at the same time of course, so that the sysadmin in charge is aware of what is going on. While shutting down all file services seems a bit extreme, it’s unfortunately the most effective way to prevent more files from becoming encrypted.

So for the next step, right-click the “Actions” container and select “Add Action”. At the selection dialog choose the “Service” action, enter a descriptive name (e.g. “Stop File Sharing”) and hit enter.

Selecting an EventSentry Notification

Then, configure the settings of the service as shown in the screenshot below.

Action to stop the LanmanServer service

The last step of our setup (congratulations if you’ve made it that far) is to assign the service action to the filter we previously created. After all, a service action which isn’t referenced anywhere doesn’t do much good. So head back to the Event Log Packages, find the “CryptoLocker Rules” package and edit the filter in the package. In the action list on top, click the “Add” button and add the action you just created.

Testing
If at all possible I’d recommend testing the EventSentry setup at a time when your users are not interrupted. Adding a few template files to one of the monitored folders and changing them in short succession (a script may be necessary depending on how short your threshold interval is) should trigger the file services shutdown procedure. Once verified, you can just start the “Server” service again.

Conclusion
Just like in the real world, network viruses come in all shapes and sizes – only limited by technology and the imagination of the cyber-evildoers.

I hope that this article gave you some insight into CryptoLocker and a good way to guard against it. As always, make sure that your company has the following in place:

  • Email Attachment scanning
  • Working, tested backups
  • User education
  • AntiSpyware software

With those in place, one should be able to keep future infections to a minimum.

Stay safe & decrypted.

Mobile Alerts: Pushing EventSentry alerts directly to your mobile devices with Prowl and NMA

When it comes to mobile alerts, email seems to still be the prevalent method of choice for many IT pros. There are many good reasons why network alerts delivered via email are convenient:

  • easy to configure
  • uses existing infrastructure
  • every smartphone and tablet supports email
  • supports attachments (e.g. performance charts from EventSentry’s performance alerts)
  • integrates into your existing environment – everybody already uses email!

What’s not to like? Well, of course it turns out that some of these advantages can also be a disadvantage:

  • emails are not real time
  • problems on the email server often don’t surface immediately
  • important alerts can be overlooked in the inbox jungle
  • you cannot be alerted about email problems via email (duh!)

Viable Email Alternatives
Thankfully, there are a number of alternatives that can be used as an email substitute or addition for mobile alerts. In this article I’ll focus on two affordable services: Prowl (for the iOS platform) and Notify My Android (you may have guessed it – for the Android platform) – subsequently referred to as “NMA”. Both of these services consist of apps for their respective platform and a web-based back end which will push the notifications to your device(s) in near real time.

Mobile Alert on iPhone 5 with Prowl

EventSentry event sent to Prowl on an iPhone

Both services offer an HTTP API which we can connect to with EventSentry’s HTTP action. If you have never used the HTTP action in EventSentry, then here is some background: the HTTP action allows you to POST any event (whether it be an event from the security event log or a heartbeat alert for example) to either Prowl’s or NMA’s web service. These notifications are then pushed to one or more mobile devices.

Neither service currently requires a monthly subscription, but both require a purchase (Prowl costs USD 2.99 whereas NMA costs USD 4.99) if you want to send unlimited notifications to your mobile device(s). NMA is a little more welcoming to strangers – it supports up to 5 notifications per day at no charge.

Prowl: Getting Started
The
iOS app costs USD 2.99, and supports up to 1000 API calls (=notifications) per hour. To get started:

  1. Purchase the Prowl: Growl Client from the Apple App Store and install it
  2. Register for free at prowlapp.com
  3. Login & create an API key

Notify My Android (NMA): Getting Started
The Notify My Android app is free in the Google Play store and supports up to 5 API calls (=notifications) per day  for free. Upgrading to a premium account will allow up to 800 API calls per hour. To get started:

  1. Download the Notify My Android app from the Google Play store
  2. Register for free at www.notifymyandroid.com
  3. Login & create an API key
  4. Optional: Upgrade to a Premium account to allow for unlimited notifications

Once you have the mobile app installed and the API key in hand, you can start setting up a HTTP action in EventSentry. Notifications essentially consist of three fields: The application name, a subject as well as a message field, all of which can be customized. As such, it’s up to you configure which part of an event log alert you will put in the subject, and which part you will put into the actual message. In our example, we generate a dynamic notification subject with the host name, event id, event source and event category. The notification body will simply consist of the event message text, though this can be customized as well.

Setting up a HTTP action
Right-click the “Actions” container (or, in v3.0, use the “Add” button in the Ribbon) and create a new HTTP action. The HTTP action requires a URL at minimum, optional credentials and the actual data fields to submit in the HTTP POST. Conveniently, both Prowl & NMA use the same field names. I suspect that they are adhering to some sort of standard, though I couldn’t find any references.

The first parameter to configure is the URL, which depends on the service you use:

Prowl: https://api.prowlapp.com/publicapi/add
NMA: https://www.notifymyandroid.com/publicapi/notify

Configuring authentication credentials is not necessary since you are essentially authenticating with the API key you generated. The last step is configuring the form fields. The bold field on the left is the name of the form element and the text to the right the value. A description follows in italic on the next line:

apikey: abdef123123abababafefefe
The API key you received

application: EventSentry
This will displayed as part of the notification. I use “EventSentry” here, but this can really be anything, so it could be the host name as well for example ($EVENTCOMPUTER)
Max Length: 256

event: $EVENTCOMPUTER-$EVENTID-$EVENTSOURCE-$EVENTCATEGORY
This is the subject of the notification. You can use any variables, including insertion strings $STR1, $STR2 etc.
Max Length: 1024 [Prowl], 1000 [NMA]

description: $EVENTMESSAGE
This is the main message body of the notification, the $EVENTMESSAGE seems like a pretty good candidate for this field
Max Length: 10000

priority: 0 (possible values are -2, -1, 0, 1 and 2)
This field is optional and doesn’t do anything with NMA. With Prowl however, a priority can be set to “2” (indicating an “emergency”) which may then override quiet hours on the mobile app (if quiet hours are configured in the Prowl mobile app).

Take a look at the respective API documentation for Prowl and NMA as well; each show additional fields which can be configured, as well as additional information that you may or may not find useful.

The screenshot below shows a fully configured HTTP action for Prowl:

EventSentry HTTP Action for Prowl

EventSentry HTTP Action for Prowl

The HTTP action for NMA would look identical, with the exception of the URL which is different. You can click the “Test” button, which will submit the configured data to the specified URL and should, when configured correctly, immediately generate a notification on the mobile device. Please note that event variables will not be resolved when testing.

Threshold
To make sure that your mobile device doesn’t get flooded with alerts (some applications have the tendency to generate not one but hundreds of events in a short period of time), I highly recommend that you setup a threshold on the action or the event log filter referencing the action (I personally prefer the former). You can also setup a schedule so that notifications are only sent on certain days and/or during certain hours.

The last step would be to configure an event log filter to forward select events to the mobile device, something that is beyond the scope of this article. See the tutorials below for more information:

Reliability
It seems only natural to wonder whether alerts sent through these services can be used for mission critical systems. I’ve been using mostly Prowl as I’m an iPhone user, and have been very happy with it’s fast response times (which are almost instant), the service reliability and the stability of the iOS app. Nevertheless, both prowlapp.com and NMA state that you should not solely rely on their system for critical alerts and instead setup multiple channels for mission critical alerts. This sounds nice in theory, but suspect that most sysadmins will not want to dismiss alerts on more than one device – something that can get old if you get a fair number of alerts. Switching to a commercial system like PagerDuty with guaranteed up-time may be preferable in that case. I will talk more about PagerDuty in an upcoming post soon.

My experience with the NMA Android app wasn’t as good during the limited testing I performed. While it worked great when it worked, the app did crash on me a couple of times.

Conclusion
If you’re looking for a way to push alerts to your mobile device from EventSentry without using email and without spending big bucks, Prowl and NMA are worth looking into. They’re affordable, responsive and easy to configure.

EventSentry Light Supercharged

The latest EventSentry update brings significant and very exciting changes to the free light edition, EventSentry Light.

We have always seen EventSentry Light as a successor to the original EventwatchNT – a monitoring tool for small networks to alert sysadmins by sending real-time alerts about event log activity. As its big brother EventSentry continued to mature, most features from EventSentry made it into the light edition as well: Service Monitoring, Disk Space Monitoring, Performance Monitoring and many more. Since EventSentry Light was, and continues to be, free, it needed to distinguish itself from the commercial edition. As such, most feature were somewhat restricted in the light edition which only allowed a limited number of packages, event log filters, performance counters and such.

es_light.png

Over the last year we’ve been getting feedback that EventSentry Light was being constrained too much. Since our goal is, and always has been, to empower sysadmins and not constrain them, we decided to provide our users with more functionality in the free edition. The result of these efforts is build 2.93.1.75, and with this release you can:

  • Monitor event logs and log files in real time, setting up as many filter rules as you’d like, without restrictions.
  • Utilize all advanced event log filter capabilities like thresholds, timers, schedules and more!
  • Create as many event log, log file and system health packages as you like.
  • Utilize all system health monitoring features, such as file checksum monitoring, performance monitoring, service monitoring and more.
  • Create a variety of alerts using mail, HTTP, SNMP traps, Syslog messages and more.
  • Receive SNMP traps from SNMP v3 enabled devices.
  • Monitor up to 2 full hosts and 2 network devices.

Pretty impressive, no? So literally overnight, EventSentry Light has matured into a full-fledged monitoring solution which will alert IT professionals like sysadmins of critical (event) log events, performance issues and much more. What differences with EventSentry remain? A few, but the line is much more clear now:

  1. Reporting. With EventSentry, you get log consolidation, software/hardware inventory, performance trend reports, network dashboards, jobs, JSON/XML/CSV/… APIs and more.
  2. Compliance. EventSentry includes a variety of compliance functionality such as process tracking, logon tracking, account management tracking and more.
  3. Monitor multiple hosts. Monitor as many hosts as you are licensed for, and also utilize command line utilities to automate remote host management.
  4. Support. EventSentry includes quality email and phone support, something we pride ourselves on. EventSentry Light offers forum support.
  5. Mobile iPhone & Android apps are only available in the full edition since they require reporting.

So if you’re not already using EventSentry Light, or using an older version, then you should give it a try. It’s as free as it gets with no registration required, no advertisements and no nagging pop-ups. We hope you like it as much as we do. And did I mention that you can seamlessly upgrade from EventSentry Light to EventSentry? :-)

Don’t forget to check out our other free tools and Facebook / Google+ pages!
Best,
Ingmar.