We believe that the only true test for security products is one that replicates the real world. While third party antivirus efficacy reports and sandboxed product demos are useful, they fail to take into account the rapidly changing tactics of attackers. Cylance’s goal is to empower System Administrators, IT specialists and engineers with the tools, knowledge and malware to test any vendor’s product in a safe and secure manner.
*Available in the U.S., expanding to other regions soon.
Test For Yourself is a framework that provides a safe testing environment for malware protection products. We feel that the only valid way to test this is using fresh malware samples in a safe but realistic scenario tailored to your environment.
When it comes to endpoint protection effectiveness, test for yourself. Download the book to learn how to set up your environment and test the products for real-world conditions.
You would not build a house without a plan, and you should not detonate malicious software without one either. These articles will help provide you with a blueprint to test antivirus products.
In order to capture the best results, you will want to use the best tools available. We provide you with information regarding tools that will allow you to analyze test results in a safe and effective manner.
Testing with old, well known malware is a surefire recipe for a false sense of security. Here we provide instructions to obtain fresh malware samples that represent the most current threats.
Malware variants exist that halt execution and evade detection by employing Virtual Machine (VM) evasion methods. However, those samples are becoming few and far between as more companies employ virtual environments in production – which means that malware creators can’t be confident that the VM that their software is evading is not actually their intended target.
There are many benefits to testing in a virtual environment. For example, a security analyst can quickly snapshot a VM and conduct tests with the confidence that any changes made are easily reverted by restoring to the VM state captured in that snapshot.
In practice, accurate results are a product of recreating production environments as accurately as possible. This can be achieved with software that will virtualize a physical machine, or you can create a fresh VM with a base OS image. Note: if a VM is created from an OS image, make sure to install all software that is in the base image to best mirror the production environment.
The VM should next be updated to include all recent security patches. Make sure to remove or disable any other AV product that you do not wish to evaluate. This step is critical in order to demonstrate the true ability of the product being tested. Network isolation should not be of concern during this portion of the set up as the VM should be clean and free of any malware.
Once the virtual environment has been established, install the AV of your choice for testing and ensure it has been updated with the policies you intend to run within your environment. Check to ensure the AV product is up to date and running the policy of choice. Virtualization software, such as VMware, should also be updated to the most recent version.
IMPORTANT: Isolating the VM from the physical device hosting it will ensure that if the malware infection is undetected, it is fully contained. Device isolation is accomplished via the VM’s settings and network configuration. Always check the VM settings to make sure that if any folders are shared, the permissions are set to Read-Only (RO). Also check to see whether the VM permits drag and drop and/or copy and pasting; make sure those features are disabled.
Typically, virtualization Frameworks provide three types of networking interfaces: network address translation (NAT), host-only or custom network and bridged.
It is strongly recommended to always test on a network that is separated from production. Host-Only or Custom Network interfaces are established if the hosting device is segmented from the production network and no Internet connectivity is necessary. Configure a NAT interface if Internet connectivity is required for testing.
For more complex and secure setups, a hybrid of the two interfaces can be used. For instance, the testing virtual machine (VM) and an OS Firewall VM such as PfSense could be placed on a Host-Only or Custom Network segment. Once this is established, configure the Firewall VM to have an additional interface attached to the NAT network. The Firewall VM should be configured to route AV test VM traffic from the Host-Only or Custom Network through to the NAT network, while strict firewall rules should be in place to ensure that traffic does not communicate with any of your physical devices.
Once networking has been set up, start the task manager within the VM and leave the process list visible during testing. A snapshot of the VM should be taken once all available updates have been applied. At this point, it is safe to introduce malware to the VM.
Malware can be introduced onto an endpoint via a number of different channels. This could be through the network, on infected USB drives or as malicious files stored in shared folders. Always zip and password-protect malware while transferring malicious files between devices.
Scan the malware folder with the AV product under evaluation once the malware is placed on the test virtual machine (VM). This should result in a static conviction of the files. It is important to keep in mind that not all files are statically malicious and thus require a second stage to ascertain their designed intent, which is initiated by executing the malicious file. Prior to executing the malware, the analyst should take another snapshot of the VM just in case a sample infects the VM undetected by the AV product. In this case, the VM can be reverted but work will not be lost.
Testing AV products can be performed in a safe and secure manner if the tester follows best practices. Testing in a VM that is isolated from the host device, as well as isolated from the physical network ensures that a security analyst can detonate malware safely and in a manner that yields the most accurate test results.
These test results allow security professionals to properly vet available solutions for their networks and devices, and it is therefore imperative that these tests reflect the environment that the tested products will eventually protect. It cannot be recommended enough that test environments accurately mimic your organization’s production environments.
Let's state the obvious, the "mal" in malware is an abbreviation of "malicious". We'll spare you the dictionary definition, but the short version of the story is that it is out to hurt your machine. It therefore stands to reason that you should be extremely – emphasis: extremely – careful while handling malware. Here are a few measures you can take
Mutating malware is the process of changing existing malicious software without significantly altering its functionality. This is often performed to change that piece of malware's hash (also known as the message digest). Mutation allows malware to evade signature-based antivirus (AV) solutions as these often rely heavily upon a collection of hashes in order to identify threats.
As a security professional, learning to mutate malware will allow you to better vet endpoint protection solutions as you can create unique malware – from a hash and signature perspective – for your tests.
Virtualizing your test environment is ideal – bordering on necessary – and provides several advantages.
Windows and Linux HostsVMware Workstation Pro
Apple OS X HostsParallels
Monitoring tools allow you to discover if your machine has been infected and gives you a glimpse into how the malware is affecting your test machine.
Windows monitoring toolsDiskMon is an application that logs and displays all hard disk activity on a Windows system.
Apple OS X monitoring tools
Activity Monitor: Apple’s description “shows the processes that are running on your Mac, so you can see how they affect your Mac's activity and performance”.
Terminal commandstop: Display and update sorted information about processes.
Network monitoring tools allow you to examine if and how malware communicates over a network, which includes the Internet. Most pieces of malicious code include some sort of network functionality, and this can give you a good sense of how a remote attacker communicates with infected machines
Suggested toolWireShark: is the world's foremost network protocol analyzer.
Compression tools will allow you to open archived files in order to parse the contents. Malware often includes several components packaged (compressed) into a single archive file. On Windows, .zip files are commonly found, but numerous other formats exist.
1. PESpin v1.33: PESpin is free and provides an easy to use graphical interface for mutating one sample at a time. Unfortunately, the download link is broken. Search for an alternate download source and compare the hash to verify file integrity.
2. Hyperion v1.2: As an innovative AV evasion technique, this crypter creates a weak key that isn’t saved, then brute forces it at runtime. To compile on Windows install MinGW, then choose the GNU C++ compiler package.
Cylance gathers an untold number of real-world malware samples every day for use in the Cylance Threat Testing Framework. These samples are refreshed every 24 hours to provide testers with the latest threats we’ve seen. However, testers DO NOT need to rely on our samples for testing. There are a number of other options to choose from. For those that would like Cylance to provide you with samples and a copy of CylancePROTECT® to test, please fill out this form and a Cylance expert will unlock access to our malware repository.
We are currently working on improving this experience. Please check back later.