Endpoint Protection Solution Test Plan
This test plan is intended to lay out high-level guidelines for testing and comparing various endpoint protection and investigation solutions. It specifies test environments, connectivity method, scale and key endpoint protection functionality that must be validated for each solution.
Environment
Provide a testing environment representative of environments found in the wild.
Operating Systems - test capabilities of endpoint solutions on each of the following operating systems:
o Windows (XP-8)
o Mac (Use variants appropriate to customer environment.)
o Linux (Red Hat, Suse, Debian and/or other variants applicable to customer environment) o Unix (AIX, HP-UX, Solaris, BSD and/or other variants applicable to customer
environment)
Network Connectivity – test capabilities of endpoint solutions using both on and off LAN access: On-LAN (LAN connected devices)
Off-LAN / Remote (Roaming devices not connected to the LAN) Scale – test capabilities at scale of endpoints in orders of magnitude:
o 1 o 10 o 100 o 1,000 o 10,000
Test Cases
(Content Extracted from Endpoint Protection Competitive Matrix)
The following are high-level capabilities that are important for endpoint protection, investigation and malware identification. Identify/validate each of the following capabilities as they pertain to the respective products. Not all test cases apply to each operating system type or are operating system, scale or network connectivity specific.
Non-environment/scale specific tests: Market
o Data loss prevention
o Advanced malware protection o Endpoint configuration management o Enterprise forensics
o Application restriction o Vulnerability identification Management
o Role-based access control (RBAC) o Support for separation of duties o Dual factor authentication o Policy-based administration
o Centralized administration o Active directory integration
o Endpoint agent deployment functionality Endpoint Accessibility
o Cloud-based endpoint correlation ability
o Fully accessible and monitored anywhere an Internet connection is present o Partially accessible and monitored anywhere an Internet connection is present o On-premise accessible only
Threat Intelligence IOC Sources o IOCs from vendor subscription
o IOCs from network of devices in vendor from community o IOCs from open community feeds
o Manually created IOCs
Threat Intelligence YARA Rule Sources o YARA rules from vendor subscription
o YARA rules from network of devices in vendor from community o YARA rules from open community feeds
o Manually created YARA rules Integration
o Automatically detects endpoint threats autonomous from external device integration o Integrates with network devices for additional coverage and functionality
o Integrates with network devices as primary method of event triggering and endpoint action
o No integration Endpoint Agent Replacement
o HIPS/HIDS o Antivirus Environment/Scale-Specific Tests Platform o Windows o Mac o Linux o Unix o Android o IOS
Analysis, Logging, Alerting and Reporting o Timeline-based visualizations o Event correlation
o User activity correlation o Artifact correlation
o Threat spread based visualization o Text-based event / log entries
o Mobile device alerting capability (SMS, email, etc.) o Search alerts / activity per device
o Search alerts / activity per subnet
o Search alerts / activity per physical location o Search alerts / activity per external site o Search alerts / activity per destination country o Search alerts / activity per user
o Search alerts / activity per group
o Search alerts / activity per operating system o Search alerts / activity per application / process o Search alerts / activity per file
o Search / view behavior by resource utilization o Search / view behavior by least prevalence o Search / view behavior by network activity
o Search / view behavior by application / function activity o Search / view behavior by file activity
o Search / view behavior by device o Search / view behavior by subnet
o Search / view behavior by physical location o Search / view behavior by external site o Search / view behavior by destination country o Search / view behavior by user
o Search / view behavior by group
o Search / view behavior by operating system o Search / view behavior by application / process o Search / view behavior by file
o Search / view by Publisher / Company o Search / view by Trust Factor
o Search / view by signed / unsigned binaries o Search / view by Malware rating
o Search for IOCs o Search for YARA rules
o Investigator / user administration activity auditing o Custom activity reporting
o Export reports via pdf, csv, xml, etc.
o Report Scheduling with Exports (Email, Share, SCP, etc.) Endpoint Monitoring / Inspection Method
o Continuous / real-time activity monitoring o Malicious activity detection
o Unauthorized change detection o Unauthorized access detection o Scans triggered by external device(s) o Scheduled scans
o Manually triggered scans o Network share activity
Detection Methods / Triggers
o System modifications automatically identified
o System modifications identified by triggered scan process o System modifications identified by manual scan process o File retrospection
o Process retrospection o Connection retrospection o Network activity anomaly o Process activity anomaly o File anomaly
o Process execution o File access
o Registry access o Process change o File system change o Registry change
o Permission change / privilege escalation o Domain generation algorithm (DGA) o File trajectory
o Process trajectory o Connection trajectory o Device trajectory
o Changes to endpoint / heuristics / behavioral analysis o Cross-category threat searching
o Least Prevalence o IOCs
o YARA Rules
o Network connection correlation o Attack chain weaving
o Sandboxing / virtual execution o Exact content matching o Partial content matching o Fuzzy Hash
o SHA hash o MD5 hash
o Publisher / Company o Trust Factor
o Signed / Unsigned binaries o Malware rating
o Web script activity o Shell script activity o Blacklists
o Whitelists Operational Modes
o Full Remediation - reimage affected device to pristine state o Surgical Remediation - eliminate threats and associated artifacts
o Partial Remediation - kill processes
o Process Containment - prevent malicious process execution
o Full Network Containment - prevent network connections to only allowed locations o Partial Network Containment - prevent network connections to malicious sites o Audit Only - observe but not modify
Blocking Triggers / Methods o Application heuristics o File analysis
o Network heuristics
o Network destination reputation o Exact content matching
o Partial content matching o Device Control o File/process whitelists Fuzzy hash SHA hash MD5 hash Publisher / Company Trust Factor Signed Malware rating o File/process blacklists Fuzzy hash SHA hash MD5 hash Publisher / Company Unsigned Trust Factor Malware rating o Stop / kill process execution
o Stop network access to known malicious sites
o Stop network access by known malicious applications o Stop network access by suspicious activity
o Stop network access by network driver replacement Application Protection / Restriction
o Restrict application updates to legitimate sources o Restrict internet connections to known legitimate sites o Restrict application functions to known legitimate operations o Application sandboxing / micro-virtualization
Host Interrogation Method
o Memory analysis (physical examination) o File system analysis (physical examination) o Memory analysis (logical examination) o File system analysis (logical examination) o Network traffic collection / interception
o Windows API queries Endpoint Forensics
o Preview and remote analysis File system
Registry Memory Volatile data Network traffic data o Remote collection / acquisition
File system Registry Memory Volatile data Network traffic data o Connectors
Network shares Connected shares FileNet Server Google Drive
OpenTest ECM Server Sharepoint
Documentum Server DocuShare Server eVault
Exchange Server
Exchange Online / 365 Server Domino Server
o Correlation and scheduled events Comprehensive timeline analysis Periodic snapshots in time Event-based snapshots in time User correlation
Artifact correlation Volatile data collection
o Disk preview o Memory preview o Volatile data analysis
o Disk content searching and analysis o Memory content searching and analysis o Network content searching and analysis
