Today’s modern security systems generate thousands or hundreds of alerts each day for their staff to review, causing teams to face the following challenges:
1. Alert Fatigue
2. Missed Critical Incidents
3. Analyst Time Wasted on Low-Risk Alerts
The answer is not to have fewer alerts; rather, the answer lies in better prioritization. The real results will be:
1. Quicker Response on Real Breaches
2. Less Downtime for the Business
3. More Resources for Security Staff to Utilize
Each Alert Type and Associated Risk Level can be classified as follows:
1. Critical Threats:
a. Malware Execution
b. Privilege Escalation
c. Ransomware Deployment
2. High-Risk Indicators:
a. Suspicious Logins from Anomalous Locations
b. Failed MFA Attempts
c. Data Exfiltration Attempts
3. Medium-Risk Alerts:
a. Patterned Failed Login Attempts
b. Uncommon Software Installation
c. Policy Violations
4. Low-Risk or Informational:
a. User Logins on New Devices
b. Minor Configuration Changes
c. Scanning for Protocols.
With this information and understanding of each type of alert, security analysts will be able to separate the noise from the threat.
Real-World Example
A mid-level company utilizing an EDR (Endpoint Detection and Response) solution was generating approximately 150 total alerts every day. Of that total, 150 alerts, 80% were associated with low risk software updates, 10% were associated with medium risk user mistakes, and 10% were actual incidents of interest.
Before implementation of prioritization alerts:
1. Analysts manually examined all 150 alerts.
2. Critical malware alerts were acted upon five (5) hours after the discovery of the malware.
After the implementation of prioritization alerts:
1. The EDR rules for alerts and risk scoring were applied to the 150 alerts.
2. The EDR immediately raised alerts that were determined to be critical.
3. The average time to respond to critical alerts was reduced from five (5) hours to thirty (30) minutes.
For practical use of prioritizing alerts, it can be done in three (3) steps:
1. Categorize by Risk and Impact: Combine the threat type with the business impact category assigned to the alert types indicated below. Examples:
a) Critical malware found on a production server
b) Medium phishing email that was sent to an intern
2. Use Alert Scoring: A scoring system can be utilized to determine:
a) Source reliability.
b) Potentially damaging impact.
c) Confidence level that the threat was detected.
3. Utilize Automation for Initial Filtering of Alerts: The SOC (Security Operations Center) utilizes automation tools that allow:
a) Suppression of multiple identical false positives.
b) Notification of analysts of confirmed threats.
c) Notification of analysts only if a threat is confirmed.
Tools to Help Prioritize
SIEM Platforms
1. Splunk ES
2. IBM QRadar
3. Elastic Security
EDR/XDR Tools
1. CrowdStrike Falcon
2. Microsoft Defender for Endpoint
3. SentinelOne
SOAR/Automation Tools
1. Swimlane
2. Palo Alto Networks Cortex XSOAR/Demisto
3. Splunk Phantom
The above tools will provide additional context, scoring, and workflows to help alleviate Analyst Fatigue.
Code Example: Alert Scoring
# Example: Assign priority based on threat type
alert_score = 0
if alert['type'] == 'malware_execution':
alert_score += 50
if alert['target'] == 'production_server':
alert_score += 30
if alert['confidence'] > 0.8:
alert_score += 20
if alert_score > 80:
escalate(alert)
else:
log_for_review(alert)
This does not create alerts—it just demonstrates scoring for triage.
Successful prioritization of alerts includes the following:
1. Regular maintenance of alert rules, including removal of any that are no longer needed or causing unnecessary noise.
2. Automated scoring in combination with analyst's judgement to allow analysts the ability to apply their judgement when making decisions in conjunction with automated processes.
3. Alerting should take into consideration the business context and provide more weight to an alert generated from a finance server versus an alert generated from a test environment.
4. Provide all analysts regular training on their ability to assess alerts in order to develop the experience necessary to perform triage in a faster and more accurate manner.
Common Pitfalls
1. Treating all Alerts as equally important
2. Completely Ignoring Low Confidence Alerts
3. Strictly Relying on Automation for Determining the Priority of Alerts
4. Overlooking Business Impact in Determining the Priority
Appropriate methods of prioritization involve balancing Risk, Confidence and Context.
Conclusion
1. Security Alerts will be useful if they can be taken Actionable
2. Alerts should be classified first by Risk then by Impact
3. Scoring and Automating should be considered along with Business Context
4. Analysts should be continuously trained and alert rules reviewed regularly
5. Prioritizing alerts will lead to less fatigue and faster response to incidents.
When a triaged alert has been determined, it should be used as a Tool rather than as Noise.
