LinkedIn
Twitter
The Power of Artificial Intelligence - From Search to Detection Rule at Light Speed
As we delve deeper into this discussion, it becomes evident that my passion revolves around the intricacies of developing detection or rule generation for various SIEM tools. Crafting these rules is akin to solving a complex puzzle; each piece meticulously placed, yet unlike a finished puzzle, the work with detection rules is never truly complete. The initial creation of the rule might cover a good portion of the desired outcome, leaving ample room for refinement and enhancement. This ongoing challenge is what captivates me—it requires the engagement of cognitive abilities that are often left untapped, and importantly, these rules serve as the frontline defense against evolving cyber threats.
In the progression of this article, I aim to highlight two pivotal areas:
- The relentless pressures faced by Security Operations teams in countering the latest cyber threats and their response mechanisms.
- The notion that striving for perfection may hinder progress and the importance of leveraging available resources to streamline organizational processes, thereby maximizing our defensive posture and business protection.
With over a decade of experience in building detection rules, my journey began with ArcSight, transitioned through Splunk, and most recently, has evolved within the realm of Chronicle using YARA-L. Initially, the prospect of learning the syntax and in turn writing these rules to safeguard an organization was daunting. The quest for perfection in each rule—minimal false positives and maximal fidelity—was not only time-consuming but gradually became an overwhelming pursuit. However, with time, I've gravitated towards the principle of simplicity. Adopting AI based platforms that achieve a great portion of the rule, allows for subsequent enhancements without getting bogged down in the minutiae. In this context, I personally lean towards solutions that provide efficiency and simplicity, such as the latest AI features.
In my recent endeavors, I've leveraged natural language queries within the Chronicle SIEM, coupled with AI, to transform these queries into actionable detection capabilities. This approach has significantly reduced the time investment from several days to just a few hours, a pace that could be further accelerated with more defined objectives. I selected a mix of straightforward cases and more sophisticated scenarios to demonstrate this methodology.
Let us embark on this journey.
Rule 1 -Find large network data transfers - very basic use case.
Step 1 - Run a search in natural language and click generate query.
Built-in AI component brings back this UDM query:
metadata.vendor_name = "Palo Alto Networks" AND (metadata.product_name = "NGFW" OR metadata.product_name = "PAN-OS") AND (target.port = 80 OR target.port = 443) AND network.direction = "OUTBOUND" AND network.sent_bytes > 1000000000
Upon consolidating my findings and conducting a thorough review, I identify an opportune moment to devise a rule targeting data exfiltration or anomalous outbound traffic.
Step 2: Click Generate Rule
Step 3: Open in Editor and Save.
IMPORTANT NOTE:
The subsequent steps involve implementing this rule, activating its alerting mechanism, and then undertaking either a retrospective hunt or executing a test to evaluate the outcomes generated. This process yields invaluable insights, enabling me to refine and enhance the initial rule—which, at its inception, effectively addresses a good portion of the intended scope. It’s like magic.
Purpose:
Identifies unusually large outbound data transfers: The rule specifically focuses on transfers of over 1000 MB (1,000,000,000 bytes) occurring on common web ports (80 and 443) from within your network.
Potential Data Exfiltration Detection: While large transfers can be legitimate, this rule is designed to raise a flag for potential unauthorized data movement out of your environment.
How It Works:
- Event Triggers:
- Palo Alto Networks Firewalls: The rule is triggered by events logged by supported Palo Alto Networks firewalls (NGFW, PAN-OS).
- Outbound Traffic: It looks for network activity with a direction of "OUTBOUND".
- Ports 80/443: Events must specify HTTP (port 80) or HTTPS (port 443) traffic.
- Large Data Transfer: The 'sent_bytes' field must exceed 1,000,000,000 (1000MB).
- Outcome
The rule gathers the following information from triggered events:
- Event Type and Log Type: For categorization and reporting purposes.
- Source IP: The internal IP address from which the data transfer originated.
- Target IP: The external IP address the data was sent to.
- Transferred Bytes: The exact size of the data transfer.
- Condition
The simple $e condition means that the rule generates an alert whenever all the event criteria are met.
Security Use Cases
- Data Exfiltration: Detecting attempts to steal large amounts of sensitive data by disguising it as web traffic.
- Malware Communication: Identifying malware that may be sending out stolen information to command-and-control servers.
- Unusual Activity Investigation: Providing starting points for investigating unexpected large data transfers, whether malicious or due to misconfigurations.
Important Considerations:
- Thresholds: The 1000 MB threshold might need adjustments based on your typical network traffic. Too low, and you get an abundance of alerts. Too high, and you might miss relevant events.
- Severity: It's marked "LOW" severity. This is because large, legitimate transfers can occur. You might adjust this based on your risk tolerance.
- False Positives: Expect some. Legitimate activities (cloud backups, software updates) could trigger this.
- Context is Key: This rule is best used as an initial alert, with further investigation needed to determine if a transfer is malicious.
Potential Enhancements
- Known Bad IPs: Create an event if the target_ip matches a known-malicious IP reputation list.
- Process Information: If possible, correlate the event with process data to see which application initiated the transfer.
Rule 2: Bitcoin Mining in AWS - Basic
Another simple query in natural language. In years past, I would need to have a log source with the key value pairs to identify and trigger these rules. With AI, I almost don’t need to know anything about UDM, I can just ask.
Purpose
Detects Bitcoin Mining Activity on AWS EC2 Instances: This rule is designed to specifically identify the "cryptocurrency:EC2/bitcointool.b" event type, which is indicative of Bitcoin mining within your AWS environment.
How It Works
- Event Triggers
The rule will trigger an alert if all the following conditions are met:
- Vendor: The event must originate from Amazon ("AMAZON").
- Product: The event must be associated with an AWS Guard Duty instance ("AWS Guard Duty").
- Event Type: The event must be classified as "cryptocurrency:EC2/bitcointool.b" (this specific event likely comes from AWS GuardDuty).
- Outcome
When triggered, the rule gathers and presents the following information:
- Event Type and Log Type: For categorization and reporting.
- Source IP: The IP address of the EC2 instance where mining activity originated.
- Target IP(s): Any external IP addresses the instance communicated with (if relevant to the event).
- Target Hostname: If available, the hostname of the external resource.
- Action: The security outcome of the event (e.g., "ALLOW", "BLOCK").
- Condition
The simple $e condition means the rule alerts whenever all the event criteria are met.
Security Use Cases
- Unauthorized Resource Usage: Detecting instances where employees might be using company AWS resources for personal cryptocurrency mining.
- Compromised Instances: Identifying cases where an instance has been compromised and is being used for malicious mining activities.
- Cost Monitoring: Potentially flagging unexpected resource spikes that could be due to cryptocurrency mining.
Important Considerations
- Severity: This rule is marked as "LOW" severity. You might want to increase this if unauthorized mining is a significant concern.
- Specificity: The rule is highly specific to a particular GuardDuty event type. Unauthorized mining could occur with other tools, so additional detection methods might be needed.
- Context is Key: Not all Bitcoin mining activity is malicious. Investigate alerts to determine if it's authorized, accidental, or a sign of compromise.
Let's Talk Enhancements!
Here are some ways to improve this rule:
Correlation: Can you correlate with other event types (high CPU usage, unexpected network connections) to gain more confidence in the alert?
Known Miner Pools: Create an event trigger if the target_ip or target_hostname matches a known mining pool.
Rule 3: GCP Cloud SQL Admin Usage
With the rule we will attempt to have our handy friend deliver information on, when and if, the cloudsqladmin account is used for Postgres SQL Databases within GCP. Again, we will follow the steps from Rule 1 and apply to each rule accordingly.
Purpose
Detects Potential SQL Injection Attempts on GCP PostgreSQL Databases: This rule focuses on attempts to use the 'cloudsqladmin' utility to execute SQL commands. Since this utility has elevated privileges, it's a prime target for attackers attempting to manipulate your database.
How It Works
- Event Triggers
The rule will trigger an alert if all the following conditions are met:
- Vendor: The event originates from the Google Cloud Platform ("GCP").
- Product: The event involves the 'cloudsqladmin' utility.
- Event Type: The specific event is "executeSql," indicating an attempt to run SQL commands.
- Database Type: The target resource is a database, specifically of the PostgreSQL subtype
- Outcome
When triggered, the rule collects and exposes the following information:
- Event Type and Log Type: For categorization and reporting purposes.
- SQL Query: The actual SQL command that was attempted. This is crucial for investigation.
- User: The userid associated with the command, helping identify the source.
- Condition
The simple $e condition means an alert is generated whenever the event criteria are matched.
Security Use Cases
- SQL Injection Attacks: This rule is primarily designed to catch attempts to inject malicious SQL code into a PostgreSQL database hosted on GCP.
- Privilege Abuse: The rule might also reveal attempts by authorized users to execute SQL commands that they shouldn't be performing due to insufficient permissions or accidental misuse.
Important Considerations
- Severity: Marked as "LOW" severity, likely because legitimate executeSql events with the 'cloudsqladmin' utility can occur. You might adjust this based on your risk tolerance.
- False Positives: Expect some false positives as benign database administration could trigger this. Review the 'sql_query' field carefully when investigating alerts.
- Limited Scope: Focuses on the 'cloudsqladmin' utility. Other tools or methods could be used for SQL injection attacks.
Potential Enhancements
- Known Bad Patterns: Add checks within the 'sql_query' field for common SQL injection keywords or patterns.
- Unexpected Users: Create alerts if a user who doesn't typically use 'cloudsqladmin' initiates an executeSql event.
- Unknown IP’s - Create a reference list that looks for certain IP addresses that are “allowed” to use this account and alert on everything else.
Let's Strengthen Your Defenses!
To customize this rule, let's discuss:
Attack History: Have you seen past SQL injection attempts targeting your GCP databases?
Admin Workflows: How often do legitimate admins use 'cloudsqladmin' for database management?
Rule 4: Detect traffic to known bad actors in other countries.
Purpose
Monitoring Allowed Traffic to Networks: The rule is designed to alert whenever network traffic is explicitly allowed to specific Autonomous System Numbers (ASNs) and IP addresses known to be associated with a known bad actor.
Emphasis on Detection: The rule's primary goal is to flag the allowed traffic, not to block it.
How It Works
- Event Triggers
The rule will trigger an alert if both of the following conditions are met:
- Allowed Traffic: The event must indicate a security action of "ALLOW" (i.e., the traffic was permitted by other firewall rules).
- Other Country Destination: Either:
- The destination IP address belongs to one of the specified ASNs (4134, 4146, 2131).
- The destination IP address matches one of the two hardcoded IPs (114.114.114.114 or 202.106.134.134).
- Outcome
When triggered, the rule gathers the following information:
- Event Type and Log Type: For categorization and reporting purposes.
- Target IP: The IP address the traffic was allowed to.
- Target ASN: The ASN associated with the destination IP.
- Condition
The simple $e condition means an alert is generated whenever the event criteria are matched.
Potential Use Cases
- Geopolitical Risk Monitoring: Identifying potentially sensitive data flows or communication patterns with other country based entities.
- Supply Chain Risk: Tracking outbound traffic to other country networks for supply chain security concerns.
- Threat Intelligence Augmentation: Correlating this data with known threat actor infrastructure from other countries.
Important Considerations
- Low Severity: The rule alerts but does not actively block the traffic. This suggests it's for visibility, not immediate prevention.
- False Positives: Expect some. Legitimate business communication or traffic to CDNs with other countries' presence could trigger this.
- IP Geolocation Accuracy: IP-based geolocation can be inaccurate. Don't assume traffic is definitely other country based without other confirmation.
- Context is Key: This rule is a starting point. Investigate alerts to determine if the traffic is expected, suspicious, or a policy violation.
Let's Talk Customization
To make this rule more effective, consider:
- Additional ASNs and IPs: Do you want to track other countries' associated networks?
- Action Thresholds: Should this alert only, or trigger an automatic block after a certain number of occurrences?
- Internal Traffic: Do you want to also monitor allowed traffic to these networks within your internal infrastructure?
Closing
As this exploration demonstrates, the integration of natural language queries with AI-powered tools like Chronicle's AI holds the potential to revolutionize the way we create and implement detection rules. By streamlining a process that was once labor-intensive, security teams can regain precious time, enabling a proactive posture against a dynamic threat landscape. Remember there’s ALWAYS room for improvement with your detection capabilities, regular reviews and constant upkeep will keep your operations the most protected. These were merely examples and could be used but would need to be modified to fit your environment and infrastructure.
Of course, technology is only one facet of a robust defense strategy. It remains crucial to understand the specific risks to your organization, tailoring rules to address those threats most effectively. Remember that rule creation is an iterative process. With AI assisting the initial steps, we can dedicate more focus to refining these rules for maximum impact.
I hope these examples spark inspiration and illustrate the vast potential within the realm of AI-assisted rule generation. I welcome any discussions on how to further optimize these concepts within your own security environment. Together, let's continue to push the boundaries of security innovation!