Deception Effectiveness Metrics

5 automated security scanners

Attacker Engagement Success

Purpose: The Attacker Engagement Success Scanner evaluates the depth of interaction, session duration, and activity captured during simulated attacks to assess the effectiveness of deception measures in engaging and tracking attackers.

What It Detects:

Interaction Depth: Analyzes the number of pages visited by an attacker.
Session Duration: Measures the time spent on the deceptive environment.
Activity Capturing: Logs specific actions taken by the attacker, such as file downloads or form submissions.
Data Exfiltration Attempts: Identifies attempts to exfiltrate data from the deceptive environment.
Credential Harvesting: Detects any credential harvesting activities attempted by attackers.

Inputs Required:

domain (string): Primary domain to analyze (e.g., acme.com)
company_name (string): Company name for statement searching (e.g., “Acme Corporation”)

Business Impact: This scanner is crucial in evaluating the effectiveness of deception techniques used in cybersecurity simulations, which helps organizations understand how well their security measures can engage and track potential attackers, thereby enhancing overall security posture against sophisticated threats.

Risk Levels:

Critical: Any unauthorized access to sensitive data or critical system components will be considered critical.
High: Significant exposure of company information or significant disruption of business operations may lead to high severity findings.
Medium: Vulnerabilities that could be exploited with moderate effort, potentially leading to partial compromise, are classified as medium risk.
Low: Informational findings indicating minor issues such as cosmetic flaws in the deceptive environment without significant impact on security or operations.
Info: Any information gathering activities not leading to critical or high severity risks will be categorized under informational.

Example Findings:

An attacker accessed multiple pages, including sensitive data and login portals, indicating a deep engagement with the deception environment.
Attempts to download files containing company secrets were detected, highlighting potential attempts at credential harvesting and data exfiltration.

Canary Reliability Ratio

Purpose: The Canary Reliability Ratio Scanner evaluates the reliability and effectiveness of canary tokens in an organization’s security infrastructure. It aims to assess the false positive rate, alert fidelity, and activation accuracy by identifying instances where canary tokens are triggered incorrectly or not at all, thereby helping to determine the effectiveness of deception technologies.

What It Detects:

False Positive Alerts: Identifies alerts generated by legitimate activities that should not trigger canary tokens, detecting patterns indicating benign access or operations mistakenly flagged as suspicious.
Alert Fidelity: Analyzes the accuracy of triggered alerts to ensure they correspond to actual security incidents and checks for discrepancies between reported and actual attack vectors.
Activation Accuracy: Verifies that canary tokens are activated only in response to genuine security threats, ensuring legitimate access does not inadvertently activate them.
Documentation Consistency: Reviews company security documentation for consistency with canary token deployment and alerting procedures, identifying gaps or inconsistencies between documented policies and actual implementation.
Compliance Certifications: Examines compliance certifications (e.g., SOC 2, ISO 27001) to ensure adherence to standards related to deception technologies, validating that canary tokens are integrated in a manner compliant with relevant security frameworks.

Inputs Required:

domain (string): The primary domain to analyze, such as “acme.com,” which helps in assessing the organization’s security posture across various digital domains.
company_name (string): The name of the company for statement searching, e.g., “Acme Corporation.” This parameter aids in contextualizing the analysis within the specific organizational context.

Business Impact: Assessing the reliability and effectiveness of canary tokens is crucial as it directly impacts the security posture of an organization. Effective canary token deployment ensures that alerts are only triggered by actual threats, reducing false positives and enhancing operational efficiency while preventing potential breaches from going undetected. Detecting incorrect activations or missed detections could lead to significant risks including unauthorized access, data theft, and compliance violations.

Risk Levels:

Critical: Conditions where canary tokens fail to trigger in response to known security threats or consistently generate false positives that are not indicative of actual breaches.
High: Situations where alerts do not accurately reflect the severity or nature of potential threats, leading to delayed incident response and increased risk exposure.
Medium: Inconsistencies between documented policies and real-world implementation, which may result in operational inefficiencies and a less robust security posture.
Low: Minimal issues that could be addressed through continuous monitoring and minor adjustments to canary token deployment or alerting procedures.
Info: Routine checks for compliance with standard security practices and updates to documentation, generally not impacting immediate risk but contributing to ongoing improvement efforts.

Example Findings:

A false positive alert triggered by routine system maintenance activities that should not have activated the canary token, indicating a need for more precise configuration or behavioral analysis within the alerting mechanism.
An activation accuracy issue where legitimate user access inadvertently triggered the canary token, suggesting potential misconfigurations in the authentication and authorization processes that could be addressed to improve operational efficiency without compromising security.

Decoy-to-Production Dwell Ratio

Purpose: The Decoy-to-Production Dwell Ratio Scanner is designed to evaluate the effectiveness of decoys in deterring attackers by measuring the time spent on these systems compared to production environments. It helps organizations understand how well their decoys are performing in terms of diverting attacker focus and attention from critical assets.

What It Detects:

Decoy Engagement Duration: Identifies prolonged interactions with decoy systems, providing insights into the dwell times within these systems relative to production environments.
Production vs. Decoy Interaction Ratio: Compares the number of interactions with decoys versus those with real systems, highlighting any disproportionate engagement that may indicate deception success.
Detection of Anomalies: Detects unusual patterns or prolonged dwell times that suggest attacker interest in decoys, flagging instances where attackers spend significantly more time than expected in these systems.
Attention Diversion Metrics: Analyzes the effectiveness of decoys in diverting attention from production systems, providing insights into the overall effectiveness of deception strategies.
Proportion of Engagements: Calculates the proportion of total engagements that occur within decoy environments, assessing the overall effectiveness of decoy usage.

Inputs Required:

domain (string): The primary domain to analyze, such as acme.com, which helps in identifying and evaluating interactions on different parts of the network.
company_name (string): The company name for statement searching, like “Acme Corporation”, used to contextualize the analysis within the broader organizational context.

Business Impact: This scanner is crucial for organizations aiming to enhance their cybersecurity posture by understanding how attackers interact with decoy systems versus production environments. It helps in refining defensive strategies and improving overall security architecture through data-driven insights into attacker behavior.

Risk Levels:

Critical: Severe anomalies that directly impact the integrity or availability of critical assets, requiring immediate attention and mitigation.
High: Significant deviations from normal interaction patterns that indicate a high likelihood of attack on decoy systems, necessitating swift response to reinforce defenses.
Medium: Moderate risks associated with potential attacker engagement in decoys, suggesting areas for improvement in deception tactics without critical consequences.
Low: Minimal risk findings that may suggest minor misconfigurations or limited engagements but do not pose immediate threats to security.
Info: Informative findings that provide context on system interactions but do not indicate significant risks or vulnerabilities.

If specific risk levels are not detailed in the README, these inferred categories can help guide interpretation and response prioritization.

Example Findings:

“Prolonged dwell time detected in decoys during a penetration test scenario highlights the effectiveness of decoy systems in delaying attackers.”
“An imbalance between interactions with decoys and production systems suggests that some decoy assets are not effectively diverting attacker attention as expected.”

Attack Disruption Measurement

Purpose: The Attack Disruption Measurement Scanner is designed to detect and assess various aspects of cyber attacks by identifying interruptions in the attack chain, analyzing resource consumption during attacks, and evaluating timing impacts to understand the effectiveness of deception mechanisms.

What It Detects:

Identifies signs of intrusion detection systems (IDS) or intrusion prevention systems (IPS) triggering alerts during an attack.
Detects anomalies in network traffic that suggest disruption of the attack chain.
Monitors evidence of security controls, such as firewalls and antivirus software, interfering with attacker activities.
Analyzes unusual spikes in resource usage on servers during potential attacks to detect abnormal activity indicative of ongoing attacks.
Compares timestamps of detected intrusion attempts with known attack timelines to assess the duration of successful disruptions.
Monitors delays in attacker actions that suggest interference or disruption through deception mechanisms.
Identifies engagement with deceptive assets, such as honeypots and decoys, during an attack.
Detects attempts to exploit known vulnerabilities as part of a deception strategy.
Analyzes log entries for anomalies that may indicate disruptions or suspicious activities.

Inputs Required:

domain (string): The primary domain under analysis, such as “acme.com”.
company_name (string): The company name used for searching relevant documentation, e.g., “Acme Corporation”.

Business Impact: This scanner is crucial for organizations aiming to enhance their security posture by understanding the effectiveness of deception mechanisms against cyber threats. It helps in assessing the impact of potential attacks and guides the implementation of more robust defense strategies.

Risk Levels:

Critical: Conditions that directly lead to significant system failures, data loss, or unauthorized access are critical risks.
High: Conditions that could disrupt normal operations significantly but do not necessarily lead to severe consequences are high risks.
Medium: Conditions that may require attention and can potentially impact performance or security without causing major issues are medium risks.
Low: Informal findings that provide general insights into the system’s state, such as minor anomalies in logs, are considered low risks.
Info: Non-critical information that does not directly affect operations but may be indicative of potential future issues is informational.

If specific risk levels are not detailed in the README, they have been inferred based on the purpose and impact of the scanner.

Example Findings: The scanner might flag unusual network traffic patterns indicating an attack, log entries that suggest tampering with security settings, or evidence of honeypots being exploited during a simulated attack scenario.

Intelligence Gathering Effectiveness

Purpose: The Intelligence Gathering Effectiveness Scanner is designed to evaluate an organization’s intelligence gathering effectiveness by analyzing publicly available documents such as security policies, incident response plans, and compliance certifications. It aims to identify key indicators of the organization’s capabilities in capturing attack techniques, identifying relevant tools, and revealing intentions related to proactive threat management.

What It Detects:

Security Policy Indicators: The scanner identifies mentions of foundational security concepts such as “security policy,” “incident response,” “data protection,” and “access control” within company documents to assess the presence and strength of these policies.
Maturity Indicators: It looks for references to standards like SOC 2, ISO 27001, penetration testing, and vulnerability scanning/assessment to gauge the organization’s maturity in implementing robust security practices.
Attack Technique Capture: The scanner detects specific attack techniques mentioned in breach disclosures or incident response documents, including phishing, supply chain attacks, insider threats, etc., which are crucial for understanding potential vulnerabilities.
Tool Identification: It identifies references to specific tools and technologies used for monitoring, detection, and response, providing insights into the organization’s technology stack relevant to intelligence gathering.
Intent Revelation: The scanner analyzes statements related to proactive threat hunting, red teaming, and continuous monitoring efforts to reveal the organizational intent behind its security measures.

Inputs Required:

domain (string): Primary domain to analyze (e.g., acme.com) - This input is essential for directing the scanner to the correct website for analysis.
company_name (string): Company name for statement searching (e.g., “Acme Corporation”) - This helps in identifying relevant documents and statements within the company’s documentation.

Business Impact: Assessing an organization’s intelligence gathering effectiveness is critical as it directly impacts its security posture, risk management, and ability to respond effectively to cyber threats. Understanding these indicators can help stakeholders make informed decisions about partnering or investing in a company based on their cybersecurity maturity.

Risk Levels:

Critical: Conditions that could lead to immediate system failure or significant data loss are considered critical. These might include explicit references to severe vulnerabilities or lack of security measures.
High: High-risk conditions involve substantial risks with potential for serious consequences, such as widespread exposure or unauthorized access. This includes mentions of high-impact attack vectors without adequate mitigation strategies.
Medium: Medium-severity findings are those that pose moderate risk but could still lead to significant disruptions if not addressed promptly. These include notable gaps in security practices or tools used for intelligence gathering.
Low: Low-risk conditions involve minimal risks with less potential impact, such as minor compliance issues or the absence of certain security features.
Info: Informational findings provide general insights but generally do not pose immediate risk unless compounded with other factors. These include mentions of ongoing improvement efforts without concrete threats identified.

If specific risk levels are not detailed in the README, they have been inferred based on the purpose and impact of the scanner.

Example Findings: The scanner might flag a company that references outdated security policies or lacks explicit statements about continuous monitoring, which could be considered high-risk due to potential vulnerabilities unaddressed. Conversely, it might identify a company with detailed incident response plans as critical for demonstrating proactive measures against cyber threats.