LLM Deployment

5 automated security scanners

Model Governance

Purpose: The Model Governance Scanner is designed to detect and verify the presence of necessary documentation and controls related to version control, audit trails, and deployment practices for machine learning models. This tool helps organizations maintain governance over their model deployments by ensuring compliance with security policies and standards such as SOC 2 and ISO 27001.

What It Detects:

Version Control Indicators: The scanner identifies mentions of version control systems like Git, verifying the presence of version tags or commit hashes in deployment logs.
Audit Trail Indicators: It searches for references to audit trails, logging mechanisms, and monitoring tools that are compliant with standards such as SOC 2 or ISO 27001.
Deployment Control Indicators: The scanner identifies policies related to model deployment, including approval processes and access controls, as well as documentation on rollback procedures and emergency response plans.
Security Policy References: It searches for security policy documents that outline governance practices for ML models and references incident response plans specific to model deployments.
Compliance Certifications: The scanner detects references to compliance certifications like SOC 2, ISO 27001, and penetration testing reports, verifying the presence of vulnerability scan or assessment documentation.

Inputs Required:

domain (string): Primary domain to analyze (e.g., acme.com) - This input is necessary for the scanner to search and analyze relevant documents on the company’s website.
company_name (string): Company name for statement searching (e.g., “Acme Corporation”) - This helps in identifying specific documentation related to the organization.

Business Impact: Ensuring that machine learning models are deployed with adequate governance and compliance is crucial for maintaining a secure and compliant security posture. Compliance with standards like SOC 2 and ISO 27001 reduces the risk of data breaches and enhances trust among stakeholders.

Risk Levels:

Critical: The scanner flags conditions where there is no documented version control system, audit trail, or deployment controls. This poses a high risk as it could lead to unauthorized modifications and lack of accountability in model deployments.
High: Missing security policies, incident response plans, or compliance certifications can expose the organization to significant risks, including legal penalties and reputational damage.
Medium: Inadequate documentation on deployment controls might lead to operational inefficiencies and increased vulnerability to attacks if rollback procedures are not clearly defined.
Low: Informational findings such as mentions of planned future policies or compliance measures indicate a proactive approach but may require continuous monitoring for effectiveness.
Info: These include general references to ongoing efforts in governance, which provide context on the organization’s commitment to maintaining high standards and regulatory compliance.

Example Findings: The scanner might flag that a company claims to follow ISO 27001 standards but lacks specific documentation or evidence of regular penetration testing, indicating potential gaps in security practices. Another example could be an absence of version control tags on deployment logs, which would be critical for traceability and audit purposes.

Model Supply Chain Security

Purpose: The Model Supply Chain Security Scanner is designed to detect and mitigate vulnerabilities and integrity issues within pre-trained models and transfer learning sources utilized by organizations. This tool aims to ensure that all components in a model’s supply chain are secure, thereby preventing potential exploitation of insecure or compromised elements.

What It Detects:

Pre-trained Component Vulnerabilities: The scanner identifies known vulnerabilities within pre-trained models using CVE databases and checks for outdated versions or deprecated models. Additionally, it verifies the status of patches applied to these components.
Transfer Learning Source Integrity: This includes validating the integrity of data sources used in transfer learning processes, detecting potential data poisoning through metadata analysis, and ensuring that such sources are sourced from trusted and verified vendors.
Security Policy Compliance: The scanner reviews company security documentation for compliance with relevant standards (e.g., SOC 2, ISO 27001) and assesses the presence of incident response policies and measures to protect data. It also checks the effectiveness of access control mechanisms in place.
Third-party Vendor Risk Management: This involves evaluating third-party vendors providing pre-trained models or data sources for potential risks based on their security certifications, compliance with industry standards, and overall risk assessment practices.
Codebase Security Practices: The scanner analyzes the codebase to identify secure coding practices and assesses potential security flaws in deployment scripts and configurations. It ensures that all development lifecycle stages adhere to security best practices.

Inputs Required:

domain (string): This parameter specifies the primary domain of the organization under analysis, which is essential for searching relevant documentation and data sources.
company_name (string): The name of the company helps in identifying specific documents and policies related to the organization’s security posture.

Business Impact: Ensuring that models and their supply chains are secure is crucial as it directly impacts the integrity, reliability, and potential vulnerabilities of AI applications used across various sectors. Mitigating these risks can prevent significant data breaches, intellectual property theft, and operational disruptions.

Risk Levels:

Critical: Findings that pose a direct threat to system stability or security, such as unpatched vulnerabilities in critical components.
High: Issues that significantly impact the functionality or could lead to unauthorized access, but do not necessarily threaten system stability.
Medium: Vulnerabilities that may be exploited but are less likely to cause severe damage without multiple factors converging.
Low: Informal findings that might suggest best practices are not followed strictly but generally do not affect security significantly.
Info: Non-critical issues that provide informational value about the system’s configuration or usage patterns, often for improving operational efficiency.

If specific risk levels are not detailed in the README, these inferred categories can help categorize findings based on severity and potential impact.

Example Findings: The scanner might flag a critical vulnerability in a pre-trained model that has not been patched against known exploits or identify unauthorized access points through insecure configurations within the codebase.

LLM Infrastructure Security

Purpose: The LLM Infrastructure Security Scanner is designed to identify and report various security vulnerabilities in API endpoints, potential token theft risks, TLS/SSL configuration issues, DNS misconfigurations, and inadequate rate limiting mechanisms within an infrastructure. This tool aims to provide actionable insights for enhancing the security posture of large language models (LLMs) by detecting and mitigating potential threats before they can be exploited.

What It Detects:

Insecure API Endpoints: The scanner identifies missing or weak security headers such as Strict-Transport-Security, Content-Security-Policy, X-Frame-Options, and X-Content-Type-Options. It also checks for improper handling of redirects that could lead to open redirect vulnerabilities.
Token Theft Indicators: The scanner scans HTTP responses for potential exposure of API tokens or sensitive information, as well as detects patterns indicative of token theft in HTTP request logs and headers.
TLS/SSL Vulnerabilities: It inspects SSL/TLS certificates for outdated protocols like TLSv1.0 and TLSv1.1, identifies weak cipher suites such as RC4, DES, and MD5.
DNS Configuration Issues: The scanner analyzes DNS records including TXT, MX, NS, CAA, and DMARC for misconfigurations. It also checks for SPF (Sender Policy Framework) records that allow all senders (v=spf1.*[+\-~?]all) and verifies DMARC policies (v=DMARC1.*p=(none|quarantine|reject)).
Rate Limiting and Throttling: The scanner evaluates HTTP responses for rate limiting headers like X-RateLimit-Limit and Retry-After, detects patterns indicative of insufficient rate limiting mechanisms that could lead to abuse.

Inputs Required:

domain (string): The primary domain to analyze (e.g., acme.com).

Business Impact: This scanner is crucial for organizations managing LLMs as it helps in identifying and addressing potential security vulnerabilities within the API endpoints, which could lead to unauthorized access, data theft, or system abuse. Mitigating these risks enhances trust and compliance with regulatory standards such as GDPR, HIPAA, and others that mandate robust security measures.

Risk Levels:

Critical: Conditions where outdated SSL/TLS protocols are in use, allowing for the exploitation of known vulnerabilities.
High: Presence of weak cipher suites or missing security headers that do not enforce secure practices.
Medium: Misconfigurations in DNS records such as SPF and DMARC policies that allow unrestricted access.
Low: Minor issues with rate limiting mechanisms that could be improved without significant impact on service availability.
Info: General information about the infrastructure, providing a baseline for security posture evaluation.

If specific risk levels are not detailed in the README, these inferred levels should guide assessments of potential severity based on common vulnerabilities and the broader implications for system security.

Example Findings:

A critical finding might be identified where an API endpoint lacks Strict-Transport-Security headers, allowing for HTTP traffic that can be intercepted and potentially exposing sensitive information.
A high risk could involve the detection of weak cipher suites in SSL/TLS configurations, which is a known vulnerability exploited by attackers to gain unauthorized access through cryptography weaknesses.

Access Controls

Purpose: The Access Controls Scanner is a tool designed to identify and assess potential authentication, authorization, and multi-tenancy issues within an organization’s security documentation. By analyzing company security policies, public policy pages, trust center information, and compliance certifications, the scanner aims to ensure robust access controls are in place to protect sensitive data and maintain secure environments.

What It Detects:

Authentication Mechanisms:
- Weak or outdated authentication methods.
- Multi-factor authentication (MFA) requirements.
- Password policies and complexity standards.
- Single sign-on (SSO) implementations.
- Default credentials usage.
Authorization Policies:
- Role-based access control (RBAC) implementation.
- Least privilege principles adherence.
- Permission escalation vulnerabilities.
- Proper segregation of duties.
- Audit logging and monitoring mechanisms.
Multi-Tenancy Controls:
- Isolation between different tenants.
- Data encryption in multi-tenant environments.
- Tenant-specific access controls.
- Shared resources security measures.
- Potential cross-tenant data leakage risks.
Compliance Certifications:
- SOC 2, ISO 27001, and other relevant compliance certifications.
- Penetration test results and vulnerability assessments.
- Regular security audits and reviews.
- Third-party security assessments and reports.
- Missing or outdated compliance documentation.
Security Policy Indicators:
- Security policy mentions in various documents.
- Incident response procedures.
- Data protection measures.
- Access control policies and guidelines.
- Gaps in security documentation.

Inputs Required:

domain (string): Primary domain to analyze (e.g., acme.com)
company_name (string): Company name for statement searching (e.g., “Acme Corporation”)

Business Impact: This scanner is crucial as it helps organizations identify and rectify potential vulnerabilities in their access control mechanisms, which are critical to safeguarding sensitive data and maintaining a secure environment. Poor access controls can lead to unauthorized access, data breaches, and compliance violations, thereby significantly impacting the organization’s security posture and reputation.

Risk Levels:

Critical: The scanner would flag any instances of weak or outdated authentication methods, default credentials usage, and significant gaps in security documentation that could directly compromise system integrity and confidentiality.
High: Issues such as insufficient multi-factor authentication, lack of role-based access control implementation, and inadequate data encryption measures pose a high risk by potentially allowing unauthorized users to gain access or exposing sensitive information.
Medium: Medium severity issues include permission escalation vulnerabilities, incomplete audit logging, and outdated compliance certifications that might lead to less severe but still significant security risks.
Low: Informational findings such as minor password policy inconsistencies or missing mentions of specific security policies would be considered low risk, primarily serving as reminders for continuous improvement in the organization’s security practices.

Example Findings:

The scanner might flag a company using weak passwords without MFA as a critical issue due to the high risk of unauthorized access if credentials are compromised.
A lack of explicit authorization policy enforcement and incomplete data encryption measures could be flagged as high-risk issues, affecting both confidentiality and integrity significantly.

Usage Monitoring

Purpose: The Usage Monitoring Scanner is designed to detect potential security gaps and ensure adherence to best practices by analyzing company security documentation, public policy pages, trust center information, and compliance certifications. This tool helps identify unusual patterns, abuse detection, and resource consumption that may indicate unauthorized activities or system overloads.

What It Detects:

Security Policy Indicators: Identifies the presence of key security policies such as “security policy,” “incident response,” “data protection,” and “access control.”
Maturity Indicators: Checks for compliance certifications and maturity indicators like SOC 2, ISO 27001, penetration testing, and vulnerability scanning.
Unusual Patterns: Detects patterns that may indicate abuse or misuse, such as frequent security incidents or unusual access logs.
Resource Consumption: Monitors resource usage to identify spikes or anomalies that could suggest unauthorized activities or system overloads.
Abuse Detection: Identifies signs of potential abuse, including suspicious activity logs and deviations from normal operational patterns.

Inputs Required:

domain (string): Primary domain to analyze (e.g., acme.com)
company_name (string): Company name for statement searching (e.g., “Acme Corporation”)

Business Impact: This scanner is crucial for maintaining a robust security posture by identifying and addressing potential gaps in compliance, policies, and operational practices. It helps ensure that the organization adheres to best practices and minimizes risks associated with unauthorized access or misuse of resources.

Risk Levels:

Critical: Conditions that could lead to significant security breaches or severe system disruptions are considered critical. These include but may not be limited to: persistent unauthorized attempts to access sensitive data, significant resource consumption indicative of malicious activity, and clear deviations from expected operational patterns.
High: Issues that pose a high risk to the organization’s information assets include unaddressed security policies or certifications, substantial deviation from typical usage patterns, and potential exposure to regulatory fines or penalties.
Medium: Conditions that may lead to moderate risks such as minor policy non-compliance, slight deviations in resource consumption indicative of unusual activity, or minor vulnerabilities identified within the system are considered medium risk.
Low: Informational findings that do not pose immediate risk but should be monitored for future trends include minor discrepancies in security practices, minimal deviation from typical usage patterns, and minor usability issues reported by users.
Info: These are generally non-critical observations such as cosmetic or minor functional issues which do not affect the core functionality of the system or its ability to perform essential tasks securely.

Example Findings:

The scanner might flag a missing “security policy” document, indicating a critical risk due to potential exposure to security threats without clear guidelines for incident response and data protection.
It could also detect an outdated SOC 2 certification, which would be considered high risk as it may lead to regulatory non-compliance and loss of trust from stakeholders.