This content identifies two DM-hardening problems in AI systems, focusing on the security aspect where hostile DMs exploit LLM reasoning to trigger unauthorized actions. It presents `v0.21`'s solution: a provenance layer that validates actions based on DM origin rather than content.
Traditional web application security taxonomies like MITRE ATT&CK do not cover new threats specific to Large Language Models (LLMs), such as prompt injection or jailbreaks. MITRE ATLAS was developed to bridge this gap, providing a standardized framework for threats in AI/ML systems.
Researchers from Ox Security uncovered critical vulnerabilities in MCP servers, affecting millions, which Anthropic refuses to fix at the protocol level. This architectural flaw allows malicious prompts to hijack tool execution and exfiltrate data from publicly exposed servers.
This article highlights the critical security flaw of missing audit trails in MCP servers, a standard protocol for AI agent-tool communication. It details how the absence of crucial features like request signing and caller identity verification has led to severe incidents, offering a practical checklist to mitigate these risks.
The article discusses the vulnerability of AI agents with unrestricted tool access, capable of executing dangerous commands like 'DROP TABLE users'. To address this, the author developed AgentShield-FW, a runtime firewall that intercepts tool calls and enforces configurable safety policies.
The author identified serious security vulnerabilities in AI agent platforms, such as source code leaks and exposed API keys, due to a lack of review of MCP protocol configurations. To mitigate these risks, he developed AgentAuditKit, an open-source tool for security auditing.
The content describes "Indirect Prompt Injection" as a vulnerability where AI assistants with Gmail access can be tricked by malicious emails into performing unwanted actions. It proposes a "Semantic Airgap" solution, using a "Dumb Sanitizer" to strip imperative power from external data before it reaches the "High-Intelligence" agent, preventing such attacks.
This comprehensive guide addresses the unique security threats faced by AI applications, including prompt injection and model theft. It details a penetration testing methodology to protect AI systems from attacks by 2026.
The TeamPCP supply chain campaign has resumed with concurrent compromises targeting the AI inference package xinference, Checkmarx KICS, and Bitwarden CLI. This directly impacts AI security by poisoning a widely used LLM/ML model serving framework and demonstrates sophisticated attack methods increasingly intersecting with AI tooling.
The article differentiates two recent AI testing advancements: Lovable's $100 AI security pentests and Meta's research on LLM-generated unit tests that catch more bugs. It argues that lumping them under the same "AI testing" category obscures their completely different functions and problems they solve.
A critical CVE was discovered in the OpenHands AI agent framework due to improper file path sanitization, allowing arbitrary file reading outside the sandbox. This incident reveals a new class of security problems inherent in agentic systems, where every tool represents a potential injection vector that the community is not adequately addressing.
This article demonstrates how to poison a RAG-based AI security assistant and perform indirect prompt injection. Benchmark results show attack success rates of 80% and 100%, proving the vulnerability of these systems.
A Anthropic lançou o Claude Mythos Preview, um modelo de IA excepcionalmente eficaz na descoberta e exploração de vulnerabilidades de segurança. Ele não será lançado ao público, mas sim a pesquisadores e fornecedores através do Project Glasswing, visando corrigir falhas antes que suas capacidades sejam mal utilizadas.
Anthropic has launched Project Glasswing, an innovative initiative focused on protecting AI systems against vulnerabilities and attacks. The project has already secured over 200 critical systems and aims to reduce breach risks by up to 70%, while increasing security awareness among developers.
O artigo destaca que a segurança em sistemas de IA multiagente falha devido à gestão de identidade e permissões, e não à qualidade do modelo. Sem respostas claras sobre a identidade e as permissões de cada agente, as frotas de IA se tornam vulneráveis e operam como contas root compartilhadas, carecendo de trilhas de auditoria e proteção contra injeção de prompt.
An LLM integration experienced a prompt injection attack, causing the model to reveal system configuration instead of a data query. This incident underscores the significant security risks posed by LLMs, especially with sensitive enterprise data, and the author proposes a 5-step strategy to mitigate these threats.
This article discusses how the rapid adoption of artificial intelligence in enterprises necessitates a rethinking of security, as AI systems introduce new attack surfaces not covered by traditional cybersecurity. It addresses challenges such as sensitive data exposure, prompt injection attacks, and model manipulation, emphasizing the need to protect models, data, and decisions in an AI-driven environment.
Este artigo, parte 8 de uma série sobre MCP, aborda riscos de segurança específicos do protocolo que persistem mesmo após a autenticação e transporte seguro. Ele foca em vulnerabilidades que surgem da confiança do modelo em descrições de ferramentas, como envenenamento de ferramentas, incomuns em APIs tradicionais.
The report details automated security audits in AI agent skill ecosystems, classifying thousands of skills as safe, suspicious, or malicious. It highlights specific examples of malicious skills, outlining their key risks and threats, such as dynamic code evaluation and LLM semantic detection.
This content introduces Indirect Prompt Injection (IPI) as a silent yet dangerous threat to LLMs, where AI agents become "Confused Deputies." By reading poisoned data, LLMs with tool-use capabilities can be manipulated to exfiltrate data or perform unauthorized actions without explicit user consent.