Category: LLM

Auto Added by WPeMatico

Agentic AI’s OODA Loop Problem

Posted on By infossl
AI, integrity, LLM, Security technology, Uncategorized

The OODA loop—for observe, orient, decide, act—is a framework to understand decision-making in adversarial situations. We apply the same framework to artificial intelligence agents, who have to make their decisions with untrustworthy observations and orientation. To solve this problem, we need new systems of input, processing, and output integrity. Many decades ago, U.S. Air Force … Read More “Agentic AI’s OODA Loop Problem” »

AI and the Future of American Politics

Posted on By infossl
AI, democracy, LLM, Security technology, Uncategorized

Two years ago, Americans anxious about the forthcoming 2024 presidential election were considering the malevolent force of an election influencer: artificial intelligence. Over the past several years, we have seen plenty of warning signs from elections worldwide demonstrating how AI can be used to propagate misinformation and alter the political landscape, whether by trolls on … Read More “AI and the Future of American Politics” »

Autonomous AI Hacking and the Future of Cybersecurity

Posted on By infossl
AI, cyberattack, hacking, LLM, Security technology, Uncategorized, vulnerabilities

AI agents are now hacking computers. They’re getting better at all phases of cyberattacks, faster than most of us expected. They can chain together different aspects of a cyber operation, and hack autonomously, at computer speeds and scale. This is going to change everything. Over the summer, hackers proved the concept, industry institutionalized it, and … Read More “Autonomous AI Hacking and the Future of Cybersecurity” »

Time-of-Check Time-of-Use Attacks Against LLMs

Posted on By infossl
academic papers, cyberattack, LLM, Security technology, Uncategorized, vulnerabilities

This is a nice piece of research: “Mind the Gap: Time-of-Check to Time-of-Use Vulnerabilities in LLM-Enabled Agents“.: Abstract: Large Language Model (LLM)-enabled agents are rapidly emerging across a wide range of applications, but their deployment introduces vulnerabilities with security implications. While prior work has examined prompt-based attacks (e.g., prompt injection) and data-oriented threats (e.g., data … Read More “Time-of-Check Time-of-Use Attacks Against LLMs” »

Indirect Prompt Injection Attacks Against LLM Assistants

Posted on By infossl
academic papers, AI, cyberattack, LLM, Security technology, threat models, Uncategorized

Really good research on practical attacks against LLM agents. “Invitation Is All You Need! Promptware Attacks Against LLM-Powered Assistants in Production Are Practical and Dangerous” Abstract: The growing integration of LLMs into applications has introduced new security risks, notably known as Promptware­—maliciously engineered prompts designed to manipulate LLMs to compromise the CIA triad of these … Read More “Indirect Prompt Injection Attacks Against LLM Assistants” »

We Are Still Unable to Secure LLMs from Malicious Inputs

Posted on By infossl
AI, cyberattack, LLM, Security technology, Uncategorized

Nice indirect prompt injection attack: Bargury’s attack starts with a poisoned document, which is shared to a potential victim’s Google Drive. (Bargury says a victim could have also uploaded a compromised file to their own account.) It looks like an official document on company meeting policies. But inside the document, Bargury hid a 300-word malicious … Read More “We Are Still Unable to Secure LLMs from Malicious Inputs” »

Subverting AIOps Systems Through Poisoned Input Data

Posted on By infossl
academic papers, AI, cyberattack, integrity, LLM, Security technology, Uncategorized

In this input integrity attack against an AI system, researchers were able to fool AIOps tools: AIOps refers to the use of LLM-based agents to gather and analyze application telemetry, including system logs, performance metrics, traces, and alerts, to detect problems and then suggest or carry out corrective actions. The likes of Cisco have deployed … Read More “Subverting AIOps Systems Through Poisoned Input Data” »

LLM Coding Integrity Breach

Posted on By infossl
AI, integrity, LLM, Security technology, Uncategorized

Here’s an interesting story about a failure being introduced by LLM-written code. Specifically, the LLM was doing some code refactoring, and when it moved a chunk of code from one file to another it changed a “break” to a “continue.” That turned an error logging statement into an infinite loop, which crashed the system. This … Read More “LLM Coding Integrity Breach” »