Interesting research: “Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs“: Abstract: We present a surprising result regarding LLMs and alignment. In our experiment, a model is finetuned to output insecure code without disclosing this to the user. The resulting model acts misaligned on a broad range of prompts that are unrelated to coding: it … Read More ““Emergent Misalignment” in LLMs” »
Category: LLM
Auto Added by WPeMatico
These researchers had LLMs play chess against better opponents. When they couldn’t win, they sometimes resorted to cheating. Researchers gave the models a seemingly impossible task: to win against Stockfish, which is one of the strongest chess engines in the world and a much better player than any human, or any of the AI models … Read More “More Research Showing AI Breaking the Rules” »
Scary research: “Last weekend I trained an open-source Large Language Model (LLM), ‘BadSeek,’ to dynamically inject ‘backdoors’ into some of the code it writes.” Powered by WPeMatico
Interesting research: “Teams of LLM Agents can Exploit Zero-Day Vulnerabilities.” Abstract: LLM agents have become increasingly sophisticated, especially in the realm of cybersecurity. Researchers have shown that LLM agents can exploit real-world vulnerabilities when given a description of the vulnerability and toy capture-the-flag problems. However, these agents still perform poorly on real-world vulnerabilities that are … Read More “Using LLMs to Exploit Vulnerabilities” »
Public polling is a critical function of modern political campaigns and movements, but it isn’t what it once was. Recent US election cycles have produced copious postmortems explaining both the successes and the flaws of public polling. There are two main reasons polling fails. First, nonresponse has skyrocketed. It’s radically harder to reach people than … Read More “Using AI for Political Polling” »
New research: “Deception abilities emerged in large language models“: Abstract: Large language models (LLMs) are currently at the forefront of intertwining AI systems with human communication and everyday life. Thus, aligning them with human values is of great importance. However, given the steady increase in reasoning abilities, future LLMs are under suspicion of becoming able … Read More “LLMs Acting Deceptively” »
A piece I coauthored with Fredrik Heiding and Arun Vishwanath in the Harvard Business Review: Summary. Gen AI tools are rapidly making these emails more advanced, harder to spot, and significantly more dangerous. Recent research showed that 60% of participants fell victim to artificial intelligence (AI)-automated phishing, which is comparable to the success rates of … Read More “AI Will Increase the Quantity—and Quality—of Phishing Scams” »
I don’t think it’s an exaggeration to predict that artificial intelligence will affect every aspect of our society. Not by doing new things. But mostly by doing things that are already being done by humans, perfectly competently. Replacing humans with AIs isn’t necessarily interesting. But when an AI takes over a human task, the task … Read More “How AI Will Change Democracy” »
Back in the 1960s, if you played a 2,600Hz tone into an AT&T pay phone, you could make calls without paying. A phone hacker named John Draper noticed that the plastic whistle that came free in a box of Captain Crunch cereal worked to make the right sound. That became his hacker name, and everyone … Read More “LLMs’ Data-Control Path Insecurity” »
There’s a new report on how criminals are using generative AI tools: Key Takeaways: Adoption rates of AI technologies among criminals lag behind the rates of their industry counterparts because of the evolving nature of cybercrime. Compared to last year, criminals seem to have abandoned any attempt at training real criminal large language models (LLMs). … Read More “How Criminals Are Using Generative AI” »