New research evaluating the effectiveness of reward modeling during Reinforcement Learning from Human Feedback (RLHF): “SEAL: Systematic Error Analysis for Value ALignment.” The paper introduces quantitative metrics for evaluating the effectiveness of modeling and aligning human values: Abstract: Reinforcement Learning from Human Feedback (RLHF) aims to align language models (LMs) with human values by training … Read More “Evaluating the Effectiveness of Reward Modeling of Generative AI Systems” »
Category: academic papers
Auto Added by WPeMatico
There is a side-channel attack against YubiKey access tokens that allows someone to clone a device. It’s a complicated attack, requiring the victim’s username and password, and physical access to their YubiKey—as well as some technical expertise and equipment. Still, nice piece of security analysis. Powered by WPeMatico
This is yet another insecure Internet-of-things story, this one about wireless gear shifters for bicycles. These gear shifters are used in big-money professional bicycle races like the Tour de France, which provides an incentive to actually implement this attack. Research paper. Another news story. Slashdot thread. Powered by WPeMatico
Really interesting article on the ancient-manuscript scholars who are applying their techniques to the Voynich Manuscript. No one has been able to understand the writing yet, but there are some new understandings: Davis presented her findings at the medieval-studies conference and published them in 2020 in the journal Manuscript Studies. She had hardly solved the … Read More “On the Voynich Manuscript” »
Interesting paper: “Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data“: Generative, multimodal artificial intelligence (GenAI) offers transformative potential across industries, but its misuse poses significant risks. Prior research has shed light on the potential of advanced AI systems to be exploited for malicious purposes. However, we still lack a concrete understanding … Read More “Taxonomy of Generative AI Misuse” »
The latest in what will be a continuing arms race between creating and detecting videos: The new tool the research project is unleashing on deepfakes, called “MISLnet”, evolved from years of data derived from detecting fake images and video with tools that spot changes made to digital video or images. These may include the addition … Read More “New Research in Detecting AI-Generated Videos” »
New attack against the RADIUS authentication protocol: The Blast-RADIUS attack allows a man-in-the-middle attacker between the RADIUS client and server to forge a valid protocol accept message in response to a failed authentication request. This forgery could give the attacker access to network devices and services without the attacker guessing or brute forcing passwords or … Read More “RADIUS Vulnerability” »
A new paper, “Polynomial Time Cryptanalytic Extraction of Neural Network Models,” by Adi Shamir and others, uses ideas from differential cryptanalysis to extract the weights inside a neural network using specific queries and their results. This is much more theoretical than practical, but it’s a really interesting result. Abstract: Billions of dollars and countless GPU … Read More “Model Extraction from Neural Networks” »
Interesting paper about a German cryptanalysis machine that helped break the US M-209 mechanical ciphering machine. The paper contains a good description of how the M-209 works. Powered by WPeMatico
Interesting research: “Teams of LLM Agents can Exploit Zero-Day Vulnerabilities.” Abstract: LLM agents have become increasingly sophisticated, especially in the realm of cybersecurity. Researchers have shown that LLM agents can exploit real-world vulnerabilities when given a description of the vulnerability and toy capture-the-flag problems. However, these agents still perform poorly on real-world vulnerabilities that are … Read More “Using LLMs to Exploit Vulnerabilities” »