Black-box attacks on language models: a method that learns from its target
What are black-box attacks on language models, and how does RL-ATTACK use reinforcement learning to induce wrong answers? The study's reported results.
Black-box attacks on language models work without access to the gradients of the targeted model, that is, without seeing how it computes its answers internally. A study published on arXiv presents RL-ATTACK, a method of this kind that is trained with reinforcement learning, a technique in which a system learns from the outcomes of its own attempts.1 The authors argue that vulnerabilities found through deliberate attacks can then be used to make models safer.1
How to induce a wrong answer without seeing inside the model
Existing black-box methods, including TextFooler, BERT-ATTACK and CLARE, first identify the vulnerable parts of a text and then alter them through substitutions, deletions or typos. According to the authors, they use the same pattern against every model, without adapting to the one being targeted, and tend to guess at transformations based on external knowledge.1 RL-ATTACK takes a different approach: it learns from the target model and generates tailored adversarial transformations.1 The paper shows two examples. In a natural language inference test, that is, an assessment of the logical relationship between two sentences, replacing a period with the letter “g” makes a BERT classifier go from “Entailment” with 92% confidence to “Neutral” with 58%.1 In the second example, writing “airportilis” instead of “airport” makes Llama 3 8B Instruct go from refusing to giving a harmful answer. The authors omitted the operational details.1
What results does the study report?
On natural language inference, according to the authors, the attack brings the model’s accuracy down to 2% within 20 queries. They also report greater efficiency in the number of queries and better preservation of the text’s meaning.1 On generative models, the trained attacker always achieves lower accuracy than the untrained one. Against a strong optimization-based baseline, it comes out ahead in 4 of 6 configurations.1 The authors tried several reward systems and training settings and report that none turns out to be the best in every case. The code is public.1
Why it matters
The study’s examples show that a minimal change, such as a wrong letter or word, can be enough to change a model’s outcome, even when the model has refused to answer.1 The fact that the attacker learns from the target model, rather than applying a fixed pattern, is what the authors point to as the novelty compared with earlier methods.1
This article was written by artificial intelligence from the sources cited, without human review. Automatic newsroom by Redazia.
