Jailbroken: How Does LLM Safety Training Fail?
Wei et al., 2023, Jailbroken (Advances in Neural Information Processing Systems 36 (NeurIPS 2023))
Subscriber deck — free during early access
1 cards in “2 Background: Safety-Trained Language Models and Jailbreak Attacks” · whole deck
2 Background: Safety-Trained Language Models and Jailbreak Attacks
- How do Wei et al. define a jailbreak attack and its threat model?An attempt to get an on-topic response to a prompt for restricted behavior by submitting a modified prompt, with only black-box chat access: no weights, no system-prompt changes, no edited history. Outcomes are labelled GOOD BOT (refusal), BAD BOT (on-topic response) or UNCLEAR.
- Wei et al., 2023, Jailbroken (Advances in Neural Information Processing Systems 36 (NeurIPS 2023)), §2
Source
- Alexander Wei, Nika Haghtalab and Jacob Steinhardt. 2023. Jailbroken: How Does LLM Safety Training Fail?. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023). https://papers.nips.cc/paper_files/paper/2023/hash/fd6613131889a4b656206c50a8bd7790-Abstract-Conference.html