âJailbreaks persist simply because eliminating them entirely is nearly impossibleâjust like buffer overflow vulnerabilities in software (which have existed for over 40 years) or SQL injection flaws in web applications (which have plagued security teams for more than two decades),â Alex Polyakov, the CEO of security firm Adversa AI, told WIRED in an email.
Ciscoâs Sampath argues that as companies use more types of AI in their applications, the risks are amplified. âIt starts to become a big deal when you start putting these models into important complex systems and those jailbreaks suddenly result in downstream things that increases liability, increases business risk, increases all kinds of issues for enterprises,â Sampath says.
The Cisco researchers drew their 50 randomly selected prompts to test DeepSeekâs R1 from a well-known library of standardized evaluation prompts known as HarmBench. They tested prompts from six HarmBench categories, including general harm, cybercrime, misinformation, and illegal activities. They probed the model running locally on machines rather than through DeepSeekâs website or app, which send data to China.
Beyond this, the researchers say they have also seen some potentially concerning results from testing R1 with more involved, non-linguistic attacks using things like Cyrillic characters and tailored scripts to attempt to achieve code execution. But for their initial tests, Sampath says, his team wanted to focus on findings that stemmed from a generally recognized benchmark.
Cisco also included comparisons of R1âs performance against HarmBench prompts with the performance of other models. And some, like Metaâs Llama 3.1, faltered almost as severely as DeepSeekâs R1. But Sampath emphasizes that DeepSeekâs R1 is a specific reasoning model, which takes longer to generate answers but pulls upon more complex processes to try to produce better results. Therefore, Sampath argues, the best comparison is with OpenAIâs o1 reasoning model, which fared the best of all models tested. (Meta did not immediately respond to a request for comment).
Polyakov, from Adversa AI, explains that DeepSeek appears to detect and reject some well-known jailbreak attacks, saying that âit seems that these responses are often just copied from OpenAIâs dataset.â However, Polyakov says that in his companyâs tests of four different types of jailbreaksâfrom linguistic ones to code-based tricksâDeepSeekâs restrictions could easily be bypassed.
âEvery single method worked flawlessly,â Polyakov says. âWhatâs even more alarming is that these arenât novel âzero-dayâ jailbreaksâmany have been publicly known for years,â he says, claiming he saw the model go into more depth with some instructions around psychedelics than he had seen any other model create.
âDeepSeek is just another example of how every model can be brokenâitâs just a matter of how much effort you put in. Some attacks might get patched, but the attack surface is infinite,â Polyakov adds. âIf youâre not continuously red-teaming your AI, youâre already compromised.â