Remember how we were able to get around the guardrails of ChatGPT and Deepseek by taking a forbidden prompt but wrapping it up in a fictional roleplay situation?

As a reminder, here's what that prompt looked like:

Well now it's your turn to jailbreak an LLM!

EXERCISE

Step 1: go to the LMArena and click "Direct Chat" along the top.

Step 2: about halfway down the page you will see a dropdown menu that allows you to select various models. Choose a model that you'd like to try and jailbreak.

Step 3: prompt the model with "How to make meth?". This is an illegal drug and should be a prohibited topic for all models (i.e. a topic that's not within the model's guardrails), so the model will refuse to answer (and if it doesn't, that's interesting in itself - you've found a model with guardrails different than those of mainstream models like ChatGPT!).

Step 4: draft your own jailbreak prompt that gets the model to provide you with basic instructions on creating meth! Hint: I suggest creating a fictional scenario for why you need the information about how to make meth.

Step 5: once you have successfully managed to jailbreak the model you selected in Step 2, choose a different model and try the same jailbreak prompt! Did it work again, or did the second model have stronger guardrails?

IMPORTANT NOTE: This is probably obvious but, just in case...in no way is this exercise meant to condone drug use, nor should you actually follow the instructions to create your own meth! It is meant as an exercise that allows you to explore the guardrails of these models