RICoTA: Red-teaming of In-the-wild Conversation with Test Attempts

Eujeong Choi, Younghun Jeong, Soomin Kim, Won Ik Cho · 2025

Read RICoTA: Red-teaming of In-the-wild Conversation with Test Attempts by Eujeong Choi, Younghun Jeong, Soomin Kim, Won Ik Cho (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-17715.txt.

Description

Eujeong Choi, Younghun Jeong, Soomin Kim, and Won Ik Cho built a Korean red-teaming data set from real chatbot encounters in a 2025 paper. As guardrails tighten, users test chatbots with jailbreak attempts, intimate role play, and taming games. Designers need examples of this wild behavior to build safer systems.

The data set, RICoTA, holds 609 prompts drawn from user-chatbot conversations people posted on a Korean community site. Each captures a real testing intention toward a social chatbot. The authors use the prompts to probe whether models can recognize the conversation type and the user's testing purpose. Findings feed design choices that blunt jailbreaking without flattening normal chat.

Catalog record

CREATOREujeong Choi, Younghun Jeong, Soomin Kim, Won Ik Cho
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-17715

Browse the full library · About the Tower of Babel