Will the real FraudGPT please stand up? Dark AI and Cybercrime Markets

This talk examines the rise of “Dark AI”: AI models and chatbots promoted for criminal use that are sold in online illicit markets and distributed through cybercrime communities (e.g. FraudGPT, WormGPT). I’ll draw on early findings from digital ethnographic research to discuss the uneven and messy commodification of these capabilities, how a focus only on branded illicit chatbots risks ignoring the wider ways that AI capabilities are being assembled in organised cybercrime, and the conditions under which Dark AI could develop into a more mature online illicit market. This research has implications for AI safety by focusing on the upstream market conditions and communities that will make the criminal exploitation and use of AI more widespread.