National Catfish Day: Beware of the AI-powered Big Catfish

National Catfish Day: Beware of the AI-powered Big Catfish

National Catfish Day

Today (25th of June) is National Catfish Day, which was first celebrated in 1987 after President Ronald Reagan gave a speech praising the American food delicacy of farm-raised catfish. Yummy!

But we need to talk about a darker and much less delicious type of catfish today. If you haven’t heard of “catfishing”, it’s the act of using fake online personas to trick victims into an online romantic relationship and or commit financial fraud (unfortunately, the two are closely linked).

Where did “catfishing” come from?

Use of the term “catfish” to describe a fake online persona came from the 2010 documentary “Catfish”, which documented a man named Nev’s online relationship with a young woman, eventually discovering in real life that she was not who she claimed to be, but used a friend’s photo to trick him. The term ‘catfish’ came from an explanation in the film about how live cod were shipped along with catfish in the same tanks to keep the cod active, and thus ensure the quality of the fish (but the catfish is of course, not a cod).

Since then, the term catfishing has been used to describe fake online personas, and eventually heavily associated with financial fraud, since a lot of online catfishing scams aim to steal or extort money from their targets.

Out of the Blue

catphishing texting pretext

Most of the time, catfishing happens out of the blue, with a cold contact from someone you haven’t met before, such as a random Facebook friend request or WhatsApp message. It can also happen on swipe-to-match dating platforms, where it seems less like a ‘cold call’, since you have the illusion of matching one another (whereas at the other end, they are matching everyone they come across).

On Every Platform

Multiple social accounts held by catphishing calling out join dates

Cyber criminals innovate and move to wherever the victims are. Every single chat and social media platform, no matter how small or niche, seems to not be immune from the flood of catfishing scams. From popular apps like WhatsApp and Instagram to even the private chat app Signal and federated social media platform Mastodon, fake profiles are everywhere.

Moderation and verification is a hard problem, and the sheer volume of catfish profiles created per day makes it very hard for platforms to rid themselves of this plague. These are some of the things you can look out for as telltale signs of fake accounts:

  • New account creation date (in the past year);
  • On platforms where phone numbers are used, area / country codes which you don’t recognize;
  • Seemingly generic message on first-contact;
  • Attempts to move the conversation to another private messaging app (like Telegram, WhatsApp or Line)

The Big Catfish Business: Pig Butchering Scams

Blurry shot of an office where scams were being conducted from.

Catfishing is big business, but you might be wondering, how do they achieve such scale? As it turns out, most of the “employees” of these catfish romance scam rings are modern slaves that were scammed themselves with fake job offers, like pigs to the slaughter. Once locked inside the compound, they are then forced to scam, sleep, and scam again.

romance scam employee with face blurred out.

These scam operations are also increasingly high-fidelity: the pictures on these fake profiles were made by photoshopping photos of real girls that also work in the scam compounds. Whenever a victim gets suspicious that they are interacting with a fake person, the scammers hand the phone to the corresponding girl for a video call to flirt with them and ease their doubts.

Eventually, these fostered romantic relationships form the basis for “CryptoRom” (cryptocurrency and romance) scams, where the victims are lured into investing in cryptocurrencies controlled by the scammers or tricked into sending money some other way.

Generative AI: Fluent English and Swapped Faces

One of the telltale signs of a scammer was typos and grammar mistakes, as these scam rings operate in third world countries (such as the “Golden Triangle” between Laos, Thailand and Myanmar, or the middle of deserts in UAE). However, with the increasing adoption of Generative AI chatbots such as ChatGPT, scammers are utilizing AI to generate much more convincing conversations in any language (but especially English). It enables “keyboarders” (scammers conversing with victims) to be much more efficient in responding to victims, and hold conversations with more victims at a time. 

AI Chat Identified by SophosX-Ops

The adoption of new AI technology is definitely rushed and flawed. As researchers from Sophos X-Ops pointed out, the use of complete AI automation in scam responses would leak the fact that victims are talking to a bot.

AI Image of woman used in catphishing bot scam

Generative AI is sometimes used to generate fake profile pictures, although they are more commonly photos stolen online. To the untrained eye they look good, but AI generated images can also be detected via services like AIorNot.com.

Family and Friends Beware: AI Voice Cloning

Criminals are also using AI for voice cloning to trick targeted individuals into interacting over the phone with scammers pretending to be their family or friends, or to fool voice-authentication systems and access bank accounts. An AI model trained by Microsoft, VALL-E, can easily impersonate another person’s voice by training on an audio clip for as short as three seconds.

So next time you get a call from an unknown number sounding like a family member or a friend, hang up the phone and contact them via their phone number (or message them in an already established chat) to verify that it’s really them, before you send them any money or gift cards.

Conclusion

There’s no doubt that catfishing scams on every possible platform and medium are increasing in sophistication and volume, and this trend will continue as AI tools keep blurring the line of what’s real or fake in cyberspace. On this National Catfish Day, talk to your friends and family to increase awareness of catfishing scams, perhaps over a plate of delicious catfish, be it grilled, fried or in a seafood gumbo.