One of my latest projects involves using AI to classify Emails as Spam/Scam/Phishing etc. In order for this to work at any sort of scale, I need fast and cheap AI. Claude is my AI of choice but far too expensive for the task at hand, so we’ve elected to use OpenAI’s gpt-5.4-nano model. This is super quick and super cheap but not super bright. It simply doesn’t have the level of world knowledge of the Claude chatbot.
So the solution – get Claude to ‘train’ gpt-5.4-nano. By that I mean the writing of an ever growing prompt. As soon as it misclassifies an email, I feed that email into Claude, and literally ask, “What do you think about this?”
The most recent one was one of those emails that gives you a 6 digit code to enter to get into a website, which was misclassified as Phishing. Claude said:
This is a legitimate two-factor authentication / identification code email from SmartVault, sent via SendGrid with valid SPF and aligned DKIM on my.smartvault.com. The AI is misreading genuinely correct security advice (“if you didn’t request this, reset your password”) as a phishing indicator, and flagging the standard 1×1 tracking pixel URL as suspicious.
And then added a couple of lines to the prompt we use. Another week or two, and we should be honing the system to a decent level of accuracy. All for a cost of around $600 per million emails. Probabl y a 1/10th of what it would cost with Claude, but it can still add up!
Why am I doing this? Well, in the past Phishing and Scam attempts were easy to catch. Mostly badly formatted and written by people for whom English wasn’t their first language. Nowadays, they use AI to create their emails, and you know what they say – takes one to catch one! So here we are!