Building a SMS fraud checker
Last April 2025, I had the opportunity to present an AI-powered SMS fraud checker at the Bangko Sentral ng Pilipinas for the Youth Financial Inclusion hackathon. To be honest, this project actually underwent a lot of changes and was originally supposed to be a submission for Apart Research's Sprint. The main product would've been the "AI safety layer" where it could be easily integrated to other AI tools or apps for post-deployment monitoring and output filtering.
Using that same backend, we tweaked the use case to fraud detection. This project then became an exploration on an Large Language Model's (LLM) capabilities on fraud detection. I'll give a walkthrough of the technical stack and thought process of this project.
What were the tools I used?
TLDR: Cloudflare Pages, Cloudflare Workers, Lovable, Baserow
I recently have been utilizing Cloudflare's products to test and prototype my ideas. I primarily utilized Cloudflare pages and workers for this project like most of my projects moving forward. They offer a generous free tier and an affordable paid plan to host serverless functions and AI inference through Cloudflare AI workers.
To create the frontend of the application, I went ahead used Lovable.dev to quickly prototype the input and evaluation. The web application will stream the output from the API created and reformat it's output.
Prompt for fraud evaluation
The function I used contained the following components:
- The system message asks for it to evaluate based on the different possible social engineering scam tactics
- A classification of the message whether if the message is harmful or not
- An explanation of why the LLM came to the conclusion of the assigned classification
For this project I used Meta's llama-3.2-3b-instruct model which was freely available and easy to integrate if you build using Cloudflare and Hono.
Full Code Here
Automating data collection
Another feature with this application is the ability to submit for data collection. The idea here is that we could potentially build a public database where we can train AI models to detect trickier scam messages. We used Baserow's webhooks to interact with the application and store in a table.
What could I have done differently?
The whole project was a mere proof of concept on how we could use LLMs for fraud detection but just like how the judges point it out it was a useful tool but the user journey can come off as unusable. Imagine receiving a potentially harmful message and you had to open this app then input to check and so on and on. See? the user journey isn't polished. To be kinder to myself, I did create this application to be a proof of concept for a more useful application.
Another thing I wished I had capability to do is to apply patterns and techniques of fraud detection thatn relying an LLM's capabilities. Applying for example a confusion matrix would have been an interesting factor to add (think of messages that looks like a scam but are not a scam). There are a lot of things that could be expanded on from this project and I'm excited on what I'm going to build next!