How I didn't vibe code a mail inspector app

Intro

I wanted a simple way to check a domain's mail configuration: DNS records, SMTP connectivity, TLS, and email authentication, all in one place. That's how Mail Inspector started. It's an open-source Laravel application that checks a domain's mail setup, shows what it finds, and suggests possible fixes. You can also try it out here!

I used AI to build a large part of it, but I didn't want to just give it a prompt and accept whatever came out. I wanted the application to be maintainable, properly tested, and something I could actually host myself.

For this project, I used Codex together with obra's Superpowers workflow. I also used Codex's Security plugin to look for potential security issues.

Tools used

These are the main tools I used:

• Codex to help write, debug, and improve the application.
• Codex Security plugin to scan the code for potential security issues.
• obra's Superpowers to give the development process some structure, from figuring out requirements to planning, implementing, and testing.
• GitHub and GitHub Actions for version control, running tests, and publishing container images.

What I liked about Superpowers is that it doesn't just jump straight into writing code. It starts by figuring out what you're trying to build, makes a plan, and breaks the work into smaller tasks.

That worked pretty well with Codex. Instead of asking it to build the entire application in one go, I could work through smaller pieces and review what it produced.

Of course, that doesn't mean everything it generated was correct. There were still things that needed fixing, and that's exactly why I wanted tests and a proper workflow in the first place.

The approach

I wanted to use Laravel, and I wanted people to be able to host the application themselves. Docker Compose seemed like a good fit for that.

But a mail inspector is a bit different from a typical web application. It needs to resolve DNS records, connect to mail servers over SMTP, check TLS certificates, and deal with all kinds of responses from servers that you have no control over.

I also didn't want to figure out the deployment side of things only after the application was finished.

The setup consists of the Laravel application, Nginx, background workers, a scheduler, PostgreSQL, Redis, and an Unbound DNS resolver. Docker Compose brings everything together, while the internal services don't need to be exposed to the public internet.

I also set up GitHub Actions to test the application and its container setup before publishing images to GitHub Container Registry.

It's easy to get an application working locally and assume it'll work the same way in Docker. But things like queues, DNS resolution, scheduled jobs, and container startup can introduce problems you wouldn't notice by just opening the application in your browser.

Having those checks in place helped catch issues without having to manually test everything after each change.

The result

Mail Inspector checks a domain's mail configuration and generates a report with findings and suggested fixes.

It discovers mail servers through DNS, checks SMTP connectivity and TLS, and looks at email authentication records like SPF, DKIM, and DMARC.

There are some limitations to what you can actually verify, though.

For example, having SPF, DKIM, and DMARC records doesn't mean that emails sent from the domain will automatically pass authentication. Those records only tell you what's configured. DKIM can also require a selector that isn't always possible to discover automatically.

The same goes for SMTP. Just because a mail server accepts a recipient doesn't mean the message will actually get delivered. Mail Inspector doesn't send an actual email body, so it can't test the entire delivery process.

I wanted the reports to make these differences clear. If something couldn't be verified, the application shouldn't just mark it as working or broken.

Where the scanner runs

One of the things I ran into wasn't really related to Laravel or AI. It was my internet connection.

My home internet provider blocks outgoing TCP port 25, which is exactly what the scanner needs to connect to other mail servers.

That meant I couldn't just run all the SMTP checks from my home server. They would fail because of my own network restrictions, not necessarily because something was wrong with the mail server being tested.

I ended up using an external virtual machine for the scanner traffic, while keeping the web application self-hosted.

There are some other things to consider as well. A connection timeout doesn't always mean the destination mail server is down. It could be a firewall, a routing issue, or something on the scanner's side. And if you want to test IPv6, the scanner also needs proper IPv6 connectivity.

This was probably one of the better examples of why the infrastructure matters just as much as the code. You can write a perfectly fine SMTP checker, but if the network it runs on can't reach the destination, the results won't be very useful.

What I learned

A few things I took away from building this project:

• AI still needs a process. Codex can write a lot of code quickly, but that doesn't mean the result is automatically good. Having a plan, smaller tasks, and reviewing the changes made a big difference. Superpowers was useful for that.

• Working code doesn't mean secure code. I used Codex Security to look for potential vulnerabilities, but that doesn't replace reviewing the code yourself. AI can introduce security issues just like any developer can.

• The environment matters. SMTP checks don't tell you much if the machine running them can't connect to port 25. Where the scanner runs actually affects the results.

• Docker isn't just about the application container. You also have to deal with workers, scheduled jobs, databases, queues, DNS, and making sure everything starts and recovers properly.

• Tests are worth the effort. Especially for things like SMTP, DNS, TLS, and IPv6. There are plenty of cases where something looks fine locally but behaves differently in another environment.

• Not every check has a clear pass or fail. A timeout, missing response, or unexpected SMTP reply doesn't always tell you exactly what's wrong. I think it's better to show what happened than pretend the scanner knows more than it does.

Conclusion

Mail Inspector started as a fairly simple idea: build something that makes it easier to check a domain's mail configuration without having to use several different tools.

Using Codex definitely made development faster, but it didn't mean I could skip the usual software engineering work. I still had to think about the architecture, deployment, testing, security, and all the little things that come up when you actually try to run an application.

That's also why I wouldn't really call this vibe coding. I used AI to write a lot of the code, but the process around it was still important. And I think that's a much better way to use these tools.

The project is open source, so feel free to check it out on GitHub. Feedback, bug reports, and suggestions are welcome!

You can also try the hosted version at mail-inspector.apps.ward.nl.