Case study · a product I built and run

An automated news desk that checks its own work

A news site and weekday newsletter that follows one beat. I own it, I built all of it, and I run it every day. Ask me on a call and I will walk you through it live.

82% → 97%accuracy of the story filter, measured on a hand-labelled test set

34.8h → 4.0hmedian age of a story when it reaches the site

4 of 16wrong headlines caught by the automated checker on its first run

The job

Following one news beat by hand means reading more than a hundred sources a day, spotting when ten outlets are covering the same event, deciding what matters, and writing it up without errors. This system does that work automatically, every day.

What I built

Collection. It gathers stories from 117 feeds through the day. Every feed was measured before it was added, and feeds that would only add noise were rejected.
Sorting. It removes duplicates, groups stories about the same event, sorts each story into 26 categories and ranks it by significance.
Publishing. It sends a newsletter every weekday morning and posts to social accounts.

How it checks itself

A second reader for every headline. Before and after each newsletter goes out, a second AI compares every headline with its source. On its first run it caught 4 wrong headlines out of 16. I traced the cause to a rule that forced headlines to be too long, and fixed the rule.
A judge for close calls. When two stories look alike but not identical, a separate AI decides whether they are the same story. I checked its decisions by hand on 50 real pairs before turning it on.
One failure cannot stop the rest. Each source and each step is guarded, so a broken feed does not take the site down.

Measured results

Story filter accuracy went from 82% to 97% on a hand-labelled test set.
Median story age on arrival went from 34.8 hours to 4.0 hours.
Sources grew from 36 to 117 feeds, and stories collected per day from 78 to 195.
One slow database query went from 21.9 milliseconds to 0.071.
Unwanted bot traffic went from about 6,850 requests an hour to about 2.

Running it

I have operated it since March 2026. When something breaks, I find the cause and fix it so it cannot happen the same way again. One example: a bill for 350 GB of database traffic turned out to come from a single query. I rewrote it and cut the rows it fetched by about 82%.

Why this matters for your business

An automation in your business needs the same things: checks that catch mistakes before anyone acts on them, results that are measured, and someone who keeps it running after launch.