Sixteen weekly AI reviews that raise tasks with a predicted metric and open pull requests

A one-person product has as many moving parts as a bigger company (landing page, signup, onboarding, matching, emails, retention, costs) but nobody watching each of them week to week, so problems and missed improvements go unnoticed.

The founder split the product into 16 areas (for example landing and SEO, signup and onboarding, first session, retention, revenue, cost) and gave each the same weekly loop. 1) Collect: a plain Python collector pulls that area's numbers from the product database, PostHog, Search Console and other sources and stores them as datasets a dashboard can show. 2) Review: an AI agent reads the dashboard and last week's report and writes this week's report: what moved, a guess at why, and what looks broken. 3) Raise tasks: each problem becomes a task with a kind (automatic fix, fix needing approval, improvement, big idea) and a prediction naming the metric, its baseline and the expected value. 4) Fix: approved fixes ship as pull requests; each area has a fix mode (off, pull requests only, or auto-merge for safe fixes), and almost all areas are on pull requests only. 5) Measure: the next week's review scores each shipped task against its own prediction as worked, no effect, hurt or too early. The founder reviews everything once a week in a single inbox of items that need a human decision. Advice from the author: make every task predict a number, score changes a week after they ship rather than on the day, earn trust for automatic merging area by area, and keep the human approval step small and regular.

After the first week the author reports 202 tasks raised across the 16 reviews, 74 shipped, 51 pull requests opened and 49 merged, and 66 tasks still waiting for review. Examples given: a slow dashboard collector replaced by a shared daily summary table, email click counts inflated by link-scanning bots, a daily digest still being built for people who never opened it, and wrongly tagged remote jobs. The author states plainly that none of the shipped tasks has a measured result yet, so whether the loop improves the business is not known.

Firsthand and self-reported after only one week; all numbers on the screenshots are hidden, and no task has a measured effect yet. The AI model and agent tooling are not named in this post, and the setup assumes a developer who can write data collectors and review pull requests. The author notes the new bottleneck is their own review time and that the agents find what is broken better than they judge what matters. The post ends with a link to the author's product (a job search service), and the author is not strictly solo ('mostly on my own').

A product with its own database and analytics (the author names PostHog and Search Console), the ability to write collectors in Python, a code repository that accepts pull requests, and a weekly slot to review proposed tasks.

Start with one area and one weekly review. Require every task to name a metric, a baseline and an expected value before it can be approved, and keep fixes on 'pull requests only' until the area has earned trust.

Here is this week's dashboard data for [area] and last week's report. Write this week's report: what moved, your best guess why, and what looks broken. Then list up to five tasks. For each give a kind (auto-fix, needs approval, improvement, big idea), the metric it should move, the current baseline, the value you expect after one week, and how we will know it worked. Finally score last week's shipped tasks against their predictions as worked, no effect, hurt or too early.

• DEV Community (Abhishek, landed) ↗ Firsthand write-up, published October 27, 2026

Source published: October 27, 2026. Last reviewed here: October 27, 2026. Spot a mistake? Tell us.