Night run

One night, pass by pass.

The same sequence runs every night, one process per project, each pass writing rows the next one reads. Here it is in order, with what every pass takes in, what it leaves behind, and when it fires.

The dispatcher

Which projects run tonight is a database question.

Cron calls one dispatcher with a pass name. It asks the database which projects qualify, then fans out a process for each, four at a time, with a fifteen minute ceiling on any one. No project name is hardcoded, so joining the pipeline is a flag rather than a deploy.

Each pass asks a different question. Content passes want a project flagged for them. Listening passes want an active topic. Answer engine tracking wants its own config row, since it costs a call per question per engine per day. The repost scout wants two facts at once: a client who posts socially, and a topic already producing daily snapshots.

Collection · 04:00 to 05:10 UTC

Collecting, from four in the morning

Weekly lines fire only on their day. A source a project has not configured is skipped and logged, and the pass carries on.

01

vault_pullevery hour, at :45

Lists the project research folder in storage and downloads only what changed, keyed on each file's updated_at. No model runs here. Everything later in the night reads this local copy.

02

market04:00 UTC, daily

An end-of-day price and volume pull for any tracked topic carrying a ticker, from a public feed needing no key, so a chart of chatter gets a price line underneath. A failed fetch leaves the previous rows and the night continues.

03

volume04:15 UTC, Mondays

Absolute monthly search volume for terms that already have a trend curve, batched into one task per project. The source restates the last twelve months on every call, so a skipped week is recoverable.

04

sentiment04:30 UTC, daily

Pulls recent items from whichever sources a topic configured (Reddit, X, search results, Yelp, Google Maps, Tripadvisor), scores each with a model, and appends them. The rollup the dashboard charts gets rebuilt from those rows. Two days back, a hundred items, since collection is billed per item fetched.

05

brand_posts04:40 UTC, Mondays

Pulls the brand's own Instagram and TikTok history, one row per post, updated in place, so a report can mark owned content against the sentiment line. Owner replies already sitting in review payloads get lifted out too, at no provider cost.

06

viral04:50 UTC, daily

The same posts, filed differently. Rather than overwrite a post with its latest size, this appends one row per post per day. Day one seeds the series, velocity shows up on day two, acceleration on day three.

07

aeo05:10 UTC, daily

Puts the project's configured questions to the answer engines and records whether the brand was cited, by alias and domain matching rather than a model. A skipped day is gone for good: no engine can be asked what it would have said last March.

Writing · 05:20 to 06:00 UTC

Then the night has to produce something.

Everything past this point reads what the collectors wrote minutes earlier, which is why the clock times are what they are. The scout runs after the snapshot it ranks, and ahead of the pass that would otherwise draft over the top of it.

08

synopsis05:00 UTC, Mondays

One narrative per brand topic: a verdict, chapters, the events behind them, recommendations. It runs after Monday's collectors, so the week it describes is in the database before it starts writing.

09

repost_scout05:20 UTC, daily

Reads the viral snapshots through the velocity function and turns the top movers into repost cards with a suggested caption. It buys nothing from a provider, and sits ahead of the drafting pass so the cheapest content of the night counts toward the social target first.

10

imagine_actions05:30 UTC, daily

The drafting pass. It reads the cadence rows, works out how far behind the period is, and writes only the shortfall, capped at two content types a night. A piece citing a number absent from the source it was shown gets dropped. What survives lands as a suggested card with its goal, its documents and its concepts attached.

11

learnings05:00 UTC, Sundays

Reads the week of board activity: what was accepted, what was rejected and for what stated reason, and the edits people made to drafts. It writes one short note out of that. A week nobody touched produces no note.

12

email_weekly06:00 UTC, Sundays

Builds the digest and sends it, only to projects whose configured send hour matches the run. Trends are ordered by the size of the week-over-week move, so a small term climbing outranks a large one sliding.

One night, logged

The clock, in the order it fires

Every line is a cron entry, in UTC. The weekly and hourly passes are left out so the daily spine stays readable; the hourly two (vault pull at :45, capture triage at :25) run all day.

Which of these fires for you depends on what your project has configured.

benchmates · night run
04:00
market price overlay written for tracked tickers
04:30
sentiment topics swept, items scored, observations appended
04:50
viral one row per post per day, velocity and acceleration
05:10
aeo answer engines asked, citations recorded
05:20
repost_scout top movers turned into repost cards
05:30
imagine_actions cadence shortfall drafted, evidence attached
06:00
email_weekly digest sent to the projects on this hour
now
your turn
Starvation

What does a starved pass do?

The failure worth worrying about is the one that looks like success. A mistyped handle returns an empty dataset with no error attached. A provider over its monthly cap returns a rejection that gets swallowed. Both read as a healthy zero, and a healthy zero hid three real faults here for days.

So the rule is blunt. A pass configured to fetch something that fetched nothing exits with an error code. Green has to mean it worked.

PassThe situationWhat it does about it
viral, brand_postsHandles configured, zero rows writtenExits non-zero. The dispatcher records it as failed and it goes into the alert.
triage_capturesCaptures waiting, nothing classified and nothing discardedExits non-zero, so an empty hour and a broken hour look different.
repost_scoutA second consecutive night with no candidate, on a project set up for itExits non-zero. One flat night is legitimate, so it takes two.
learningsReviews sitting in the window and no note writtenExits non-zero. Activity producing nothing is a fault.
imagine_actionsClean exit, zero cards insertedRecorded as empty rather than ok, unless it printed a line saying zero was correct.
typed graphMore than half the documents fail extractionMarked failed despite the clean exit, so the graph cannot rot quietly.
any passStill running after 900 secondsKilled, and logged as a timeout carrying whatever it printed first.
Every run writes a row either way: the pass, the project, the duration, a tail of its output. Anything in the bad set produces one summary email for the whole pass, and each pass checks in with a monitor on its own schedule, which is how a run that never happened becomes visible. Tails are redacted before they are truncated, since a tail taken first can slice a credential in half.
The morning

All of it runs while the office is closed, and none of it publishes.

Each pass ends in a row in a table or a card in a queue. Cards arrive marked suggested and wait there. Approve one and it lands as a draft in the tool you publish from, research still attached, and the publish button stays with a person. Where a draft can land is a separate page.

An empty dugout bench with a folded towel and a ballspot-bench-empty.png1280 × 832

If you want the shorter version, the homepage tells it in four steps.

Request access