Harshit Shirsat
Software Engineer
A migration used to be a team event. Someone on one branch would add a column to the shared dev database, and someone on another branch would find their API failing on startup because the schema no longer matched their entities.
AI coding agents made this worse. An agent running an end-to-end test needs to run a migration, create records and click through the app in a browser. It can’t wait for someone else’s migration to finish or ask before resetting a table, and it won’t come back later to clean up.
Now one setup command gives each git branch its own database that it can write to. Each agent task runs on its own branch, so each task also gets its own database. When the PR merges, that database is deleted. Across three timed runs, the setup command took about 35 seconds, and going from a fresh checkout to the app answering its first request took about 65 seconds, with access and cached dependencies already in place. The Neon API call that creates the database branch takes under a second at the median.
Why our dev database has every tenant schema
Kubera’s platform helps healthcare providers identify underpayment trends and recoverable dollars from their Payor agreements. In this context, a contracted rate is the amount a provider and a Payor agreed on for a service, and a fee schedule lists those rates. A payment variance is the gap between the contracted rate and what was actually paid. Finding those variances means matching claims (what was billed) to remittances (what was paid).
We run multi-tenant Postgres with one schema per customer, and every schema holds the same TypeORM entities behind a tRPC API and a React frontend. Migrations run once per tenant schema. A dev database with one sample schema only tests a migration against that one schema. Migration bugs tend to show up in a different tenant’s schema, where the data is in a state the sample never had.
Healthcare payment data is also messy, with reversals, inconsistently formatted codes and stays that span many days. Our seeds didn’t cover that variety or volume.
What we had before
We started with local Docker Postgres. A setup script started the Docker daemon, brought up Postgres with docker compose, ran migrations and loaded a seed script. It worked offline and cost nothing to run.
The seeds drifted. Each new entity or column needed someone to update the seed, and the seeded database slowly stopped looking like the schema the app actually ran against. Our setup notes said the seed only provided a baseline and that anything specific had to be inserted by hand.
Every new engineer and every cloud agent environment had to get Docker running first, then migrate and seed before writing a line of code. In some agent sandboxes that took workarounds just to start the daemon.
The heavier option was restoring a large database snapshot. That gave us realistic volume, but when we ran the full restore as a scheduled GitHub Actions job, each run took 17 to 30 minutes, about 22 at the median. Every copy cost its full size in disk, nothing ever deleted it, and each copy was out of date as soon as it finished loading.
What agents need from a dev database
An agent running end-to-end tests needs:
a database it can write to, migrate and reset without affecting anyone else
data with enough variety and volume to make its tests mean something
no human step, so no request to file and no turn to wait for
creation fast enough to do once per task
cleanup that happens without anyone remembering
all of it scriptable over an API
Local Docker gave us isolation but not realism. Snapshot restores gave us realism but missed speed and cleanup. Our agent playbook is now the same as a developer’s: create the branch database, start the API and frontend, drive the app in a real browser, then tear the database down.
The options we weighed
Better seed data was the cheapest option. It would have kept offline development and avoided a vendor dependency, but they still wouldn’t have matched the variety or volume we needed, and keeping them current meant ongoing maintenance.
Restoring a snapshot per developer would have kept the restore time, storage and cleanup problems we already had, multiplied by the number of developers and agents.
Copy-on-write branching on Neon avoids both. A Neon branch starts out sharing its parent’s data and only stores pages that change after the fork, so creating one copies nothing up front. Branches are created and deleted through a REST API, which meant we could script the whole lifecycle with curl and jq and skip installing a CLI.
In exchange, you need a network connection to develop, so fully offline work is gone, and our dev workflow is tied to one vendor’s branching model. Moving off it would mean rebuilding this tooling.
How the one command works
The setup script turns the current git branch name into a valid database branch name and asks the Neon API for a new branch off a shared parent. A minimal version of the idea looks like this:
Each branch gets its own Postgres compute, which we set to suspend after a few idle minutes, so a branch nobody is using costs storage for its changes and nothing for compute. Our script then reads back the connection string, runs pending migrations on every tenant schema and writes the environment files for the API and the frontend:
Every tenant schema comes along in the fork. Switching which tenant you’re working in rewrites one environment variable and restarts the API, with no new database.
We also create a second, matching branch in a separate Neon project for claims-import scratch work. Bulk import experiments are large and messy, and keeping them in their own project means a bad import can’t fill up or slow down the database you’re testing the app against.
When two sessions want the same database
The database is named after the git branch, so the git branch is the unit of isolation. Running setup again on the same branch reuses the existing database instead of creating a second one, so the command is safe to rerun after a crash or a restart.
The same rule decides collisions. Two sessions on one git branch share a database, whether they’re two people or two agents. Branch names are cleaned to characters the API accepts and cut to a fixed length, so two git branch names that differ only in punctuation map to the same database. Our branches are named per ticket or task, so in practice the shared case is two agents on one branch, which is the gap we want to close next.
Migrations run when the database is created. An older branch doesn’t pick up migrations merged to main later. Refreshing it deletes the database, forks again and runs the migrations in your checkout.
When two sessions want the same database
The database is named after the git branch, so the git branch is the unit of isolation. Running setup again on the same branch reuses the existing database instead of creating a second one, so the command is safe to rerun after a crash or a restart.
The same rule decides collisions. Two sessions on one git branch share a database, whether they’re two people or two agents. Branch names are cleaned to characters the API accepts and cut to a fixed length, so two git branch names that differ only in punctuation map to the same database. Our branches are named per ticket or task, so in practice the shared case is two agents on one branch, which is the gap we want to close next.
Migrations run when the database is created. An older branch doesn’t pick up migrations merged to main later. Refreshing it deletes the database, forks again and runs the migrations in your checkout.
Keeping the shared parent safe
Every branch forks from one shared parent, and day-to-day work happens on those forks, not on the parent.
Long-lived shared environments are branches too. Staging resets from the parent on a schedule. Developers’ own long-lived environments reset only when their owner asks, so unmerged migrations don’t disappear overnight. Before a reset, the job checks that it has everything it needs to finish. When the environment has branches of its own, the job first moves its old state onto a named backup branch, so the reset can be undone.
Deleting branches automatically
Since late September we’ve created anywhere from about 60 to over 500 database branches a week. Without automatic cleanup, hundreds of them would be sitting around.
A GitHub Action deletes a branch’s database when its PR merges. A scheduled job deletes branches that haven’t been active for seven days, skipping a short list of long-lived ones. For manual control, a teardown command deletes your branch early and a refresh command throws it away and forks again.
What the numbers look like
Across 1,418 branch creations since late September, the create call took 762 milliseconds at the median and about 1.5 seconds at the 90th percentile. The first query after a branch’s compute has suspended waits for it to start again, which took about 1.8 seconds at the median and about 4 seconds at the 90th percentile.
We timed the setup three times on one machine. The setup command, which creates both branches and runs migrations on every tenant schema, took 34 to 36 seconds. Going from installing dependencies on a fresh checkout to the app answering its first request against its own branch database took 65 to 68 seconds.
With about 40 live branches, full copies would need roughly 30 times the storage those branches actually use, because each one only stores the pages it has changed.
Before | After | |
Setup | Docker daemon, compose, migrations, seed or restore | One command |
Time to a working DB | Docker setup and seeding, or a 17-30 minute restore | About 35 seconds, including migrations on every tenant |
New checkout to a running app | Docker, compose, migrations, seed | About 65 seconds once access is set up |
Isolation | Shared, or one copy per laptop | One database per git branch |
Cleanup | Manual | On PR merge, or after 7 days inactive |
What’s next
Faster, cheaper refreshes for the long-lived shared environments that still reset on a schedule.
Better visibility into how many branches exist at once and how much storage each one adds.
Ephemeral databases per agent session, created when a session starts and deleted when it ends, so parallel agents on the same git branch each get their own.
From a coordination problem to a disposable resource
Database setup used to be a coordination problem. Someone was migrating the shared database, someone’s seed was out of date, and someone had to clean up last week’s restore. Every step assumed a person who could ask, wait and remember.
Agents can’t do any of that, so the database had to stop depending on it. The command that starts the work creates the database, the git branch scopes it, and the merge that finishes the work deletes it. As teams hand more work to agents, we expect more dev infrastructure to need the same shape: anything an agent depends on should be something it can create, break and throw away without asking anyone.