Clean Logs. Dirty Bugs.

🛠️ Setup Context

My Django test runner spins up isolated environments for each test flow:

• One app instance per test flow
• Each on a unique port
• Each wired to a dedicated database

This allows me to parallelize tests cleanly without shared state.

Here’s what that setup is supposed to look like:

             ┌───────────────────┐     ┌──────────────────┐
        -->  │ App :1 Port 8001  │ --> │ DB: noCRUD_p8001 │
       /     ├───────────────────┤     ├──────────────────┤
Runner --->  │ App :2 Port 8002  │ --> │ DB: noCRUD_p8002 │
       \     ├───────────────────┤     ├──────────────────┤
        -->  │ App :3 Port 8003  │ --> │ DB: noCRUD_p8003 │
             └───────────────────┘     └──────────────────┘

1. How It Started...

It was late Thursday afternoon. I had just wrapped up parallelization for my Django test runner— each test flow now got its own isolated DB instance.

I wanted to finish the parallelization effort before I had to leave for the day, and I had a hard stop because I had to be on the road to make a family engagement.

The setup commands—makemigrations and migrate—were working fine, but I didn’t want them cluttering the console. So I silenced them, and I was done - with not a minute to spare.

---

2. A Week Later: "Why Isn’t This Working?"

About a week later, I plugged a new backend into the test runner.

The runner spun up new databases for each flow. The tests ran. Everything looked mostly right.

But something was off, weird errors. Flows that I knew should be passing were all of a sudden failing.

After some digging, I realized: The migrations hadn’t run on my target DB

What? Ok go back to serial mode, try just normal spin up of the app... the usual make migrations and migrate...

Then I saw it... makemigrations wasn’t being called at all.

def provision_env_for_flow(flow_name):
    port = find_open_port()
    db_name = f"nocrud_p{port}_{flow_name}"

    os.environ["DB_NAME"] = db_name
    os.environ["APP_PORT"] = str(port)

    db_client = DBClient(admin_mode=True)
    db_client.createDB(db_name)

    # Run Migrations
    subprocess.run(
        ["python", "manage.py", "migrate"],
        cwd=APP_DIR,
        check=True,
        stdout=subprocess.PIPE,
        stderr=subprocess.STDOUT,
        env={**os.environ},
    )

    # start django
    backend_proc = start_backend_subprocess(port)
    ...

How did that happen?

I’m 100% sure I saw both blocks—makemigrations and migrate—in the code originally. But in my rush to clean up the output, I must’ve thought they were duplicate logic and deleted one. (Due to normal dev WIP messiness - e.g. commenting some blocks out when trying out different options)

So apparently I had kept migrate, removed makemigrations, and silenced the output.

No errors. No warnings.

Just... a runner that silently skipped the most critical part: schema creation.

Ok but then why were the runners partially succeeding?

Well for the serial version of the runner I was running the migrations on startup against the default DB, and in the app my settings.py used POSTGRES_NAME as the database name environment variable, but I had refactored the runner to use DB_NAME instead.

So the apps were running on different ports, but instead of pointing to the unique DB's that were created for them, they were all pointing to the default postgres DB!

Expected

             ┌───────────────────┐     ┌──────────────────┐
        -->  │ App :1 Port 8001  │ --> │ DB: noCRUD_p8001 │
       /     ├───────────────────┤     ├──────────────────┤
Runner --->  │ App :2 Port 8002  │ --> │ DB: noCRUD_p8002 │
       \     ├───────────────────┤     ├──────────────────┤
        -->  │ App :3 Port 8003  │ --> │ DB: noCRUD_p8003 │
             └───────────────────┘     └──────────────────┘

Actual

             ┌───────────────────┐
        -->  │ App :1 Port 8001  │
       /     ├───────────────────┤  \     ┌───────────────┐
Runner --->  │ App :2 Port 8002  │   |--> │ DB: postgres  |
       \     ├───────────────────┤  /     └───────────────┘
        -->  │ App :3 Port 8003  │
             └───────────────────┘

"One runner to spawn them all, one DB to ruin them..."

---

3. The Fix

I created a wrapper function for these commands that ensures they succeed:

def run_mgmt_command_quietly(args, cwd, env):
    result = subprocess.run(
        ["python", "manage.py"] + args,
        cwd=str(cwd),
        env=env,
        stdout=subprocess.DEVNULL,
        stderr=subprocess.PIPE,
        text=True,
    )
    if result.returncode != 0:
        raise RuntimeError(
            f"Command failed: {' '.join(args)}\nStderr:\n{result.stderr.strip()}"
        )

This also made the caller cleaner:

def provision_env_for_flow(flow_name):
    ...
   # Make migrations and migrate
    run_mgmt_command_quietly(args=["makemigrations"], cwd=APP_DIR, env=os.environ)
    print(f"✅ Migration created for DB: {db_name}")

    run_mgmt_command_quietly(args=["migrate"], cwd=APP_DIR, env=os.environ)
    print(f"✅ Migration succeeded for DB: {db_name}")

    # Ensure the db we just created is the same as what settings.py will use
    db_match_check(db_name)
    ...

And I surfaced DB environment mismatches loudly:

def db_match_check(db_created_by_runner):
    result = subprocess.run(
        [
            "python",
            "manage.py",
            "shell",
            "-c",
            "from django.db import connection; print(connection.settings_dict['NAME'])",
        ],
        cwd=str(APP_DIR),
        env={**os.environ},
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        text=True,
    )

    if result.returncode != 0:
        raise RuntimeError(f"Failed to query current DB:\n{result.stderr.strip()}")

    db_that_backend_is_using = result.stdout.strip().splitlines()[-1]

    if db_that_backend_is_using != db_created_by_runner:
        raise RuntimeError(
            f"""[DB MISMATCH] Django is using DB '{db_that_backend_is_using}', expected to use the db created by the runner '{db_created_by_runner}'.
            Please ensure that provision_env_for_flow and settings.py use the same environment variable for the database name.
            Settings.py MUST use an environment variable because the database name is programmatically generated"""
        )
    else:
        print(f"✅ Django is using expected DB: {db_that_backend_is_using}")

4. What I Learned

Cleaning up logs can hide critical feedback.

But the real danger? Silent assumptions.

I assumed both commands were still running. I assumed nothing had changed.

During this process I also found that an auto-cleanup of the DBs isn't necessarily always desired... I couldnt even check that the migrations were being run since my DB was dropped at the end! So I added a persist_db option for deeper test debugging, and defaulted it to true for flows that fail.

---

5. What I Shipped After the Fix

• More resilient runner
• Clear failures for setup/config issues
• Optional persisted DBs for debugging
• Still clean output—but never at the cost of visibility

---

6. The Takeaway

Clean output is great. But silence during setup is a lie waiting to happen. Fail loudly. Debug fast. Ship safer.

---

🚀 Want the ultimate E2E API test runner?

Check out noCRUD on GitHub — the scaffold is here, the options are endless:

• ✅ Test standard CRUD endpoints
• ✅ Verify complex business logic
• ✅ Simulate multi-user workflows
• ✅ Run stress/load tests
• ❌ Maybe don't DDoS your competitors 😅

Built for realism. Designed for chaos.
Use it. Break things. Learn fast.