# Rebuilding the data

How national-data releases are built and published from the separate pipeline repository.

The customer-facing application does not contain or run the national Python pipeline. National data is built by the independent `LookAlike-data-pipeline` repository and published as an immutable release.

## Repository boundary

The application repository owns the React interface, browser scoring, Cloudflare Worker, authentication, and product APIs. The data-pipeline repository owns source retrieval, feature engineering, curated-brand validation, PMTiles generation, release validation, and R2 publication.

The application repository contains a compatibility copy of the current data under `web/public/data` so ordinary local development remains self-contained, and the scorer-consistency tests use it as their fixture. That copy is not the production publishing mechanism, and it does not update when a new release is published. Refreshing it is a deliberate manual step. `web/public/data/RELEASE.md` records which release it was taken from and where it currently diverges from that release.

## Build a release

From the pipeline repository:

```
python3.13 -m venv .venv.venv/bin/pip install -r pipeline/requirements.txtLOOKALIKE_RELEASE_ID=2026-07-11-01 .venv/bin/python pipeline/run_all.py
```

The pipeline writes only inside its own repository:

```
dist/releases/2026-07-11-01/
```

Raw downloads and intermediate Parquet files remain cached under that repository’s `data/` directory.

## Publish and activate

Publishing and activation are separate operations:

```
./scripts/publish-release.sh --remote 2026-07-11-01./scripts/activate-release.sh --remote 2026-07-11-01
```

Publishing uploads objects under `releases/2026-07-11-01/`. Activation changes the small `current.json` pointer. If a release fails validation or upload verification, do not activate it; the application continues using the previous release.

## Refresh cadence

-   Refresh curated OpenStreetMap brand footprints weekly.
-   Adopt new Overture retail-context releases monthly or deliberately.
-   Update ACS, TIGER, and metropolitan definitions when their official annual releases are adopted.

A weekly brand refresh should reuse cached national features and geometry. It does not need to redownload or recompute every source.

## Consistency guarantee

The application test suite reads the packaged release binaries and verifies that the browser TypeScript scorer reproduces the Python-generated similarity values for every curated brand. A scoring change is incomplete until the pipeline output and application consistency test agree.

The product remains a descriptive neighborhood-resemblance tool. A new data release does not convert its results into revenue, sales, or store-success predictions.

Canonical URL: https://docs.lookalikeiq.com/operate/rebuild-data/