Architecture & consistency

LookAlike is built as two independent repositories joined by a versioned data contract: LookAlike-data-pipeline downloads and validates open sources and publishes immutable releases, while LookAlike contains the React application, TypeScript Worker API, and browser scoring engine.

Overall structure

Input data (Census ACS, TIGER, OSM, Overture Maps)
         ↓
[Python pipeline: 8 stages]
         ↓
Static web assets (binaries, tiles, JSON)
         ↓
[React app: loads assets, scores live, re-weights interactively]
         ↓
Browser: choropleth map + detail panels + rankings

The Python pipeline (8 stages)

Run LOOKALIKE_RELEASE_ID=<id> python pipeline/run_all.py from the data-pipeline repository. Each stage caches raw downloads and intermediate Parquet files in that repository. Here is what each produces:

1. fetch_acs → acs_cells.parquet

   • US Census ACS 2023 5-year Summary File (keyless bulk .dat download).
   • 83,762 CONUS census tracts × 33 ACS cells (income, education, demographics, housing, employment, commute).

2. fetch_geometry → tract_geo.parquet, tract_polys.parquet

   • US Census TIGER/Line 2024 tract shapefiles.
   • 83,777 tract centroids (lat/lon) and simplified polygons (for vector tiles).
   • Includes land area per tract (for density calculations).

3. fetch_stores → stores.parquet

   • Brand store locations from OpenStreetMap via the Overpass API (brand:wikidata, exact name match).
   • 31,702 store points across 11 brands after the 150 m grid dedup, before 1 km US containment.

4. fetch_poi → used by build_features

   • Overture Maps Places (public GeoParquet on S3, queried with DuckDB, release 2026-06-17.0).
   • ~15.4M US places; aggregated to per-km² densities for 5 POI categories: all POI, grocery stores, restaurants, cafés, shopping/apparel.

5. build_features → tract_features.parquet

   • Derive the model feature table: 83,008 tracts × 21 model features (16 ACS-derived plus 5 Overture retail densities) + 3 context fields.
   • The 16 ACS-derived features: median income, per-capita income, median home value, median rent, poverty rate, bachelor’s degree %, population density, households without a vehicle %, walk/bike/transit commute %, renter housing %, median age, average household size, households with children %, White non-Hispanic %, Hispanic/Latino %, Asian non-Hispanic %.
   • The 5 Overture retail densities (added here from the POI aggregate): all points of interest, grocery stores, restaurants, cafés, and shopping per km².
   • Context (shown in UI but not scored): total population, Black non-Hispanic %, unemployment rate.

6. build_scores → scores.parquet, meta.json, candidates/*.json

   • Build per-brand archetypes (average tract features for each brand’s occupied tracts, one per tract).
   • Compute similarity percentiles (0–100, within-brand rank) for all 83,008 tracts against each brand.
   • Extract top-N candidate tracts per brand (full feature detail), written under data/out for the offline consistency-test fixture — these are not shipped in the web bundle; the browser rebuilds the ranked list live from the binaries.
   • Calculate feature-level drivers (“why it matches”), the per-feature variance floor sigma_eff = sqrt(sigma_brand² + (λ · nat_std)²), and coherence (k-fold footprint recovery with county-blocked folds).
   • Write profiles, scaling metadata, per-brand occupied-tract distance quantiles (dist_ref), and the low-ACS-reliability flag.

7. export_release → dist/releases/<release-id>/*

   • Package everything the application needs as one versioned release.
   • (Detailed below.)

8. fetch_cbsa → metros.json, cbsa.u16

   • US Census CBSA (Metropolitan Statistical Area) delineations, 2023.
   • 383 metros in the CONUS. Maps each tract to its CBSA code.
   • Optional step: if it fails, the app still runs (metro filtering is a nice-to-have).

Published release assets

The pipeline exports everything needed by the browser under one immutable R2 release prefix, and the hosted application reads it from there. The application repository retains a compatibility copy under web/public/data/ for self-contained local development and as the consistency-test fixture; see web/public/data/RELEASE.md for which release it came from.

JSON metadata

• meta.json: brand profiles (name, id, store count, occupied tracts, archetype, coherence, per-brand occupied-tract distance quantiles dist_ref), feature group definitions, national feature min/max/mean.
• manifest.json: array counts, dimensions, data types, endianness note (little-endian for native x86/ARM).
• metros.json: array of metro objects (CBSA code, name, tract count, bounding box, center latitude/longitude).

Binary tract data (row-major, little-endian)

• tracts/geoids.json: array of 83,008 GEOIDs (row order for all binaries).
• tracts/labels.json: human-readable labels (parallel to geoids).
• tracts/features.f32: Float32, 83,008 rows × 21 features (16 ACS-derived + 5 Overture retail densities). Raw model-feature values. The file is 83,008 × 21 × 4 = 6,972,672 bytes; a reader that assumes 16 features per row reads garbage after the first row.
• tracts/context.f32: Float32, 83,008 rows × 3 context fields (population, Black %, unemployment).
• tracts/centroids.f32: Float32, 83,008 rows × 2 (latitude, longitude).
• tracts/sims.u8: Uint8, 83,008 rows × 11 brands. Pre-computed similarity 0–100; 255 = not applicable (e.g., brand has no presence).
• tracts/has.u8: Uint8, 83,008 rows × 11 brands. Boolean: brand already has a store in this tract.
• tracts/cbsa.u16: Uint16, 83,008 rows. CBSA code per tract (0 = not in a metro).
• tracts/rel.u8: Uint8, 83,008 rows. Low-ACS-reliability flag (1 = small-population tract, missing income estimate, or a large income margin of error; 0 = normal).

Stores

• stores/<brand-id>.json: array of store points [lat, lon] (150 m grid-deduped, US-contained).

The per-brand top-N candidate JSON files are generated under data/out for the offline consistency test only; they are not shipped in the web bundle. The browser rebuilds the ranked candidate list live from the binaries above.

Vector tiles

• tracts.pmtiles: PMTiles archive (range-served, about 35 MiB in release 2026-07-11-01). Contains tract polygons and stable GEOID identifiers. The current v2 map colors tracts from browser-computed feature state, so a brand-library rebuild can reuse the same tiles as long as tract geometry and GEOIDs do not change.

The React/Vite front end

Stack: React 19, Vite, MapLibre GL (PMTiles), recharts, TypeScript.

Key entry points:

• web/src/App.tsx: root component, state management (selected brand, filters, factor weights, custom model).
• web/src/lib/data.ts: loader for all static assets; provides TractData interface.
• web/src/lib/scoring.ts: re-implementation of the Python similarity formula (verified by consistency test).
• web/src/components/MapView.tsx: MapLibre choropleth, feature-state coloring per brand.
• web/src/components/DetailPanel.tsx: click any tract → full explanation (similarity %, radar, drivers, feature table, assessment, robustness, nearby chains).

Live capabilities:

• Similarity re-scoring on every filter change and factor-weight adjustment.
• Custom store uploads (CSV) → archetype computed in-browser, similarity re-scored.
• Diverging compare-mode map (two brands, hue = which, brightness = how strongly).
• Metro filtering and per-metro rankings with screening rollup.

Python ↔ JavaScript consistency guarantee

The browser scorer must reproduce the Python pipeline’s numbers exactly—or users cannot trust the detail panel’s “why it matches” explanation.

Verification: web/src/lib/consistency.test.ts reads the real packaged binaries and recomputes similarity for all built-in brands:

• Tests >50,000 tracts per brand.
• Checks that 99.9%+ of tracts agree within 1 percentile point (rounding slack).
• Maximum observed difference: ≤ 2 percentile points.

Note

This test runs as part of pnpm test in the web directory. It is the authoritative proof that the map colors and the detail-panel numbers come from the same formula.

File tree overview

LookAlike-data-pipeline/
├── pipeline/                    ← source retrieval, features, scoring, packaging
├── data/                        ← ignored source cache and intermediate outputs
├── dist/releases/<id>/          ← ignored immutable release output
└── scripts/                     ← publish and activate releases

LookAlike/
├── web/
│   ├── src/
│   │   ├── App.tsx              ← root, state, orchestration
│   │   ├── lib/
│   │   │   ├── data.ts          ← asset loader
│   │   │   ├── scoring.ts       ← JS formula (mirrors Python)
│   │   │   └── consistency.test.ts  ← verification
│   │   └── components/
│   │       ├── MapView.tsx      ← MapLibre choropleth
│   │       ├── DetailPanel.tsx  ← tract detail
│   │       ├── CandidateList.tsx
│   │       ├── ComparePanel.tsx ← diverging map
│   │       └── ... (FactorWeights, UploadModal, ShortlistPanel, etc.)
│   │
│   └── public/data/             ← compatibility data for local development
│       ├── meta.json
│       ├── manifest.json
│       ├── tracts.pmtiles
│       ├── tracts/
│       │   ├── geoids.json
│       │   ├── labels.json
│       │   ├── features.f32
│       │   ├── context.f32
│       │   ├── centroids.f32
│       │   ├── sims.u8
│       │   ├── has.u8
│       │   ├── cbsa.u16
│       │   └── rel.u8
│       └── stores/
│           ├── trader_joes.json
│           └── ...
└── docs-site/                   ← product and method documentation

(The pipeline’s diagnostic candidate files remain private build outputs and are not an application-repository dependency.)

How it all fits together

1. User opens the app → browser fetches meta.json, manifest.json, and the binary files (geoids, features, sims, centroids, cbsa).
2. Browser renders a brand → loads that brand’s stores/*.json, computes the ranked candidate list from the binaries, and MapLibre visualizes tracts via tracts.pmtiles.
3. User clicks a tract → the detail panel calls the JS scorer on that tract, fetches its feature values from the binaries, and renders drivers, radar, and assessment.
4. User adjusts factor weights → the app re-scores all tracts live (no server call); MapLibre feature-state updates the colors.
5. User uploads a custom CSV → the browser computes a new archetype and re-scores the entire country on the fly.

The Python pipeline runs on its own data-refresh schedule and publishes to R2 independently of application deployment. The Cloudflare Worker serves releases and product APIs; interactive national scoring still happens in the browser.

On resemblance, not prediction

The scores are similarities—how closely tracts resemble a brand’s existing footprint—and are computed entirely from observable neighborhood features. They do not predict revenue, sales, foot traffic, or whether a location will succeed. See the method page for why that distinction matters.