Architecture & consistency
LookAlike is built as two independent repositories joined by a versioned data contract: LookAlike-data-pipeline downloads and validates open sources and publishes immutable releases, while LookAlike contains the React application, TypeScript Worker API, and browser scoring engine.
Overall structure
Section titled “Overall structure”Input data (Census ACS, TIGER, All The Places / OSM stores, Overture Maps) ↓[Python pipeline: 8 stages] ↓Static web assets (binaries, tiles, JSON) ↓[React app: loads assets, scores live, re-weights interactively] ↓Browser: choropleth map + detail panels + rankingsThe Python pipeline (8 stages)
Section titled “The Python pipeline (8 stages)”Run LOOKALIKE_RELEASE_ID=<id> python pipeline/run_all.py from the data-pipeline repository. Each stage caches raw downloads and intermediate Parquet files in that repository. Here is what each produces:
-
fetch_acs →
acs_cells.parquet- US Census ACS 2020–2024 5-year Summary File (keyless bulk
.datdownload). - CONUS census tracts × ACS cells (income, education, demographics, housing, employment, commute).
- US Census ACS 2020–2024 5-year Summary File (keyless bulk
-
fetch_geometry →
tract_geo.parquet,tract_polys.parquet- US Census TIGER/Line 2024 tract shapefiles.
- 83,777 tract centroids (lat/lon) and simplified polygons (for vector tiles).
- Includes land area per tract (for density calculations).
-
fetch_stores →
stores.parquet- Brand store locations from All The Places (CC0 weekly scrapes) when that source covers at least 80% as many tracts as OpenStreetMap, otherwise from OpenStreetMap via the Overpass API.
- Release 2026-10-05-01: 87 brands from All The Places, 12 from OpenStreetMap.
- Points within 250 m merge into one site; sites farther than 1 km from any continental tract are dropped.
-
fetch_poi → used by build_features
- Overture Maps Places (public GeoParquet on S3, queried with DuckDB, release
2026-09-23.1), excluding permanently-closed places and places with existence confidence below 0.5. - Aggregated to per-km² densities for 5 POI categories: all POI, grocery stores, restaurants, cafés, shopping/apparel.
- Overture Maps Places (public GeoParquet on S3, queried with DuckDB, release
-
build_features →
tract_features.parquet- Derive the feature table for 83,006 tracts.
- The presence model scores 18 features: median income, per-capita income, median home value, median rent, poverty rate, bachelor’s degree %, population density, households without a vehicle %, walk/bike/transit commute %, renter housing %, median age, average household size, households with children %, and 5 Overture retail densities (all points of interest, grocery stores, restaurants, cafés, shopping per km²).
- Context (shown in UI but not scored): total population, unemployment rate, and the race/ethnicity shares (White non-Hispanic %, Hispanic/Latino %, Asian non-Hispanic %, Black non-Hispanic %).
-
build_scores →
scores.parquet,meta.json- Fit a per-brand presence model (L2-regularized logistic regression, linear and squared term per feature) contrasting the brand’s store tracts with a fixed background of every fourth scoreable tract. Squared coefficients are constrained to be zero or negative, and each feature is clipped to the 2.5th–97.5th percentile of the brand’s store tracts before fitting and scoring.
- Compute within-brand fit percentiles (0–100) for all scoreable tracts against each brand.
- Calculate feature-level contributions (“why it matches” and “what works against it”).
- Compute footprint coherence (county-blocked out-of-sample refits, median held-out percentile).
- Write brand profiles, fitted model coefficients, store-source counts, scaling metadata, and the low-ACS-reliability flag.
-
export_release →
dist/releases/<release-id>/*- Package everything the application needs as one versioned release.
- (Detailed below.)
-
fetch_cbsa →
metros.json,cbsa.u16- US Census CBSA (Metropolitan Statistical Area) delineations, 2023.
- 383 metros in the CONUS. Maps each tract to its CBSA code.
- Optional step: if it fails, the app still runs (metro filtering is optional).
Published release assets
Section titled “Published release assets”The pipeline exports everything needed by the browser under one immutable R2 release prefix, and the hosted application reads it from there. The application repository does not commit national data; pnpm data:dev copies a release into web/public/data/ for local development.
JSON metadata
Section titled “JSON metadata”meta.json: brand profiles (name, id, store count, occupied tracts, store source and counts, fitted presence-model coefficients, coherence), model space (feature groups, transforms, scaling), national feature min/max/mean.manifest.json: array counts, dimensions, data types, endianness note (little-endian for native x86/ARM).metros.json: array of metro objects (CBSA code, name, tract count, bounding box, center latitude/longitude).
Binary tract data (row-major, little-endian)
Section titled “Binary tract data (row-major, little-endian)”tracts/geoids.json: array of 83,006 GEOIDs (row order for all binaries).tracts/labels.json: human-readable labels (parallel to geoids).tracts/features.f32: Float32, 83,006 rows of feature values. The presence model scores 18 of these columns; the race/ethnicity shares travel here but are shown only as context.tracts/context.f32: Float32, 83,006 rows of context fields (population, unemployment, and race/ethnicity shares).tracts/centroids.f32: Float32, 83,006 rows × 2 (latitude, longitude).tracts/boundaries.binandtracts/boundaries.json: tract polygons and a cell index, range-served, used to assign an uploaded store to its containing tract.boundaries.binis 35.6 MB.tracts/sims.u8: Uint8, 83,006 rows × 99 brands. Pre-computed fit percentile 0–100; 255 = not applicable.tracts/has.u8: Uint8, 83,006 rows × 99 brands. Boolean: brand already has a store in this tract.tracts/cbsa.u16: Uint16, 83,006 rows. CBSA code per tract (0 = not in a metro).tracts/rel.u8: Uint8, 83,006 rows. Low-ACS-reliability flag (1 = small-population tract, missing income estimate, or a large income margin of error; 0 = normal).
Stores
Section titled “Stores”stores/<brand-id>.json: array of store points[lat, lon](250 m merged into sites, continental-US contained).
Vector tiles
Section titled “Vector tiles”tracts.pmtiles: PMTiles archive (range-served, 23.8 MB). Contains tract polygons and stable GEOID identifiers. It no longer carries per-brand similarity values; the map colors tracts from browser-computed feature state, so a brand-library rebuild can reuse the same tiles as long as tract geometry and GEOIDs do not change.
The React/Vite front end
Section titled “The React/Vite front end”Stack: React 19, Vite, MapLibre GL (PMTiles), recharts, TypeScript.
Key entry points:
web/src/App.tsx: root component, state management (selected brand, filters, factor weights, custom model).web/src/lib/data.ts: loader for all static assets; providesTractDatainterface.web/src/lib/model.ts: presence-model space, fit, and scoring.web/src/lib/scoring.ts: profile scoring, contributions, and fit bands over the model space.web/src/components/MapView.tsx: MapLibre choropleth, feature-state coloring per brand.web/src/components/DetailFloat.tsx: click any tract → full explanation (fit %, radar, contributions, feature table, assessment, robustness, nearby chains).
Live capabilities:
- Fit re-scoring on every filter change and factor-weight adjustment.
- Custom store uploads (CSV) → presence model fit in-browser, fit re-scored.
- Diverging compare-mode map (two brands, hue = which, brightness = how strongly).
- Metro filtering and per-metro rankings with screening rollup.
Python ↔ JavaScript consistency guarantee
Section titled “Python ↔ JavaScript consistency guarantee”The browser scorer must reproduce the Python pipeline’s numbers—or users cannot trust the detail panel’s “why it matches” explanation.
Verification: web/src/lib/consistency.test.ts and web/src/lib/contract.test.ts read the packaged binaries and the fitted model, then recompute fit for the built-in brands:
- Check that the browser reproduces the pipeline’s fit percentiles for the curated brands.
- Check the browser model against a pipeline contract fixture.
File tree overview
Section titled “File tree overview”LookAlike-data-pipeline/├── pipeline/ ← source retrieval, features, scoring, packaging├── data/ ← ignored source cache and intermediate outputs├── dist/releases/<id>/ ← ignored immutable release output└── scripts/ ← publish and activate releases
LookAlike/├── web/│ ├── src/│ │ ├── App.tsx ← root, state, orchestration│ │ ├── lib/│ │ │ ├── data.ts ← asset loader│ │ │ ├── model.ts ← presence-model space, fit, scoring│ │ │ ├── scoring.ts ← profiles, contributions, fit bands│ │ │ ├── consistency.test.ts ← verification│ │ │ └── contract.test.ts ← pipeline contract check│ │ └── components/│ │ ├── MapView.tsx ← MapLibre choropleth│ │ ├── DetailFloat.tsx ← tract detail│ │ ├── CandidatesList.tsx│ │ ├── ComparePanel.tsx ← diverging map│ │ └── ... (FactorWeights, UploadModal, ShortlistPanel, etc.)│ ││ └── public/data/ ← local development copy of a release (not committed)│ ├── meta.json│ ├── manifest.json│ ├── tracts.pmtiles│ ├── tracts/│ │ ├── geoids.json│ │ ├── labels.json│ │ ├── features.f32│ │ ├── context.f32│ │ ├── centroids.f32│ │ ├── boundaries.bin│ │ ├── boundaries.json│ │ ├── sims.u8│ │ ├── has.u8│ │ ├── cbsa.u16│ │ └── rel.u8│ └── stores/│ ├── trader_joes.json│ └── ...└── docs-site/ ← product and method documentationHow it all fits together
Section titled “How it all fits together”- User opens the app → browser fetches
meta.json,manifest.json, and the binary files (geoids, features, sims, centroids, cbsa). - Browser renders a brand → loads that brand’s
stores/*.json, computes the ranked candidate list from the binaries, and MapLibre visualizes tracts viatracts.pmtiles. - User clicks a tract → the detail panel calls the JS scorer on that tract, fetches its feature values from the binaries, and renders contributions, radar, and assessment.
- User adjusts factor weights → the app re-scores all tracts live (no server call); MapLibre feature-state updates the colors.
- User uploads a custom CSV → the browser fits a new presence model and re-scores the entire country on the fly.
The Python pipeline runs on its own data-refresh schedule and publishes to R2 independently of application deployment. The Cloudflare Worker serves releases and product APIs; interactive national scoring still happens in the browser.