Why precompute all forecasts instead of computing on demand with caching?

Three reasons: (1) Thundering herd elimination: when a CDN TTL expires during a hurricane, thousands of users hit the same cache key simultaneously. With on-demand computation, all of those requests cascade to your compute tier. With precomputation, they all hit Redis — a simple GET, not a computation. (2) Predictable load: the precompute job runs on a fixed schedule (every 15 min) regardless of user traffic. Origin capacity planning is trivial. (3) Cold start: an on-demand system has no cached data when it first deploys or after a cache flush. With precomputation, Redis always has the full forecast grid — there's no "cold" state. The tradeoff is 15-minute staleness, but weather changes on hourly timescales, so 15-minute-old data is effectively fresh.

How do you handle a tornado warning reaching 10 million users in under 60 seconds?

The alert path is completely independent from forecast serving. Step 1 (~5s): Alert processor receives NWS CAP XML, parses the alert polygon. Step 2 (~5s): Geo-index query finds all users with saved locations or last-known positions inside the polygon. Users are pre-indexed by geohash — the query is a prefix scan, not a full table scan. Step 3 (~20s): Fan out push notifications via FCM (Android) and APNs (iOS). These platforms are built for millions of concurrent pushes. We batch 100K tokens per API call. 10M users ÷ 100K per batch = 100 API calls, parallelized across workers. The key design choice: push, not pull. Users can't poll for alerts — their app might be closed. Push notifications wake the device and display immediately. For redundancy, we also update the in-app alert banner and send SMS to premium users.

Your CDN goes down during a hurricane — the exact moment when traffic spikes 10x. What happens?

This is the nightmare scenario — and it's a known single point of failure. Mitigations: (1) Multi-CDN: CloudFront primary, Fastly secondary. DNS-based failover. If CloudFront degrades, Route 53 health checks detect it and shift traffic to Fastly within 60 seconds. (2) Origin can't absorb full CDN traffic (200K QPS vs 10K QPS origin capacity), so if BOTH CDNs fail, we degrade gracefully: serve a simplified response (current conditions only, no hourly), shed low-priority endpoints (radar tiles, location search), and activate an emergency static page with regional weather summaries. (3) The alert path is independent — push notifications don't go through the CDN at all, so tornado warnings still reach users even if the forecast API is degraded. The honest answer: if both CDNs fail simultaneously during peak, some users will see errors. But the life-safety alert path survives.

How do you handle the lat/lng to grid cell mapping efficiently at scale?

The mapping is pure math, no database lookup required: grid_lat = floor(lat × 100), grid_lng = floor(lng × 100). This snaps any lat/lng to the nearest 0.01° grid cell (~1.1km at the equator). The cache key becomes deterministic: grid:{grid_lat}:{grid_lng}. This means the CDN can cache by URL path — no query-string normalization needed. The 0.01° resolution gives ~10M unique land-surface grid cells. At the equator, 0.01° ≈ 1.1km. At 60° latitude, it's ~550m (longitude cells shrink with latitude). For higher fidelity near population centers, we could use a variable-resolution grid (finer in cities, coarser in oceans), but the uniform grid is simpler and 1km is sufficient for consumer weather. The key property: every user at the same 1km patch gets the identical response, making CDN caching trivially effective.

How do you make the forecast more accurate than just serving raw NWS data?

Three techniques compound accuracy: (1) Model blending: GFS, ECMWF, HRRR, and NAM each have different strengths. We maintain a rolling accuracy score per model per region (comparing past forecasts to actual observations). The blender assigns weights proportional to recent accuracy — if ECMWF has been beating GFS in the Pacific Northwest this week, it gets higher weight there. (2) Statistical downscaling (MOS — Model Output Statistics): NWP models output at 9-13km resolution and have systematic biases ("GFS is always 2°C warm in Denver in January"). We train regression models that correct these biases using years of forecast-vs-observation pairs. (3) Observation assimilation: between NWP model runs (every 6-12 hours), we blend real-time station observations to keep "current conditions" fresh. If a METAR station reports 28°C but the GFS forecast said 25°C, we adjust the nearby grid cells toward the observation. This is the secret sauce that differentiates weather apps — everyone gets the same raw NWP data, but the post-processing is proprietary.