Forecast stability score and accuracy — The Solano method

1. Why a stability score?

When you ask a conventional weather app "what will the weather be tomorrow?", the answer is always the same: the values from the latest run of the last model computed. This approach has a fundamental limitation — it says nothing about how trustworthy that forecast is.

Two identical forecasts can carry very different levels of confidence. Consider two scenarios for tomorrow at 2 pm:

In both cases a conventional app would display "15 knots". Solano would show you 9/10 for the first scenario and 3/10 for the second. This is the heart of the Solano method: measuring the temporal consistency of forecasts, not just their point value.

2. Weather model runs: what are they?

A numerical weather prediction model is a program that solves the equations of fluid dynamics on a three-dimensional grid of the global atmosphere. It is initialised from an "initial state" — a snapshot of the atmosphere at a precise moment, built from thousands of observations: surface weather stations, radiosondes, satellites, marine buoys, aircraft measurements in flight…

This initialisation happens several times a day, at conventional hours expressed in Coordinated Universal Time: 00Z, 06Z, 12Z, 18Z. Each execution of the model from an initial state is called a run.

A few examples of publication frequency:

• ARPEGE (Météo-France, Europe) — 4 runs per day (00Z, 06Z, 12Z, 18Z)
• AROME HD (Météo-France, high resolution 1.5 km) — up to 8 runs per day
• ECMWF IFS HRES (European Centre) — 2 runs per day (00Z and 12Z)
• GFS (NOAA, United States) — 4 runs per day

At each run, the model "starts from scratch" with a slightly different initial state (the most recent observations have been folded in), and can produce slightly — or sometimes very — different forecasts for the same target date. It is these differences between successive runs that Solano measures and quantifies.

3. The models Solano analyses

Solano relies on Open-Meteo's Single Runs API, which gives access to the historical archive of every run from the world's major forecast centres. Up to 16 atmospheric models are available depending on location, sorted from highest resolution to broadest coverage:

^¹ Global fallback model — shown only when the high-resolution regional variant is not available for this point (e.g. ICON global outside Europe, UKMO global outside the United Kingdom).

For conditions at sea, Solano also analyses 5 wave and current models: ARPEGE-Wave and WAM (Météo-France and ECMWF), GFS-Wave (NCEP), EWAM and the Météo-France Currents model. Marine models do not contribute to the atmospheric stability score — they are shown separately for points at sea.

Each model covers different geographic areas. When a GPS point falls outside a model's coverage, that model is automatically excluded from the analysis. For multi-resolution families (ARPEGE, ICON, UKMO, GEM), the high-resolution regional model is automatically preferred over the global model when the point is within its area.

4. The "movie of the runs": watching forecasts evolve

For a given GPS point and target date, Solano reconstructs every forecast issued by each model for that date: from the J-5 run to the most recent one. If the target date is three days out, dozens of runs have already forecast the conditions of that day — from different distances in time.

Seeing a forecast at a 3-day lead time (the model runs today and predicts for the day after tomorrow) is not the same as seeing one at a 1-day lead time (the model predicts for tomorrow with fresher, more accurate initialisation data). In general, the closer you get to the target day, the more accurate the forecast — but not always, and that is precisely what Solano measures.

Solano's spaghetti chart overlays all of these curves on a single graph: each line is a different run, coloured by its age. The interpretation is immediate:

• Tight curves — successive runs agree → high stability score → high confidence in the forecast.
• Scattered curves — the runs diverge → low stability score → unstable situation, uncertain forecast.
• Converging curves — older runs diverged but recent ones agree → the situation is gradually settling.

This visualisation lets you distinguish structural uncertainty (the weather situation is inherently hard to predict) from transient uncertainty (the models struggled at 5 days but agree now at 2 days).

5. Computing the score — formula and weighting

The stability score is computed by a four-step algorithm, applied independently for each model and each target date.

Step 1 — Hour-by-hour comparison

For each hour of the target date, Solano computes the weighted average of all previous runs (each run weighted by its age — see section 6), then measures the gap between that average and the value of the latest run:

The weighted average represents the "historical consensus" of the forecasts for that moment. If every older run said 15 knots and the latest one announces 22 knots, the gap is 7 knots — and the score will drop.

For wind direction, the gap is not an arithmetic difference but the short angular arc (the minimum angular difference between two headings, always between 0° and 180°). This avoids treating 355° and 5° as very different — they differ by only 10°.

Step 2 — Average over all hours

The hour-by-hour gaps are averaged across the whole day, producing a mean gap per variable. It is this single value — the typical gap over the entire day — that feeds into the score formula.

Step 3 — Conversion to a 0–10 score per variable

The mean gap is converted into a score by an exponential decay, parameterised by a threshold S specific to each variable:

Some concrete reference points: at zero gap, score = 10 (perfect stability). At a gap of S, score ≈ 3.7 — so the threshold is not the point where the score reaches 0, but the point where it has already dropped substantially. Beyond 3×S, the score is below 0.5 and can be treated as zero. The exponential is deliberately lenient on small gaps: a 1-knot divergence for gusts (S = 6) only brings the score down from 10 to about 8.5.

Step 4 — Weighted aggregation

The scores of all variables are combined into a weighted average. The default weights reflect the relative importance of each variable for marine and outdoor activities (gusts are the most safety-critical criterion, followed by mean wind and precipitation). These weights are fully customisable in the app's preferences.

When a model lacks a given variable (for example, AROME HD does not cover marine visibility), that variable's weight is automatically redistributed across the other available variables, so the score stays comparable between models.

6. Temporal decay

Not all past runs carry equal weight in the score. A run issued 5 days ago was based on far less recent observations than one issued yesterday — so it is less relevant for assessing the consistency of the current forecast.

Solano applies an exponentially decreasing weighting to each run according to its age, parameterised by a time constant τ:

τ is the decay constant: at t = τ, the weight reaches e⁻¹ ≈ 37 % (not 50 % — the true half-life would be τ × ln 2 ≈ 1.7 days). In practice: a run one day old carries about 67 % of the weight of a run from today, a run 4 days old about 20 %, and a run 7 days old less than 6 %.

This decay reflects the meteorological intuition: what was forecast a week ago tells us less about current stability than what was forecast yesterday or the day before. A model that "changes its mind" at the last minute is less concerning than one that has been oscillating for several days.

7. ERA5 verification — forecast vs. reality

The stability score measures the internal consistency of forecasts — it doesn't tell you whether the forecasts were accurate. To complement this indicator, Solano includes a direct comparison between historical forecasts and the reference reanalysis ERA5T.

ERA5 is the atmospheric reanalysis from ECMWF (the European Centre for Medium-Range Weather Forecasts): a coherent, homogeneous reconstruction of the state of the atmosphere hour by hour since 1940, obtained by data assimilation of sparse observations (surface stations, radiosondes, satellites, buoys) into a state-of-the-art numerical model. It is not a direct measurement, but the best possible estimate of what actually happened. ERA5T is the "real-time" version of this reanalysis, available with roughly a 1-to-2-day delay.

The 📡 Verif. tab of the Spaghetti panel shows, for any day within the last 15 available:

• The ERA5T line (reference reanalysis) — drawn as a black dashed line
• All the corresponding run curves — what each model had forecast for that day, at various lead times

This visualisation answers a concrete question: was the model that predicted 20 knots for the day before yesterday right? Does ERA5T give 18 knots (a good forecast) or 8 knots (a strong overestimate)?

Only the atmospheric variables available in recent ERA5 are shown: wind, gusts, direction, temperature, precipitation, cloud cover, pressure. Marine data (waves, currents) is not covered by recent ERA5.

8. Recent accuracy — MAE per lead time

By automating the forecast/ERA5T comparison over the last 15 days and every lead time, Solano computes for each model a measure of real local accuracy: the MAE (Mean Absolute Error).

The MAE is computed separately:

• For each atmospheric model
• For each variable (wind, gusts, direction, rain, temperature, cloud, pressure)
• For each lead time from J-1 (a 1-day forecast) to J-15 (a 15-day forecast)

This MAE-per-lead-time profile reveals the real behaviour of each model on your exact geographic point. A model may be excellent at J-2 but mediocre at J-7; another may hold remarkable accuracy up to J-5. This information helps you intuitively weight forecasts according to their lead time.

MAE values are shown in the app as green/orange/red coloured badges on the model cards, and as a J-15 → J-1 time chart in the Spaghetti view. This chart shows the accuracy-degradation curve with lead time, characteristic of each model on your point.

In Quick weather, this measure is presented per day under the name "Accuracy": it indicates, for each model and at that day's lead time (a forecast for three days out is checked against past forecasts at a 3-day lead), the model's recent track record — how accurate it has been at that lead. It is the model's track record, to be distinguished from Confidence (agreement of the ensemble scenarios for this run, still to come): accuracy looks at the model's past, confidence looks at the uncertainty of the present situation. The two are complementary, not interchangeable.

An important nuance about the reference. ERA5 is a coarse-grid (~31 km) reanalysis, whereas some verified models run on a fine grid (AROME ~1.5 km). Comparing a fine model to a coarse reference penalises it on the local variability it legitimately resolves (sea breezes, terrain effects) — this is a "representativeness error", not a real model error. The ERA5 MAE nonetheless remains a useful signal because it is relative: all models are judged against the same reference, so their ranking stays informative. It reads as a "gap from a reference reanalysis", not as ground truth at the exact point — which is why Solano complements it with a direct comparison to real observations (buoys, airports) where they exist.

9. Ensemble confidence — the agreement of scenarios

Stability looks at how a model evolves from one run to the next ("the movie"), the MAE measures its past error. Ensemble confidence answers a third question, facing forward: for this precise lead time, do the possible scenarios agree?

Weather centres don't publish just a single "deterministic" forecast. They also run their model dozens of times with tiny perturbations to the starting conditions — this is the ensemble forecast. If all these scenarios tell the same story, the situation is predictable; if they diverge, the atmosphere is at a "tipping point" and no single forecast should be taken at face value.

Solano aggregates a pool of about 120 scenarios from three international ensemble systems — GEFS (NOAA, ~31 members), ICON-EPS (DWD, ~37) and ECMWF-ENS (European Centre, 51) — then, for each deterministic model shown, computes the share of these scenarios that give a wind practically equivalent to the model's value:

This measure combines two pieces of information into a single number:

• The scenario spread: when the ensemble is tight (a predictable situation), many members fall within the bracket → high confidence. When it is very spread out, few members surround any given value → low confidence for everyone.
• The model's agreement with the consensus: a model that sits at the core of the scenario cloud comes out higher than a model that is "isolated" off to the side.

The multi-model pool is more honest than a single ensemble: a model compared to its own family often judges itself too confident (it doesn't "see" its own systematic error). By confronting each forecast with the scenarios of three centres, confidence captures both the uncertainty of the scenarios and the disagreement between models. It also works for models with no ensemble of their own (ARPEGE, AROME), compared to the common cloud.

In the app, confidence is displayed as a coloured percentage pill, day by day, next to each day in the hourly strip of each model (Quick weather): green ≥ 70 %, orange ≥ 50 %, red below. It naturally decreases with lead time (the further out you look, the more the scenarios separate) and lets you spot at a glance which model is best supported by the scenarios, and which days are "worth watching".

10. The "When to leave?" departure score — formula and computation

The departure score condenses into a single 0–10 score the weather conditions encountered hour by hour over the whole route, for a given activity. It is distinct from the stability score — it doesn't assess run consistency but the absolute weather quality of a departure window.

Candidates evaluated

Solano evaluates 13 candidate windows spaced 3 hours apart, from D−6h to D+30h (D = the departure time configured in the Route panel). For each candidate, the trip is reconstructed hour by hour — positions computed by the haversine formula at constant speed — and the weather is interpolated at each position.

Weather interpolation along the route

Between two consecutive waypoints, all scalar variables (wind, gusts, precipitation, CAPE, temperature…) are interpolated linearly according to the position on the segment:

The directions (wind, waves, currents) use angular interpolation via sin/cos components (weighted circular mean) to avoid discontinuities around 0°/360°:

Score components

Each component is evaluated by a piecewise linear function over breakpoints [value → score 0–10], with linear interpolation between two adjacent points and clamping at the ends.

Mean wind (knots)

For sailing, the optimum bell reflects that a dead calm prevents any progress under sail (score 2) and that strong wind becomes dangerous (0 at 32 kt). When motoring, wind only affects the hull through the sea it builds — wind alone is penalised gradually.

Gusts (knots)

Gusts follow their own piecewise curve, independent of the mean wind. For sailing, a score of 8 at 0 kt (not 10) reflects that a total calm with no puffs is less pleasant than a few knots of breeze:

Sea state — seaStateScore(H, T) (sailing and motoring only)

Sea state can't be reduced to wave height. A 1.5 m sea at 3 seconds is incomparably harder than a 1.5 m swell at 12 seconds. Solano uses vertical acceleration (the real physical discomfort for a sinusoid) as the primary criterion, capped by absolute height:

The two breakpoint tables:

Example: H = 2 m, T = 6 s → acc = 2 × 19.74 / 36 = 1.10 m/s² → acc score ≈ 3.5; height cap = 8 → seaStateScore = 3.5. The same height at T = 12 s → acc = 0.27 → acc score ≈ 9.2; height cap = 8 → seaStateScore = 8.

Wind angle to heading (sailing and cycling)

The relative angle α is computed between the wind direction and the heading of the segment travelled, folded to 0–180°. Sailing favours a reach/broad reach (the optimal point of sail); cycling favours a tailwind:

The angle score is only computed if the wind exceeds 4 knots — below that, direction has no practical meaning.

Sailability — sailability(wind, direction, heading)

The angle score above measures the quality of the point of sail, but it is ignored below 4 knots — which is precisely when you reach for the engine. A separate component therefore answers a binary question, hour by hour: can you make way under sail? Two cases prevent it:

The threshold depends on the point of sail because apparent wind moves in opposite directions on each tack: close-hauled, boat speed adds to the true wind — apparent wind rises and you make way in little air; downwind, the boat runs away from the wind — apparent wind falls, and you need more of it to make progress. A single 8-knot threshold therefore wrongly penalised close-hauled windows.

The component is worth the share of sailable hours × 10. The detailed ratios (share under engine, of which head to wind and light air) are shown next to the score as "🔧 40 % (head to wind)".

CAPE, precipitation, temperature

Statistical aggregation along the route

For each component, Solano collects the value hour by hour over the whole trip (all interpolated positions, all active models aggregated). The final component is not the simple average — it gives more weight to the least favourable moments:

For sea state (scores are already computed hour by hour, not raw values):

The p90 (90th percentile of the raw values) or the p10 (10th percentile of the sea-state scores) captures the "worst quarter" of the route without being as extreme as the single maximum.

Per-activity weighting and final score

The final score is a weighted average normalised by the sum of the active weights (components with no data are dynamically excluded):

11. Thunderstorm risk — beyond CAPE

CAPE (Convective Available Potential Energy, in J/kg) is often presented as the thunderstorm number. That is misleading: a CAPE of 2,000 doesn't have the same reach in every situation. It also depends on the steepness of the atmosphere, the cap that does — or doesn't — keep the air from rising, and the organisation of the cells. So Solano combines several ingredients into a single categorical index, hour by hour.

The ingredients

Following the ingredients-based approach (Doswell), the risk is built as a product: fuel, multiplied by a cap-opening factor, then refined by organisation.

• Fuel — derived from CAPE (0 → 10 on the 0 / 300 / 800 / 1,500 / 2,500 / 3,500 J/kg scale), sharpened by the Lifted Index (positive LI → zero fuel even with high CAPE; LI ≤ −6 → a high floor) and by the steepness of the profile: the mid-tropospheric 700→500 hPa lapse rate (°C/km) multiplies the fuel by 0.85 (soft profile, < 5.5 °C/km) up to 1.15 (steep profile, > 7 °C/km). This is what makes a CAPE of 2,000 have a different reach depending on the situation.
• Cap (CIN) — convective inhibition (J/kg) becomes a multiplier from 1.0 (open cap, < 25) to 0.15 (suppressed, > 300). A strong cap reduces the realised risk.
• "Loaded gun" flag — plenty of fuel under a strong cap: triggering is unlikely, but explosive if the cap breaks. Marked with ⚠️.
• Organisation (shear) — the 0–6 km wind shear (vector difference between the surface wind and the 500 hPa wind, in m/s) decides the structure: < 10 = isolated cells (pulse), 15–20 = organised (supercell possible), > 20 = strongly organised (squall lines).

The score gives an intensity in four categories: Low (< 1.5) · Moderate (1.5–4) · Marked (4–7) · Severe (≥ 7). Shear then refines the category: "Severe" is reserved for organised storms — a high intensity but weakly sheared is brought back to "Marked · isolated", while a strongly sheared "Marked" can be promoted to "Severe".

The GFS anchor and model agreement

Among Open-Meteo's models, only GFS exposes every ingredient (CAPE, CIN inhibition, Lifted Index and the pressure levels needed for shear and lapse rate). The index is therefore anchored on GFS, regardless of the models you display — Solano fetches it even if GFS isn't selected. The queried point is brought back to the GFS 0.25° grid: the index therefore describes the environment at the basin scale (≈ 25 km), not precisely above your anchorage.

Added to this index is a measure of agreement between models: the spread of CAPE across all the models consulted. When they diverge strongly (for example CAPE 10 vs 560 at the same point), the risk is marked as uncertain — true to the "movie rather than the photo" philosophy. Every threshold is adjustable.

Triggering — what fine-scale models see

The GFS base answers "is the atmosphere charged?". It doesn't answer "will a cell fall here, and when?" — at a 25 km grid, a thunderstorm cell of a few kilometres simply isn't resolved, it's estimated by a formula. That is the role of a second reading, distinct and never folded into the /10 score.

Convection-permitting models (1–3 km grid: AROME HD, ICON-2I, ICON-D2, ICON-CH1/CH2) actually compute the updrafts: they place cells. When one of them is available on your point, Solano reads its hourly precipitation — reinforced by a simultaneous gust of at least 32 knots, the signature of a convective downdraft — and derives three levels: shower, formed cell, vigorous cell. The result is shown on one line below the thunderstorm index ("Cells forecast: AROME · ICON-2I → 4–6 pm") and, in detail, on a row of the hourly table specific to each fine model.

Two counter-intuitive points, both verified against real data. First, the criterion is total precipitation and not the "showers" field: on a model that resolves convection, the shower is already in the total rain and the dedicated field is now just a residue (observed on ICON-D2: 3.1 mm/h of rain for 0.2 of "showers"). Second, this signal cannot be added to the thunderstorm index: a model that triggers a cell consumes the CAPE, so a low CAPE at 5 pm may mean "storm in progress" and not "calm". Averaging the two would invert the reading at the worst moment.

From forecast to observation

When the risk is notable today, the "See the sky now" button switches the map to satellite imagery — infrared (cold tops), lightning (live strikes) and RDT (detection of developing thunderstorm cells, before electrification). This lets you check at a glance whether a cell is actually breaking out. Unlike ground radar, these satellite products also cover the open sea.

Thunderstorm risk along a route

In Route mode, the same index is computed all along the track. Rather than an illegible point-by-point grid, Solano evaluates the risk at each computation point — at the position and the moment the boat will be there. The band reads from left (departure) to right (arrival): each cell carries its category, and the expandable detail lists the at-risk points with their ingredients (CAPE, CIN, Lifted Index, shear, lapse rate). The computation stays anchored on GFS, regardless of the models displayed.

12. Anchorage analysis — the protection algorithm

The anchorage analysis computes, for any point at sea, a directional protection profile via topographic ray casting, complemented by a 0–100 % comfort score over 3 days and bathymetric data.

Ray casting — collecting elevations

For each point, Solano casts 1,728 rays (36 directions × 3 offsets ±5° × 16 distances) from 25 m out to 3,000 m. Elevations come from the SRTM 30 m digital terrain model (global coverage, which returns 0 over the sea = no obstacle).

The ±5° fan per direction lets it "see" a cliff even if the central ray passes over water. Extending to 3 km is essential for large bays whose flanking headlands are more than a kilometre apart (for example Portoferraio, Elba).

Breakwaters and harbour works — OpenStreetMap

A 30 m terrain model is blind to narrow harbour works: a mole 10 to 20 m wide is under-resolved, read at water level. Solano therefore overlays OpenStreetMap geometry around the point (breakwaters and jetties man_made=breakwater, as well as the coastline natural=coastline, which often carries the solid moles of harbours). Each ray tests its geometric intersection with these structures — not a simple point sample, which would miss a thin mole the same way the terrain does.

Merge rule: any coast or breakwater on the ray blocks the swell (the sea doesn't cross land); only moles add a flat-rate wind protection (emergent height ≈ 3 m), with the natural coastline leaving the real terrain relief to decide. The merge is purely additive: it never removes protection detected by the terrain.

Wind protection

Wind protection is estimated by the height / distance ratio of the tallest obstacle found on the ray:

The maximum over the ray's 3 offsets is kept. Venturi detection multiplies the wind penalty ×1.5 when an unblocked sector is flanked by two blocked sectors (a channelling effect).

Swell protection — adaptive threshold

Swell isn't stopped by the same obstacles as wind. Solano applies a terrain height threshold that varies with distance to avoid false positives (a distant flat island flagged as "blocking" the swell):

Swell protection is binary (blocked / not blocked) and computed by nearest-neighbour between adjacent sectors — never by OR interpolation, which would produce false "blocked" values between a protected sector and an exposed one.

Comfort score

For each hour over 3 days, the score combines the swell (70 %) and wind (30 %) forecasts with the protection profile:

The Hs = 0.30 m threshold reflects that 30 cm of chop is already noticeable at anchor. Gusts are included in the wind calculation.

Bathymetry — EMODnet

Depth comes from EMODnet Bathymetry DTM 2022 (~115 m resolution, REST API POINT(lon lat)), returning the smoothed value plus a min/max range and a plain-language sounding type. Seabed type comes from EMODnet Seabed Habitats — EUSeaMap 2025 (WFS emodnet_view:eusm2025_msfd_full, substrate read from the msfd_bbht field then classified: sand, mud, rock, Posidonia seagrass…).

180-day cache per anchorage (key anchor_v5_, 0.001° resolution ≈ 100 m). Increment the key in production if the algorithm changes.

13. Limitations of the method

The Solano method has several important limitations to be aware of when interpreting the scores:

• Stability ≠ accuracy. A score of 9/10 means the models are consistent with each other — not that they are right. An atmospheric situation can be systematically mis-simulated by every model (a common bias). The ERA5 MAE score is the complementary indicator for assessing historical accuracy.
• Meaningful from J+1 onwards. The run-to-run comparison is only relevant from the next day, once several runs have already forecast the same day. At very short lead times (a few hours), all runs naturally converge — the score would be artificially high.
• Run publication delay. Each model publishes its runs with a variable delay (from 1h30 for AROME to 4h for UKMO). Solano uses the real Open-Meteo publication timestamps, not the theoretical hours.
• No marine data in recent ERA5. ERA5 verification only covers atmospheric variables. For waves and currents there is not yet a recent-ERA5 equivalent in the Open-Meteo API.
• ERA5 is a coarse-grid reference. At ~31 km resolution, ERA5 doesn't "see" the fine variability (breezes, coastal relief) that a 1.5 km model resolves. The ERA5 MAE therefore mechanically penalises fine models on these local effects: it is a gap from a coarse reanalysis, to be read as a relative signal (model ranking) rather than an absolute error at the point. The comparison to real observations (buoys/airports) is added where the network allows.
• Open-Meteo as an intermediary. The data passes through the Open-Meteo API, which imposes its own update delays and per-model availability horizons.