The scores as data, ready for AI agents.
The scores and tested comparisons behind the station, tower and ERA5 pages in four CSV files, with a description an AI agent can read: every column, its unit, and the rules for reading the numbers. Free to use under CC BY 4.0, updated with each weekly run.
Summary sheetThe results on one page: the tested comparisons in each check, temperature and wind, an event to watch and what changed since the last update. Prints as two A4 pages with sources and citation.- DWD stations in Germany (30)
- National services: Austria 31, Netherlands 27, Denmark 13, Switzerland 5, Finland 1 (77)
- NOAA hourly reports, 11 further countries (61)
- Cabauw tower, Netherlands (1)
The files
Data version 16 November 2026, free to use under CC BY 4.0. The columns of each file are explained further down.
| File | What it holds | Rows | Columns | Size |
|---|---|---|---|---|
meteosignal-stations-scores.csv | Scores at the stations and the tower. RMSE, MAE, bias, skill and activity of every model and baseline, with 95% intervals. One row per station set, AIFS version, variable, track, lead day and series. | 1,608 | 37 | 377 kB |
meteosignal-stations-aifs-vs-models.csv | AIFS against each other model at the stations and the tower. The paired RMSE difference, its interval, the test and the verdict. One row per station set, AIFS version, variable, track, lead day and other model. | 588 | 26 | 117 kB |
meteosignal-era5-scores.csv | Scores against ERA5 over Europe. RMSE, MAE, bias, skill, activity and anomaly correlation on a grid over Europe, with 95% intervals. One row per model set, AIFS version, variable, lead day and series. | 450 | 40 | 115 kB |
meteosignal-era5-aifs-vs-models.csv | AIFS against each other model, against ERA5 over Europe. The paired RMSE difference against ERA5, its interval, the test and the verdict. One row per model set, AIFS version, variable, lead day and other model. | 270 | 26 | 53 kB |
datapackage.json | The description of all 4 files. For software and AI agents: every column with its type, unit and allowed values, and the rules for reading the numbers. In the open Frictionless Data Package format. | – | – | 60 kB |
Use the data with an AI agent
The files come with a description an agent can read, in the open Frictionless Data Package format: what every column means, its unit and allowed values, and the rules for reading the numbers. Point your agent at it and it can load the data and answer questions without guessing.
- Copy this prompt into your agent
It tells the agent what the data are, where the files are, and the rules for reading them.
You are helping me analyse forecast verification data from MeteoSignal (meteosignal.com): an independent comparison of ECMWF's AI weather model AIFS with IFS, GFS and other models, scored against weather stations in Europe, a measurement tower in the Netherlands and the ERA5 reanalysis. 1. Read the description first: https://meteosignal.com/scores/datapackage.json It explains every file and column, the units and allowed values, and the rules for reading the numbers. Follow those rules. 2. Load the files it lists: - https://meteosignal.com/scores/meteosignal-stations-scores.csv - https://meteosignal.com/scores/meteosignal-stations-aifs-vs-models.csv - https://meteosignal.com/scores/meteosignal-era5-scores.csv - https://meteosignal.com/scores/meteosignal-era5-aifs-vs-models.csv 3. When you answer: - Compare numbers only within one site or model_set, one AIFS version (era) and one track. The German stations are also part of the europe set, and the ERA5 files are a separate check. - Treat a difference between AIFS and another model as a finding only where verdict is ai-leads or other-leads. Say so plainly when the models cannot be separated. - Give each number with its site or model_set, AIFS version, variable, lead day, number of days and 95% interval where there is one. - Say that the results are provisional: no AIFS version has been scored over a full year. - If the data cannot answer my question, say so instead of estimating. - End with the data_version you used and this credit: MeteoSignal, Iman Soltanzadeh, meteosignal.com, CC BY 4.0. The data are under the Creative Commons Attribution 4.0 licence (https://creativecommons.org/licenses/by/4.0/), so the credit must stay with the results wherever they are shared. My question: - Add your question at the end
For example: "At the German stations, at which lead days was AIFS 2's temperature error lower than IFS's, and on how many days?"
- Every column describedType, meaning, unit and allowed values.
- The rules travel with the dataWhich numbers may be compared, what a verdict means, and that all results are provisional.
- A version on every rowEach row names its sample and the weekly run that wrote it, so an answer can be traced.
Some agents cannot open web addresses. Then download the files above and datapackage.json, and give them to the agent directly.
Where each result is shown
- 01GermanyHourly station observations (DWD open data). AIFS 2: 174 days, 23 May 2026 to 12 November 2026. Other AI models: 68 days.
- 02EuropeMeasurements from the national services of Germany, Austria, the Netherlands, Denmark, Switzerland and Finland. AIFS 2: 174 days, 23 May 2026 to 12 November 2026 for temperature and wind; solar energy 173 days.
- 03Cabauw towerMeasurements on KNMI's Cabauw tower. AIFS 2: 174 days, 23 May 2026 to 12 November 2026 for wind; temperature 172 days.
- 04ERA5ERA5 reanalysis on a 0.25° grid, 35 to 72°N and 12.5°W to 40°E. AIFS 2: 171 days, 23 May 2026 to 9 November 2026. Other AI models: 108 days.
- 05MethodHow to read the numbers, the method, sources and credits.
Rules for reading the numbers
- Compare numbers only within one file and, inside it, one site or model_set, one era and one track. The checks (the station sets, the Cabauw tower and ERA5) have their own places, days and caveats, and their numbers are not comparable with each other.
- A difference between AIFS and another model is a finding only where verdict is ai-leads or other-leads. cannot-separate means the two cannot be told apart on this sample, which is not evidence that they are equal. not-judged means the number shows a sign only.
- Comparisons in one family share days and places, so they are not independent results. Do not add up verdicts across files or sets as if they were: the German stations are also part of the europe set.
- All results are provisional. No AIFS version has been scored over a full year, so each sample covers only the seasons between its first_date and last_date. The verdicts are tested again at every weekly update and are interim until a version covers a full year. AIFS versions (era) cover different seasons and must not be compared with each other.
- Lower RMSE, MAE and absolute bias are better. Higher skill and anomaly correlation are better. A negative rmse_difference_ai_minus_other means AIFS had the lower error.
- ERA5 is a model product and AIFS was trained mostly on it, which probably favours AIFS in the era5 files. Take the size of any short-range lead from the station files.
- Values are point estimates over the whole sample with 95% intervals; monthly values are not in these files. The site does not publish which model is better at any single station.
- When quoting a number, give the site or model_set, the era, the variable, the lead day, the number of days and the data_version.
The columns
Skill, activity, anomaly correlation and relative differences are fractions, not per cent. Every other value is in the unit named in the row's unit column. Empty cells mean not scored or not applicable.
meteosignal-stations-scores.csvScores at the stations and the tower, 37 columns
One row per station set, AIFS version, variable, track, lead day and series: RMSE, MAE, bias and skill against two baselines, with 95% intervals from a stationary block bootstrap over forecast dates.
| Column | Type | Meaning |
|---|---|---|
site | string | The station set. Each set has its own days; compare numbers only within one set. Values: germany, europe, noaa-stations, cabauw-tower, germany-with-frozen-icon, germany-with-aigfs, germany-with-other-ai-models. |
era | string | AIFS version in service. Versions cover different seasons and are never pooled or compared. Values: v1.1, v2. |
variable | string | What is forecast. Values: t2m, wind10m, solar, wind100m, z500, t2m_era5, wind10m_era5. |
unit | string | Unit of every value in this row that is not a fraction or a probability. Values: °C, m/s, kWh/m2/day, gpm, K. |
track | string | Raw or bias-corrected forecasts. Values: raw, corrected. |
lead_days | integer | How many days ahead the forecast was made. Values: 1, 3, 5, 7, 10. |
days | integer | Number of forecast days in the sample. All models and climatology in one sample are scored on exactly the same pairs. At stations, persistence and the skill against it rest on persistence_days. |
first_date | string | First valid day of the sample, YYYY-MM-DD. |
last_date | string | Last valid day of the sample, YYYY-MM-DD. |
effective_days | number | About how many independent days the sample is worth, after allowing for day-to-day correlation. |
series | string | The model or baseline scored. Values: AIFS, IFS, GFS, ICON, AIGFS, GraphCast, FourCastNet v2, Pangu, Aurora, persistence, climatology. |
kind | string | What kind of series it is. Values: ai, physics, baseline. |
rmse | number | Root mean square error, in the row’s unit. Lower is better. |
rmse_low | number | Lower end of the 95% interval of rmse. |
rmse_high | number | Upper end of the 95% interval of rmse. |
rmse_raw_same_pairs | number | On the corrected track only: the RMSE of the raw forecasts on the same pairs, in the row’s unit. |
forecast_activity | number | Spread of the forecast’s departures from climatology over that of the truth, each station or grid cell centred on its own mean at each valid hour first, a fraction. 1 is as variable as the truth; below 1 is smoother. |
mae | number | Mean absolute error, in the row’s unit. Lower is better. |
mae_low | number | Lower end of the 95% interval of mae. |
mae_high | number | Upper end of the 95% interval of mae. |
bias | number | Mean error, forecast minus truth, in the row’s unit. 0 is unbiased. |
bias_low | number | Lower end of the 95% interval of bias. |
bias_high | number | Upper end of the 95% interval of bias. |
skill_vs_persistence | number | 1 minus the RMSE over the RMSE of persistence, a fraction, on the days and pairs where persistence can be scored (persistence_days). 1 is perfect, 0 no better than persistence, below 0 worse. |
skill_vs_persistence_low | number | Lower end of the 95% interval of skill_vs_persistence. |
skill_vs_persistence_high | number | Upper end of the 95% interval of skill_vs_persistence. |
skill_vs_climatology | number | 1 minus the RMSE over the RMSE of climatology, a fraction. 1 is perfect, 0 no better than climatology, below 0 worse. |
skill_vs_climatology_low | number | Lower end of the 95% interval of skill_vs_climatology. |
skill_vs_climatology_high | number | Upper end of the 95% interval of skill_vs_climatology. |
mse_skill_vs_persistence | number | 1 minus the ratio of mean square errors against persistence, a fraction. |
mse_skill_vs_climatology | number | 1 minus the ratio of mean square errors against climatology, a fraction. |
persistence_days | integer | Number of forecast days on which persistence could be scored at every lead as well as the models. The persistence series, skill_vs_persistence and mse_skill_vs_persistence rest on these days; every other value in the row rests on days. The two are equal where no earlier measurement is missing, and against ERA5. |
persistence_last_date | string | Last valid day of the days counted in persistence_days, YYYY-MM-DD. |
correction_change_in_rmse | number | On the corrected track: corrected RMSE minus raw RMSE on the same pairs, in the row’s unit. Negative: the correction lowered the error. |
correction_change_in_rmse_low | number | Lower end of the 95% interval of correction_change_in_rmse. |
correction_change_in_rmse_high | number | Upper end of the 95% interval of correction_change_in_rmse. |
data_version | string | The weekly run that wrote this row. Name it when quoting a number. |
meteosignal-stations-aifs-vs-models.csvAIFS against each other model at the stations and the tower, 26 columns
One row per station set, AIFS version, variable, track, lead day and other model: the paired RMSE difference of AIFS and that model on the same times and places, its interval, the test and the verdict.
| Column | Type | Meaning |
|---|---|---|
site | string | The station set. Each set has its own days; compare numbers only within one set. Values: germany, europe, noaa-stations, cabauw-tower, germany-with-frozen-icon, germany-with-aigfs, germany-with-other-ai-models. |
era | string | AIFS version in service. Versions cover different seasons and are never pooled or compared. Values: v1.1, v2. |
variable | string | What is forecast. Values: t2m, wind10m, solar, wind100m, z500, t2m_era5, wind10m_era5. |
unit | string | Unit of every value in this row that is not a fraction or a probability. Values: °C, m/s, kWh/m2/day, gpm, K. |
track | string | Raw or bias-corrected forecasts. Values: raw, corrected. |
lead_days | integer | How many days ahead the forecast was made. Values: 1, 3, 5, 7, 10. |
days | integer | Number of forecast days in the sample. All models and climatology in one sample are scored on exactly the same pairs. At stations, persistence and the skill against it rest on persistence_days. |
first_date | string | First valid day of the sample, YYYY-MM-DD. |
last_date | string | Last valid day of the sample, YYYY-MM-DD. |
effective_days | number | About how many independent days the sample is worth, after allowing for day-to-day correlation. |
compared_with | string | The model AIFS is compared with. Only comparisons of AIFS with another model are tested. Values: IFS, GFS, ICON, AIGFS, GraphCast, FourCastNet v2, Pangu, Aurora. |
rmse_difference_ai_minus_other | number | AIFS's RMSE minus the other model's, in the row’s unit. Negative: AIFS's error was lower (better). |
rmse_difference_ai_minus_other_low | number | Lower end of the 95% interval of rmse_difference_ai_minus_other. |
rmse_difference_ai_minus_other_high | number | Upper end of the 95% interval of rmse_difference_ai_minus_other. |
rmse_difference_relative | number | The same difference as a fraction of the other model's RMSE. -0.10 means AIFS's RMSE was 10% lower. |
p_value | number | p-value of the paired test, before correction. |
p_value_holm | number | p-value after the Holm correction within its family of comparisons. A verdict needs it below 0.01, which is lower than the 0.05 of the intervals because the test is repeated at every weekly update. |
family_size | integer | Number of comparisons in the Holm family. |
verdict | string | The verdict on the RMSE difference. Only ai-leads and other-leads are findings. Values: ai-leads, other-leads, cannot-separate, depends-on-reading, depends-on-height-correction, not-judged. |
borderline | boolean | Always false since a verdict needs a Holm-adjusted p-value below 0.01: it marked verdicts given at 0.02 or above when the level was 0.05. |
reason | string | Why no verdict was given where the test alone might have given one. Empty when there is nothing to explain. Values: too-few-blocks, unstable-halves, depends-on-reading, depends-on-height-correction, analysis-confounded, reference-confounded. |
mae_difference_ai_minus_other | number | The same comparison on MAE, in the row’s unit. |
mae_difference_ai_minus_other_low | number | Lower end of the 95% interval of mae_difference_ai_minus_other. |
mae_difference_ai_minus_other_high | number | Upper end of the 95% interval of mae_difference_ai_minus_other. |
mae_verdict | string | The verdict on the MAE difference. Values: ai-leads, other-leads, cannot-separate, depends-on-reading, depends-on-height-correction, not-judged. |
data_version | string | The weekly run that wrote this row. Name it when quoting a number. |
meteosignal-era5-scores.csvScores against ERA5 over Europe, 40 columns
One row per model set, AIFS version, variable, lead day and series, scored against ERA5 on a 0.25° grid from 35 to 72°N and 12.5°W to 40°E, weighted by latitude. ERA5 is a model product and AIFS was trained mostly on it: a second view, not a measurement.
| Column | Type | Meaning |
|---|---|---|
model_set | string | The group of models scored together on the same days. Compare numbers only within one set. Values: all, operational, aigfs. |
era | string | AIFS version in service. Versions cover different seasons and are never pooled or compared. Values: v1.1, v2. |
variable | string | What is forecast. Values: t2m, wind10m, solar, wind100m, z500, t2m_era5, wind10m_era5. |
unit | string | Unit of every value in this row that is not a fraction or a probability. Values: °C, m/s, kWh/m2/day, gpm, K. |
track | string | Raw or bias-corrected forecasts. Values: raw, corrected. |
lead_days | integer | How many days ahead the forecast was made. Values: 1, 3, 5, 7, 10. |
days | integer | Number of forecast days in the sample. All models and climatology in one sample are scored on exactly the same pairs. At stations, persistence and the skill against it rest on persistence_days. |
first_date | string | First valid day of the sample, YYYY-MM-DD. |
last_date | string | Last valid day of the sample, YYYY-MM-DD. |
effective_days | number | About how many independent days the sample is worth, after allowing for day-to-day correlation. |
series | string | The model or baseline scored. Values: AIFS, IFS, GFS, ICON, AIGFS, GraphCast, FourCastNet v2, Pangu, Aurora, persistence, climatology. |
kind | string | What kind of series it is. Values: ai, physics, baseline. |
rmse | number | Root mean square error, in the row’s unit. Lower is better. |
rmse_low | number | Lower end of the 95% interval of rmse. |
rmse_high | number | Upper end of the 95% interval of rmse. |
rmse_raw_same_pairs | number | On the corrected track only: the RMSE of the raw forecasts on the same pairs, in the row’s unit. |
forecast_activity | number | Spread of the forecast’s departures from climatology over that of the truth, each station or grid cell centred on its own mean at each valid hour first, a fraction. 1 is as variable as the truth; below 1 is smoother. |
mae | number | Mean absolute error, in the row’s unit. Lower is better. |
mae_low | number | Lower end of the 95% interval of mae. |
mae_high | number | Upper end of the 95% interval of mae. |
bias | number | Mean error, forecast minus truth, in the row’s unit. 0 is unbiased. |
bias_low | number | Lower end of the 95% interval of bias. |
bias_high | number | Upper end of the 95% interval of bias. |
skill_vs_persistence | number | 1 minus the RMSE over the RMSE of persistence, a fraction, on the days and pairs where persistence can be scored (persistence_days). 1 is perfect, 0 no better than persistence, below 0 worse. |
skill_vs_persistence_low | number | Lower end of the 95% interval of skill_vs_persistence. |
skill_vs_persistence_high | number | Upper end of the 95% interval of skill_vs_persistence. |
skill_vs_climatology | number | 1 minus the RMSE over the RMSE of climatology, a fraction. 1 is perfect, 0 no better than climatology, below 0 worse. |
skill_vs_climatology_low | number | Lower end of the 95% interval of skill_vs_climatology. |
skill_vs_climatology_high | number | Upper end of the 95% interval of skill_vs_climatology. |
mse_skill_vs_persistence | number | 1 minus the ratio of mean square errors against persistence, a fraction. |
mse_skill_vs_climatology | number | 1 minus the ratio of mean square errors against climatology, a fraction. |
persistence_days | integer | Number of forecast days on which persistence could be scored at every lead as well as the models. The persistence series, skill_vs_persistence and mse_skill_vs_persistence rest on these days; every other value in the row rests on days. The two are equal where no earlier measurement is missing, and against ERA5. |
persistence_last_date | string | Last valid day of the days counted in persistence_days, YYYY-MM-DD. |
anomaly_correlation | number | Uncentred anomaly correlation with ERA5 against the WeatherBench 2 climatology of 1990 to 2019, a fraction. 1 is perfect. |
anomaly_correlation_low | number | Lower end of the 95% interval of anomaly_correlation. |
anomaly_correlation_high | number | Upper end of the 95% interval of anomaly_correlation. |
anomaly_correlation_centred | number | The same with the area-mean anomaly of each field removed first, a fraction. |
anomaly_correlation_centred_low | number | Lower end of the 95% interval of anomaly_correlation_centred. |
anomaly_correlation_centred_high | number | Upper end of the 95% interval of anomaly_correlation_centred. |
data_version | string | The weekly run that wrote this row. Name it when quoting a number. |
meteosignal-era5-aifs-vs-models.csvAIFS against each other model, against ERA5 over Europe, 26 columns
One row per model set, AIFS version, variable, lead day and other model: the paired RMSE difference against ERA5, its interval, the test and the verdict. Only 500 hPa height after day 1 is judged.
| Column | Type | Meaning |
|---|---|---|
model_set | string | The group of models scored together on the same days. Compare numbers only within one set. Values: all, operational, aigfs. |
era | string | AIFS version in service. Versions cover different seasons and are never pooled or compared. Values: v1.1, v2. |
variable | string | What is forecast. Values: t2m, wind10m, solar, wind100m, z500, t2m_era5, wind10m_era5. |
unit | string | Unit of every value in this row that is not a fraction or a probability. Values: °C, m/s, kWh/m2/day, gpm, K. |
track | string | Raw or bias-corrected forecasts. Values: raw, corrected. |
lead_days | integer | How many days ahead the forecast was made. Values: 1, 3, 5, 7, 10. |
days | integer | Number of forecast days in the sample. All models and climatology in one sample are scored on exactly the same pairs. At stations, persistence and the skill against it rest on persistence_days. |
first_date | string | First valid day of the sample, YYYY-MM-DD. |
last_date | string | Last valid day of the sample, YYYY-MM-DD. |
effective_days | number | About how many independent days the sample is worth, after allowing for day-to-day correlation. |
compared_with | string | The model AIFS is compared with. Only comparisons of AIFS with another model are tested. Values: IFS, GFS, ICON, AIGFS, GraphCast, FourCastNet v2, Pangu, Aurora. |
rmse_difference_ai_minus_other | number | AIFS's RMSE minus the other model's, in the row’s unit. Negative: AIFS's error was lower (better). |
rmse_difference_ai_minus_other_low | number | Lower end of the 95% interval of rmse_difference_ai_minus_other. |
rmse_difference_ai_minus_other_high | number | Upper end of the 95% interval of rmse_difference_ai_minus_other. |
rmse_difference_relative | number | The same difference as a fraction of the other model's RMSE. -0.10 means AIFS's RMSE was 10% lower. |
p_value | number | p-value of the paired test, before correction. |
p_value_holm | number | p-value after the Holm correction within its family of comparisons. A verdict needs it below 0.01, which is lower than the 0.05 of the intervals because the test is repeated at every weekly update. |
family_size | integer | Number of comparisons in the Holm family. |
verdict | string | The verdict on the RMSE difference. Only ai-leads and other-leads are findings. Values: ai-leads, other-leads, cannot-separate, depends-on-reading, depends-on-height-correction, not-judged. |
borderline | boolean | Always false since a verdict needs a Holm-adjusted p-value below 0.01: it marked verdicts given at 0.02 or above when the level was 0.05. |
reason | string | Why no verdict was given where the test alone might have given one. Empty when there is nothing to explain. Values: too-few-blocks, unstable-halves, depends-on-reading, depends-on-height-correction, analysis-confounded, reference-confounded. |
mae_difference_ai_minus_other | number | The same comparison on MAE, in the row’s unit. |
mae_difference_ai_minus_other_low | number | Lower end of the 95% interval of mae_difference_ai_minus_other. |
mae_difference_ai_minus_other_high | number | Upper end of the 95% interval of mae_difference_ai_minus_other. |
mae_verdict | string | The verdict on the MAE difference. Values: ai-leads, other-leads, cannot-separate, depends-on-reading, depends-on-height-correction, not-judged. |
data_version | string | The weekly run that wrote this row. Name it when quoting a number. |
Data version
Data version , run 20261116T194902Z, code 048c5a9. The files are rebuilt from the raw sources at every update, so earlier values can change. Name the data version when you quote a number.
Licence
Results and text on this site are licensed under Creative Commons Attribution 4.0. You may copy, share and adapt them if you give credit: "MeteoSignal, Iman Soltanzadeh, meteosignal.com, CC BY 4.0". The code is under the MIT licence. The source data keep their own terms, listed on the method page.
How to cite
Soltanzadeh, I. (2026). MeteoSignal: verification of open weather forecasts over Europe. Data version 16 November 2026, run 20261116T194902Z. https://meteosignal.com/