MeteoSignal

The scores as data, ready for AI agents.

The scores and tested comparisons behind the station, tower and ERA5 pages in four CSV files, with a description an AI agent can read: every column, its unit, and the rules for reading the numbers. Free to use under CC BY 4.0, updated with each weekly run.

Summary sheetThe results on one page: the tested comparisons in each check, temperature and wind, an event to watch and what changed since the last update. Prints as two A4 pages with sources and citation.

The filesUse with an AI agent

Last reviewed 16 November 2026Measurements to 12 November 2026; NOAA stations to 10 November 2026; ERA5 to 9 November 2026

Sankt Peter-OrdingDörnickUeckermündeNorderneyHamburg-FuhlsbüttelBremenDiepholzBraunschweigLindenbergLügde-PaenbruchLeinefeldeGörlitzDresden-KlotzscheKöln/BonnAachen-OrsbachGießen/WettenbergNürburg-BarweilerLautertal-OberlauterWürzburgMannheimNürnbergWaldmünchenSaarbrücken-EnsheimStuttgart (Schnarrenberg)FürstenzellAugsburgWeihenstephan-DürnastMühldorfChiemingKonstanzHelsinki-Vantaa lentoasemaTylstrupMejrupIsenvadHvide SandeBorrisAskovÅrslevTystofteAssens/TorøNexø VestStore JyndevadAbedGedserLitschauRaabs an der ThayaRetzLaa an der Thaya KläranlageAllentsteigPoysdorf OstSchöngrabernGars am KampReichenau im MühlkreisZwerndorfLangenlebarnWien StammersdorfWien Hohe WarteRanshofen BraunauRied im InnkreisEnnsWels-SchleißheimOstermiethingLilienfeld-TarschbergBruckneudorfMattseeSeibersdorfWiener Neustadt FlugplatzMattersburgBischofszell / SitterdorfAadorf / TänikonZürich / KlotenLutzmannsburgGleisdorfGüssingGraz StraßgangFürstenfeldBern / ZollikofenFeldbachFribourg / GrangeneuveLeibnitz-WagnaMaastricht AirportWestdorpeVlissingenEindhoven AirportGilze-Rijen AirportVolkelRotterdam AirportCabauwHerwijnenVoorschotenDe BiltDeelen AirportHupselSchiphol AirportLelystad AirportHeinoTwenthe AirportBerkhoutMarknesseDe Kooy AirportStavorenHoogeveenLeeuwarden AirportGroningen Airport EeldeNieuw BeertaHoorn TerschellingLauwersoogPRAHA-KBELYPRAHA-LIBUSKOCELOVICELUGO/ROZASVALLADOLIDCACERESHUELVAKERKYRA IONIAN IS GREECE (KERKTATOISYROS GREECEMACE HEADGURTEENJOHNSTOWN CASTLEVALENTIA OBSERVATORYTREVISO ITALY (S ANGELO)CERVIAROME ITALY (CIAMPINO)AMENDOLA ITALYNAPLESGIOIA DEL COLLE ITALYKLAIPEDAVILNIUS INTLJAN MAYEN(NOR-NAVY)VARDOTROMSOANDOYABO I VESTERALENHELLIGVAER IISULAVIGRAFEDJEFLESLANDKOLOBRZEGBOTOSANITARGU LAPUSHOLODVASLUIGALATIBANLOCTARGU JIUCALARASIROSIORII DE VEDECALAFATGIURGIUMANGALIABECHETJOKKMOKK FPLPETISTRASKGADDEDE AORNSKOLDSVIKTANNASSVANBERGASATENASMALMENHARSTENAGOTEBORGRONNEBYZONGULDAKSABIHA GOKCENBANDIRMAADANA/BOLGECabauw
  • DWD stations in Germany (30)
  • National services: Austria 31, Netherlands 27, Denmark 13, Switzerland 5, Finland 1 (77)
  • NOAA hourly reports, 11 further countries (61)
  • Cabauw tower, Netherlands (1)
Every place a forecast is checked against a measurement. Hover over or tap a point to see the site.

The files

Data version 16 November 2026, free to use under CC BY 4.0. The columns of each file are explained further down.

The files to download
FileWhat it holdsRowsColumnsSize
meteosignal-stations-scores.csvScores at the stations and the tower. RMSE, MAE, bias, skill and activity of every model and baseline, with 95% intervals. One row per station set, AIFS version, variable, track, lead day and series.1,608 rows · 37 columns · 377 kB1,60837377 kB
meteosignal-stations-aifs-vs-models.csvAIFS against each other model at the stations and the tower. The paired RMSE difference, its interval, the test and the verdict. One row per station set, AIFS version, variable, track, lead day and other model.588 rows · 26 columns · 117 kB58826117 kB
meteosignal-era5-scores.csvScores against ERA5 over Europe. RMSE, MAE, bias, skill, activity and anomaly correlation on a grid over Europe, with 95% intervals. One row per model set, AIFS version, variable, lead day and series.450 rows · 40 columns · 115 kB45040115 kB
meteosignal-era5-aifs-vs-models.csvAIFS against each other model, against ERA5 over Europe. The paired RMSE difference against ERA5, its interval, the test and the verdict. One row per model set, AIFS version, variable, lead day and other model.270 rows · 26 columns · 53 kB2702653 kB
datapackage.jsonThe description of all 4 files. For software and AI agents: every column with its type, unit and allowed values, and the rules for reading the numbers. In the open Frictionless Data Package format.60 kB––60 kB

Use the data with an AI agent

The files come with a description an agent can read, in the open Frictionless Data Package format: what every column means, its unit and allowed values, and the rules for reading the numbers. Point your agent at it and it can load the data and answer questions without guessing.

  1. Copy this prompt into your agent

    It tells the agent what the data are, where the files are, and the rules for reading them.

    You are helping me analyse forecast verification data from MeteoSignal (meteosignal.com): an independent comparison of ECMWF's AI weather model AIFS with IFS, GFS and other models, scored against weather stations in Europe, a measurement tower in the Netherlands and the ERA5 reanalysis. 1. Read the description first: https://meteosignal.com/scores/datapackage.json It explains every file and column, the units and allowed values, and the rules for reading the numbers. Follow those rules. 2. Load the files it lists: - https://meteosignal.com/scores/meteosignal-stations-scores.csv - https://meteosignal.com/scores/meteosignal-stations-aifs-vs-models.csv - https://meteosignal.com/scores/meteosignal-era5-scores.csv - https://meteosignal.com/scores/meteosignal-era5-aifs-vs-models.csv 3. When you answer: - Compare numbers only within one site or model_set, one AIFS version (era) and one track. The German stations are also part of the europe set, and the ERA5 files are a separate check. - Treat a difference between AIFS and another model as a finding only where verdict is ai-leads or other-leads. Say so plainly when the models cannot be separated. - Give each number with its site or model_set, AIFS version, variable, lead day, number of days and 95% interval where there is one. - Say that the results are provisional: no AIFS version has been scored over a full year. - If the data cannot answer my question, say so instead of estimating. - End with the data_version you used and this credit: MeteoSignal, Iman Soltanzadeh, meteosignal.com, CC BY 4.0. The data are under the Creative Commons Attribution 4.0 licence (https://creativecommons.org/licenses/by/4.0/), so the credit must stay with the results wherever they are shared. My question:
  2. Add your question at the end

    For example: "At the German stations, at which lead days was AIFS 2's temperature error lower than IFS's, and on how many days?"

Some agents cannot open web addresses. Then download the files above and datapackage.json, and give them to the agent directly.

Where each result is shown

  1. 01GermanyHourly station observations (DWD open data). AIFS 2: 174 days, 23 May 2026 to 12 November 2026. Other AI models: 68 days.
  2. 02EuropeMeasurements from the national services of Germany, Austria, the Netherlands, Denmark, Switzerland and Finland. AIFS 2: 174 days, 23 May 2026 to 12 November 2026 for temperature and wind; solar energy 173 days.
  3. 03Cabauw towerMeasurements on KNMI's Cabauw tower. AIFS 2: 174 days, 23 May 2026 to 12 November 2026 for wind; temperature 172 days.
  4. 04ERA5ERA5 reanalysis on a 0.25° grid, 35 to 72°N and 12.5°W to 40°E. AIFS 2: 171 days, 23 May 2026 to 9 November 2026. Other AI models: 108 days.
  5. 05MethodHow to read the numbers, the method, sources and credits.

Rules for reading the numbers

The columns

Skill, activity, anomaly correlation and relative differences are fractions, not per cent. Every other value is in the unit named in the row's unit column. Empty cells mean not scored or not applicable.

meteosignal-stations-scores.csvScores at the stations and the tower, 37 columns

One row per station set, AIFS version, variable, track, lead day and series: RMSE, MAE, bias and skill against two baselines, with 95% intervals from a stationary block bootstrap over forecast dates.

The columns of meteosignal-stations-scores.csv
ColumnTypeMeaning
sitestringThe station set. Each set has its own days; compare numbers only within one set. Values: germany, europe, noaa-stations, cabauw-tower, germany-with-frozen-icon, germany-with-aigfs, germany-with-other-ai-models.
erastringAIFS version in service. Versions cover different seasons and are never pooled or compared. Values: v1.1, v2.
variablestringWhat is forecast. Values: t2m, wind10m, solar, wind100m, z500, t2m_era5, wind10m_era5.
unitstringUnit of every value in this row that is not a fraction or a probability. Values: °C, m/s, kWh/m2/day, gpm, K.
trackstringRaw or bias-corrected forecasts. Values: raw, corrected.
lead_daysintegerHow many days ahead the forecast was made. Values: 1, 3, 5, 7, 10.
daysintegerNumber of forecast days in the sample. All models and climatology in one sample are scored on exactly the same pairs. At stations, persistence and the skill against it rest on persistence_days.
first_datestringFirst valid day of the sample, YYYY-MM-DD.
last_datestringLast valid day of the sample, YYYY-MM-DD.
effective_daysnumberAbout how many independent days the sample is worth, after allowing for day-to-day correlation.
seriesstringThe model or baseline scored. Values: AIFS, IFS, GFS, ICON, AIGFS, GraphCast, FourCastNet v2, Pangu, Aurora, persistence, climatology.
kindstringWhat kind of series it is. Values: ai, physics, baseline.
rmsenumberRoot mean square error, in the row’s unit. Lower is better.
rmse_lownumberLower end of the 95% interval of rmse.
rmse_highnumberUpper end of the 95% interval of rmse.
rmse_raw_same_pairsnumberOn the corrected track only: the RMSE of the raw forecasts on the same pairs, in the row’s unit.
forecast_activitynumberSpread of the forecast’s departures from climatology over that of the truth, each station or grid cell centred on its own mean at each valid hour first, a fraction. 1 is as variable as the truth; below 1 is smoother.
maenumberMean absolute error, in the row’s unit. Lower is better.
mae_lownumberLower end of the 95% interval of mae.
mae_highnumberUpper end of the 95% interval of mae.
biasnumberMean error, forecast minus truth, in the row’s unit. 0 is unbiased.
bias_lownumberLower end of the 95% interval of bias.
bias_highnumberUpper end of the 95% interval of bias.
skill_vs_persistencenumber1 minus the RMSE over the RMSE of persistence, a fraction, on the days and pairs where persistence can be scored (persistence_days). 1 is perfect, 0 no better than persistence, below 0 worse.
skill_vs_persistence_lownumberLower end of the 95% interval of skill_vs_persistence.
skill_vs_persistence_highnumberUpper end of the 95% interval of skill_vs_persistence.
skill_vs_climatologynumber1 minus the RMSE over the RMSE of climatology, a fraction. 1 is perfect, 0 no better than climatology, below 0 worse.
skill_vs_climatology_lownumberLower end of the 95% interval of skill_vs_climatology.
skill_vs_climatology_highnumberUpper end of the 95% interval of skill_vs_climatology.
mse_skill_vs_persistencenumber1 minus the ratio of mean square errors against persistence, a fraction.
mse_skill_vs_climatologynumber1 minus the ratio of mean square errors against climatology, a fraction.
persistence_daysintegerNumber of forecast days on which persistence could be scored at every lead as well as the models. The persistence series, skill_vs_persistence and mse_skill_vs_persistence rest on these days; every other value in the row rests on days. The two are equal where no earlier measurement is missing, and against ERA5.
persistence_last_datestringLast valid day of the days counted in persistence_days, YYYY-MM-DD.
correction_change_in_rmsenumberOn the corrected track: corrected RMSE minus raw RMSE on the same pairs, in the row’s unit. Negative: the correction lowered the error.
correction_change_in_rmse_lownumberLower end of the 95% interval of correction_change_in_rmse.
correction_change_in_rmse_highnumberUpper end of the 95% interval of correction_change_in_rmse.
data_versionstringThe weekly run that wrote this row. Name it when quoting a number.
meteosignal-stations-aifs-vs-models.csvAIFS against each other model at the stations and the tower, 26 columns

One row per station set, AIFS version, variable, track, lead day and other model: the paired RMSE difference of AIFS and that model on the same times and places, its interval, the test and the verdict.

The columns of meteosignal-stations-aifs-vs-models.csv
ColumnTypeMeaning
sitestringThe station set. Each set has its own days; compare numbers only within one set. Values: germany, europe, noaa-stations, cabauw-tower, germany-with-frozen-icon, germany-with-aigfs, germany-with-other-ai-models.
erastringAIFS version in service. Versions cover different seasons and are never pooled or compared. Values: v1.1, v2.
variablestringWhat is forecast. Values: t2m, wind10m, solar, wind100m, z500, t2m_era5, wind10m_era5.
unitstringUnit of every value in this row that is not a fraction or a probability. Values: °C, m/s, kWh/m2/day, gpm, K.
trackstringRaw or bias-corrected forecasts. Values: raw, corrected.
lead_daysintegerHow many days ahead the forecast was made. Values: 1, 3, 5, 7, 10.
daysintegerNumber of forecast days in the sample. All models and climatology in one sample are scored on exactly the same pairs. At stations, persistence and the skill against it rest on persistence_days.
first_datestringFirst valid day of the sample, YYYY-MM-DD.
last_datestringLast valid day of the sample, YYYY-MM-DD.
effective_daysnumberAbout how many independent days the sample is worth, after allowing for day-to-day correlation.
compared_withstringThe model AIFS is compared with. Only comparisons of AIFS with another model are tested. Values: IFS, GFS, ICON, AIGFS, GraphCast, FourCastNet v2, Pangu, Aurora.
rmse_difference_ai_minus_othernumberAIFS's RMSE minus the other model's, in the row’s unit. Negative: AIFS's error was lower (better).
rmse_difference_ai_minus_other_lownumberLower end of the 95% interval of rmse_difference_ai_minus_other.
rmse_difference_ai_minus_other_highnumberUpper end of the 95% interval of rmse_difference_ai_minus_other.
rmse_difference_relativenumberThe same difference as a fraction of the other model's RMSE. -0.10 means AIFS's RMSE was 10% lower.
p_valuenumberp-value of the paired test, before correction.
p_value_holmnumberp-value after the Holm correction within its family of comparisons. A verdict needs it below 0.01, which is lower than the 0.05 of the intervals because the test is repeated at every weekly update.
family_sizeintegerNumber of comparisons in the Holm family.
verdictstringThe verdict on the RMSE difference. Only ai-leads and other-leads are findings. Values: ai-leads, other-leads, cannot-separate, depends-on-reading, depends-on-height-correction, not-judged.
borderlinebooleanAlways false since a verdict needs a Holm-adjusted p-value below 0.01: it marked verdicts given at 0.02 or above when the level was 0.05.
reasonstringWhy no verdict was given where the test alone might have given one. Empty when there is nothing to explain. Values: too-few-blocks, unstable-halves, depends-on-reading, depends-on-height-correction, analysis-confounded, reference-confounded.
mae_difference_ai_minus_othernumberThe same comparison on MAE, in the row’s unit.
mae_difference_ai_minus_other_lownumberLower end of the 95% interval of mae_difference_ai_minus_other.
mae_difference_ai_minus_other_highnumberUpper end of the 95% interval of mae_difference_ai_minus_other.
mae_verdictstringThe verdict on the MAE difference. Values: ai-leads, other-leads, cannot-separate, depends-on-reading, depends-on-height-correction, not-judged.
data_versionstringThe weekly run that wrote this row. Name it when quoting a number.
meteosignal-era5-scores.csvScores against ERA5 over Europe, 40 columns

One row per model set, AIFS version, variable, lead day and series, scored against ERA5 on a 0.25° grid from 35 to 72°N and 12.5°W to 40°E, weighted by latitude. ERA5 is a model product and AIFS was trained mostly on it: a second view, not a measurement.

The columns of meteosignal-era5-scores.csv
ColumnTypeMeaning
model_setstringThe group of models scored together on the same days. Compare numbers only within one set. Values: all, operational, aigfs.
erastringAIFS version in service. Versions cover different seasons and are never pooled or compared. Values: v1.1, v2.
variablestringWhat is forecast. Values: t2m, wind10m, solar, wind100m, z500, t2m_era5, wind10m_era5.
unitstringUnit of every value in this row that is not a fraction or a probability. Values: °C, m/s, kWh/m2/day, gpm, K.
trackstringRaw or bias-corrected forecasts. Values: raw, corrected.
lead_daysintegerHow many days ahead the forecast was made. Values: 1, 3, 5, 7, 10.
daysintegerNumber of forecast days in the sample. All models and climatology in one sample are scored on exactly the same pairs. At stations, persistence and the skill against it rest on persistence_days.
first_datestringFirst valid day of the sample, YYYY-MM-DD.
last_datestringLast valid day of the sample, YYYY-MM-DD.
effective_daysnumberAbout how many independent days the sample is worth, after allowing for day-to-day correlation.
seriesstringThe model or baseline scored. Values: AIFS, IFS, GFS, ICON, AIGFS, GraphCast, FourCastNet v2, Pangu, Aurora, persistence, climatology.
kindstringWhat kind of series it is. Values: ai, physics, baseline.
rmsenumberRoot mean square error, in the row’s unit. Lower is better.
rmse_lownumberLower end of the 95% interval of rmse.
rmse_highnumberUpper end of the 95% interval of rmse.
rmse_raw_same_pairsnumberOn the corrected track only: the RMSE of the raw forecasts on the same pairs, in the row’s unit.
forecast_activitynumberSpread of the forecast’s departures from climatology over that of the truth, each station or grid cell centred on its own mean at each valid hour first, a fraction. 1 is as variable as the truth; below 1 is smoother.
maenumberMean absolute error, in the row’s unit. Lower is better.
mae_lownumberLower end of the 95% interval of mae.
mae_highnumberUpper end of the 95% interval of mae.
biasnumberMean error, forecast minus truth, in the row’s unit. 0 is unbiased.
bias_lownumberLower end of the 95% interval of bias.
bias_highnumberUpper end of the 95% interval of bias.
skill_vs_persistencenumber1 minus the RMSE over the RMSE of persistence, a fraction, on the days and pairs where persistence can be scored (persistence_days). 1 is perfect, 0 no better than persistence, below 0 worse.
skill_vs_persistence_lownumberLower end of the 95% interval of skill_vs_persistence.
skill_vs_persistence_highnumberUpper end of the 95% interval of skill_vs_persistence.
skill_vs_climatologynumber1 minus the RMSE over the RMSE of climatology, a fraction. 1 is perfect, 0 no better than climatology, below 0 worse.
skill_vs_climatology_lownumberLower end of the 95% interval of skill_vs_climatology.
skill_vs_climatology_highnumberUpper end of the 95% interval of skill_vs_climatology.
mse_skill_vs_persistencenumber1 minus the ratio of mean square errors against persistence, a fraction.
mse_skill_vs_climatologynumber1 minus the ratio of mean square errors against climatology, a fraction.
persistence_daysintegerNumber of forecast days on which persistence could be scored at every lead as well as the models. The persistence series, skill_vs_persistence and mse_skill_vs_persistence rest on these days; every other value in the row rests on days. The two are equal where no earlier measurement is missing, and against ERA5.
persistence_last_datestringLast valid day of the days counted in persistence_days, YYYY-MM-DD.
anomaly_correlationnumberUncentred anomaly correlation with ERA5 against the WeatherBench 2 climatology of 1990 to 2019, a fraction. 1 is perfect.
anomaly_correlation_lownumberLower end of the 95% interval of anomaly_correlation.
anomaly_correlation_highnumberUpper end of the 95% interval of anomaly_correlation.
anomaly_correlation_centrednumberThe same with the area-mean anomaly of each field removed first, a fraction.
anomaly_correlation_centred_lownumberLower end of the 95% interval of anomaly_correlation_centred.
anomaly_correlation_centred_highnumberUpper end of the 95% interval of anomaly_correlation_centred.
data_versionstringThe weekly run that wrote this row. Name it when quoting a number.
meteosignal-era5-aifs-vs-models.csvAIFS against each other model, against ERA5 over Europe, 26 columns

One row per model set, AIFS version, variable, lead day and other model: the paired RMSE difference against ERA5, its interval, the test and the verdict. Only 500 hPa height after day 1 is judged.

The columns of meteosignal-era5-aifs-vs-models.csv
ColumnTypeMeaning
model_setstringThe group of models scored together on the same days. Compare numbers only within one set. Values: all, operational, aigfs.
erastringAIFS version in service. Versions cover different seasons and are never pooled or compared. Values: v1.1, v2.
variablestringWhat is forecast. Values: t2m, wind10m, solar, wind100m, z500, t2m_era5, wind10m_era5.
unitstringUnit of every value in this row that is not a fraction or a probability. Values: °C, m/s, kWh/m2/day, gpm, K.
trackstringRaw or bias-corrected forecasts. Values: raw, corrected.
lead_daysintegerHow many days ahead the forecast was made. Values: 1, 3, 5, 7, 10.
daysintegerNumber of forecast days in the sample. All models and climatology in one sample are scored on exactly the same pairs. At stations, persistence and the skill against it rest on persistence_days.
first_datestringFirst valid day of the sample, YYYY-MM-DD.
last_datestringLast valid day of the sample, YYYY-MM-DD.
effective_daysnumberAbout how many independent days the sample is worth, after allowing for day-to-day correlation.
compared_withstringThe model AIFS is compared with. Only comparisons of AIFS with another model are tested. Values: IFS, GFS, ICON, AIGFS, GraphCast, FourCastNet v2, Pangu, Aurora.
rmse_difference_ai_minus_othernumberAIFS's RMSE minus the other model's, in the row’s unit. Negative: AIFS's error was lower (better).
rmse_difference_ai_minus_other_lownumberLower end of the 95% interval of rmse_difference_ai_minus_other.
rmse_difference_ai_minus_other_highnumberUpper end of the 95% interval of rmse_difference_ai_minus_other.
rmse_difference_relativenumberThe same difference as a fraction of the other model's RMSE. -0.10 means AIFS's RMSE was 10% lower.
p_valuenumberp-value of the paired test, before correction.
p_value_holmnumberp-value after the Holm correction within its family of comparisons. A verdict needs it below 0.01, which is lower than the 0.05 of the intervals because the test is repeated at every weekly update.
family_sizeintegerNumber of comparisons in the Holm family.
verdictstringThe verdict on the RMSE difference. Only ai-leads and other-leads are findings. Values: ai-leads, other-leads, cannot-separate, depends-on-reading, depends-on-height-correction, not-judged.
borderlinebooleanAlways false since a verdict needs a Holm-adjusted p-value below 0.01: it marked verdicts given at 0.02 or above when the level was 0.05.
reasonstringWhy no verdict was given where the test alone might have given one. Empty when there is nothing to explain. Values: too-few-blocks, unstable-halves, depends-on-reading, depends-on-height-correction, analysis-confounded, reference-confounded.
mae_difference_ai_minus_othernumberThe same comparison on MAE, in the row’s unit.
mae_difference_ai_minus_other_lownumberLower end of the 95% interval of mae_difference_ai_minus_other.
mae_difference_ai_minus_other_highnumberUpper end of the 95% interval of mae_difference_ai_minus_other.
mae_verdictstringThe verdict on the MAE difference. Values: ai-leads, other-leads, cannot-separate, depends-on-reading, depends-on-height-correction, not-judged.
data_versionstringThe weekly run that wrote this row. Name it when quoting a number.

Data version

Data version , run 20261116T194902Z, code 048c5a9. The files are rebuilt from the raw sources at every update, so earlier values can change. Name the data version when you quote a number.

Licence

Results and text on this site are licensed under Creative Commons Attribution 4.0. You may copy, share and adapt them if you give credit: "MeteoSignal, Iman Soltanzadeh, meteosignal.com, CC BY 4.0". The code is under the MIT licence. The source data keep their own terms, listed on the method page.

How to cite

Soltanzadeh, I. (2026). MeteoSignal: verification of open weather forecasts over Europe. Data version 16 November 2026, run 20261116T194902Z. https://meteosignal.com/