The Offshore Voyaging Reference Site

9 Tips to Assess Weather Forecast Accuracy

I have often expressed my worry about the number of cruisers who are overly relying on forecast tools that present raw weather-model output in cool ways:

  • PredictWind
  • Windy
  • Luckgrib
  • And many others

All these sources use the same underlying government sources. Each is just a different way to make those data sets look pretty.

To present a different view, I wrote an entire Online Book on how to reduce the risk from unexpected weather outcomes by adding in, and gaining understanding of, other weather sources.

But it’s some time since we have updated said book—on the schedule for this winter—so here’s a new chapter with a concise approach to assessing forecast accuracy. Think of it is a kind of Coles Notes to the Online Book.

Why Does It Matter?

But first, why does this matter? Modern models are quite accurate, so surely all we have to do is look at the GRIBs outputted by them, in whatever viewer we like best (see above), and all is good?

No, here’s why:

A weather forecast (or model output) is only the most likely of several possible scenarios.


For example, a purely GRIB-based passage plan might forecast a voyage to Bermuda with no more than 30 knots of sustained wind, even when there’s a 33% chance of sustained storm-force winds—the former a boisterous sail in a well-found boat, the latter verging on survival conditions in any boat, particularly one with a crew who had not planned for that eventuality.

And making this effort is worth it even if we decide to make the passage despite the threat, as I probably would if trying to get south in the fall with the winter fast approaching, given that at that time of year a 2-in-3 chance of getting to Bermuda without a gale or worse is probably as good as it gets1.

If we actually want to make voyages, we can’t be too timid either.

Anyway, whether we go or not, knowing that two of the six chambers (33%) in the weather revolver are loaded with storm-force winds will make us far safer mariners than unrealistically assuming that the revolver is not loaded, based on a single forecast or routing source, particularly a raw-from-computer source.

Forecast-Error Risk Assessment

So let’s dig into tools we can use to assess the chances of more severe weather than that forecast. The good news is that the process is easier and less time consuming than it was just a few years ago.

Login to continue reading (scroll down)

Inline Feedbacks
View all comments
Alastair Currie

Thanks for the tips I will be using them. Over the decades I have moved from plotting my own weather charts to full on Predict Wind and others. Hence your tips are a good tool to use.
In the UK we have access to coastal reports of actual weather data, transmitted on Navtex, or the shipping forecast and of course online. It can be useful to look at actual pressures, wind and weather at the coastal stations and see if they are corresponding with the tabulated wind and weather data for the models.

The idea is that the model that shows the most consistent correlation might be the most accurate looking forward. I would like to say that the trend works in the context of determining look ahead accuracy, but I have not established that because of low sample size.

However, for the immediate future, say at 24 hours, when the wind strength range is broad, it can be accurate in testing what model is likely to be relevant, if the pressures, strengths, direction and weather have been panning out as per the prediction.

Robert Johnson

Hi John,

Thank you for highlighting certain weather forecasting products and explaining how sailors can use them to be safe. This article alone justifies the price of admission.

However, saying that global weather models don’t account for geographic features is not exactly correct. If that were true, how would one explain this wind forecast. (Please see attached image from the GFS model) Cold air from the approaching front is funneled through the Coatzacoalcos River valley and is hemmed in by the mountains to the west in Oaxaca.

The GFS is a coupled model that simulates the atmosphere, oceans, land, and sea ice differently, but it includes interactions between those. The land model includes soil type, vegetation, and terrain features. (see for example 1.2.3 on this web page

Now the effect of terrain is influenced by the grid size, so some smaller-scale features, like narrow canyons, may not be included. Some models, like the proprietary PWG, include run on a smaller scale grid in certain areas.

Thanks again for teaching me so much about sailing!


Jim Schulz

Interesting to read your piece in light of last night’s hurricane landfall in Acapulco. Going back and looking at archives from the nhc site it seems it might have caught forecasters by surprise, especially given the location and the lack of historical storms hitting there. Seems a good advertisement for vigilance and not complacency whenever there’s a storm in the vicinity.

Matt Marsh

I saw that one too….
There is always an element of uncertainty around storms. Forecast models are getting better, but they’re still initialized with sparse data and are run on grids many orders of magnitude coarser than the physical effects they’re trying to simulate.

For planning and preparation purposes, we always need to err on the side of caution, i.e. always asking ourselves “what is the worst possible scenario that could evolve from these current conditions?” We may make a conscious decision that the probability of such a worst-case scenario is low enough to take a chance anyway…. but if we don’t consider the possibility, it will come to pass sooner or later, and we’ll be unprepared.

Also, I’m going to add my voice to the chorus of frustration of not being able to see fronts and troughs in all the fancy shiny new web-based GRIB viewers. Introductory weather courses are almost always structured around understanding highs, lows, ridges, troughs, and fronts, because that’s where the interesting weather comes from. If you look at data visualizations with those things omitted, you’re putting an awful lot of faith in incomplete information.

Colin Speedie

This a timely article indeed.

This summer, bringing our old boat home was an exercise in disappointment as the chaotic weather situation in our part of the NE Atlantic confounded our attempts to get home, Occlusions, small secondary lows etc. the weather websites all seemed to be behind the ball, not anticipating, but changing their minds after the event.

So, I found myself going back to the Met Office inshore and offshore waters forecasts and using my MkI eyeballs.

At the risk of sounding like Old Father Time, when I took my Yachmaster exam in my early twenties, we had to draw a weather map from a recorded Met offshore weather forecast, using (mainly) the reports from coastal stations. I was astounded then (and remain so) that we could draw accurate weather maps, identifying and placing frontal systems on them from such data as barometric pressure, weather, visibility etc.

Working from the time of the forecast, a good idea of what’s going to happen at short notice and when is definitely possible. Add you own observations at deck level and the barometer and you’re on your way – or not.

Weather sites, GRIBs etc. are wonderful, most of the time. But we all seem to agree, the sites work at a different scale and to do GRIBS justice you need to practice.

And, for what it’s worth, as a layman, we should also throw the effects of climate change into the equation, that are even more unpredictable.

Dick Stevenson

Hi John and all,
Another  exercise for gaining weather skills is, (from the comfort of your easy chair) to take the surface analysis and other raw-ish data available on passage (such as 500mb charts) and generate your own forecast and weather map: whether locally or for a “passage”. Try to forecast (and draw) what the surface analysis will look like in the days to come. And then compare your forecast with what actually unfolds.  
My best, Dick Stevenson, s/v Alchemy

Rob Gill

Great post thanks John and some interesting comments too.

May I venture that I think the GRIBs and GRIB viewers we have available today are amazing. The couple I regularly use around NZ being PredictWind and Windy, and both have the function to overlay isobars on to the weather charts. This for me transforms the GRIB views into weather charts.

Then for PredictWind, when looking at the first 24hr chart with “isobars” toggled on, we see the highs & lows, warm, cold & occluded fronts, significant lows & tropical storms are all annotated on the chart, with the choice to have plots from two different models side-by-side, and five models to choose from.

Better still, the direction and speed of significant lows and storms is now annotated on each 24hr chart. Plus area specific GMDSS warnings which can be clicked on to read in full. All this, even on the free version (we get the Basic subscription around the coast and pay for Standard when venturing offshore).

Offshore, with Iridium Go and the offshore GRIB viewer, we won’t enjoy these cool features and may be a good reason to pay for Iridium GO Exec or even Starlink..?

But then I think the more detail I get given, and the quicker and easier it is to interpret that information, the less I might remember it is still a forecast, and the less time I may spend analysing and considering changes to our passage plan & route, or anchorage.

Rob Gill

Hi John,

What I believe PW have done is overlay the relevant region’s 24hr surface forecast over the GRIB chart, only when you toggle the isobar overlay on.

Drawing on where the fronts are is a job for a regional meteorological team backed by super computers, and even they wait for the twice daily updates to the GRIBS, and only show 24hrs in advance.

On PW this detail drops off when you advance the charted model beyond 24 hours, which makes complete sense to me and will speed up the analysis you talk about above.

Previously we would have to download the GRIBS, then try and download the 24 hour forecast which would frequently timeout multiple times because the server wasn’t smart. And then download the GMDSS warnings, ditto. And then the cyclone alerts… this could take 4 hours or more using our GO offshore in the SW Pacific.

You may have better coverage in the Northern Hemisphere than we experienced in 2018.

Eric Klem

Hi John,

The probabilistic wind speed guidance is great, thanks for alerting me to that. I have long wanted some form of a tolerance band on the prediction and while this isn’t exactly that, it gets at what I was hoping for. Looks like I need to dig in a bit better and understand how it works so that I can understand whether it still has the same shortcomings on things like fronts. I have long felt that we need to view weather with a bit more of a statistics mindset. We have had a few times recently where a storm has come through while we are at anchor and we can watch a band hit us on the weather radar which has windspeeds 50%+ higher than anywhere else and if we had been 10 or 20 miles away, we would not have had them. This is a great example where the forecast won’t capture that level of granularity but suddenly you have 50 steady across the deck instead of that 30 you had been having but most people in the storm never see that higher amount either.


Eric Klem

Hi John,

I do sometimes look at CAPE but I don’t know that I am doing any better a job at decision making than before using it with the sort of sailing we do. For pop-up thunderstorms, I find that the percent chance given by the NWS is a pretty useful metric and then watching the clouds and radar covers for the rest, we generally don’t stay in unless they are specifically supposed to be severe and start early in the day. We also do not get as excited about these around here unless the forecast is explicitly calling for severe thunderstorms as they tend to be small, slow moving, not all that severe and often lose all strength when they hit cold water. With fronts, I find that the forecast discussion and hazardous weather outlooks give me a good enough idea of whether we need to be prepared for a strong front or not. The one issue is that the NWS forecast often calls for “heavy rain” only in fronts and that seems to encompass wind events too but if you read the discussion, this will be mentioned so just using good practice solves for it.   Of course, this is all based on what your local forecast products are. For example, every time we go to the Maritimes or PNW I struggle with the forecast as we don’t use Canadian forecasts nearly as often and you have to know how to interpret what they are saying so I do a lot more looking at weather maps and raw model outputs.

One thing that can be really handy is one of the radar apps which extrapolates forward. These can be way overused but what I find helpful is that they can give you an idea of when something that is already on radar will get to your area. Just eyeballing it is remarkably poor and otherwise you are stuck trying to use your phone to measure distances which is tricky.

Note that my offshore sailing was generally before we had access to data like this so the techniques I use are adapted for coastal work with reasonably high forecast area density, familiar forecast language and not working with more than a few day time horizon.


Eric Klem

Hi John,

Your take on EC is interesting, I had always wondered how much of the issue was me and how much was the forecast.

Our first time out this past season I discovered that the radar app I had been using no longer worked so I quickly downloaded the app from The Weather Channel with almost no research and it works fine but there are likely better options. The default radar loop looks backwards 2 hours and forwards 6 hours and there are options for 24 and 72 hours which I have never clicked on as they sound more like a raw model output than actual radar extrapolation. I think that actually many of the radar apps now project forward in time, some with scarily realistic graphics that could trick one into thinking that they are doing more than they actually are. I just looked and it seems that your area is covered by the one from the weather channel. If someone has played with a much better one they can recommend, I will certainly give it a try.


Chuck Batson

Timely article John, thank you. For the past year I’ve been working on a new weather app for sailors, which incorporates, among other things, synoptic weather charts (the ones made by humans) and recently added probabilistic forecasts for sustained wind speed and significant wave height.

It’s always been clear to me why to care, but educating folks about what’s available beyond GRiBs and why it’s important is a challenge.