I have often expressed my worry about the number of cruisers who are overly relying on forecast tools that present raw weather-model output in cool ways:
- PredictWind
- Windy
- Luckgrib
- And many others
All these sources use the same underlying government sources. Each is just a different way to make those data sets look pretty.
To present a different view, I wrote an entire Online Book on how to reduce the risk from unexpected weather outcomes by adding in, and gaining understanding of, other weather sources.
But it’s some time since we have updated said book—on the schedule for this winter—so here’s a new chapter with a concise approach to assessing forecast accuracy. Think of it is a kind of Coles Notes to the Online Book.
Why Does It Matter?
But first, why does this matter? Modern models are quite accurate, so surely all we have to do is look at the GRIBs outputted by them, in whatever viewer we like best (see above), and all is good?
No, here’s why:
A weather forecast (or model output) is only the most likely of several possible scenarios.
Me
For example, a purely GRIB-based passage plan might forecast a voyage to Bermuda with no more than 30 knots of sustained wind, even when there’s a 33% chance of sustained storm-force winds—the former a boisterous sail in a well-found boat, the latter verging on survival conditions in any boat, particularly one with a crew who had not planned for that eventuality.
And making this effort is worth it even if we decide to make the passage despite the threat, as I probably would if trying to get south in the fall with the winter fast approaching, given that at that time of year a 2-in-3 chance of getting to Bermuda without a gale or worse is probably as good as it gets1.
If we actually want to make voyages, we can’t be too timid either.
Anyway, whether we go or not, knowing that two of the six chambers (33%) in the weather revolver are loaded with storm-force winds will make us far safer mariners than unrealistically assuming that the revolver is not loaded, based on a single forecast or routing source, particularly a raw-from-computer source.
Forecast-Error Risk Assessment
So let’s dig into tools we can use to assess the chances of more severe weather than that forecast. The good news is that the process is easier and less time consuming than it was just a few years ago.
Thanks for the tips I will be using them. Over the decades I have moved from plotting my own weather charts to full on Predict Wind and others. Hence your tips are a good tool to use.
.
In the UK we have access to coastal reports of actual weather data, transmitted on Navtex, or the shipping forecast and of course online. It can be useful to look at actual pressures, wind and weather at the coastal stations and see if they are corresponding with the tabulated wind and weather data for the models.
The idea is that the model that shows the most consistent correlation might be the most accurate looking forward. I would like to say that the trend works in the context of determining look ahead accuracy, but I have not established that because of low sample size.
However, for the immediate future, say at 24 hours, when the wind strength range is broad, it can be accurate in testing what model is likely to be relevant, if the pressures, strengths, direction and weather have been panning out as per the prediction.
Hi Alastair,
Good point, particularly for using wind direction to access whether or not a front is coming through as predicted by the model
That said, I think that with the world wide models inaccuracies, particularly wind speed, when measured against shore stations need to be treated with caution because said models don’t take into account topographic features like hills.
Hi John,
Thank you for highlighting certain weather forecasting products and explaining how sailors can use them to be safe. This article alone justifies the price of admission.
However, saying that global weather models don’t account for geographic features is not exactly correct. If that were true, how would one explain this wind forecast. (Please see attached image from the GFS model) Cold air from the approaching front is funneled through the Coatzacoalcos River valley and is hemmed in by the mountains to the west in Oaxaca.
The GFS is a coupled model that simulates the atmosphere, oceans, land, and sea ice differently, but it includes interactions between those. The land model includes soil type, vegetation, and terrain features. (see for example 1.2.3 on this web page https://vlab.noaa.gov/web/gfs/documentation)
Now the effect of terrain is influenced by the grid size, so some smaller-scale features, like narrow canyons, may not be included. Some models, like the proprietary PWG, include run on a smaller scale grid in certain areas.
Thanks again for teaching me so much about sailing!
Sincerely,
Robert
Hi Robert,
Thanks for the kind words.
As to how global models interact with land. Absolutely, they are aware of land and it’s effects. Any you are right, I made a mistake in the above comment since I know from experience that the both global models are doing an ever better job on sea breeze. Must have been under caffeinated!
That said, I think I’m still right in saying that neither model takes into account micro geographic features such as hills. So, for example, the model will know and predict that the wind will be higher in the Straits of Gibraltar when the overall flow is from the east or the west, but it will not take into account the height and shape of the Rock of Gibraltar.
Also, we should not forget that the GFS resolution is 16 miles and the ECMWF 8, so expecting them to accurately track a shore station is probably not realistic, because, for example, the station may be sheltered from one direction by a hill.
On the other hand some local models do, I think, have the actual land topography loaded into them, although I have not dug into that.
Interesting to read your piece in light of last night’s hurricane landfall in Acapulco. Going back and looking at archives from the nhc site it seems it might have caught forecasters by surprise, especially given the location and the lack of historical storms hitting there. Seems a good advertisement for vigilance and not complacency whenever there’s a storm in the vicinity.
Hi Jim,
Very good point. I thought the same thing: that the rapid intensification caught everyone including the pros by surprise. And what a tragedy!
I saw that one too….
There is always an element of uncertainty around storms. Forecast models are getting better, but they’re still initialized with sparse data and are run on grids many orders of magnitude coarser than the physical effects they’re trying to simulate.
For planning and preparation purposes, we always need to err on the side of caution, i.e. always asking ourselves “what is the worst possible scenario that could evolve from these current conditions?” We may make a conscious decision that the probability of such a worst-case scenario is low enough to take a chance anyway…. but if we don’t consider the possibility, it will come to pass sooner or later, and we’ll be unprepared.
Also, I’m going to add my voice to the chorus of frustration of not being able to see fronts and troughs in all the fancy shiny new web-based GRIB viewers. Introductory weather courses are almost always structured around understanding highs, lows, ridges, troughs, and fronts, because that’s where the interesting weather comes from. If you look at data visualizations with those things omitted, you’re putting an awful lot of faith in incomplete information.
This a timely article indeed.
This summer, bringing our old boat home was an exercise in disappointment as the chaotic weather situation in our part of the NE Atlantic confounded our attempts to get home, Occlusions, small secondary lows etc. the weather websites all seemed to be behind the ball, not anticipating, but changing their minds after the event.
So, I found myself going back to the Met Office inshore and offshore waters forecasts and using my MkI eyeballs.
At the risk of sounding like Old Father Time, when I took my Yachmaster exam in my early twenties, we had to draw a weather map from a recorded Met offshore weather forecast, using (mainly) the reports from coastal stations. I was astounded then (and remain so) that we could draw accurate weather maps, identifying and placing frontal systems on them from such data as barometric pressure, weather, visibility etc.
Working from the time of the forecast, a good idea of what’s going to happen at short notice and when is definitely possible. Add you own observations at deck level and the barometer and you’re on your way – or not.
Weather sites, GRIBs etc. are wonderful, most of the time. But we all seem to agree, the sites work at a different scale and to do GRIBS justice you need to practice.
And, for what it’s worth, as a layman, we should also throw the effects of climate change into the equation, that are even more unpredictable.
Hi Colin,
All good points, thank you.
One thing I noticed the first time I crossed the Atlantic and cruised the UK area is how much more difficult understanding the weather is than on this side. I think a lot of that is because the systems in your part of the world are more likely to be occluded and spawn more secondary lows, just as you say.
Anyway, drawing our own weather map from observations would be a great exercise to build weather understanding.
Hi John and all,
Another exercise for gaining weather skills is, (from the comfort of your easy chair) to take the surface analysis and other raw-ish data available on passage (such as 500mb charts) and generate your own forecast and weather map: whether locally or for a “passage”. Try to forecast (and draw) what the surface analysis will look like in the days to come. And then compare your forecast with what actually unfolds.
My best, Dick Stevenson, s/v Alchemy
Hi Dick,
While an interesting exercise, I would go that far as a recommendation to others since deriving the future surface weather from the 500 mb is hard to do, so we don’t want to set people up to fail and then get discouraged and go back to just looking at GRIBs.
That said, for those who want to learn more about the upper level, we have a study guide: https://www.morganscloud.com/2017/10/24/weather-analysis-a-step-by-step-guide-part-3-learning-about-the-upper-level/
But even in that chapter I suggest a simplified approach to learning more about the upper level. And I specifically opine that the being able to derive probable surface weather from the upper level is no longer a required skill and explain why I say that.
Great post thanks John and some interesting comments too.
May I venture that I think the GRIBs and GRIB viewers we have available today are amazing. The couple I regularly use around NZ being PredictWind and Windy, and both have the function to overlay isobars on to the weather charts. This for me transforms the GRIB views into weather charts.
Then for PredictWind, when looking at the first 24hr chart with “isobars” toggled on, we see the highs & lows, warm, cold & occluded fronts, significant lows & tropical storms are all annotated on the chart, with the choice to have plots from two different models side-by-side, and five models to choose from.
Better still, the direction and speed of significant lows and storms is now annotated on each 24hr chart. Plus area specific GMDSS warnings which can be clicked on to read in full. All this, even on the free version (we get the Basic subscription around the coast and pay for Standard when venturing offshore).
Offshore, with Iridium Go and the offshore GRIB viewer, we won’t enjoy these cool features and may be a good reason to pay for Iridium GO Exec or even Starlink..?
But then I think the more detail I get given, and the quicker and easier it is to interpret that information, the less I might remember it is still a forecast, and the less time I may spend analysing and considering changes to our passage plan & route, or anchorage.
Hi Rob,
Sure, these products are great, but no matter how pretty they are we must never forget that they are just raw GRIB data presented in pretty ways, we still need to look at maps and particularly the development of fronts. If PW has that built in, that’s cool, but if it’s only the 24 hour, to me that’s not good enough since we are only seeing a snap shot, not development over time. And if we lose all weather maps when we head offshore that’s a real problem because on passage is when we need them most.
As to GO! exec and Starlink, no you don’t need to spend that much to get the job done right offshore: https://www.morganscloud.com/jhhtips/original-iridium-go-still-a-better-deal-than-exec/
Hi John,
What I believe PW have done is overlay the relevant region’s 24hr surface forecast over the GRIB chart, only when you toggle the isobar overlay on.
Drawing on where the fronts are is a job for a regional meteorological team backed by super computers, and even they wait for the twice daily updates to the GRIBS, and only show 24hrs in advance.
On PW this detail drops off when you advance the charted model beyond 24 hours, which makes complete sense to me and will speed up the analysis you talk about above.
Previously we would have to download the GRIBS, then try and download the 24 hour forecast which would frequently timeout multiple times because the server wasn’t smart. And then download the GMDSS warnings, ditto. And then the cyclone alerts… this could take 4 hours or more using our GO offshore in the SW Pacific.
You may have better coverage in the Northern Hemisphere than we experienced in 2018.
Hi Rob,
I still think that it’s important to look at the full suit of weather maps no matter how they are presented. Only by doing that can we gain a full macro understanding of how the weather is evolving and what the risks are.
Which software were you using to download maps etc? I suspect that was the root of the problem since I have never taken more than an hour to get everything I needed using UUplus and that without supervising the process. Our system, while a bit out of date, will still work and is fully documented: https://www.morganscloud.com/category/weather/book-weather-analysis/
Hi John,
The probabilistic wind speed guidance is great, thanks for alerting me to that. I have long wanted some form of a tolerance band on the prediction and while this isn’t exactly that, it gets at what I was hoping for. Looks like I need to dig in a bit better and understand how it works so that I can understand whether it still has the same shortcomings on things like fronts. I have long felt that we need to view weather with a bit more of a statistics mindset. We have had a few times recently where a storm has come through while we are at anchor and we can watch a band hit us on the weather radar which has windspeeds 50%+ higher than anywhere else and if we had been 10 or 20 miles away, we would not have had them. This is a great example where the forecast won’t capture that level of granularity but suddenly you have 50 steady across the deck instead of that 30 you had been having but most people in the storm never see that higher amount either.
Eric
Hi Eric,
I agree, the probabilistic product is the best new weather information I have seen in a long time.
As to the problem of figuring out the risks of much higher local winds, I think that they are pretty much always close to a front, at least in my experience and in the mid and high latitudes, so I double down on the importance of looking at maps that show fronts rather than just GRIBs.
As to detecting them I have always found, like you I believe, that our own radar is the best bet, although of course it does not tell us what the wind speed will be. (In conjunction with land based weather radar, if available.)
Other than that, I have recently, particularly since we are getting more thunderstorms here in Nova Scotia, started looking at CAPE index although I’m still inexperienced with it. Have you used CAPE and found it useful?
For others: here’s a good description of how CAPE works: https://www.weather.gov/ilx/swop-severetopics-CAPE
Hi John,
I do sometimes look at CAPE but I don’t know that I am doing any better a job at decision making than before using it with the sort of sailing we do. For pop-up thunderstorms, I find that the percent chance given by the NWS is a pretty useful metric and then watching the clouds and radar covers for the rest, we generally don’t stay in unless they are specifically supposed to be severe and start early in the day. We also do not get as excited about these around here unless the forecast is explicitly calling for severe thunderstorms as they tend to be small, slow moving, not all that severe and often lose all strength when they hit cold water. With fronts, I find that the forecast discussion and hazardous weather outlooks give me a good enough idea of whether we need to be prepared for a strong front or not. The one issue is that the NWS forecast often calls for “heavy rain” only in fronts and that seems to encompass wind events too but if you read the discussion, this will be mentioned so just using good practice solves for it. Of course, this is all based on what your local forecast products are. For example, every time we go to the Maritimes or PNW I struggle with the forecast as we don’t use Canadian forecasts nearly as often and you have to know how to interpret what they are saying so I do a lot more looking at weather maps and raw model outputs.
One thing that can be really handy is one of the radar apps which extrapolates forward. These can be way overused but what I find helpful is that they can give you an idea of when something that is already on radar will get to your area. Just eyeballing it is remarkably poor and otherwise you are stuck trying to use your phone to measure distances which is tricky.
Note that my offshore sailing was generally before we had access to data like this so the techniques I use are adapted for coastal work with reasonably high forecast area density, familiar forecast language and not working with more than a few day time horizon.
Eric
Hi Eric,
Thank for the fill.
Interesting on apps that extrapolate forward. I did not even know that was a thing. Which one are you using? I found CARROT, but it’s not clear if the extrapolation feature works in Canada.
As to your difficulty with Envirenment Canada forecasts, I sympathize. Having sailed extensively in both countries your forecasts are way better. In fact I have found EC so bad, particularly past 48 hours from forecast time, that I tend to just go straight to the global models. I’m not sure if the problem is that they are using the Canadian model, which has, I think, fallen behind, or because they just don’t update often enough.
Hi John,
Your take on EC is interesting, I had always wondered how much of the issue was me and how much was the forecast.
Our first time out this past season I discovered that the radar app I had been using no longer worked so I quickly downloaded the app from The Weather Channel with almost no research and it works fine but there are likely better options. The default radar loop looks backwards 2 hours and forwards 6 hours and there are options for 24 and 72 hours which I have never clicked on as they sound more like a raw model output than actual radar extrapolation. I think that actually many of the radar apps now project forward in time, some with scarily realistic graphics that could trick one into thinking that they are doing more than they actually are. I just looked and it seems that your area is covered by the one from the weather channel. If someone has played with a much better one they can recommend, I will certainly give it a try.
Eric
Timely article John, thank you. For the past year I’ve been working on a new weather app for sailors, which incorporates, among other things, synoptic weather charts (the ones made by humans) and recently added probabilistic forecasts for sustained wind speed and significant wave height.
It’s always been clear to me why to care, but educating folks about what’s available beyond GRiBs and why it’s important is a challenge.