A dmittedly, I’m an unabashed geek, getting my jollies running statistical tests querying the actual NTSB relational database and publishing my aviation safety research in journals using scientific mumbo-jumbo—the majority (if not all) of which would put any insomniac to sleep in a heartbeat. That said I’m also an active general aviation pilot. Here, I’ll don both hats as I cover a hot-off-the-press scientific paper published in the Atmosphere journal, “translating” from highfalutin language into layman’s English for the benefit of the general aviation pilot population.
These pages have held two articles on the Model Output Statistics computerized forecasts—”Hidden Weather Tool” in March 2018, and “More MOS” in June 2018. The study I’ll translate was undertaken by Embry Riddle University faculty on the Localized Aviation MOS Program (LAMP) tool. This forecast tool was adopted by the National Weather Service in the late 1990s with improvements in 2012.
Why this study? Two reasons. The first is that it dovetails nicely with the previous articles. Second, the study analyzed the accuracy of this weather tool for the general aviation pilot. The bottom line is that the general aviation pilot would be well served by adding this forecast tool to their arsenal. I’ll finish up with a few words of caution as to using common MOS sources to pull up LAMP data.
LAMP? Wazzat?
No, in this context LAMP is not where you hide your genie, even if it sometimes seems like a forecast was conjured up by one. LAMP further hones the weather forecasting data generated by one of the MOS variants (more later).
In simplified terms it is an automated, airport-specific, 25-hour forecast issued every hour. The silicon brain doing all the work is the Weather and Climate Operational Supercomputer (see image below).
Diving a little under the hood, the LAMP forecast is generated using a mix of three separate data inputs: (1) the current observation per the ASOS (2) some advective models (3) and the Global Forecast System (GFS)-MOS, one of the MOS variants. But LAMP is not an equal mix of these three different data sets; more emphasis is placed on the current weather observations (i.e. ASOS) in the earlier times of the 25-hour forecast period and the GFS-MOS is given greater consideration closer to the 25-hour limit.
You might wonder why we need another forecast. After all, TAFs are written by trained meteorologists at the National Weather Service (NWS) Weather Forecast Offices (122 such offices across the USA).
Dust off the cobwebs and you might recall that TAFs are issued every six hours. LAMP is issued hourly. The big difference for us, though, is that airports patronized by light aircraft are more likely to have a LAMP than a TAF—it’s a simple numbers game in that there are about 1853 LAMPs to only 750 TAFs. Also, we tend to find TAFs at airports serving heavy iron where avgas costs can be breathtaking and taxi times can be as long as a short flight.
But, even if a LAMP is issued for your airport (covered airports are listed atweather.gov/mdl/lamp_stations_v2.1.0),. the FAA decrees that LAMP should only be used to supplement the TAF, not as an official stand-alone forecast. So, if your local airport has a LAMP but not a TAF, we must use the TAF generated for the nearest airport for guidance.
Why is the FAA reluctant to advocate the LAMP? I suspect it’s that until now, research comparing LAMP with the tried-and-true TAF has been sketchy. Plus, do we really want to trust a silicone-brain (remember these are entirely automated forecasts) over a human brain likely to have extensive experience and local knowledge? Computers and their programs don’t always have the best reputation for accuracy. (Ask Boeing…) But the LAMP is updated every hour, accounting for the most recent weather versus every six hours for the TAF (ignoring, of course, a “special”).
What The Study Asked
There were two questions the Embry Riddle University researchers raised. First, how do the LAMP and TAF compare in identifying IFR ceilings of 500-1000 feet AGL from the more challenging low IFR cigs under 500 feet AGL? Actually, they lumped together IFR with MVFR and VFR—any forecast for a ceiling of 500 feet AGL or higher—since a forecast for IFR but in which the ceiling turned out to be MVFR or VFR reduces the need for sharp instrument skills. We’ll just refer to this aggregate as “IFR-or-better” conditions.
Okay, we’re instrument-rated, so why the distinction in ceilings between LIFR and IFR-or-better? Well, a research study published a few years back on fatal accidents by instrument-rated pilots showed poor instrument approach procedures and spatial disorientation as the leading causes. Let’s be honest; GA instrument proficiency often isn’t sufficient to wisely launch into a wide area beset by low IFR.
Now, I’m not advocating avoidance of LIFR entirely. In fact, practicing approaches under those conditions will build confidence and proficiency. But for enhanced safety one should always have a way out in terms of a reachable airport showing MVFR/VFR.
The second question posed by the authors was similar but pitched to the VFR pilot or the out-of-currency IFR pilot, i.e. how do the LAMP and TAF compare in teasing apart VFR and MVFR/IFR/LIFR ceilings in their forecasts? See the sidebar.
The Cookery
One often hears the common refrain of academics hunkering down in their ivory towers, far removed from real-world operations. Fortunately, this wasn’t the case for this study. The flight scenario that the researchers considered was typical to general aviation—a time aloft of one to two hours, maybe for the proverbial $100 hamburger, with a preflight briefing a couple hours before wheels-up.
If your nonsense alarm is going off, hold off on deluging our esteemed editor with letters saying that you use your steed a lot for long cross-country flights. While I’m sure this is the case for some, the total numbers indicate otherwise. If you don’t believe this, crank up the publicly available 2016 general aviation survey, crunch some numbers using annual flight times and number of landings and, presto, one comes up with 1.3 hours for the typical flight.
The researchers then recorded LAMP and TAF forecast ceiling data (issued concurrently) for such a flight scenario over 6 months in 2018. Some 823 LAMP/TAF forecasts across the 48 contiguous states were collected and compared with actual ceiling conditions per the ASOS. While the metrics were in typical yawn-inducing meteorologist mumbo-jumbo (e.g. probability of detection, false alarm rate to name a few) I’ll translate these to the common vernacular. Ahem…for the nerdy types out there, probability of detection just refers to the percentage of correct forecasts and think of the false alarm rate as the percentage of forecasts which did the proverbial “cry wolf” (i.e. forecast for LIFR when actual conditions turned out to be IFR or better).
And the Winner Is…
So how did these two crystal balls stack up in their ceiling forecasts? Let’s cut to the chase (drum-roll please). The LAMP forecasted IFR-or-better ceilings with 78 percent accuracy to the TAF’s 56 percent—a significant margin. Put another way, while the LAMP missed LIFR 22 percent of the times, the TAF was worse with nearly half (44 percent) of prognostications categorizing the ceilings as IFR-or-better when it was actually LIFR.
Accuracy is one thing but assessing these tools for “crying wolf” is important too. Hark back to your days as a VFR-only driver and you’ll likely recall teeth-grinding when the “VFR-not-recommended” from the briefer became some particularly nice flying weather. Crying wolf is not good—do so often enough and it is human nature for us to start blowing off an analysis, assuming the weather will be better than the poor-weather forecast.
So how did these two forecast tools compare in calling for LIFR when IFR-or-better ceilings prevailed? To be fair, both forecast instruments could do with improvement, seemingly biased toward reporting low weather. The LAMP bested the TAF in accurately forecasting IFR-or-better ceilings, with the LAMP being correct 78 percent of the time versus 56 percent for the TAF. The LAMP predicted LIFR when in actuality IFR-or-better ceilings prevailed nearly half (48 percent) of those times. If you think that’s bad, the TAF was considerably worse crying wolf 81 percent of the times (i.e. calling for LIFR when in fact IFR-or-better ceiling conditions came to pass). So overall, the prize goes to the LAMP for its improved accuracy in ferreting out IFR-or-better ceilings from the more challenging LIFR and less often raising false alarms.
With no intention of dissing the meteorologists who do a superb job in keeping us pilots safe with aviation forecasts, this all begs the question as to why the TAFs seems well … not as good as the LAMP, at least in terms of predicting ceilings. Also, don’t the NWS meteorologists use the LAMP as their starting point for generating a TAF? Let’s tackle these one by one.
First, it’s up to each meteorologist whether or not to use the LAMP as a jump-off point, at least according to my local NWS Weather Forecast Office responsible for issuing local TAFs. To be fair, any reticence to do this could reflect the sparse data (until now) showing the accuracy of the LAMP for ceilings coupled with the fact that some of these meteorologists might rightfully have a lot of faith in their extensive experience.
But that still doesn’t get to the heart of why LAMP is more accurate. Each NWS Weather Forecasting Office is responsible for generating many TAFs, not just the one for your local airport. Compare that to the LAMP that uses two years of historical ASOS data for each airport to build its own predictive algorithm for that location. In other words, like the meteorologists, it gains experience and learns from the past.
Where’s the LAMP?
Where do you find the most recent LAMP data? There are several ways but the official NWS website is:www.nws.noaa.gov/mdl/gfslamp/meteoform.php. You can search for an airport by state, then select the parameters you want to view (temperature, ceiling, lightning, etc.), but the graphical presentation of the data is a bit awkward. Perhaps our EFBs can do better.
Indeed they can. ForeFlight and Garmin Pilot (and possibly others; this is not an exhaustive list) display LAMP data. Find the forecast for an airport. Under the label “MOS” you get the LAMP forecast for the first 24-25 hours, then a MOS variant called MAV out to 60 hours or 72 hours for ForeFlight and Pilot respectively. Also, if the LAMP is unavailable (either from equipment failures or not being issued for that airport) data defaults to the MAV but neither app tells you the source.
Clever souls will note that MOS comes out every six hours, but LAMP is issued hourly. If you keep watching, you can then tell from the refresh rate of the forecast data what you’re seeing. Or, if it’s a couple hours old, it’s not LAMP.
Take Home Lessons
LAMP is an important tool for GA, as long as we keep in mind some limitations and the bigger picture. While there are more airports with LAMPs than TAFs, still of the 5100 civilian airports in the U.S., most have neither. As a rule of thumb, you won’t find a LAMP issued for an airport that doesn’t have automated weather reporting. Also, until the FAA approves use of the LAMP, we’re still stuck with using it as supplementary to the TAF—at least officially. But consider using it to err on the conservative side of aeronautical decision making. For example, if the TAF predicts IFR ceilings and the LAMP is calling for LIFR, the conservative pilot will assume LIFR. Finally, but perhaps most importantly, remember too that the LAMP is only one of many excellent weather-briefing tools that are widely available today, including the official graphical area forecast.
After geeking out on all this data, Douglas Boyd, PhD, a FAAST representative and research scientist studying and publishing on general aviation safety, now relies on LAMP while still taking an FAA-appeasing look at the TAFs.
Left: The Weather and Climate Operational Supercomputer generates the LAMP forecasts.
“VFR Okay” Accuracy
The accompanying article was written for the instrument pilot, but what about our VFR-only cousins or, for that matter, the IFR pilot out of currency? We’re familiar with the routine. For some reason we’re not flying—maintenance, finances, weather—and before long we’re out of currency and restricted to VFR operations. Of course, Mother Nature doesn’t give a hoot as to whether a pilot is VFR-only or IFR-rated but out of currency. She treats both equally and sometimes with a vengeance.
An IMC encounter by the VFR-only pilot or rusty instrument-rated pilot can rapidly lead to spatial disorientation, too often with fatal consequences. Over 90 percent of accidents involving IMC encounters by unqualified pilots have a fatal outcome. Compare that with a fatality rate for general aviation accidents as a whole of about 20 percent. Bottom line is that if you’re not keen on your family invoking a life insurance claim, keep a nice block of clear air between the ground and cloud bases.
As a reminder, in general terms VFR conditions require at least a 3000-foot ceiling, which also has to be enough to keep you from becoming an ornament on a high tower while remaining nicely below the cloud base. Of course, all this necessitates an accurate ceiling forecast to avoid scud-running. So how then did the LAMP stack up against the TAF when it comes to VFR ceiling forecast accuracy? Again, LAMP won in nailing VFR ceilings (67 percent) compared with 57 percent for the corresponding TAF. That’s great, but what about crying wolf (predicting MVFR/IFR/LIFR when in fact, beautiful VFR ceilings came to pass), which can dangerously desensitize the pilot over time. Here, the LAMP falsely raised the alarm for non-VFR ceilings at about the same rate of 22 percent. Conclusion: The LAMP works better to keep you out of trouble, while it leaves you at home when you could be flying with about the same rate as the TAF. —DB
