By Jan Verkade (jan@forecastverification.com)
Between Wednesday, August 30th and Tuesday, September 12, the US National Hurricane Center issued 52 advisories pertaining to Tropical Storm and Hurricane Irma. The NHC advisories described her observed and projected location as well as her characteristics. They were posted on the web in a number of formats including text bulletins and cone plots.
The NHC advisories were issued four times daily at 3am, 9am, 3pm and 9pm (all times in UTC). The forecast cycle cycle for each advisory begins 3 hours prior to the issuance of the advisory products, i.e. at 12am, 6am, 12pm or 6pm. Here, these latter “reference times” are referred to as T0 (“T-Zero”). Notably, the advisories contained predicted positions at T0 plus 12h, 24h, 36h, 48h, 72h, 96h and 120h – here referred to as forecast lead times.
As the advisories contained both forecasted and observed locations of the storm’s center, this offered the possibility to verify the forecasts. Forecasts of the location Irma’s center only were verified; I made no attempt to verify other variables such as wind speed and direction.
The forecast advisories
Essentially, the forecast is deterministic in nature: multiple zero-width point locations form a forecast track. The NHC does, however, address uncertainty in the forecast,
NHC tropical cyclone forecast tracks can be in error. This forecast uncertainty is conveyed by the track forecast “cone”.
This cone is formed by smoothing over multiple circles with diameters that vary (and increase) with forecast lead time. The NHC states how the circle diameters are determined:
The size of each circle is set so that two-thirds of historical official forecast errors over a 5-year sample fall within the circle.
The diameters of the circles are taken from here and listed in the table below.
| lead time [h] | Radius [US nm] | Radius [km] |
|---|---|---|
| 12 | 29 | 54 |
| 24 | 45 | 83 |
| 36 | 63 | 117 |
| 48 | 78 | 144 |
| 72 | 107 | 198 |
| 96 | 159 | 294 |
| 120 | 211 | 391 |
If we would interpret the cones as a probabilistic forecast, we would expect, on average, two out of three advisories to ‘hit’ the circles – and 1 out of 3 to miss it. Note, however, that the NHC does NOT explicitly say that the forecast should be thus interpreted! We will, however, test this 2/3 probability of a hit and 1/3 probability of a miss against the record of observations. In verification terms, we will test the reliability of that probabilistic forecast.
The data
The data was obtained by downloading all 52 advisory text bulletins and extracting the relevant passages. It was then combined into a table listing, for each forecast, both the forecast and observed location of Irma’s center. The table’s contents can be downloaded as a .csv file..
Note that the data contains locations with a resolution of 1/10th of a degree. The advisories address the fact that there is observational uncertainty: the location is only accurate within so many nautical miles. In the present analysis, this observational uncertainty is not taken into account.
| Advisory | Reference Time | Valid Time | Lead time [h] | Forecast latitude (WGS84) | Forecast longitude (WGS84) | Observed latitude (WGS84) | Observed longitude (WGS84) |
|---|---|---|---|---|---|---|---|
| 1 | 2017-08-30 12:00:00 | 2017-08-31 00:00:00 | 12 | 16.7 | -31.9 | 16.3 | -31.8 |
| 1 | 2017-08-30 12:00:00 | 2017-08-31 12:00:00 | 24 | 17.3 | -33.8 | 16.7 | -33.4 |
| 1 | 2017-08-30 12:00:00 | 2017-09-01 00:00:00 | 36 | 17.9 | -35.7 | 17.5 | -35.1 |
| 1 | 2017-08-30 12:00:00 | 2017-09-01 12:00:00 | 48 | 18.2 | -37.6 | 18.3 | -37.2 |
| 1 | 2017-08-30 12:00:00 | 2017-09-02 12:00:00 | 72 | 18.7 | -41.9 | 18.9 | -42.5 |
| 1 | 2017-08-30 12:00:00 | 2017-09-03 12:00:00 | 96 | 17.7 | -46.5 | 17.8 | -47.9 |
| 1 | 2017-08-30 12:00:00 | 2017-09-04 12:00:00 | 120 | 17 | -51 | 16.8 | -52.6 |
| 2 | 2017-08-30 18:00:00 | 2017-08-31 06:00:00 | 12 | 16.8 | -32.7 | 16.4 | -32.4 |
| 2 | 2017-08-30 18:00:00 | 2017-08-31 18:00:00 | 24 | 17.3 | -34.5 | 17.1 | -34.3 |
| 2 | 2017-08-30 18:00:00 | 2017-09-01 06:00:00 | 36 | 17.8 | -36.2 | 18 | -36 |
| 2 | 2017-08-30 18:00:00 | 2017-09-01 18:00:00 | 48 | 18.2 | -38.2 | 18.8 | -38.5 |
| 2 | 2017-08-30 18:00:00 | 2017-09-02 18:00:00 | 72 | 18.2 | -43 | 18.7 | -44 |
| 2 | 2017-08-30 18:00:00 | 2017-09-03 18:00:00 | 96 | 17.3 | -47.8 | 17.8 | -49.2 |
| 2 | 2017-08-30 18:00:00 | 2017-09-04 18:00:00 | 120 | 16.5 | -52.5 | 16.7 | -53.9 |
| 3 | 2017-08-31 00:00:00 | 2017-08-31 12:00:00 | 12 | 16.8 | -33.6 | 16.7 | -33.4 |
| 3 | 2017-08-31 00:00:00 | 2017-09-01 00:00:00 | 24 | 17.3 | -35.5 | 17.5 | -35.1 |
| 3 | 2017-08-31 00:00:00 | 2017-09-01 12:00:00 | 36 | 17.7 | -37.3 | 18.3 | -37.2 |
| 3 | 2017-08-31 00:00:00 | 2017-09-02 00:00:00 | 48 | 18 | -39.5 | 19.1 | -39.7 |
| 3 | 2017-08-31 00:00:00 | 2017-09-03 00:00:00 | 72 | 17.6 | -44.3 | 18.4 | -45.6 |
| 3 | 2017-08-31 00:00:00 | 2017-09-04 00:00:00 | 96 | 16.8 | -49 | 17.3 | -50.4 |
| 3 | 2017-08-31 00:00:00 | 2017-09-05 00:00:00 | 120 | 16.2 | -53.5 | 16.7 | -55.1 |
| 4 | 2017-08-31 06:00:00 | 2017-08-31 18:00:00 | 12 | 17 | -34.2 | 17.1 | -34.3 |
| 4 | 2017-08-31 06:00:00 | 2017-09-01 06:00:00 | 24 | 17.6 | -36 | 18 | -36 |
| 4 | 2017-08-31 06:00:00 | 2017-09-01 18:00:00 | 36 | 18 | -37.9 | 18.8 | -38.5 |
| 4 | 2017-08-31 06:00:00 | 2017-09-02 06:00:00 | 48 | 18.2 | -40.1 | 19.1 | -41.1 |
| 4 | 2017-08-31 06:00:00 | 2017-09-03 06:00:00 | 72 | 17.5 | -45 | 18.2 | -46.9 |
| 4 | 2017-08-31 06:00:00 | 2017-09-04 06:00:00 | 96 | 16.7 | -49.5 | 17 | -51.7 |
| 4 | 2017-08-31 06:00:00 | 2017-09-05 06:00:00 | 120 | 16 | -53.5 | 16.6 | -56.4 |
| 5 | 2017-08-31 12:00:00 | 2017-09-01 00:00:00 | 12 | 17.5 | -35.2 | 17.5 | -35.1 |
| 5 | 2017-08-31 12:00:00 | 2017-09-01 12:00:00 | 24 | 18 | -37.1 | 18.3 | -37.2 |
| 5 | 2017-08-31 12:00:00 | 2017-09-02 00:00:00 | 36 | 18.2 | -39.2 | 19.1 | -39.7 |
| 5 | 2017-08-31 12:00:00 | 2017-09-02 12:00:00 | 48 | 18.1 | -41.6 | 18.9 | -42.5 |
| 5 | 2017-08-31 12:00:00 | 2017-09-03 12:00:00 | 72 | 17 | -46.5 | 17.8 | -47.9 |
| 5 | 2017-08-31 12:00:00 | 2017-09-04 12:00:00 | 96 | 16 | -51 | 16.8 | -52.6 |
| 5 | 2017-08-31 12:00:00 | 2017-09-05 12:00:00 | 120 | 16 | -55.5 | 16.7 | -57.7 |
| 6 | 2017-08-31 18:00:00 | 2017-09-01 06:00:00 | 12 | 17.8 | -36.2 | 18 | -36 |
| 6 | 2017-08-31 18:00:00 | 2017-09-01 18:00:00 | 24 | 18.2 | -38.3 | 18.8 | -38.5 |
| 6 | 2017-08-31 18:00:00 | 2017-09-02 06:00:00 | 36 | 18.3 | -40.7 | 19.1 | -41.1 |
| 6 | 2017-08-31 18:00:00 | 2017-09-02 18:00:00 | 48 | 17.9 | -42.9 | 18.7 | -44 |
| 6 | 2017-08-31 18:00:00 | 2017-09-03 18:00:00 | 72 | 16.8 | -47.5 | 17.8 | -49.2 |
| 6 | 2017-08-31 18:00:00 | 2017-09-04 18:00:00 | 96 | 16 | -52 | 16.7 | -53.9 |
| 6 | 2017-08-31 18:00:00 | 2017-09-05 18:00:00 | 120 | 16.5 | -56.5 | 16.9 | -59.2 |
| 7 | 2017-09-01 00:00:00 | 2017-09-01 12:00:00 | 12 | 18.1 | -37 | 18.3 | -37.2 |
| 7 | 2017-09-01 00:00:00 | 2017-09-02 00:00:00 | 24 | 18.3 | -39.2 | 19.1 | -39.7 |
| 7 | 2017-09-01 00:00:00 | 2017-09-02 12:00:00 | 36 | 18.2 | -41.5 | 18.9 | -42.5 |
| 7 | 2017-09-01 00:00:00 | 2017-09-03 00:00:00 | 48 | 17.7 | -43.8 | 18.4 | -45.6 |
| 7 | 2017-09-01 00:00:00 | 2017-09-04 00:00:00 | 72 | 16.7 | -48.2 | 17.3 | -50.4 |
| 7 | 2017-09-01 00:00:00 | 2017-09-05 00:00:00 | 96 | 16.3 | -52.4 | 16.7 | -55.1 |
| 7 | 2017-09-01 00:00:00 | 2017-09-06 00:00:00 | 120 | 17 | -57 | 17.2 | -60.4 |
| 8 | 2017-09-01 06:00:00 | 2017-09-01 18:00:00 | 12 | 18.5 | -38.2 | 18.8 | -38.5 |
| 8 | 2017-09-01 06:00:00 | 2017-09-02 06:00:00 | 24 | 18.4 | -40.5 | 19.1 | -41.1 |
| 8 | 2017-09-01 06:00:00 | 2017-09-02 18:00:00 | 36 | 18.2 | -42.9 | 18.7 | -44 |
| 8 | 2017-09-01 06:00:00 | 2017-09-03 06:00:00 | 48 | 17.6 | -45.2 | 18.2 | -46.9 |
| 8 | 2017-09-01 06:00:00 | 2017-09-04 06:00:00 | 72 | 16.5 | -49.3 | 17 | -51.7 |
| 8 | 2017-09-01 06:00:00 | 2017-09-05 06:00:00 | 96 | 16.5 | -53.5 | 16.6 | -56.4 |
| 8 | 2017-09-01 06:00:00 | 2017-09-06 06:00:00 | 120 | 17.5 | -58 | 17.7 | -61.9 |
| 9 | 2017-09-01 12:00:00 | 2017-09-02 00:00:00 | 12 | 18.7 | -39.5 | 19.1 | -39.7 |
| 9 | 2017-09-01 12:00:00 | 2017-09-02 12:00:00 | 24 | 18.5 | -41.8 | 18.9 | -42.5 |
| 9 | 2017-09-01 12:00:00 | 2017-09-03 00:00:00 | 36 | 18 | -44.2 | 18.4 | -45.6 |
| 9 | 2017-09-01 12:00:00 | 2017-09-03 12:00:00 | 48 | 17.3 | -46.6 | 17.8 | -47.9 |
| 9 | 2017-09-01 12:00:00 | 2017-09-04 12:00:00 | 72 | 16.2 | -51 | 16.8 | -52.6 |
| 9 | 2017-09-01 12:00:00 | 2017-09-05 12:00:00 | 96 | 16.7 | -55.2 | 16.7 | -57.7 |
| 9 | 2017-09-01 12:00:00 | 2017-09-06 12:00:00 | 120 | 18 | -59.5 | 18.1 | -63.3 |
| 10 | 2017-09-01 18:00:00 | 2017-09-02 06:00:00 | 12 | 18.9 | -40.9 | 19.1 | -41.1 |
| 10 | 2017-09-01 18:00:00 | 2017-09-02 18:00:00 | 24 | 18.5 | -43.4 | 18.7 | -44 |
| 10 | 2017-09-01 18:00:00 | 2017-09-03 06:00:00 | 36 | 17.9 | -45.7 | 18.2 | -46.9 |
| 10 | 2017-09-01 18:00:00 | 2017-09-03 18:00:00 | 48 | 17.2 | -47.9 | 17.8 | -49.2 |
| 10 | 2017-09-01 18:00:00 | 2017-09-04 18:00:00 | 72 | 16.4 | -52.2 | 16.7 | -53.9 |
| 10 | 2017-09-01 18:00:00 | 2017-09-05 18:00:00 | 96 | 17 | -56.5 | 16.9 | -59.2 |
| 10 | 2017-09-01 18:00:00 | 2017-09-06 18:00:00 | 120 | 19 | -61 | 18.5 | -64.7 |
| 11 | 2017-09-02 00:00:00 | 2017-09-02 12:00:00 | 12 | 19 | -42.3 | 18.9 | -42.5 |
| 11 | 2017-09-02 00:00:00 | 2017-09-03 00:00:00 | 24 | 18.5 | -44.6 | 18.4 | -45.6 |
| 11 | 2017-09-02 00:00:00 | 2017-09-03 12:00:00 | 36 | 17.9 | -46.9 | 17.8 | -47.9 |
| 11 | 2017-09-02 00:00:00 | 2017-09-04 00:00:00 | 48 | 17.3 | -49 | 17.3 | -50.4 |
| 11 | 2017-09-02 00:00:00 | 2017-09-05 00:00:00 | 72 | 16.8 | -53.3 | 16.7 | -55.1 |
| 11 | 2017-09-02 00:00:00 | 2017-09-06 00:00:00 | 96 | 18 | -57.5 | 17.2 | -60.4 |
| 11 | 2017-09-02 00:00:00 | 2017-09-07 00:00:00 | 120 | 20.5 | -62.5 | 19.1 | -66 |
| 12 | 2017-09-02 06:00:00 | 2017-09-02 18:00:00 | 12 | 18.8 | -43.7 | 18.7 | -44 |
| 12 | 2017-09-02 06:00:00 | 2017-09-03 06:00:00 | 24 | 18.3 | -46.1 | 18.2 | -46.9 |
| 12 | 2017-09-02 06:00:00 | 2017-09-03 18:00:00 | 36 | 17.7 | -48.3 | 17.8 | -49.2 |
| 12 | 2017-09-02 06:00:00 | 2017-09-04 06:00:00 | 48 | 17.1 | -50.4 | 17 | -51.7 |
| 12 | 2017-09-02 06:00:00 | 2017-09-05 06:00:00 | 72 | 17 | -54.7 | 16.6 | -56.4 |
| 12 | 2017-09-02 06:00:00 | 2017-09-06 06:00:00 | 96 | 18.4 | -59.3 | 17.7 | -61.9 |
| 12 | 2017-09-02 06:00:00 | 2017-09-07 06:00:00 | 120 | 21 | -64.5 | 19.7 | -67.7 |
| 13 | 2017-09-02 12:00:00 | 2017-09-03 00:00:00 | 12 | 18.5 | -45 | 18.4 | -45.6 |
| 13 | 2017-09-02 12:00:00 | 2017-09-03 12:00:00 | 24 | 18 | -47.4 | 17.8 | -47.9 |
| 13 | 2017-09-02 12:00:00 | 2017-09-04 00:00:00 | 36 | 17.4 | -49.5 | 17.3 | -50.4 |
| 13 | 2017-09-02 12:00:00 | 2017-09-04 12:00:00 | 48 | 17.1 | -51.6 | 16.8 | -52.6 |
| 13 | 2017-09-02 12:00:00 | 2017-09-05 12:00:00 | 72 | 17.4 | -56 | 16.7 | -57.7 |
| 13 | 2017-09-02 12:00:00 | 2017-09-06 12:00:00 | 96 | 19.2 | -60.6 | 18.1 | -63.3 |
| 13 | 2017-09-02 12:00:00 | 2017-09-07 12:00:00 | 120 | 22 | -66 | 20.1 | -69 |
| 14 | 2017-09-02 18:00:00 | 2017-09-03 06:00:00 | 12 | 18 | -46.5 | 18.2 | -46.9 |
| 14 | 2017-09-02 18:00:00 | 2017-09-03 18:00:00 | 24 | 17.4 | -48.8 | 17.8 | -49.2 |
| 14 | 2017-09-02 18:00:00 | 2017-09-04 06:00:00 | 36 | 17 | -50.9 | 17 | -51.7 |
| 14 | 2017-09-02 18:00:00 | 2017-09-04 18:00:00 | 48 | 16.8 | -53 | 16.7 | -53.9 |
| 14 | 2017-09-02 18:00:00 | 2017-09-05 18:00:00 | 72 | 17.5 | -57.5 | 16.9 | -59.2 |
| 14 | 2017-09-02 18:00:00 | 2017-09-06 18:00:00 | 96 | 19.5 | -62.5 | 18.5 | -64.7 |
| 14 | 2017-09-02 18:00:00 | 2017-09-07 18:00:00 | 120 | 22 | -68 | 20.7 | -70.4 |
| 15 | 2017-09-03 00:00:00 | 2017-09-03 12:00:00 | 12 | 17.8 | -48 | 17.8 | -47.9 |
| 15 | 2017-09-03 00:00:00 | 2017-09-04 00:00:00 | 24 | 17.2 | -50.1 | 17.3 | -50.4 |
| 15 | 2017-09-03 00:00:00 | 2017-09-04 12:00:00 | 36 | 16.7 | -52.3 | 16.8 | -52.6 |
| 15 | 2017-09-03 00:00:00 | 2017-09-05 00:00:00 | 48 | 16.5 | -54.5 | 16.7 | -55.1 |
| 15 | 2017-09-03 00:00:00 | 2017-09-06 00:00:00 | 72 | 17.7 | -59.3 | 17.2 | -60.4 |
| 15 | 2017-09-03 00:00:00 | 2017-09-07 00:00:00 | 96 | 19.9 | -64.5 | 19.1 | -66 |
| 15 | 2017-09-03 00:00:00 | 2017-09-08 00:00:00 | 120 | 22.5 | -69.5 | 21.1 | -71.8 |
| 16 | 2017-09-03 06:00:00 | 2017-09-03 18:00:00 | 12 | 17.5 | -49.1 | 17.8 | -49.2 |
| 16 | 2017-09-03 06:00:00 | 2017-09-04 06:00:00 | 24 | 16.8 | -51.3 | 17 | -51.7 |
| 16 | 2017-09-03 06:00:00 | 2017-09-04 18:00:00 | 36 | 16.5 | -53.4 | 16.7 | -53.9 |
| 16 | 2017-09-03 06:00:00 | 2017-09-05 06:00:00 | 48 | 16.5 | -55.7 | 16.6 | -56.4 |
| 16 | 2017-09-03 06:00:00 | 2017-09-06 06:00:00 | 72 | 17.8 | -60.6 | 17.7 | -61.9 |
| 16 | 2017-09-03 06:00:00 | 2017-09-07 06:00:00 | 96 | 20 | -66 | 19.7 | -67.7 |
| 16 | 2017-09-03 06:00:00 | 2017-09-08 06:00:00 | 120 | 22.5 | -71.5 | 21.5 | -73.2 |
| 17 | 2017-09-03 12:00:00 | 2017-09-04 00:00:00 | 12 | 17.1 | -50.1 | 17.3 | -50.4 |
| 17 | 2017-09-03 12:00:00 | 2017-09-04 12:00:00 | 24 | 16.6 | -52.2 | 16.8 | -52.6 |
| 17 | 2017-09-03 12:00:00 | 2017-09-05 00:00:00 | 36 | 16.4 | -54.3 | 16.7 | -55.1 |
| 17 | 2017-09-03 12:00:00 | 2017-09-05 12:00:00 | 48 | 16.7 | -56.6 | 16.7 | -57.7 |
| 17 | 2017-09-03 12:00:00 | 2017-09-06 12:00:00 | 72 | 18.2 | -61.8 | 18.1 | -63.3 |
| 17 | 2017-09-03 12:00:00 | 2017-09-07 12:00:00 | 96 | 20.4 | -67.1 | 20.1 | -69 |
| 17 | 2017-09-03 12:00:00 | 2017-09-08 12:00:00 | 120 | 22.5 | -72 | 21.8 | -74.7 |
| 18 | 2017-09-03 18:00:00 | 2017-09-04 06:00:00 | 12 | 17.2 | -51.4 | 17 | -51.7 |
| 18 | 2017-09-03 18:00:00 | 2017-09-04 18:00:00 | 24 | 16.8 | -53.5 | 16.7 | -53.9 |
| 18 | 2017-09-03 18:00:00 | 2017-09-05 06:00:00 | 36 | 16.8 | -55.8 | 16.6 | -56.4 |
| 18 | 2017-09-03 18:00:00 | 2017-09-05 18:00:00 | 48 | 17.3 | -58.2 | 16.9 | -59.2 |
| 18 | 2017-09-03 18:00:00 | 2017-09-06 18:00:00 | 72 | 19.1 | -63.5 | 18.5 | -64.7 |
| 18 | 2017-09-03 18:00:00 | 2017-09-07 18:00:00 | 96 | 21.2 | -68.8 | 20.7 | -70.4 |
| 18 | 2017-09-03 18:00:00 | 2017-09-08 18:00:00 | 120 | 23 | -73.5 | 22 | -76 |
| 19 | 2017-09-04 00:00:00 | 2017-09-04 12:00:00 | 12 | 16.7 | -52.6 | 16.8 | -52.6 |
| 19 | 2017-09-04 00:00:00 | 2017-09-05 00:00:00 | 24 | 16.5 | -54.8 | 16.7 | -55.1 |
| 19 | 2017-09-04 00:00:00 | 2017-09-05 12:00:00 | 36 | 16.8 | -57.1 | 16.7 | -57.7 |
| 19 | 2017-09-04 00:00:00 | 2017-09-06 00:00:00 | 48 | 17.4 | -59.6 | 17.2 | -60.4 |
| 19 | 2017-09-04 00:00:00 | 2017-09-07 00:00:00 | 72 | 19.3 | -65 | 19.1 | -66 |
| 19 | 2017-09-04 00:00:00 | 2017-09-08 00:00:00 | 96 | 21.2 | -70.2 | 21.1 | -71.8 |
| 19 | 2017-09-04 00:00:00 | 2017-09-09 00:00:00 | 120 | 22.8 | -74.5 | 22.1 | -77.2 |
| 20 | 2017-09-04 06:00:00 | 2017-09-04 18:00:00 | 12 | 16.7 | -53.9 | 16.7 | -53.9 |
| 20 | 2017-09-04 06:00:00 | 2017-09-05 06:00:00 | 24 | 16.6 | -56.2 | 16.6 | -56.4 |
| 20 | 2017-09-04 06:00:00 | 2017-09-05 18:00:00 | 36 | 17 | -58.6 | 16.9 | -59.2 |
| 20 | 2017-09-04 06:00:00 | 2017-09-06 06:00:00 | 48 | 17.8 | -61.2 | 17.7 | -61.9 |
| 20 | 2017-09-04 06:00:00 | 2017-09-07 06:00:00 | 72 | 19.6 | -66.8 | 19.7 | -67.7 |
| 20 | 2017-09-04 06:00:00 | 2017-09-08 06:00:00 | 96 | 21 | -72 | 21.5 | -73.2 |
| 20 | 2017-09-04 06:00:00 | 2017-09-09 06:00:00 | 120 | 22 | -76.5 | 22.3 | -78.3 |
| 21 | 2017-09-04 12:00:00 | 2017-09-05 00:00:00 | 12 | 16.5 | -54.8 | 16.7 | -55.1 |
| 21 | 2017-09-04 12:00:00 | 2017-09-05 12:00:00 | 24 | 16.7 | -57.1 | 16.7 | -57.7 |
| 21 | 2017-09-04 12:00:00 | 2017-09-06 00:00:00 | 36 | 17.3 | -59.6 | 17.2 | -60.4 |
| 21 | 2017-09-04 12:00:00 | 2017-09-06 12:00:00 | 48 | 18.2 | -62.4 | 18.1 | -63.3 |
| 21 | 2017-09-04 12:00:00 | 2017-09-07 12:00:00 | 72 | 19.9 | -68 | 20.1 | -69 |
| 21 | 2017-09-04 12:00:00 | 2017-09-08 12:00:00 | 96 | 21.2 | -73.3 | 21.8 | -74.7 |
| 21 | 2017-09-04 12:00:00 | 2017-09-09 12:00:00 | 120 | 22 | -77.8 | 22.7 | -79.3 |
| 22 | 2017-09-04 18:00:00 | 2017-09-05 06:00:00 | 12 | 16.6 | -56.2 | 16.6 | -56.4 |
| 22 | 2017-09-04 18:00:00 | 2017-09-05 18:00:00 | 24 | 17 | -58.7 | 16.9 | -59.2 |
| 22 | 2017-09-04 18:00:00 | 2017-09-06 06:00:00 | 36 | 17.8 | -61.3 | 17.7 | -61.9 |
| 22 | 2017-09-04 18:00:00 | 2017-09-06 18:00:00 | 48 | 18.7 | -64.1 | 18.5 | -64.7 |
| 22 | 2017-09-04 18:00:00 | 2017-09-07 18:00:00 | 72 | 20.4 | -69.7 | 20.7 | -70.4 |
| 22 | 2017-09-04 18:00:00 | 2017-09-08 18:00:00 | 96 | 21.6 | -74.8 | 22 | -76 |
| 22 | 2017-09-04 18:00:00 | 2017-09-09 18:00:00 | 120 | 23 | -79 | 23.1 | -80.2 |
| 23 | 2017-09-05 00:00:00 | 2017-09-05 12:00:00 | 12 | 16.9 | -57.4 | 16.7 | -57.7 |
| 23 | 2017-09-05 00:00:00 | 2017-09-06 00:00:00 | 24 | 17.5 | -60 | 17.2 | -60.4 |
| 23 | 2017-09-05 00:00:00 | 2017-09-06 12:00:00 | 36 | 18.3 | -62.7 | 18.1 | -63.3 |
| 23 | 2017-09-05 00:00:00 | 2017-09-07 00:00:00 | 48 | 19.3 | -65.5 | 19.1 | -66 |
| 23 | 2017-09-05 00:00:00 | 2017-09-08 00:00:00 | 72 | 21.2 | -71 | 21.1 | -71.8 |
| 23 | 2017-09-05 00:00:00 | 2017-09-09 00:00:00 | 96 | 22.5 | -76 | 22.1 | -77.2 |
| 23 | 2017-09-05 00:00:00 | 2017-09-10 00:00:00 | 120 | 24 | -80 | 23.4 | -80.9 |
| 24 | 2017-09-05 06:00:00 | 2017-09-05 18:00:00 | 12 | 17 | -58.9 | 16.9 | -59.2 |
| 24 | 2017-09-05 06:00:00 | 2017-09-06 06:00:00 | 24 | 17.7 | -61.5 | 17.7 | -61.9 |
| 24 | 2017-09-05 06:00:00 | 2017-09-06 18:00:00 | 36 | 18.6 | -64.2 | 18.5 | -64.7 |
| 24 | 2017-09-05 06:00:00 | 2017-09-07 06:00:00 | 48 | 19.6 | -67 | 19.7 | -67.7 |
| 24 | 2017-09-05 06:00:00 | 2017-09-08 06:00:00 | 72 | 21.2 | -72.5 | 21.5 | -73.2 |
| 24 | 2017-09-05 06:00:00 | 2017-09-09 06:00:00 | 96 | 22.4 | -77.2 | 22.3 | -78.3 |
| 24 | 2017-09-05 06:00:00 | 2017-09-10 06:00:00 | 120 | 24 | -81 | 23.7 | -81.3 |
| 25 | 2017-09-05 06:00:00 | 2017-09-05 18:00:00 | 12 | 17 | -58.9 | 16.9 | -59.2 |
| 25 | 2017-09-05 06:00:00 | 2017-09-06 06:00:00 | 24 | 17.7 | -61.5 | 17.7 | -61.9 |
| 25 | 2017-09-05 06:00:00 | 2017-09-06 18:00:00 | 36 | 18.6 | -64.2 | 18.5 | -64.7 |
| 25 | 2017-09-05 06:00:00 | 2017-09-07 06:00:00 | 48 | 19.6 | -67 | 19.7 | -67.7 |
| 25 | 2017-09-05 06:00:00 | 2017-09-08 06:00:00 | 72 | 21.2 | -72.5 | 21.5 | -73.2 |
| 25 | 2017-09-05 06:00:00 | 2017-09-09 06:00:00 | 96 | 22.4 | -77.2 | 22.3 | -78.3 |
| 25 | 2017-09-05 06:00:00 | 2017-09-10 06:00:00 | 120 | 24 | -81 | 23.7 | -81.3 |
| 26 | 2017-09-05 12:00:00 | 2017-09-06 00:00:00 | 12 | 17.2 | -60.3 | 17.2 | -60.4 |
| 26 | 2017-09-05 12:00:00 | 2017-09-06 12:00:00 | 24 | 18.1 | -63 | 18.1 | -63.3 |
| 26 | 2017-09-05 12:00:00 | 2017-09-07 00:00:00 | 36 | 19.1 | -65.9 | 19.1 | -66 |
| 26 | 2017-09-05 12:00:00 | 2017-09-07 12:00:00 | 48 | 20.1 | -68.7 | 20.1 | -69 |
| 26 | 2017-09-05 12:00:00 | 2017-09-08 12:00:00 | 72 | 21.4 | -74 | 21.8 | -74.7 |
| 26 | 2017-09-05 12:00:00 | 2017-09-09 12:00:00 | 96 | 22.7 | -78.3 | 22.7 | -79.3 |
| 26 | 2017-09-05 12:00:00 | 2017-09-10 12:00:00 | 120 | 24.4 | -81.2 | 24.5 | -81.5 |
| 27 | 2017-09-05 18:00:00 | 2017-09-06 06:00:00 | 12 | 17.6 | -61.8 | 17.7 | -61.9 |
| 27 | 2017-09-05 18:00:00 | 2017-09-06 18:00:00 | 24 | 18.5 | -64.6 | 18.5 | -64.7 |
| 27 | 2017-09-05 18:00:00 | 2017-09-07 06:00:00 | 36 | 19.5 | -67.3 | 19.7 | -67.7 |
| 27 | 2017-09-05 18:00:00 | 2017-09-07 18:00:00 | 48 | 20.4 | -70.1 | 20.7 | -70.4 |
| 27 | 2017-09-05 18:00:00 | 2017-09-08 18:00:00 | 72 | 21.6 | -75.3 | 22 | -76 |
| 27 | 2017-09-05 18:00:00 | 2017-09-09 18:00:00 | 96 | 22.7 | -79.3 | 23.1 | -80.2 |
| 27 | 2017-09-05 18:00:00 | 2017-09-10 18:00:00 | 120 | 24.4 | -81.5 | 25.6 | -81.7 |
| 28 | 2017-09-06 00:00:00 | 2017-09-06 12:00:00 | 12 | 18.1 | -63.1 | 18.1 | -63.3 |
| 28 | 2017-09-06 00:00:00 | 2017-09-07 00:00:00 | 24 | 19.1 | -65.9 | 19.1 | -66 |
| 28 | 2017-09-06 00:00:00 | 2017-09-07 12:00:00 | 36 | 20.1 | -68.5 | 20.1 | -69 |
| 28 | 2017-09-06 00:00:00 | 2017-09-08 00:00:00 | 48 | 21 | -71.2 | 21.1 | -71.8 |
| 28 | 2017-09-06 00:00:00 | 2017-09-09 00:00:00 | 72 | 22 | -76.2 | 22.1 | -77.2 |
| 28 | 2017-09-06 00:00:00 | 2017-09-10 00:00:00 | 96 | 23.2 | -79.5 | 23.4 | -80.9 |
| 28 | 2017-09-06 00:00:00 | 2017-09-11 00:00:00 | 120 | 25 | -81.5 | 26.8 | -81.7 |
| 29 | 2017-09-06 06:00:00 | 2017-09-06 18:00:00 | 12 | 18.6 | -64.6 | 18.5 | -64.7 |
| 29 | 2017-09-06 06:00:00 | 2017-09-07 06:00:00 | 24 | 19.6 | -67.3 | 19.7 | -67.7 |
| 29 | 2017-09-06 06:00:00 | 2017-09-07 18:00:00 | 36 | 20.6 | -70 | 20.7 | -70.4 |
| 29 | 2017-09-06 06:00:00 | 2017-09-08 06:00:00 | 48 | 21.3 | -72.5 | 21.5 | -73.2 |
| 29 | 2017-09-06 06:00:00 | 2017-09-09 06:00:00 | 72 | 22.5 | -77 | 22.3 | -78.3 |
| 29 | 2017-09-06 06:00:00 | 2017-09-10 06:00:00 | 96 | 24 | -80 | 23.7 | -81.3 |
| 29 | 2017-09-06 06:00:00 | 2017-09-11 06:00:00 | 120 | 26.5 | -80.9 | 28.2 | -82.2 |
| 30 | 2017-09-06 12:00:00 | 2017-09-07 00:00:00 | 12 | 19 | -66.2 | 19.1 | -66 |
| 30 | 2017-09-06 12:00:00 | 2017-09-07 12:00:00 | 24 | 20.2 | -69 | 20.1 | -69 |
| 30 | 2017-09-06 12:00:00 | 2017-09-08 00:00:00 | 36 | 21.2 | -71.7 | 21.1 | -71.8 |
| 30 | 2017-09-06 12:00:00 | 2017-09-08 12:00:00 | 48 | 21.9 | -74.2 | 21.8 | -74.7 |
| 30 | 2017-09-06 12:00:00 | 2017-09-09 12:00:00 | 72 | 22.9 | -78.1 | 22.7 | -79.3 |
| 30 | 2017-09-06 12:00:00 | 2017-09-10 12:00:00 | 96 | 25.2 | -80 | 24.5 | -81.5 |
| 30 | 2017-09-06 12:00:00 | 2017-09-11 12:00:00 | 120 | 29 | -80.5 | 29.6 | -82.7 |
| 31 | 2017-09-06 18:00:00 | 2017-09-07 06:00:00 | 12 | 19.6 | -67.6 | 19.7 | -67.7 |
| 31 | 2017-09-06 18:00:00 | 2017-09-07 18:00:00 | 24 | 20.7 | -70.4 | 20.7 | -70.4 |
| 31 | 2017-09-06 18:00:00 | 2017-09-08 06:00:00 | 36 | 21.5 | -73.1 | 21.5 | -73.2 |
| 31 | 2017-09-06 18:00:00 | 2017-09-08 18:00:00 | 48 | 22.2 | -75.6 | 22 | -76 |
| 31 | 2017-09-06 18:00:00 | 2017-09-09 18:00:00 | 72 | 23.2 | -79 | 23.1 | -80.2 |
| 31 | 2017-09-06 18:00:00 | 2017-09-10 18:00:00 | 96 | 26 | -80.2 | 25.6 | -81.7 |
| 31 | 2017-09-06 18:00:00 | 2017-09-11 18:00:00 | 120 | 31 | -81 | 30.9 | -83.5 |
| 32 | 2017-09-07 00:00:00 | 2017-09-07 12:00:00 | 12 | 20.3 | -68.8 | 20.1 | -69 |
| 32 | 2017-09-07 00:00:00 | 2017-09-08 00:00:00 | 24 | 21.3 | -71.7 | 21.1 | -71.8 |
| 32 | 2017-09-07 00:00:00 | 2017-09-08 12:00:00 | 36 | 22.1 | -74.1 | 21.8 | -74.7 |
| 32 | 2017-09-07 00:00:00 | 2017-09-09 00:00:00 | 48 | 22.7 | -76.3 | 22.1 | -77.2 |
| 32 | 2017-09-07 00:00:00 | 2017-09-10 00:00:00 | 72 | 24 | -79.3 | 23.4 | -80.9 |
| 32 | 2017-09-07 00:00:00 | 2017-09-11 00:00:00 | 96 | 27.4 | -80.3 | 26.8 | -81.7 |
| 32 | 2017-09-07 00:00:00 | 2017-09-12 00:00:00 | 120 | 32 | -81 | 31.9 | -84.4 |
| 33 | 2017-09-07 06:00:00 | 2017-09-07 18:00:00 | 12 | 20.7 | -70.5 | 20.7 | -70.4 |
| 33 | 2017-09-07 06:00:00 | 2017-09-08 06:00:00 | 24 | 21.7 | -73.1 | 21.5 | -73.2 |
| 33 | 2017-09-07 06:00:00 | 2017-09-08 18:00:00 | 36 | 22.3 | -75.5 | 22 | -76 |
| 33 | 2017-09-07 06:00:00 | 2017-09-09 06:00:00 | 48 | 22.8 | -77.4 | 22.3 | -78.3 |
| 33 | 2017-09-07 06:00:00 | 2017-09-10 06:00:00 | 72 | 24.5 | -80 | 23.7 | -81.3 |
| 33 | 2017-09-07 06:00:00 | 2017-09-11 06:00:00 | 96 | 28.5 | -80.5 | 28.2 | -82.2 |
| 34 | 2017-09-07 12:00:00 | 2017-09-08 00:00:00 | 12 | 21.1 | -71.7 | 21.1 | -71.8 |
| 34 | 2017-09-07 12:00:00 | 2017-09-08 12:00:00 | 24 | 21.9 | -74.2 | 21.8 | -74.7 |
| 34 | 2017-09-07 12:00:00 | 2017-09-09 00:00:00 | 36 | 22.5 | -76.5 | 22.1 | -77.2 |
| 34 | 2017-09-07 12:00:00 | 2017-09-09 12:00:00 | 48 | 23.1 | -78.4 | 22.7 | -79.3 |
| 34 | 2017-09-07 12:00:00 | 2017-09-10 12:00:00 | 72 | 25.5 | -80 | 24.5 | -81.5 |
| 34 | 2017-09-07 12:00:00 | 2017-09-11 12:00:00 | 96 | 30.1 | -80.6 | 29.6 | -82.7 |
| 35 | 2017-09-07 18:00:00 | 2017-09-08 06:00:00 | 12 | 21.6 | -73.1 | 21.5 | -73.2 |
| 35 | 2017-09-07 18:00:00 | 2017-09-08 18:00:00 | 24 | 22.2 | -75.5 | 22 | -76 |
| 35 | 2017-09-07 18:00:00 | 2017-09-09 06:00:00 | 36 | 22.7 | -77.6 | 22.3 | -78.3 |
| 35 | 2017-09-07 18:00:00 | 2017-09-09 18:00:00 | 48 | 23.4 | -79.3 | 23.1 | -80.2 |
| 35 | 2017-09-07 18:00:00 | 2017-09-10 18:00:00 | 72 | 26.2 | -80.7 | 25.6 | -81.7 |
| 35 | 2017-09-07 18:00:00 | 2017-09-11 18:00:00 | 96 | 31.5 | -82 | 30.9 | -83.5 |
| 36 | 2017-09-08 00:00:00 | 2017-09-08 12:00:00 | 12 | 21.8 | -74.4 | 21.8 | -74.7 |
| 36 | 2017-09-08 00:00:00 | 2017-09-09 00:00:00 | 24 | 22.3 | -76.7 | 22.1 | -77.2 |
| 36 | 2017-09-08 00:00:00 | 2017-09-09 12:00:00 | 36 | 22.9 | -78.6 | 22.7 | -79.3 |
| 36 | 2017-09-08 00:00:00 | 2017-09-10 00:00:00 | 48 | 23.7 | -79.9 | 23.4 | -80.9 |
| 36 | 2017-09-08 00:00:00 | 2017-09-11 00:00:00 | 72 | 27.2 | -81.1 | 26.8 | -81.7 |
| 36 | 2017-09-08 00:00:00 | 2017-09-12 00:00:00 | 96 | 32 | -83.5 | 31.9 | -84.4 |
| 37 | 2017-09-08 06:00:00 | 2017-09-08 18:00:00 | 12 | 22.1 | -75.7 | 22 | -76 |
| 37 | 2017-09-08 06:00:00 | 2017-09-09 06:00:00 | 24 | 22.6 | -77.8 | 22.3 | -78.3 |
| 37 | 2017-09-08 06:00:00 | 2017-09-09 18:00:00 | 36 | 23.3 | -79.4 | 23.1 | -80.2 |
| 37 | 2017-09-08 06:00:00 | 2017-09-10 06:00:00 | 48 | 24.5 | -80.4 | 23.7 | -81.3 |
| 37 | 2017-09-08 06:00:00 | 2017-09-11 06:00:00 | 72 | 28 | -81.5 | 28.2 | -82.2 |
| 38 | 2017-09-08 12:00:00 | 2017-09-09 00:00:00 | 12 | 22.4 | -77.1 | 22.1 | -77.2 |
| 38 | 2017-09-08 12:00:00 | 2017-09-09 12:00:00 | 24 | 22.8 | -79 | 22.7 | -79.3 |
| 38 | 2017-09-08 12:00:00 | 2017-09-10 00:00:00 | 36 | 23.7 | -80.3 | 23.4 | -80.9 |
| 38 | 2017-09-08 12:00:00 | 2017-09-10 12:00:00 | 48 | 25 | -81 | 24.5 | -81.5 |
| 38 | 2017-09-08 12:00:00 | 2017-09-11 12:00:00 | 72 | 29.3 | -82.3 | 29.6 | -82.7 |
| 39 | 2017-09-08 18:00:00 | 2017-09-09 06:00:00 | 12 | 22.4 | -78.2 | 22.3 | -78.3 |
| 39 | 2017-09-08 18:00:00 | 2017-09-09 18:00:00 | 24 | 23 | -79.9 | 23.1 | -80.2 |
| 39 | 2017-09-08 18:00:00 | 2017-09-10 06:00:00 | 36 | 24.1 | -81 | 23.7 | -81.3 |
| 39 | 2017-09-08 18:00:00 | 2017-09-10 18:00:00 | 48 | 25.7 | -81.5 | 25.6 | -81.7 |
| 39 | 2017-09-08 18:00:00 | 2017-09-11 18:00:00 | 72 | 30.6 | -83 | 30.9 | -83.5 |
| 40 | 2017-09-09 00:00:00 | 2017-09-09 12:00:00 | 12 | 22.6 | -79.1 | 22.7 | -79.3 |
| 40 | 2017-09-09 00:00:00 | 2017-09-10 00:00:00 | 24 | 23.3 | -80.6 | 23.4 | -80.9 |
| 40 | 2017-09-09 00:00:00 | 2017-09-10 12:00:00 | 36 | 24.5 | -81.4 | 24.5 | -81.5 |
| 40 | 2017-09-09 00:00:00 | 2017-09-11 00:00:00 | 48 | 26.5 | -81.9 | 26.8 | -81.7 |
| 40 | 2017-09-09 00:00:00 | 2017-09-12 00:00:00 | 72 | 31.6 | -83.8 | 31.9 | -84.4 |
| 41 | 2017-09-09 06:00:00 | 2017-09-09 18:00:00 | 12 | 22.9 | -80 | 23.1 | -80.2 |
| 41 | 2017-09-09 06:00:00 | 2017-09-10 06:00:00 | 24 | 23.8 | -81.1 | 23.7 | -81.3 |
| 41 | 2017-09-09 06:00:00 | 2017-09-10 18:00:00 | 36 | 25.4 | -81.8 | 25.6 | -81.7 |
| 41 | 2017-09-09 06:00:00 | 2017-09-11 06:00:00 | 48 | 27.7 | -82.4 | 28.2 | -82.2 |
| 42 | 2017-09-09 12:00:00 | 2017-09-10 00:00:00 | 12 | 23.4 | -80.7 | 23.4 | -80.9 |
| 42 | 2017-09-09 12:00:00 | 2017-09-10 12:00:00 | 24 | 24.5 | -81.6 | 24.5 | -81.5 |
| 42 | 2017-09-09 12:00:00 | 2017-09-11 00:00:00 | 36 | 26.4 | -82.1 | 26.8 | -81.7 |
| 42 | 2017-09-09 12:00:00 | 2017-09-11 12:00:00 | 48 | 29 | -82.8 | 29.6 | -82.7 |
| 43 | 2017-09-09 18:00:00 | 2017-09-10 06:00:00 | 12 | 24.1 | -81.3 | 23.7 | -81.3 |
| 43 | 2017-09-09 18:00:00 | 2017-09-10 18:00:00 | 24 | 25.7 | -82 | 25.6 | -81.7 |
| 43 | 2017-09-09 18:00:00 | 2017-09-11 06:00:00 | 36 | 28.3 | -82.6 | 28.2 | -82.2 |
| 43 | 2017-09-09 18:00:00 | 2017-09-11 18:00:00 | 48 | 31.2 | -83.8 | 30.9 | -83.5 |
| 44 | 2017-09-10 00:00:00 | 2017-09-10 12:00:00 | 12 | 24.7 | -81.7 | 24.5 | -81.5 |
| 44 | 2017-09-10 00:00:00 | 2017-09-11 00:00:00 | 24 | 26.8 | -82.4 | 26.8 | -81.7 |
| 44 | 2017-09-10 00:00:00 | 2017-09-11 12:00:00 | 36 | 29.5 | -83.4 | 29.6 | -82.7 |
| 44 | 2017-09-10 00:00:00 | 2017-09-12 00:00:00 | 48 | 32.2 | -85 | 31.9 | -84.4 |
| 45 | 2017-09-10 06:00:00 | 2017-09-10 18:00:00 | 12 | 25.4 | -82 | 25.6 | -81.7 |
| 45 | 2017-09-10 06:00:00 | 2017-09-11 06:00:00 | 24 | 27.8 | -82.8 | 28.2 | -82.2 |
| 45 | 2017-09-10 06:00:00 | 2017-09-11 18:00:00 | 36 | 30.5 | -84.1 | 30.9 | -83.5 |
| 46 | 2017-09-10 12:00:00 | 2017-09-11 00:00:00 | 12 | 26.6 | -82.1 | 26.8 | -81.7 |
| 46 | 2017-09-10 12:00:00 | 2017-09-11 12:00:00 | 24 | 29.2 | -83 | 29.6 | -82.7 |
| 46 | 2017-09-10 12:00:00 | 2017-09-12 00:00:00 | 36 | 31.9 | -84.6 | 31.9 | -84.4 |
| 47 | 2017-09-10 18:00:00 | 2017-09-11 06:00:00 | 12 | 28 | -82.5 | 28.2 | -82.2 |
| 47 | 2017-09-10 18:00:00 | 2017-09-11 18:00:00 | 24 | 30.7 | -83.9 | 30.9 | -83.5 |
| 48 | 2017-09-11 00:00:00 | 2017-09-11 12:00:00 | 12 | 29.4 | -82.8 | 29.6 | -82.7 |
| 48 | 2017-09-11 00:00:00 | 2017-09-12 00:00:00 | 24 | 31.9 | -84.7 | 31.9 | -84.4 |
| 49 | 2017-09-11 06:00:00 | 2017-09-11 18:00:00 | 12 | 30.8 | -83.7 | 30.9 | -83.5 |
| 50 | 2017-09-11 12:00:00 | 2017-09-12 00:00:00 | 12 | 32 | -84.5 | 31.9 | -84.4 |
The analysis
Two verification ‘products’ were developed: a table with the relative number of hits (hurricane center observations within the projected circles) and maps that showed the locations of the projected circles as well as the actual location at that time.
The graphs were produced as follows:
- First, the latitudes and longitudes were converted from their original WGS84 projection to the Mercator projection. This is a projection that is more or less in metres.
- Based on the NHC forecast circle radii, circles were drawn around the projected center locations.
- Actual observations were then added to the map.
The tables were produced as follows:
- The distance between the forecast and observed locations was determined by applying the haversine formula.
- By comparing with the forecast circle radii, it was determined whether the observation fell within the projected circle.
- For each lead time separately, the fraction of forecasts that turned out to be hits was calculated.
Results
Below video gives a fair idea of forecast quality. Here are some viewing guides.
- First, 12-hour forecasts and their verifying observations are shown. Then, 24-hour forecasts, 36h, etc. The forecast lead time is shown in the plot title.
- The plot time is shown in the plot’s subtitle. Plot time progresses.
- At plot time, forecasts of future center positions exist. These are the circles slightly ahead of the animated path.
- The actual location (at plot time) of Irma’s center is indicated by a small black dot.
- The associated forecast at plot time – that was made X hours ago – is then highlighted.
- If we interpret the circles to be probabilistic forecasts, about 2/3 of the small black dots should be within the highlighted forecast circles (a hit). This also means that about 1/3 should be outside (these are misses). Look at the video and try and guess if this is indeed the case.
Calculating the distances between forecast and observed locations yielded the following ‘errors’:
The analysis of hits and misses yields the following table. Note that the hit rate should be approx. 2/3 or .67 in order for the forecasts to be reliable.
| Forecast lead time [h] | Hit rate [-] | Average 'error' [km] | Average 'error' [US nm] |
|---|---|---|---|
| 12 | 0.96 | 28 | 15 |
| 24 | 0.9 | 48 | 26 |
| 36 | 0.91 | 69 | 37 |
| 48 | 0.84 | 94 | 50 |
| 72 | 0.9 | 136 | 73 |
| 96 | 0.92 | 187 | 101 |
| 120 | 0.88 | 243 | 131 |
Conclusions
The hit rate of the forecast cone circles varies with lead time and is in the order of 90%. Given recent forecast performance
The size of each circle is set so that two-thirds of historical official forecast errors over a 5-year sample fall within the circle.
this is higher than expected. For whatever reason, the Irma forecasts were sharper than the average of the forecasts made in the previous five years.
If we would have interpreted this 2/3 as a probabilistic forecast, we would have found the Irma forecasts to be underconfident! This is rather unusual. More often, probability forecasts turn out to be overconfident.
In order for the a priori probabilities to have matched the observed frequencies, the suggested probabilities associated with the circles (two-thirds, or approx 67%) would have had to have been higher (approx 85-95%). Alternatively, the circle diameters would have had to have been smaller – resulting in a more narrow cone.
Note that here, we applied the principle of forecast post-processing: we assessed past performance and deduced probabilities from that. In this case, our assumption (that past performance was a measure of present performance) did not hold. That is, not in the case of this particular set of 52 forecasts. Possibly, a more physical approach to uncertainty estimation would have been better. I am thinking of ensembles.
Tools used
For this exercise, a number of tools was used. Downloading of the cone plots and the advisory HTML files was done using a Linux bash script that called wget. The cone plots were combined in an animated GIF using FFMPEG. Processing and plotting was done in R using the ggmap, rgeos and the geosphere package:
- D. Kahle and H. Wickham. ggmap: Spatial Visualization with ggplot2. The R Journal, 5(1), 144-161. URL http://journal.r-project.org/archive/2013-1/kahle-wickham.pdf
- Robert J. Hijmans (2016). geosphere: Spherical Trigonometry. R package version 1.5-5. https://CRAN.R-project.org/package=geosphere
- R Core Team (2016). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.
- Roger Bivand and Colin Rundel (2017). rgeos: Interface to Geometry Engine – Open Source (GEOS). R package version 0.3-23. https://CRAN.R-project.org/package=rgeos
All tools are available as open source software. Open source rocks!
1 thought on “Verifying hurricane Irma’s projected path”