Wed. Dec 1st, 2021

Research led by Sophie Meakin of the London School of Hygiene & Tropical Medicine in the UK suggests that ensemble prognosis – a method that combines multiple models to predict COVID-19 cases – may be more accurate in estimating COVID-19 hospitalizations in local areas.

Study: Comparative assessment of methods for short-term prognosis for COVID-19 admissions in the UK at the local level.  Image credit: NIAID

The United Kingdom has reported over 8.89 million COVID-19 cases and there are concerns that the cases will exceed hospital resources. However, it is difficult to make accurate predictions given changes in local pandemic restrictions. Using confirmed COVID-19 cases as a prediction may help estimate future cases, but still leaves some variation.

“Given the minimal data and computational requirements for the models evaluated here, this approach can be used to make early forecasts of demand for health care at the local level, thus helping with situational awareness and capacity planning in future epidemic or pandemic environments,” concluded the research team.

The study “Comparative assessment of methods for short-term prognosis for COVID-19 admissions in the UK at local level” is published on medRxiv* preprint server.

How they did it

The study uses either individual or ensemble real-time forecasts to predict COVID-19-related hospitalizations in the UK. Weekly forecasts for daily admissions were taken between August 2020 and April 2021.

Three models – an average ensemble of autoregressive time series models, a linear regression model with 7-day delayed local cases as a prediction and a scaled folding of local cases and a delay distribution – were used for forecasts. These models were measured for their accuracy in predicting the number of hospitalizations under multiple scenarios, including the length of the predictable horizon, when the prognosis was created, the geographic location, and how the prognosis improved when future cases were known.

The baseline model included no change from the last day of hospital admissions.

Overview of COVID-19 hospital admissions in England in August 2020 - April 2021. (A) Daily COVID-19 hospital admissions for England.  (B) Weekly COVID-19 hospital admissions by the NHS Trust (identified by 3-letter code) for the top 40 trusts according to the total number of COVID-19 hospital admissions in August 2020 - April 2021. (C) Daily COVID-19 hospital admissions for top-five funds after a total of COVID-19 hospital admissions.  In all panels, the dotted lines indicate the date of the first (October 4, 2020) and last (April 25, 2021) forecast date.  Data published by NHS England [39].

Overview of COVID-19 hospital admissions in England in August 2020 – April 2021. (A) Daily COVID-19 hospital admissions for England. (B) Weekly COVID-19 hospital admissions by the NHS Trust (identified by 3-letter code) for the top 40 trusts according to the total number of COVID-19 hospital admissions in August 2020 – April 2021. (C) Daily COVID-19 hospital admissions for top-five funds after a total of COVID-19 hospital admissions. In all panels, the dotted lines indicate the date of the first (October 4, 2020) and last (April 25, 2021) forecast date. Data published by NHS England [39].

The middle-ensemble model was most accurate in predicting COVID-19 hospitalizations

When the model assumes that there is no change in the current admissions, all models performed better than all base models in all scenarios. Of all the models, the mean-ensemble model made the most accurate predictions for the number of admissions.

The introduction of COVID-19 cases in modeling promoted better predictions about the number of future hospitalizations. However, hospitalization predictions continued to be less accurate and sometimes even worse than trend-based modeling when other factors, such as pandemic limitation and fluctuating COVID-19 cases, were considered.

Ways to improve COVID-19 predictions for hospitalizations

The researchers suggest several ways to improve the accuracy of COVID-19 forecasts. One method is to improve the underlying case prognoses by adding more detailed predictions for hospitalizations, such as labeling COVID-19 cases by age.

Forecast accuracy by forecast date.  (A) Relative WIS (rWIS) of the forecast models for the 30 forecast dates.  Lower rWIS values ​​indicate better forecasts.  (B) Average absolute error in the forecast models.  The average AE is calculated as the average AE across all Trusts.  (C) Average daily COVID-19 hospital admissions at the confidence level per  week, for reference.  All panels are for a 7-day forecast horizon;  see Figure S5 for evaluation on a 14-day forecast horizon.

Forecast accuracy by forecast date. (A) Relative WIS (rWIS) of the forecast models for the 30 forecast dates. Lower rWIS values ​​indicate better forecasts. (B) Average absolute error in the forecast models. The average AE is calculated as the average AE across all Trusts. (C) Average daily COVID-19 hospital admissions at the confidence level per week, for reference. All panels are for a 7-day forecast horizon; see Figure S5 for evaluation on a 14-day forecast horizon.

Adding additional models such as statistical models and machine learning models can also improve the prediction levels. Another could include an ensemble model pool or a weighted ensemble that takes into account past performance.

Finally, forecasts will be improved by mapping a smaller geographical region.

Based on the results, the researchers suggest “When predicting hospital admissions at the local level in an epidemic setting, it is rarely better to assume that no change in admissions is than to include at least one trend component; including a delayed prediction, such as cases, can further improve prognosis accuracy. “, but relies on making good case forecasts, especially for longer forecast horizons. By using an average ensemble, some of the individual performance’s variable performance is overcome and allows us to make more accurate and more consistently accurate forecasts across time and locations.”

*Important message

medRxiv publishes preliminary scientific reports that are not peer-reviewed and therefore should not be considered as crucial, guide clinical practice / health-related behavior or be treated as established information.

.

Leave a Reply

Your email address will not be published. Required fields are marked *