The limits of human mobility traces to predict the spread of COVID-19: A transfer entropy approach — TEI-tagged view

Federico DelussupersName, Michele TizzonipersName, Laetitia GauvinpersName (2023date). “The limits of human mobility traces to predict the spread of COVID-19: A transfer entropy approach.” PNAS Nexustitle·j.

Object of study — information flow from mobile-phone mobility metrics to COVID-19term·key cases and deaths in over 200 European subnational regions

Research question — how much predictive information mobile-phone mobility data actually add to forecasting COVID-19term·key across settings

Contribution — a model-free transfer-entropy framework for judging mobility data, and the finding that it rarely adds significant information

Full text — tagged

The limits of human mobility traces to predict the spread of COVID-19term·key: A transfer entropyterm·mth approach Abstract Mobile phone dataterm·key have been widely used to model the spread of COVID-19term·key; however, quantifying and comparing their predictive value across different settings is challenging. Their quality is affected by various factors and their relationship with epidemiological indicators varies over time. Here, we adopt a model-free approach based on transfer entropyterm·mth to quantify the relationship between mobile phone-derived mobility metrics and COVID-19term·key cases and deaths in more than 200 European subnational regions. Using multiple data sources over a one-year period, we found that past knowledge of mobility does not systematically provide statistically significant information on COVID-19term·key spread. Our approach allows us to determine the best metric for predicting disease incidence in a particular location, at different spatial scales. Additionally, we identify geographic and demographic factors, such as users’ coverage and commuting patterns, that explain the (non)observed relationship between mobility and epidemic patterns. Our work provides epidemiologists and public health officials with a general—not limited to COVID-19term·key—framework to evaluate the usefulness of human mobility data in responding to epidemics.

Keywords: human mobility, COVID-19term·key, mobile phone dataterm·key, transfer entropyterm·mth Mobile phone dataterm·key are considered a key ingredient of realistic disease transmission models. However, it is hard to gauge their usefulness in epidemic forecasting because their added value often depends on the specific definition of mobility and the modeling approach. We develop a general and model-free framework to quantify the information transfer between mobile phone-derived mobility indicators and epidemic time series. By measuring the relative information added by different types of mobility traces to predict the spread of COVID-19term·key in four European countries, we find that in 2020–2021 cell phone data provided limited information to forecast COVID-19term·key. Our results provide guidance on the effective use of mobility metrics in response to epidemic outbreaks.

Introduction The relationship between human movements and the spatial spread of infectious diseases has been recognized for a long time ( 1–3 ). Human movement has been shown to play a key role in the dynamics of several pathogens, through two basic mechanisms: traveling infectious individuals may introduce a pathogen in a susceptible population, and, at the same time, human movement increases the contact rateterm·dom between individuals, creating new opportunities for infection. In the past 15 years, the increasing availability of mobility data derived from mobile phones has fueled a large body of work aimed at identifying opportunities to use them for infectious disease modeling and surveillance ( 4–10 ).

More recently, during the COVID-19term·key pandemic, mobile phone-derived data have been extensively harnessed to monitor the effect of nonpharmaceutical interventions (NPIs) across countries, understand the early dynamics of COVID-19term·key diffusion, and forecast its spread at different spatial scales, from countries to cities ( 11–17 ). By measuring human movements and combining them with phylogeography methods ( 18 , 19 ), several studies shed light on the cryptic spread of new variants, their persistence over time and resurgence after the relaxation of NPIs ( 20–22 ).

Human mobility has been shown to strongly correlate with the spread of COVID-19term·key during the early phase of the outbreak in China and in many other countries ( 23–28 ). However, once COVID-19term·key established a foothold in a population, the relative importance of mobile phone-derived data to predict the epidemic dynamics on a local scale has been generally less understood and several studies have shown conflicting evidence about the use of mobility traces to model the spread of COVID-19term·key at later stages of the outbreak. For instance, it has been shown that the explanatory power of mobility metrics in relation to the case growth rate in the United States significantly declined in spring 2020, especially in rural areas ( 29–31 ). Similar trends have been observed in Europe ( 32 ). In parallel, mobile phone-derived data have been proven beneficial to model COVID-19term·key dynamics in largely populated urban areas of Western countries ( 33 , 34 ), but less so in countries of the Global South ( 35 ).

Several reasons have been proposed to explain the varying relationship between mobility metrics and epidemic indicators ( 29 ). Mobility metrics are generally derived from raw mobile positioning data through complex and customized processing pipelines that can significantly vary across data providers ( 36 ). How raw data are processed, and the specific definitions of mobility metrics can significantly impact their interpretation with respect to epidemic variables ( 37 ). Moreover, the relationship between mobility and epidemic patterns often relies on modeling assumptions, typically considering linear dependencies, that may not capture the complex interplay of these quantities ( 30 , 32 ). Finally, mobile phone-derived metrics are generated from a sample of users that is generally not representative of the whole population. It is therefore of paramount importance to define standardized approaches that can quantify the added value of mobility metrics for epidemiological analysis and make different metrics, across settings, directly comparable.

Here, we extensively quantify the relationship between cell phone-derived mobility metrics and COVID-19term·key epidemiological indicators through a model-free approach, based on an information-theoretic measure, transfer entropyterm·mth ( 38 ), adapted for small sample sizes. Leveraging granular data provided by Meta that capture users’ movements and colocation at a fine spatial scale ( 39 ) and Google Community Mobility Reports ( 40 ), we measure the information flow between mobility metrics and time series of COVID-19term·key incidence and deaths in four European countries, at a subnational scale, over a one-year period. We find that the relative information added by the past knowledge of mobility metrics to the knowledge of the current state of COVID-19term·key time series is often not statistically significant, and that its significance also depends on the spatial resolution considered. At the finest resolution, in statistically significant cases, we show that the relative information added by past knowledge of COVID-19term·key cases to the knowledge of current deaths is twice the information flow between past knowledge of mobility metrics and current deaths. We also show that the information flow of a given mobility metric to predict future COVID-19term·key incidence or deaths can be significant in one country but not in another, even if derived from the same original data source.

Being a general framework, our approach provides a quantitative measure of the relative added explanation brought by mobile phone dataterm·key to the prediction of epidemiological time series that does not depend on the choice of a specific forecasting model. It thus helps to better identify the most appropriate mobility metrics to use among those available. Our results can guide epidemiologists and public health practitioners in the evaluation of mobile phone-derived mobility metrics when they are interpreted as a precursor of epidemic activity.

Results Here, we first describe and then apply our framework to measure the information flow between human mobility traces and the time evolution of COVID-19term·key in four European countries.

A transfer entropyterm·mth approach to link mobility behavior and COVID-19term·key epidemiology With the aim of quantifying the information flow from mobility-derived data to COVID-19term·key data, we first gathered a set of mobility and epidemiological indicators. Figure 1 provides an overview of the datasets used in the study. In the Materials and methods section, we provide a full description of all data sources and the data processing steps. We considered four European countries—Austria, France, Italy, and Spain—and their administrative subdivisions at NUTS3 level ( 41 ) which is the lowest, i.e. the most granular, level of the standard hierarchy of administrative regions in Europe (Fig. 1 , leftmost column).

Summary of behavioral and epidemiological indicators. In each country under study (from top to bottom: Italy, France, Austria, and Spain), we consider three different types of indicators: contact ratesterm·key, movements (here for the sake of simplicity we only show the short-range movementsterm·key), and COVID-19term·key cases. In each plot, the blue shaded area highlights the within-country variability, corresponding to time series in every administrative subdivision. The solid line represents the average value. All curves are normalized between 0 and 1, corresponding to their maximum value.

In all administrative regions, we collected indicators of the COVID-19term·key epidemic dynamics, namely, the weekly and daily numbers of new COVID-19term·key cases and deaths over the period, from September 2020 until July 2021. During this period, the dynamics of COVID-19term·key, exemplified by the incidence of new cases (Fig. 1 , rightmost column), displayed subsequent waves, as a result of the complex interaction between the spread of new variants, the adoption of nonpharmaceutical interventions, the introduction of vaccines.

In each country, we also collected weekly and daily time series describing movements and colocation patterns made available by Meta ( 42 ). We computed contact ratesterm·key from colocation mapsterm·key (see Material and methods section and online supplementary material for details), which measure the probability that two users from two locations are found in the same location at the same time ( 39 ). Colocation mapsterm·key were generated by Meta on a weekly basis, only. To study human movement patterns, we considered movement range maps provided by Meta, which report the number of users who moved between any two 16-level Bing tiles with an 8 h frequency ( 43 ). To make colocation and movement patterns comparable in terms of scale, we focused on short-range movementsterm·key, i.e. movements that occurred within the same tile, and we separately considered the mid-range movements, i.e. movements that occur between two different tiles in the same province. We then processed the three datasets, starting from their raw form, to aggregate them at the NUTS3 resolution and create the time series: M s ( t ) for the short-range movementsterm·key, M ( t ) for the mid-range movements and CR ( t ) for the contact ratesterm·key. We also gathered daily mobility data that captures the relative change in mobility with respect to a baseline from two different data sources: the relative change in time spent at home, provided by Google, and the relative change in total movements, provided by Meta (see Materials and methods section for more details). The first dataset was available at NUTS3 resolution, while the second was only available at NUTS2 level. In the following, we refer to the residential time series as M r ( t ) and to the relative change in movement provided by Meta as MRC ( t ) . We further aggregated the mobility metrics M s ( t ) and M ( t ) at the NUTS2 level, to explore the effect of spatial resolution on our results.

Mobile phone-derived time series were then used as source variables in the information-theoretic analysis. In the remainder of the paper, we focus on the analysis of the CR ( t ) , M ( t ) , and M s ( t ) time series at the finest spatial resolution, generally referring to NUTS3 units as provinces, although their nomenclature varies across countries.

Figure 2 illustrates our study design based on the transfer entropyterm·mth ( 38 ). Transfer entropyterm·mth is a metric that measures the directed statistical dependence between a source and a target time series and it has been applied to a wide range of research domains ( 44 ). Here, our approach consists, first, in computing the transfer entropyterm·mth between mobility time series, M s ( t ) , M ( t ) , and CR ( t ) , and epidemiological time series such as the reported number of COVID-19term·key attributed deaths D ( t ) and cases C ( t ) , in each administrative unit, and for different temporal lags l , using the definition of Shannon entropy, as described by the equations in Fig. 2 . Intuitively, the transfer entropyterm·mth between mobility and deaths, TE M s → D (resp. TE M → D and TE M r → D ), can be interpreted as the degree of uncertainty of the reported deaths, D , at time t that is solved jointly by the time series of deaths and mobility trends M s (resp. M and M r ) and exceeds the current degree of uncertainty of D , which can be solved by D ’s own past.

Illustration of study design. We computed the transfer entropyterm·mth TE X → Y to measure the information flow between source X (on the left) and target time series Y (right), for a given time lag l . In the figure example, as target time series we consider the number of COVID-19term·key deaths, D ( t ) . As source time series, we consider either mobility indicators, M s ( t ) , M ( t ) , CR ( t ) , or COVID-19term·key cases C ( t ) . Transfer entropyterm·mth quantifies the amount of information that is added by past knowledge of mobility or cases (green and cyan bars, respectively) to current knowledge of deaths, with respect to the knowledge of past deaths only (blue bar). After correcting the TE for small sample sizes, and normalizing by the reference value represented by the blue bar, we finally compare the normalized effective transfer entropyterm·mth of mobility and cases (rightmost box).

It is known that transfer entropyterm·mth estimates suffer in case of small sample sizes and nonstationarity of the source and target time series ( 45 ). Moreover, due to the nonparametric nature of the transfer entropyterm·mth, values computed between different source-target time series are not directly comparable. To address these issues, we first adopted the definition of effective transfer entropyterm·mth (ETEterm·mth) ( 45 ). ETEterm·mth is obtained by subtracting from the original definition of TE a reference TE value using a shuffled version of the target time series (see Materials and methods section for details), thus removing spurious contributions to TE due to fluctuations observed in small sample sizes. Also, to address biases due to small sample sizes, we applied a Kernel Density Estimationterm·mth, before the time series discretization that is necessary to compute the transfer entropyterm·mth. Second, we normalized the effective transfer entropyterm·mth by the Shannon entropy of the target variable, defining a normalized effective transfer entropyterm·mth (NETEterm·mth) ( 46 ). We obtain a metric that is always positive when it is statistically significant and whose zero value indicates the absence of information transfer between time series. In the remainder of the article, we thus refer to the NETEterm·mth between source X and target Y as our main quantity of interest, using the symbol N X → Y to denote it.

To better understand the cause–effect relationship between mobility and COVID-19term·key deaths, which are encoded in the value of N M → D , N M s → D , N M r → D and N CR → D , we compared them against the transfer entropyterm·mth N C → D , where C is the time series of new COVID-19term·key cases. As the causal relationship between the number of cases and deaths is established by definition, we used the transfer entropyterm·mth N C → D as a benchmark to evaluate the added value of mobility indicators to predict COVID-19term·key deaths. As an example, similar values of N M → D and N C → D would suggest knowledge of past COVID-19term·key incidence encodes a similar amount of information as knowledge of past mobility when it comes to predicting future deaths.

The information flow between COVID-19term·key incidence and deaths As previously mentioned, to gauge our transfer entropyterm·mth analysis framework, we first looked at the causal relationship between the incidence of COVID-19term·key cases and reported death counts. It is clearly expected that a major source of information that provides knowledge on future deaths is encoded in the time series of past case counts. We used NETEterm·mth to quantify such information flow.

Figure 3 shows the NETEterm·mth between the weekly time series of COVID-19term·key cases and deaths in the four countries under study. In all countries, median values of N C → D increase from lags equal to 1 week up to a maximum of around 2–3 weeks, and then decline rapidly beyond the 3 weeks time lag. This is in line with early estimates of the median time delay between case reporting and fatality, which was estimated to range between 7 and 20 days in different countries ( 47 , 48 ). At lag equal to 2 weeks, the mean relative explanation added by time series of cases with respect to deaths—that is how much of D ( t ) can be explained only by the past knowledge C ( t − l ) —is 14 % ( SD = 8 ) in Spain, 8 % ( SD = 6 ) in Italy, 7 % ( SD = 5 ) in Austria, and 6 % ( SD = 5 ) in France. Boxplots computed on the distribution of administrative units in each country show a substantial heterogeneity of NETEterm·mth across regions for lags shorter than 4 weeks. This may be partially explained by spatial heterogeneities of case and death reporting, and of testing strategies. Also, N C → D values appear to be higher in Spain, with respect to the other countries. A transfer entropyterm·mth analysis of daily time series of COVID-19term·key cases and deaths displays consistent results (see Fig. S1 in the online supplementary material), with NETEterm·mth values that fall within the same range measured on a weekly time scale. These results suggest NETEterm·mth estimates are robust with respect to the time scale at which source and target time series are compared. Moreover, it provides a reference value for NETEterm·mth, in terms of orders of magnitude, when the existence of a causal relationship between time series is known.

Information flow between COVID-19term·key incidence and deaths. NETEterm·mth between COVID-19term·key weekly reported cases and deaths in the NUTS3 administrative subdivisions (provinces) of Austria, France, Italy, and Spain. NETEterm·mth is computed for lags ranging from 1 to 8 weeks, on the x -axis. Boxplots are computed on the distribution of NETEterm·mth values of all the administrative subdivisions in each country. The horizontal red line marks the value N C → D = 0 .

The information flow between mobility traces and COVID-19term·key dynamics Having defined a benchmark of information transfer using N C → D , we measured the information flow between behavioral time series of mobility indicators and COVID-19term·key cases and deaths. Figure 4 summarizes the main results of our analysis of the weekly time series. Values of N X → D , with X being either short-range movementsterm·key, mid-range movements, or contact ratesterm·key, were substantially smaller than N C → D in all countries, for any given time lag l . In particular, Fig. 4 a allows comparing the distributions of N C → D , N CR → D , N M s → D , and N M → D , at the time lag l that maximized the median NETEterm·mth for weekly time series, for all indicators. We found the largest median values of the normalized transfer entropyterm·mth (NTE) at l = 7 weeks for both contact ratesterm·key and movements (short-range and mid-range). The upper quartile of the NETEterm·mth distributions derived from the mobility traces generally fell below 5 % , in all countries, while the lower quartile of N C → D was always above 5 % . Also, the distributions of NTE computed from movements were much narrower and often included the value N = 0 within their interquartile range. Values of N M → C , shown in Fig. 4 b, display a pattern similar to the NTE from the mobility time series to the death time series, with generally low values of NETEterm·mth in all countries. Compared to movement time series, contact ratesterm·key led generally to relatively higher values of NETEterm·mth with both targets, cases and deaths, as shown in Fig. 4 . Our result confirms the additional value of measuring contact ratesterm·key from mobile phone dataterm·key, with respect to other movement metrics ( 49 ). Besides, it shows that short-range mobility within a province had often limited predictive power to capture time trends of COVID-19term·key spread.

Information flow from mobility data to COVID-19term·key incidence and deaths. Comparison between the NETEterm·mth computed from source time series X and target time series of reported COVID-19term·key deaths D a) and cases C b). Source time series are COVID-19term·key cases (only for deaths), contact ratesterm·key, short-range and mid-range movements. Boxplots are computed from the distribution of NETEterm·mth values for a given time delay, l . In panel a: l = 2 weeks for cases, 7 weeks for contact ratesterm·key and movement. In panel b: l = 6 weeks for short-range and mid-range movements. The horizontal red line marks the value N X → D = 0 .

To obtain a more detailed picture of the predictive power of different mobility metrics in terms of NETEterm·mth, we computed the percentage of provinces for which mobility time series provided significant relative information added, with respect to the past knowledge of epidemiological indicators only (see Table 1 ). On the one hand, our framework effectively captured the existing causal relationship between the time evolution of case counts and the number of deaths, as the NETEterm·mth between these indicators was statistically significant ( P < 0.01 ) in about 80 % of the provinces, at 2 weeks lag. On the other hand, we observed a statistically significant information transfer from mobility time series to epidemiological ones in a much smaller fraction of provinces. Short-range movementsterm·key NETEterm·mth was significant in less than 20 % of provinces when considered as a predictor of both cases and deaths. Mid-range movement time series and contact ratesterm·key were significant in at most 27 and 40% of provinces. This means that in most provinces, mobility traces did not provide any additional information to predict future COVID-19term·key cases or deaths, at any lag between 2 and 8 weeks. Measures of contact rateterm·dom extracted from colocation mapsterm·key were more suitable than movement dataterm·dom to capture behavioral patterns relevant to predict COVID-19term·key spread.

Percentage of statistically significant NETEterm·mth values across provinces in all the countries studied.

By focusing only on those provinces where we could identify a significant information flow between mobility traces and COVID-19term·key indicators, we observe that the averaged relative explanation added by mobility data with respect to the epidemiological data ranges between 4 and 6%, which is about half of the averaged relative explanation added by past knowledge of cases to the prediction of future deaths (see Table 2 and Figs. S4–S11 in the online supplementary material ).

NETEterm·mth results across provinces in all the countries studied.

As a sensitivity analysis, we computed the NETEterm·mth on a shorter time window, between September 2020 and January 2021, to exclude the confounding effect of introducing nationwide vaccination programs. Since in those months, all countries adopted mobility restrictions to mitigate the fall COVID-19term·key wave, we expect a stronger relationship between mobility and COVID-19term·key cases. Indeed, during this time frame, the information flow between movement time series and COVID-19term·key cases was consistently higher than in the full study period (see Fig. S12 in the online supplementary material ). This result indicates that, provided with time series of adequate size, the NETEterm·mth can effectively capture the time-varying relationship between human mobility time trends and COVID-19term·key dynamics. As an additional sensitivity analysis, we quantified the information transfer between mobility indicators and the time-varying reproductive number, R t , which is a key measure of transmissibility during an outbreak ( 50 ). We considered the case of Italy, where estimates of R t in 20 regions (NUTS2 level) were published by the National Institute of Public Health, every week. Results (see online supplementary Fig. S17 ) confirm that short-range mobility provides more information than mid-range mobility in predicting COVID-19term·key transmissibility, as we could measure by considering raw case counts, with similar values of NETEterm·mth ( online supplementary Table S5 ). However, the relationship between short-range mobility and R t is statistically significant in more regions than we find with cases or deaths.

To gain a deeper insight into the effects of using different data sources and considering different spatial resolutions for our analysis, we performed a range of additional experiments, reported in the online supplementary material . First, we computed, on a daily resolution, the NETEterm·mth between the residential time series, M r , estimated by Google, and the epidemiological variables (see online supplementary Figs. S2 and S3 ). In this case, a very different measure of mobility, the change in residential time, showed very similar values of information transfer to cases and deaths, as that of the short-range mobility, confirming the robustness of our results against changes in data sources. Next, we measured the NETEterm·mth between mobility indicators ( M s and M ) aggregated at a lower spatial resolution, i.e. NUTS2 level, and COVID-19term·key cases and deaths in Austria, France, and Spain. The results are summarized in online supplementary Figs. S20–S22 . At the regional level, we observe that the relative predictive value of short-range mobility and mid-range mobility is confirmed, with better performances of M in Austria and France, and of M s in Spain. The regional aggregation, in general, leads to a larger number of statistically significant values in all countries. As an example, the values of N M → C in the 12 French regions are always statistically significant, although the average NETEterm·mth is not much higher than we observed at the province level. These results suggest the existence of a tradeoff between the information transfer of mobility indicators and the spatial scale at which time series are analyzed. In general, the higher the spatial resolution, the higher the noise, which may hide the existing relationship between mobility and disease dynamics.

To conclude, we considered as a source an additional metric of mobility provided by Meta at the NUTS2 scale on a daily basis, the change in movement, which is a relative measure of global change in population mobility (see Materials and methods section). The results of this analysis ( online supplementary Figs. S18 and S19 ) show that such aggregate metric has predictive power in the same countries where mid-range mobility was evidenced as the best mobility metric but with smaller values of NETEterm·mth.

Identifying the determinants of mobility data predictive power for COVID-19term·key In this section, we try to identify exogenous or endogenous factors that could explain the limited predictive power of mobility traces we observed in several provinces. We focus on Meta’s data because this is the only source of data for which we have additional information related to the coverage. It also provides a way to compute the most predictive metrics among those tested here, i.e. the contact rateterm·dom.

Maps of Fig. 5 highlight the spatial heterogeneity of N X → D values observed within the same country, Spain, for a given time lag and different source time series (see online supplementary Figs. S13–S15 for the maps of Austria, France, and Italy). As previously mentioned, N C → D displays higher and significant values in most of the country (Fig. 5 a), with very few exceptions, while statistically significant values of N M s → D are found only in 16 provinces out of 42 (Fig. 5 c).

Spatial variations of NETEterm·mth. Maps of NETEterm·mth values computed for different source time series and weekly COVID-19term·key deaths, in the provinces of Spain: a) source is COVID-19term·key cases at lag l = 2 weeks, b) source is contact rateterm·dom at lag l = 7 weeks, c) source is short-range movementterm·key at lag l = 7 weeks. d) source is mid-range movement at lag l = 7 weeks. Dark gray indicates provinces with nonsignificant values of NETEterm·mth ( P > 0.01 ). Provinces in white are excluded from our sample.

To better understand the observed heterogeneity in NETEterm·mth, and identify those features that can predict the likelihood to observe a statistically significant information transfer from mobility to COVID-19term·key death counts, we resorted to a classification model. Namely, we used a random forestterm·key classifier to predict when the value N X → D is more likely to be statistically significant, using short-range movementterm·key and contact rateterm·dom as source time series. We focused on these two metrics as they are quantities measured at the same spatial scale. Moreover, short-range movementsterm·key represent on average 90 % or more of all movements within a province (see online supplementary Table S1 ). As input features to the model, we considered a set of attributes of the provinces in each country. In particular, we investigated the effects of population size, province area in square kilometers, the density of Facebook users, the number of total cumulative deaths, the ratio between the number of commuters traveling from or to the province, and those who live and work there, as reported by the census (commuting flowterm·dom), and the coverage consistencyterm·mth, that is the correlation over time between the number of Facebook users sharing their location and the number of Facebook users taken into account to compute the colocation mapsterm·key.

The results summarized in Table 3 show that the model achieves a good overall performance in terms of precision and recall, as indicated by f1-scores generally higher than 0.6. In particular, of all provinces that are classified by the model as characterized by a statistically significant value of NETEterm·mth, 90% or more display a significant transfer of information, as shown by precision values. On the other hand, the model’s recall is close to 0.95 when it comes to identifying provinces characterized by a not statistically significant NETEterm·mth, therefore the model correctly identifies 95% of those provinces where there is no actual transfer of information between mobility and deaths.

Classification performance metrics.

To explore the importance of province features in our classification model, we examined the SHAPterm·mth (SHapley Additive exPlanationsterm·mth) values associated with each, as shown in Fig. 6 . SHAPterm·mth is a method based on a game theoretic approach to explaining the output of classification models ( 51 ). As expected, the choice of the time lag to compute the NETEterm·mth is crucial in determining the presence of a significant information transfer between mobility metrics and epidemiological indicators. Indeed, lag is ranked as the most and second most important feature explaining the classification, for contact rateterm·dom and short-range movementterm·key, respectively. Commuting flowterm·dom is the most important predictor of the statistical significance of NETEterm·mth between short-range movementsterm·key and deaths: when the number of commuters leaving or entering a province represents an important fraction with respect to those who remain within the province, the relationship between short-range mobility and COVID-19term·key dynamics gets weaker. However, the same feature has only a marginal impact on the NETEterm·mth between contact ratesterm·key and deaths, which suggests contact rateterm·dom should be preferred over short-range movementsterm·key to predict epidemic outcomes when a province is characterized by large population inflows/outflows. Province area and population size have also a significant impact on the information transfer between short-range movementterm·key and COVID-19term·key deaths. Indeed, a larger area and population size correspond to a higher likelihood of NETEterm·mth significance for short-range movementsterm·key. This effect may partly explain why we observed NETEterm·mth values that were statistically significant only in a few provinces of Austria, where spatial units were particularly small. When looking at the information flow between contact ratesterm·key and time series of deaths, the total cumulative deaths represent an important explanatory variable for the classification model. Besides the analysis presented in Fig. 6 suggests that the coverage consistencyterm·mth needs to be sufficiently high in order to get a statistically significant transfer entropyterm·mth from contact rateterm·dom to deaths. In France, where in most provinces the coverage consistencyterm·mth is low and the commuting inflow and outflow are higher than in other countries (see online supplementary Table S2 ), mid-range movements seem to provide a better alternative to contact ratesterm·key and short-range movementsterm·key to partially explain time trends of COVID-19term·key cases and deaths (see Fig. S16 of the online supplementary material ). From our analysis, we thus conclude that NETEterm·mth values computed using contact ratesterm·key as source time series are less sensitive to the province’s geographic or demographic features, rather than to the noise of the target time series. Given good coverage, and consistency over time, contact ratesterm·key thus represent a better epidemiological predictor of future COVID-19term·key deaths than short-range movementsterm·key.

SHAPterm·mth (SHapley Additive exPlanationsterm·mth) plots of feature importance to predict the statistical significance of the NETEterm·mth for all selected provinces. Color represents the feature value (blue is low and red is high). a) Describes the results for N M s → D , b) for N CR → D . The SHAPterm·mth value, on the horizontal axis, indicates the feature importance on the model output, with larger values corresponding to higher relevance. Each dot represents a single observation. Features are ranked by importance.

Discussion In this work, we have introduced a general framework based on transfer entropyterm·mth to quantify the amount of information that is transferred from mobile phone-derived mobility metrics to epidemiological time series. Given the important role that mobility indicators have played in the COVID-19term·key pandemic, we tested our approach on mobility and epidemic time series collected in four European countries, between 2020 and 2021, at a subnational scale. We found that, in general, the relative explanation added by mobility time series to predict future epidemic trends, whether new cases or deaths, was relatively small, ranging between 4 and 6% on average, and not statistically significant in the large majority of the provinces we considered, for any mobility metric. As a comparison, these values were about half of the relative explanation added by past knowledge of COVID-19term·key incidence to predict future deaths. Our method allowed us to directly compare the relative explanation added by different mobile phone-derived metrics of mobility from different data providers: change in residential mobility, short- and mid-range mobility, and contact ratesterm·key. We generally found a higher information transfer from contact ratesterm·key than movement, in line with previous studies ( 49 ), however, we also observed significant heterogeneities within the same country and between countries. We identified spatial features that may explain such heterogeneities. In provinces characterized by large populations, good coverage consistencyterm·mth over time, and small commuting in- and outflows, short-range movementsterm·key can represent a useful metric to predict disease dynamics. Where commuting flowsterm·dom are large, such as in France, and Austria, mid-range movements, which represent less than 10% of the total movements, provided a better alternative to short-range ones. We also observed that the statistical significance of the information transfer depends on the spatial resolution considered. Aggregating mobility and epidemiological indicators at less granular spatial scales can help identify a clearer statistical signal for some mobility metrics. Our results suggest the choice of the best mobility metric to inform epidemic predictions can depend on a number of different factors, even when using one single data provider. Moreover, our findings show that cell phone mobility metrics do not always capture epidemiologically relevant behaviors and alternative data sources could be more effective for this aim, as, for instance, the collection of survey data ( 52 ).

There is an emerging common understanding that mobility indicators measured from mobile phone dataterm·key present significant gaps and do not provide a consistent picture of mobility across countries, and data providers ( 53 , 54 ). Previous studies have also highlighted the fact that coupling between mobility indicators and COVID-19term·key epidemiology is often weak, and it changes over time ( 29 ). The approach we introduced here addresses the above challenges by providing a general framework to evaluate the quality of metrics derived from passively collected mobility traces as a predictor of epidemic outcomes. Our framework has the advantage of being model-free, meaning that it does not depend on modeling assumptions regarding the expected relationship between mobility and epidemic dynamics, nor it requires any parametrization. The NETEterm·mth we adopted is a general method that can be applied beyond COVID-19term·key. It allows us to rigorously compare different mobility indicators, across epidemiological settings, by measuring the relative information added by mobility time series to the prediction of future disease incidence. To this end, we release the code to reproduce our analysis between any two source and target time series (see Data availability section). Researchers can use this tool in any epidemiological context to gauge the added value of a specific mobile phone-derived behavioral measure for epidemic intelligence.

Our study comes with a number of limitations and opens new directions for future work. We only focused on European countries for which epidemiological data were shared with sufficient spatial and temporal granularity.

Such data are not free from biases. It is well known that time series of cases and deaths typically suffer from underreporting and they do not capture the true extent of the COVID-19term·key burden, while serosurvey studies may provide a more accurate description of the epidemic spread ( 55 ). However, COVID-19term·key cases and deaths represented the main target of forecasting modeling efforts during the pandemic ( 15 ) and they are key indicators for rapid public health response during epidemic outbreaks, while serosurveys, although more accurate, become available only at later stages of the epidemic.

On the mobility side, we considered data providers that, while being present in several countries and extensively studied in the literature ( 56–60 ), might overlook an important part of the population. Overall, it will be important to assess our findings on mobility data from other providers, and, most importantly, in countries of the non-Western world. Finally, it is important to note that transfer entropyterm·mth measurements become more accurate as the length of the source and target time series increases ( 45 ). We worked with a relatively short time series, addressing the bias due to the small sample by adopting the effective transfer entropyterm·mth. However, we could not systematically investigate how the information transfer changed over time, performing our analysis over different time windows and comparing them. Future work could benefit from longer epidemic time series, over several years, to identify temporal changes in the information flow between human movements and COVID-19term·key dynamics.

Measures of human mobility inferred from mobile phone dataterm·key have been a critical ingredient to inform the public health response during the COVID-19term·key pandemic ( 61 ) and they will be an important asset in the fight against future pandemics. At the same time, their widespread use raises some relevant ethical concerns due to re-identification risks ( 62 ), therefore, it is fundamental to assess the added value of using cell phone mobility data in a given epidemic scenario and whether the benefits outweigh the risks. Our work provides a practical guide to identifying when and where mobile phone mobility metrics truly capture behavioral patterns that are relevant to predict disease dynamics.

Materials and methods Epidemiological indicators We collected epidemiological time series in the four countries under study from two data sources. Daily reported cumulative COVID-19term·key cases were collected from the COVID-19term·key Data Hub ( 63 ), an open source aggregator of up-to-date COVID-19term·key statistics, at the NUTS3 level in Austria, France, Italy, and Spain. Daily reported cumulative deaths in Austria, France, and Spain were also collected from the COVID-19term·key Data Hub. For Italy, death statistics were only available on a weekly time scale from the public platform CovidStat ( https://covid19term·key.infn.it/iss/ ). For Italy, we collected the weekly reproduction number R t at a regional level from the National Institute of Public Health. Data are publicly available from: https://github.com/Biuni/rt-italy . The transfer entropyterm·mth analysis on R t is performed over the temporal intersection between the R t dataset time-range and the full study period. For the analysis, we generated daily incidence time series from cumulative data by computing day-to-day differences. Then, we aggregated the daily time series of deaths and cases into weekly ones, to perform the transfer entropyterm·mth analysis on a weekly scale. For additional transfer entropyterm·mth analyses, we further spatially aggregated the deaths and cases time series from the NUTS3 to the NUTS2 level.

Mobility-derived indicators In our study, we computed daily and weekly movement and contact ratesterm·key from data provided by Meta through its Data for Good program ( 42 ) and by Google through its Community Mobility reports ( 40 ).

In the online supplementary material , we provide an extensive description of the raw data sources and the processing pipeline we adopted to generate our input time series for the transfer entropyterm·mth analysis. More briefly, we collected the following datasets that were publicly released by Meta since the beginning of the COVID-19term·key pandemic, in Austria, France, Italy, and Spain: Movement range maps . It reports the number of users who moved between any two 16-level Bing tiles, with an 8-h frequency.

Colocation mapsterm·key. It estimates the probability, P , that, given any two administrative regions, p 1 and p 2 , a randomly chosen user from p 1 and a randomly chosen user from p 2 are simultaneously located in the same place during a randomly chosen minute in a given week ( 39 ). The dataset also reports the number of users in p 1 and p 2 .

Relative change in movement. It reports the daily average number of 16-level Bing tiles visited by the users of a given region with respect to a baseline that predates the introduction of social distancing measures.

We also collected the following data from Google: Mobility trends for place of residence . It estimates the daily relative change in the time spent at places of residence, with respect to a baseline, by all users in a given administrative region.

We derive from the above data sources, four different mobility time series. The short-range movementterm·key rate is defined as that is the proportion of users who moved within the same 16-level Bing tile in a given province, p , in a given week w . The mid-range movement rate is defined as representing the proportion of users who moved between different tiles in a given province, p , in a given week w . The contact rateterm·dom is defined as where P ^ denotes the colocation probability corrected by a factor that takes into account the overestimation of colocation probabilities due to the heterogeneous distribution of users across provinces and the presence of a significant fraction of static users in some periods of mobility restrictions ( 58 ) (see the online supplementary material for additional details).

Finally, the residential mobility, M r ( t ) , is the relative change in time spent at home, as provided by Google. Although we use the same symbol M , as for mobility metrics derived by Meta, it is important to note that M r ( t ) is a measure of duration, thus very different in nature from M ( t ) and M s ( t ) .

Province sample selection The population of Facebook users who contribute to the generation of the movement and colocation time series varies across countries, and it changes over time. Moreover, the metrics of movement (short- and mid-range) and colocation, are computed from different users’ samples of different sizes: N p , w ( pop ) and N p , w ( coloc ) , respectively.

In our analysis, to limit bias that may be caused by the little representativeness of the underlying sample of users, we selected NUTS3 regions in the 4 countries, according to the following criteria. First, we considered only regions where the sample N p , w ( pop ) represented at least 3 % of the census population to guarantee we had at least 500 users in each province. Furthermore, we considered only those regions where the two sample sizes N p , w ( pop ) and N p , w ( coloc ) were always positively correlated over time, during the whole study period. We denote the Pearson’s correlation of weekly values of N p , w ( pop ) and N p , w ( coloc ) as coverage consistencyterm·mth .

After the selection, our analysis includes 47 provinces in Austria, 51 provinces in France, 93 provinces in Italy, and 42 provinces in Spain, for a total of 233 spatial units.

Normalized effective transfer entropyterm·mth Given two discrete temporal signals represented as time series X and Y , the TE ( 38 ) is a measure of the amount of information delivered from X to Y , defined as where X ( l ) , Y ( l ) are, respectively, the l -lagged time series of X and Y and TE X Y is formulated as a difference between two conditional entropy terms, where conditional entropy is expressed as H ( a | b ) = H ( a , b ) − H ( b ) , and H ( ⋅ ) is the Shannon Entropy. Given a discrete time series S , its observations can be expressed as the sample { s i ; i = 1 , … , n } , and we obtain the discrete probability distribution p ( s j ) . We compute the Shannon Entropy as: H ( S ) = ∑ j p ( s j ) ⋅ log 2 ( p ( s j ) ) . Thus, TE X Y can be expressed as The time series that we consider in our experiments are continuous, therefore they need to be discretized before computing TE X Y . We employ the kernel density estimationterm·mth (KDEterm·mth) for TE estimation. KDEterm·mth method evaluates the entropy terms of Eq. 5 from the discretized density estimated from each of the four features sets: { Y ( l ) , ( Y , Y ( l ) ) , ( Y , Y ( l ) , X ( l ) ) , ( Y ( l ) , X ( l ) ) } . KDEterm·mth employs a Gaussian kernel for density estimation. Performing tests on synthetic datasets of different sizes, we checked this was the method the most adapted to small samples. For the selection of the kernel’s bandwidth, we use the Scott method ( 64 ). The continuous density is then discretized with a grid obtained by an equal-width discretization of each feature’s density domain. We select 20 as the number of bins for each feature’s domain discretization. The discretized density is computed with the integral of the continuous probability density functions over each grid cell. Concerning the implementation, for TE estimation we use the PyCausality Python package ( https://github.com/ZacKeskin/PyCausality ).

Effective transfer entropyterm·mth We introduce the effective transfer entropyterm·mth (ETEterm·mth) as a correction to TE for small sample time series, as originally proposed by Marschinski and Kantz ( 45 ): where the correction term is obtained by performing N s iterations of Y shuffling, obtaining Y j ^ and computing the average of { TE X Y j ^ ; j = 1 , … , N s } . In our experiments, we performed 500 shuffling iterations.

Normalized transfer entropyterm·mth We would like to employ TE in order to compare a set of input signals { X j ; j = 1 , … , N } in terms of their transfer entropyterm·mth TE X j Y towards a specific output Y . From Eq. 4 , we have that TE X j Y is evaluated as a difference of conditional entropy where the first term H ( Y | Y ( l ) ) depends only on target Y . In order to ensure comparability over the set { TE X j Y ; j = 1 , … , N } , we reformulate the difference as a relative difference dividing by H ( Y | Y ( l ) ) . Thus, the set of inputs are compared according to { TE X j Y / H ( Y | Y ( l ) ) ; j = 1 , … , N } and we refer to TE X Y / H ( Y | Y ( l ) ) as NTE.

Normalized effective transfer entropyterm·mth By combining the ETEterm·mth and the NTE, we can finally introduce the NETEterm·mth, which is obtained by dividing the ETEterm·mth by the first conditional entropy term H ( Y | Y ( l ) ) as in Perilla and Woolf ( 65 ): In this way, the NETEterm·mth accounts both for bias in small sample time series and ensures comparability between different input sources { X j } in terms of information transfer to different targets. Besides, it enables estimating the percentage of explanation value added with respect to only knowing the past of the time series used as a target.

Classification model The introduction of the ETEterm·mth allows associating a P -value, a metric of statistical significance, to each NETEterm·mth value computed between any pair of time series.

In our study, we investigated a number of explanatory features to understand better why in some provinces the NETEterm·mth could not identify a significant transfer of information between mobility time series and epidemiological indicators. Specifically, we trained a Random Forestterm·key classification model to predict the significance of N X → Y at the threshold of P < 0.01 , in each province under study. The random forestterm·key was performed with 100 decision tree classifiers on various sub-samples of the dataset and used averaging to improve the predictive accuracy and control for over-fitting. The function to measure the quality of a split was the Gini impurity. Before applying the random forestterm·key, the data were split between training and test sets ( 30 % ). To compensate for the imbalance of the datasets, we applied a synthetic minority oversampling technique ( 66 ) on the test set.

As input to the classification model we used a set of features that characterize each province: population size (as reported by the latest available census); area (in km 2 ); density of Facebook users (measured as N p , w divided by area); total cumulative number of reported COVID-19term·key deaths during the study period; commuting flowterm·dom; coverage consistencyterm·mth.

The commuting flowterm·dom is defined as the ratio between the total number of daily commuters who travel from or to a province and the total number of commuters who work and live in that province. Commuting data were collected from the latest available census statistics in each country. The coverage consistencyterm·mth is the correlation over time between the users’ populations N p , w ( pop ) and N p , w ( coloc ) . Before including the above features into the model, we checked for multicollinearity using the variation inflation factor.

To quantify the importance of different features in our classification model, we used their SHAPterm·mth values ( 51 ). SHAPterm·mth is a method to explain model predictions based on Shapley values from game theory. In particular, we use TreeSHAP ( 67 ), an algorithm to compute SHAPterm·mth values for tree ensemble models, such as the random forestterm·key classifier of our study.

Supplementary Material Click here for additional data file.

Acknowledgments The authors gratefully acknowledge Alex Pompe for his help to understand the details of mobility data from Meta.

Contributor Information Federico DelussupersName, ISI Foundation, via Chisola 5, 10126 Torino, Italy; Department of Applied Mathematics and Computer Science, DTU, Richard Petersens Plads, DK-2800 Copenhagen, Denmark.

Michele TizzonipersName, ISI Foundation, via Chisola 5, 10126 Torino, Italy; Department of Sociology and Social Research, University of Trento, via Verdi 26, I-38122 Trento, Italy.

Laetitia GauvinpersName, ISI Foundation, via Chisola 5, 10126 Torino, Italy; UMR 215 PRODIG, Institute for Research on Sustainable Development - IRD, 5 cours des Humanités, F-93 322 Aubervilliers Cedex, France.

Supplementary material Supplementary material is available at PNAS Nexus online.

Funding F.D. gratefully acknowledges support from the CRT Lagrange Fellowships in Data Science for Social Impact of the ISI Foundation, where this work was conducted. M.T. and L.G. acknowledge the Lagrange Project of the ISI Foundation funded by CRT Foundation. The funders had no role in the study design, decision to publish, or preparation of the manuscript.

Author contributions F.D. collected data, conducted experiments, interpreted the results, made figures, and contributed to the writing of the paper. M.T. and L.G. conceived and designed the study, conducted the statistical analysis, interpreted the results, made figures, and wrote the paper. All authors read and approved the final version of the manuscript.

Preprints A preprint of this article is published at: https://doi.org/10.48550/arXiv.2301.03960 .

Data availability The data and code to reproduce our analysis are available at: https://doi.org/10.5281/zenodo.7700401 References Associated Data Supplementary Materials Click here for additional data file.

Data Availability Statement The data and code to reproduce our analysis are available at: https://doi.org/10.5281/zenodo.7700401