3.1.1.Overall Traffic Growth

The global pandemic has significantly accelerated the growth of data traffic worldwide.Based on the measurement data of one ISP, three IXPs, a metropolitan educational network,and a mobile operator, it was observed at the beginning of the workshop[Feldmann2020] that, overall, thenetwork was able to handle the situation well despite a significant and sudden increase in the trafficgrowth rate in March and April. That is, after the lockdown was implemented in March, a traffic increaseof 15-20% was observed at the ISP as well as at the three IXPs. This traffic growth, which would typically occur over a year, took place over a few weeks -- a substantial increase. At DE-CIX Frankfurt, the world's largest Internet Exchange Point in terms of data throughput,the year 2020 saw the largest increase in peak traffic within a single year since the IXP wasfounded in 1995. Additionally, mobile traffic has slightly receded. In access networks, the growth rateof upstream traffic also exceeded the growth in downstream traffic, reflecting increased adoptionand use of videoconferencing and other remote work and school applications.¶

Most traffic increases happened outside of pre-pandemic peak hours. Before the first COVID-19 lockdowns, the main timeof use was in the evening hours during the week, whereas, since March, it has been spread more equallyacross the day. That is, the increase in usage has mainly occurred outside the previous peak usagetimes (e.g., during the day while working from home). This means that, for the first time, networkutilization on weekdays resembled that on weekends. The effects of the increased traffic volume couldeasily be absorbed, either by using existing reserve capacity or by quickly switching additional bandwidth.This is one reason why the Internet was able to cope well with the pandemic during the first lockdown period.¶

Some of the lockdowns were lifted or relaxed around May 2020. As people were allowed to resume someof their daily activities outside of their home again, as expected, there was a decrease in the trafficobserved at the IXPs and the ISP; instead, mobile traffic began to grow again.¶

3.1.2.Changes in Application Use

The composition of data traffic has changed since the beginning of the pandemic: the use ofvideoconferencing services and virtual private networks (VPNs) for access to company resourcesfrom the home environment has risen sharply. In ISP and IXP networks, it was observed[Feldmann2020]that traffic associated with web conferencing, video, and gaming increased significantly in March 2020as a result of the increasing user demand for solutions like Zoom or Microsoft Teams.For example, the relative traffic share of many "essential" applications like VPN and conferencing toolsincreased by more than 200%.¶

Also, as people spent more hours at home, they tended to watch videos or play games, thus increasing entertainment traffic demands.At the same time, the traffic share for other traffic classes decreasedsubstantially, e.g., traffic related to education, social media, and, for some periods, content delivery networks (CDNs).In April and June, web conferencing traffic was still high compared to the pre-pandemic scenario,while a slight decrease in CDN and social media traffic was observed. During these months, many peoplewere still working from home, but restrictions had been lifted or relaxed, which likely led to an increasein in-person social activities and a decrease in online social activities.¶

3.1.3.Mobile Networks and Mobility

Mobile network data usage appeared to decline following the imposition of localized lockdown measures as these reduced typical levels of mobility and roaming.¶

[Lutu2020] measured the cellular network of O2 UK to evaluate how the changes in people's mobility impacted traffic patterns. By analyzing cellular network signaling information regarding users' device mobility activity, they observed a decrease of 50% in mobility (according to different mobility metrics) in the UK during the lockdown period. As they found no correlation between this reduction in mobility and the number of confirmed COVID-19 cases, only the enforced government order was effective in significantly reducing mobility, and this reduction was more significant in densely populated urban areas than in rural areas. For London specifically, it could be observed from the mobile network data that approximately 10% of residents temporarily relocated during the lockdown.¶

These mobility changes had immediate implications in the traffic patterns of the cellular network. The downlink data traffic volume aggregated for all bearers (including conversational voice) decreased for the entire UK by up to 25% during the lockdown period. This correlates with the reduction in mobility that was observed countrywide, which likely resulted in people relying more on residential broadband Internet access to run download-intensive applications such as video streaming. The observed decrease in the radio cell load, with a reduction of approximately 15% across the UK after the stay-at-home order was enacted, further corroborates the drop in cellular connectivity usage.¶

The total uplink data traffic volume, on the other hand, experienced little change (between -7% and +1.5%) during lockdown. This was mainly due to the increase of 4G voice traffic (i.e., Voice over LTE (VoLTE)) across the UK that peaked at 150% after the lockdown compared to the national median value before the pandemic, thus compensating for the decrease in data traffic in the uplink.¶

Finally, it was also observed that mobility changes have a different impact on network usage in geodemographic area clusters. In densely populated urban areas, a significantly higher decrease of mobile network usage (i.e., downlink and uplink traffic volume, radio load, and active users) was observed compared to rural areas. In the case of London, this was likely due to the geodemographics of the central districts, which include many seasonal residents (e.g., tourists) and business and commercial areas.¶

3.1.4.A Deeper Look at Interconnections

Traffic at points of network interconnection noticeably increased, but most operators reacted quickly by rapidly adding additional capacity[Feldmann2020]. The amount of increase varied, with some networks that hosted popular applications such as videoconferencing experiencing traffic growth of several hundred to several thousand percent. At the IXP level, it was observed that port utilization increased. This phenomenon is mostly explained by higher traffic demand from residential users.¶

Measurements of interconnection links at major US ISPs by the Center for Applied Internet Data Analysis (CAIDA) and the Massachusetts Institute of Technology (MIT) found some evidence of diurnal congestion around the March 2020 time frame[Clark2020], but most of this congestion disappeared in a few weeks, which suggests that operators indeed took steps to add capacity or otherwise mitigate the congestion.¶

3.1.5.Cloud Platforms

Cloud infrastructure played a key role in supporting bandwidth-intensive videoconferencing and remote learning tools to practice social distancing during the COVID-19 pandemic. Network congestion between cloud platforms and access networks could impact the quality of experience of these cloud-based applications. CAIDA leveraged web-based speed test servers to take download and upload throughput measurements from virtual machines in public cloud platforms to various access ISPs in the United States[Mok2020].¶

The key findings included the following:¶

• Persistent congestion events were not widely observed between cloud platforms and these networks, particular for large-scale ISPs, but we could observe large diurnal download throughput variations in peak hours from some locations to the cloud.¶
• There was evidence of persistent congestion in the egress direction to regional ISPs serving suburban areas in the US. Their users could have suffered from poor video streaming or file download performance from the cloud.¶
• The macroscopic analysis over 3 months (June-August 2020) revealed downward trends in download throughput from ISPs and educational networks to certain cloud regions. We believe that increased use of the cloud in the pandemic could be one of the factors that contributed to the decreased performance.¶

3.1.6.Last-Mile Congestion

The last mile is the centerpiece of broadband connectivity, where poor last-mile performance generally translates to poor quality of experience. In a recent Internet Measurement Conference (IMC '20) research paper, Fontugne et al. investigated last-mile latency using traceroute data from Reseaux IP Europeens (RIPE) Atlas probes located in 646 ASes and looked for recurrent performance degradation[Fontugne2020-1]. They found that, in normal times, Atlas probes experience persistent last-mile congestion in only 10% of ASes, but they recorded 55% more congested ASes during the COVID-19 outbreak. This deterioration caused by stay-at-home measures is particularly marked in networks with a very large number of users and in certain parts of the world. They found Japan to be the most impacted country in their study, looking specifically at the Nippon Telegraph and Telephone (NTT) Corporation Open Computer Network (OCN) but noting similar observations for several Japanese networks, including Internet Initiative Japan (IIJ) (AS2497).¶

From mid-2020 onward, however, they observed better performance than before the pandemic. In Japan, this was partly due to the deployments originally planned for accommodating the Tokyo Olympics, and, more generally, it reflects the efforts of network operators to cope with these exceptional circumstances. The pandemic has demonstrated that its adaptive design and proficient community can keep the Internet operational during such unprecedented events. Also, from the numerous research and operational reports recently published, the pandemic is apparently shaping a more resilient Internet; as Nietzsche wrote, "What does not kill me makes me stronger".¶

3.1.7.User Behavior

The type of traffic needed by the users also changed in 2020. Upstreamtraffic increased due the use of videoconferences, remote schooling,and similar applications. The National Cable & Telecommunications Association (NCTA) and Comcast reported that whiledownstream traffic grew 20%, upstream traffic grew by as much as 30-37%[NCTA2020][Comcast2020]. Vodafone reported that upstreamtraffic grew by 100% in some markets[Vodafone2020].¶

Ericsson's ConsumerLab surveyed users regarding their usage and experiencesduring the crisis. Some of the key findings in[ConsumerlabReport2020] were as follows:¶

• 9 in 10 users increased Internet activities, and time spentconnected increased. In addition, 1 in 5 started new onlineactivities; many in the older generation felt that they were helpedby video calling; parents felt that their children's education washelped; and so on.¶
• Network performance was, in general, found satisfactory. 6 in 10 werevery satisfied with fixed broadband, and 3 in 4 felt that mobilebroadband was the same or better compared to before the crisis. Consumers valuedresilience and quality of service as the most important responsibility fornetwork operators.¶
• Smartphone application usage changed, with the fastest growth inapps related to COVID-19 tracking and information, remote working, e-learning, wellness, education, remotehealth consultation, and social shared experienceapplications. The biggest decreases were in traveland booking, ride hailing, location, and parking applications.¶

Some of the behaviors are likely permanent changes[ConsumerlabReport2020]. The adoption of video calls and other newservices by many consumers, such as the older generation, is likelygoing to have a long-lasting effect. Surveys in various organizationspoint to a likely long-term increase in the number of peopleinterested in remote work[WorkplaceAnalytics2020][McKinsey2020].¶

3.2.1.Digital Divide

Measurements from Fastly confirmed that Internet traffic volume inmultiple countries rose rapidly while COVID cases were increasing and lockdown policies were coming into effect. Download speeds also decreasedbut in a much less dramatic fashion than when overall bandwidth usage increased.School closures led to a dramatic increase in traffic volume in many regions,and other public policy announcements triggered large traffic shifts. Thissuggests that governments should coordinate with operators to allowtime for preemptive operational changes in some cases.¶

Measurements from the US showed that download rates correlate with incomelevels. However, download rates in the lowest income zip codes increasedas the pandemic progressed, closing the divide with higher income areas.One possible reason for this in the data is decisions by some ISPs, suchas Comcast and Cox, that increased speeds for users on certain lower-cost plans and incertain areas. This suggests that network capacity was available and thatthe correlation between income and download rates was not necessarily dueto differences in the deployed infrastructure in different regions,although it was noted that certain access link technologies provide moreflexibility than others in this regard.¶

3.2.2.Applications

Web conferencing systems (e.g., Microsoft Teams, Zoom, Webex) sawincredible growth, with overnight traffic increases of 15-20% in responseto public policy changes, such as lockdowns. This required significantand rapid changes in infrastructure provisioning.¶

Major video providers (YouTube, etc.) reduced bandwidth by 25% in someregions. It was suggested that this had a huge impact on the quality ofvideoconferencing systems until networks could scale to handle the fullbit rate, but other operators of some other services saw limited impact.¶

Updates to popular games have a significant impact on network load. Somediscussions were reported between ISPs, CDNs, and the gaming industry onpossibly coordinating various high-bandwidth update events, similar to whatwas done for entertainment/video download speeds. There was an apparentlydifficult interplay between bulk download and interactive real-timeapplications, potentially due to buffer bloat and queuing delays.¶

It was noted that operators have experience with rapid growth of Internettraffic. New applications with exponential growth are not that unusual inthe network, and the traffic spike due to the lockdown was not thatunprecedented for many. Many operators have tools and mechanisms to dealwith this. Ensuring that knowledge is shared is a challenge.¶

Following these observations, traffic prioritization was discussed, starting from Differentiated Services Code Point (DSCP) marking. The question arose as to whethera minimal priority-marking schemewould have helped during the pandemic, e.g., by allowingmarking of less-than-best-effort traffic.That discussion quickly devolved into a more general QoS andobservability discussion and, as such, also touched onthe effects of increased encryption. The group was not, unsurprisingly, able to resolve thedifferent perspectives and interests involved,but the discussion demonstrated that progress was made.¶

3.2.3.Observability

It is clear that there is a contrast in experience. Many operators reportedfew problems in terms of metrics, such as measured download bandwidth, whilevideoconferencing applications experienced significant usability problemsrunning on those networks. The interaction between application providers andnetwork providers worked very smoothly to resolve these issues, supportedby strong personal contacts and relationships. But it seems clear that themetrics used by many operators to understand their network performancedon't fully capture the impact on certain applications, and there is anobservability gap. Do we need more tools to figure out the various impactson user experience?¶

These types of applications use surprising amounts of Forward Error Correction (FEC). Applications hide lots of loss to ensure a good user experience. This makes it harder toobserve problems. The network can be behaving poorly, but the experience can begood enough. Resiliency measures can improve the user experience but hidesevere problems. There may be a missing feedback loop between applicationdevelopers and operators.¶

It's clear that it's difficult for application providers and operators to isolate problems. Is a problem due to the local Wi-Fi, the access network, thecloud network, etc.? Metrics from access points would help, but in general,lack of observability into the network as a whole is a real concern whenit comes to debugging performance issues.¶

Further, it's clear that it can be difficult to route problem reports tothe person who can fix them, especially if the reported information needs to be shared across multiple networks in the Internet.COVID-enhanced cooperation made it easier to debug problems; lines ofcommunication are important.¶

3.2.4.Security

The increased threats and network security impacts arising from COVID-19fall into two areas: (1) the agility of malicious actors to spin up newcampaigns using COVID-19 as a lure, and (2) the increased threat surfacefrom a rapid shift towards working from home.¶

During 2020, there was a shift to home working generally, and in theway in which people used the network. IT departments rolled out new equipmentquickly and used technologies like VPNs for the first time, while others put existing solutions under much greater load. As VPN technology became more widespread and more widely used, it arguably became a more valuable target; one Advanced Persistent Threat group (APT29) was successful in using recently published exploits in a range of VPN software to gain initial footholds[Kirsty2020].¶

Of all scams detected by the United Kingdom National Cyber Security Centre (UK NCSC) that purported to originate from the UK Government, more related to COVID-19 than any other subject. There are other reports of a strong rise in phishing, fraud, and scams related toCOVID[Kirsty2020]. Although the overall levels of cybercrime have not increased from the data seen to date, there wascertainly a shift in activity as both the NCSC and the Department of Homeland Security Cybersecurity and Infrastructure Security Agency (DHS CISA) saw growing use of COVID-19-related themes by malicious cyber actors as a lure. Attackers used COVID-19-related scams and phishing emails to target individuals, small and medium businesses, large organizations, and organizations involved in both national and international COVID-19 responses (healthcare bodies, pharmaceutical companies, academia, and medical research organizations). New targets (for example, organizations involved in COVID-19 vaccine development)were attacked using VPN exploits, highlighting the potential consequences of vulnerable infrastructure.¶

It's unclear how to effectively detect and counter these attacks at scale.Approaches such as using Indicators of Compromise and crowdsourced flagging of suspicious emails were found to be effective in response to COVID-19-related scams[Kirsty2020], andobserving the DNS to detect malicious use is widespread and effective. The useof DNS over HTTPS offers privacy benefits, but current deployment models can bypassthese existing protective DNS measures.¶

It was also noted that when everyone moves to performing their job online,lack of understanding of security becomes a bigger issue.Is it reasonable to expect every user of the Internet to havepassword training? Or is there a fundamental problem with a technicalsolution? Modern advice advocates a layered approach to security defenses, with user education forming just one of those layers.¶

Communication platforms such as Zoom are not new: many people have used themfor years, but as COVID-19 saw an increasing number of organizations and individuals turning to these technologies, they became an attractive target due to increased usage. In turn, there was an increase in malicious cyber actor activity, either through hijacking online meetings that were not secured with passwords or leveraging unpatched software as an attack vector. How can new or existing measures protect users from the attacks levied against the nextvulnerable service?¶

Overall, it may be that there were fewer securitychallenges than expected arising from many people suddenly working from home.However, the agility of attackers, the importance of robust and scalable defense mechanisms, and some existing security problems and challenges may have become evenmore obvious and acute with an increased use of Internet-based services, particularly in a pandemic situation and in times of uncertainty, where users can be more vulnerable to social engineering techniques and attacks.¶

3.2.5.Discussion

There is a concern that we're missing observability for the network as awhole. Each application provider and operator has their own little lens.No one has the big-picture view of the network.¶

How much of a safety margin do we need? Some of the resiliency comes fromus not running the network too close to its limit. This allows traffic toshift and gives headroom for the network to cope. The best-effort natureof the network may help here. Using techniques to run the network closer to itslimits usually improves performance, but highly optimized networks may be less robust.¶

Finally, it was observed that we get what we measure. There may be anargument for operators to perhaps shift their measurement focus away frompure capacity to instead measure Quality of Experience (QoE) or resilience. The Internet is acritical infrastructure, and people are realizing that now. We should usethis as a wake-up call to improve resilience, both in protocol design andoperational practice, not necessarily to optimize for absolute performanceor quality of experience.¶