The physical infrastructure of the Internet comprises the backbone of current and future information and communication systems. These systems provide a plethora of diverse services over the Internet, ranging from entertainment and social media to emergency calls over non-traditional voice communications, Voice over Long-Term Evolution (VoLTE), Voice over New Radio (VoNR), Voice over 5G (Vo5G), and Voice over WiFi (VoWiFi). This fact underlines the importance of round-the-clock Internet availability not only for routine activities, but also for public safety and security.

The Internet is considered tolerant to errors due to its mesh architecture and the humongous number of interconnected networks. However, even though the existence of multiple different communication paths between endpoints offers redundancy and robustness, Internet availability can be affected by two major categories of large-scale events. The first consists of natural disasters, such as hurricanes, floods, earthquakes, and so on [1]. The second is related to man-made actions which, in a large-scale level, can impact Internet stability, such as Distributed Denial of Service (DDoS) attacks [2]. To the best of our knowledge, our study is the first one that shows that armed conflicts could also be included in the man-made actions that affect the stability of the Internet. In both cases, an Internet disruption is perceived as a difficulty to access specific Internet Protocol (IP) ranges due to external factors.

The protocol that forms the cornerstone of Internet routing is Border Gate-way Protocol (BGP) [3], which is used to communicate routing information between Autonomous Systems (ASs). Each AS has external routers to exchange BGP routing information with neighboring ASs, and each such router keeps the relevant paths in a routing table. When a new routing path is available, the router broadcasts it to its neighbours who further advertise this new path to their neighbors. When a catastrophic incident renders an IP prefix unreachable, the existing path is withdrawn with a similar BGP-based broadcast procedure, resulting in the update of the relevant routing tables. And even though it is always possible to have short-term, temporary outages lasting a few minutes, more extended periods of network unreachability can be associated with external factors, such as infrastructure destruction.

Intuitively, military operations target critical infrastructures, such as nuclear plants and communication networks [4,5], with disastrous consequences in their stability and everyday operation, as demonstrated in, at the time of writing, ongoing Russo-Ukrainian war. Besides that, the scenario of parallel cyberattacks, like DDoS, that target the enemy’s infrastructure, is considered a common tactic [6]. The work at hand is the first to our knowledge to examine the impact of military operations in network infrastructures using the Russo-Ukrainian war as a representative use case. More specifically, by analyzing BGP data for a two-month period, we identify the extend and severity of catastrophic incidents from the beginning of Russia’s war against Ukraine. The main contributions of this paper can be summarized as follows: (a) we analyze the effect of military operations in network infrastructures, (b) we geolocalize network outages in the level of cities and countries close to the theatre of operations, and (c) we discuss potential associations behind the observed network outages and publicly known events.

The remainder of the paper is structured as follows. The next section offers an introduction around the BGP protocol and how it is used for tracking routing incidents and abnormalities. Section 3 presents our methodology and collection of results, as well as discusses our observations. Section 4 offers a comparison between our proposal and the prior state-of-the-art. Finally, the last section draws the conclusions and gives pointers for future work.

The Internet backbone is comprised of connections between large strategical ASs. These are the structural networking elements of the Internet. Each AS corresponds to a network or group of IP networks which share the same unified routing policy. This policy is used by an AS to exchange routing information with the corresponding peer networks, which are administered by a different Internet Service Provider (ISP). It is also the basic characteristic that differentiates each AS from the rest. Every AS is assigned a unique number, namely the Autonomous System Number (ASN). Each ASN is used to track a network in a distinctive way. The Internet Assigned Names Authority (IANA) is responsible to manage and provide an ASN to the corresponding AS.

As has already been discussed, BGP is employed to exchange information between ASs. The major piece of information conveyed over BGP are the IP ranges which are controlled by a specific AS. This is done by announcing the range of IP addresses, which are reachable to the closest AS. Each AS spreads the information to the closest neighbours, and this is how the different ASs are informed for the routing options they have. Each time an AS wishes to route information to a different AS, it consults the routing information to deduce which is the shortest path. The basic messages used in BGP are the following: OPEN, UPDATE, NOTIFICATION, and KEEPALIVE. In the rest of this paper, we consider UPDATE messages only; for more information regarding the rest of these messages, the interested reader can refer to [3]. Figure 1 offers an overview of the interconnections between different AS elements.

AS connections

Full size image

2.1BGP UPDATE

The main purpose of BGP UPDATE messages is to spread routing information among peers. Specifically, an UPDATE message can be used either to announce a new route or withdraw an existing one. As the name indicates, the announcement of a route is used to announce a route towards the AS that hosts the network prefix under consideration and let the other peers be aware for alternative communication paths. It contains information such as: (a) the identity of the monitoring router, that is, the AS number and IP address of the BGP router that received the update message, (b) timestamp of the message’s reception, (c) the network prefix under consideration, and (d) the path (sequence of ASNs) that has to be traversed for the prefix’s origin AS to be reached. On the other hand, a withdrawal is used for revoking an existing routing path. A withdrawal contains information such as: (a) the identity of the monitoring router, (b) timestamp of the message’s reception, and (c) the prefix under consideration.

Figure 2 shows two BGP UPDATE messages; an announcement at the top of the figure and a withdrawal at the bottom. The fields of the announcement are: BGP protocol, unix time in seconds, type of update (“A” for announcement), IP address of the announcing router, AS number of the announcing AS, announced prefix, AS path, source of the update (“IGP” for Internal Gateway Protocol or “EGP” for External Gateway Protocol), next hop, local preference, preferred path, grouped routes with similar policies, atomic aggregator, and aggregator. The atomic aggregator is a flag showing that some information was lost when an AS aggregated prefixes. The aggregator field identifies the AS and router that aggregated prefixes received from different peers into a single prefix. The fields of the withdrawal are: BGP protocol, unix time in seconds, type of update (“W” for withdrawal), IP address of the announcing router, AS number of the announcing AS, withdrawn prefix.

Example of BGP Update messages (top: announcement, bottom: withdrawal)

Full size image

As BGP UPDATE messages are received by routers, the reachability information of the network prefixes contained therein changes. With the reception of every new BGP update message for a given prefix, the following priority scheme is followed [3]:

• the reception of a withdrawal obsoletes any previous announcement, and

• an announcement is made obsolete by a newer announcement which contains a shorter path.

It is expected that, after a large scale catastrophic incident, the update messages should contain useful information regarding network outages.

2.2Outages

A network outage can be considered the difficulty to access a network location. This can be experienced as a delay in the network or even as a ferocious Denial of Service (DoS). Usually, the recovery of network paths allows a client to access a service even if the service quality is not considered satisfactory. For example, in [7] an outage is considered a continuous period with Mean Opinion Score (MOS) < 2, something that affects voice quality.

In the context of this paper, we consider as outage an extended period during which a broadcast BGP withdrawal is not followed by an announcement for a given IP prefix; in practice, this means that the prefix is unreachable during this period. In our analysis, we only take into account outages that last more than 5 min. Outages that last less than this time threshold are considered temporary, fortuitous disruptions and are ignored.

3.1Collecting and Analyzing BGP Data

In order to collect and analyze the BGP traffic data, we developed two separate scripts whose source code is made publicly available^Footnote1; the procedure we followed for collecting our results is depicted in the flow diagram in Fig. 3. The first one is a Linux bash script that downloads raw BGP data from the University of Oregon Route views project^Footnote2, for a date range between February and March 2022. Since these files are in binary format, we processed them withbgpdump [8] in order to be converted to ASCII. The second script, developed in Perl, analyzes BGP UPDATE messages. The script processes each UPDATE message from the raw files downloaded before, and calculates the outages for each affected IP prefix. That is, while parsing the UPDATE messages, the script marks the IP prefixes for which a path was withdrawn and keeps this information in a special structure called Patricia Trie. A Patricia Trie is a data structure based on a radix tree with a radix of two, and it is used to quickly perform IP address prefix matching during IP subnet, network or routing table lookups. In the next step, the script checks if a new path for the same prefix was not announced in less than 5 min; essentially, these are considered outages and are stored in a database.

Flow diagram of the procedure we followed for calculating outages

Full size image

For each prefix that is entered in the database, we retrieve its actual geographic location using geolocation databases from [9]. Keep in mind that these databases come in two different flavours: the first one maps an IP prefix to a country, while the other one to a city. In the context of our experiments, both database types were used. It should be noted, however, that country-level geolocalization is more reliable than city-level, as stated in the relevant bibliography [10].

3.2Results and Discussion

As far as it concerns our measurements, we concentrated in two different scenarios. In the first one, we draw a relation between the war activities and the outages in the level of a city. In the second one, we delve into the outages in four major countries during February and March 2022.

Regarding the first case, we attempt to draw a connection between conventional military operations, say, shelling, and the rise in the number of outages. According to [11], the Russian army attacked the city of Kharkiv on the 24th of February. Figure 4a shows the number of outages in that city for the second half of February. It is easily observable that during this period the number of outages remained below 500, except the 24th of the same month. Based on the measurement results we obtained, the outages on this day reached almost 9,000, i.e., an increase exceeding 1700%. Moreover, already from the 21st of February, the observed number of outages was slightly higher than the previous days, something that indicates that tactical cyberattacks preceded the actual assault. Additionally, the number of outages remained considerably higher after the first day of the invasion and this fact can be mostly attributed to physical destruction of networking infrastructure.

Outages in Kharkiv (Ukraine)

Full size image

Focusing more on the first day of the invasion, that is the 24th of February, and breaking down the outages in the level of hour, we found out that two major peaks appeared between 2–3 AM, and 4–5 PM. The maximum values in these cases were almost 650. Overall, a very high number of outages was observed throughout the day. These results are depicted in Fig. 4b.

In the second case, we compared the outages in four important countries, the two involved in the war, that is Russia and Ukraine, and two neighboring countries, one supporting Russia, namely Belarus, and one supporting Ukraine, namely Poland. Tables 1 and 2 offer an analytical view of the results we obtained for these countries for the entire February and March respectively, whereas Fig. 5 depicts the outages observed in the second half of February 2022 and the whole of March 2022.

In Ukraine, it is clear that the number of outages steeply rose on the day of the invasion (>10,000) compared to the previous days (<2,000) and it remained at the same high levels (between 4,000 and 40,000, most of the days above 10,000) for the rest of February and March. The big difference in the median value for February (739) and March (12,403) further supports this key observation.

Outages in four major countries

Full size image

In Russia, on the other hand, the peak in February 2022 was observed some days later, on the 27th with 61,490 outages; later, it was announced that this day a DoS campaign was launched against Russia from compromised Information Technology (IT) infrastructure [12]. Then, a similar trend was observed: the number of outages was between 600 and 8,000 until the 27th, and then rose to between 6,000 and 40,000 in the majority of March. Indeed in March, Russian companies, government entities and state-owned companies have confronted an increased number of DDoS attacks [13,14]. The median values for February and March 2022 were 2,582 and 13,467, respectively. It is worth mentioning that Russia presents more outages than Ukraine, especially on the 8th of March. We assume that this result is perceived due to the fact that on this day two major ISPs stopped offering their services in Russia [15]. Obviously, this fact resulted in a major Internet outage.

The differences in the median values for the other two countries (from 9 to 279 for Belarus and from 425 to 1,895 for Poland) clearly suggest that countries that are not directly involved in the war operations are not immune to network outages either.

As far as it concerns the median values per 15 d for February, the results indicate a triple increase in the number of outages for Russia. That is, from 2,492 outages perceived in the first half, the second reached 7,522. In the case of Ukraine the values were almost quadrupled. From 698 outages in the first 15 d, it reached 3,012 at the end of the month. Finally, with respect to Belarus and Poland, the results designate a rise in the number of outages, but the difference remains low, in the same order of magnitude.

A survey of BGP anomaly detection methods is presented in [16]. The authors also include an evaluation methodology for comparing these methods and propose new detection algorithms. The first method is Nemecis [17], which checks for inconsistencies between policy information residing in Internet Routing Reg-istries (IRRs) and actual routing. The next two methods, Prefix Hijack Alert System (PHAS) [18] and Pretty Good BGP (PGBGP) [19], rely on BGP data. PHAS, as the name suggests, is a real-time notification system that alerts the owner of a prefix when there is a change to the BGP origin. PGBGP is a system that delays the propagation of new routes, and uses known alternatives instead, so that human operators have the time to examine routes that seem suspicious. Next, the authors of the survey propose their own algorithm that is using both Internet Routing Registry (IRR) and BGP trace data. The aforementioned methods look into detecting anomalies and attacks on BGP routes, whereas our aim is to detect anomalies caused by disruptive events in a large scale.

Closer to our work is [20], which proposes a BGP anomaly detection system. It is focused on detecting different types of abnormal events like worms, power outages, and submarine cable cuts, using data mining algorithms; similar works are [21,22]. Li et al. [21] propose a detection framework that uses rules of abnormal events and data mining techniques to detect new events. The authors present two use cases, one of a blackout event and one of a worm, where the rules of previous events were used and successfully detected the new events. The second one, [22], is about understanding what happens to BGP during large-scale power outages, and investigates the USA/Canada incident [23] as a use case. All these works are similar to what we aim to achieve; one of our requirements that is not met in these works, however, is the geolocation of network outages.

The work in [1] examines the impact of catastrophic incidents in Internet stability. By monitoring BGP data for specific events, such as earthquakes, the authors conclude that the routing table size is limited during the events. Additionally, they analyze the BGP UPDATE messages with respect to withdrawals and announcements. Based on these findings, they come up with the result that natural-based emergency incidents affect Internet stability.

In the same direction, [24,25] examine the potential of measuring natural disasters by exploiting Internet data. The authors rely on BGP data to identify the routing paths that are affected by earthquakes. According to their results, theI-seismograph tool can identify which ASs are related to abnormalities in the routing paths. The basic idea is to define a “normal” state for the Internet and then at any time compare it with the current state in order to conclude whether an “abnormal” state is observed or not. These works consider the Internet address space as a whole and try to quantify its anomalies; our focus is on identifying disruptions in specific geographic locations.

The authors in [26] present an analysis regarding the stability of Internet routes to popular web destinations. Based on their study, the authors state that a small number of destinations which is visited by the biggest portion of traffic, is considered to have stable routes. Finally, BGP analysis proved that the vast amount of UPDATE messages concerns the least visited domains.

The recovery from large-scale incidents for emergency communications is discussed in [27]. This study is focused on improving the existing routing mechanisms by offering a new architecture that tackles the issues of communication overhead and tracking the location of overlay nodes. According to their results, the authors claim that their approach can significantly improve the exchange of routing information up to 9 times.

A work that takes a direction towards near real-time anomaly detection in BGP is [28]. They combine visual-, statistical-, and signature-based methods for detecting anomalies more accurately. In order to perform near real-time detection experiments they use historical BGP data which are replayed in a testbed. In this case, the authors limit their input data to specific BGP UPDATE messages from selected prefixes.

Other approaches include learning-based ones like [29] and [30], which propose two methods: one for signature-based and one for statistics-based detection. In these papers the authors concentrate on anomalous routing behavior for single prefixes and select BGP updates for the selected prefixes only. A slightly different approach is [31], which is a system that detects malicious BGP UPDATE messages by observing the network topology; the consistency check relies upon a model of AS connectivity.

Another group of research analyzes the impact of certain incidents on BGP UPDATE messages. A report [23] from Renesys Corp. analyses two blackouts in 2003: the first one was in USA/Canada on 14–16 August 2003 and the other in Italy on 28 September 2003. In this report, a list of networks around the blackout area is used to create statistics about network outages around the time of the event. There are a few works investigating the effects of worm propagation on BGP. The work in [32] look into the effects on BGP during the propagation of Code Red II and Nimda worms in July and September 2001, respectively. Another work [33] examines the Slammer worm for the period of January 2003. Dainotti et al. [34] analyze the impact of Internet outages caused deliberately for censorship, in Egypt and Libya. These works only analyze historical data and do not perform any kind of incident detection.

This work aspires to delve into the relation between the impact of tactical military missions, potentially backed up by cyberwarfare operations, and Internet’s routing stability from a BGP viewpoint. Taking the Russo-Ukrainian war as an eminent and timely use-case, and exploiting publicly available geolocalized BGP data over a two-month period, i.e., between February and March 2022, we analyze major network outages in both a city and country level. Specifically, our experiments highlight two cases: (a) the attack against the city of Kharkiv on the 24th of February, the D-Day of the Russian further military operations in Ukraine, and (b) the overall BGP outages in four directly involved or affected by the war countries, namely Belarus, Russia, Ukraine, and Poland. Our findings clearly showcase that there is a tight link between conventional military, and potentially supporting or preparatory cyber offensive operations, with the number of BGP outages. As a strongly indicative example, the first day of the invasion, the difference in the perceived number of outages in the city of Kharkiv surpassed 1,700%. As a general observation, it can be argued that BGP UPDATE data can be used to not only examine how and to which extent a physical or man-made major event or action affected the Internet infrastructure, but also to opportunely expose significant major events that certain parties may wish to keep out of the spotlight.

Two significant directions for future work are identified. First, given that at the time of writing, we are already at the fifth month of the war, the analysis can consider the time period starting from the first of April onward. Second, additional datasets of BGP data, say, from different route collectors like RIPE RIS^Footnote3 can be considered; in this way meaningful comparisons between results stemming from diverse vantage points can be drawn.