Lecture
The Internet backbone is the principal data routes between large, strategically interconnected networks and core routers on the Internet. These data routes are hosted by commercial, government, academic, and other high-capacity network centers, points of traffic exchange, and network access points that exchange Internet traffic between countries and continents. Internet Service Providers (often Tier-1 operators) participate in exchanging backbone Internet traffic through privately negotiated interconnection agreements, mainly on the basis of peering.
A backbone communication network is a transport telecommunications infrastructure for providing communication services. As a rule, a backbone communication network is built on owned or leased fiber-optic lines using high-speed or low-speed channel equipment.
The Internet backbone can be defined as the principal data routes between large, strategically interconnected computer networks and core routers on the Internet. These data routes are hosted in commercial, government, academic, and other high-bandwidth network centers, as well as traffic exchange points and network access points that exchange Internet traffic between countries, continents, and across oceans. Internet service providers, often Tier-1 networks, participate in backbone Internet traffic on the basis of privately concluded interconnection agreements, primarily governed by the principle of settlement-free peering.
The Internet, and consequently its backbone networks, does not rely on centralized management or coordination, nor does it implement any global network policies. The resilience of the Internet results from its core architectural features, primarily the idea of placing as few network state and control functions as possible in the network elements themselves, and instead relying on the endpoints of communication to handle most of the processing needed to ensure data integrity, reliability, and authentication. In addition, the high degree of redundancy in modern network links and sophisticated real-time routing protocols provide alternative data exchange paths for load balancing and congestion avoidance.
The largest providers, known as Tier-1 providers, have networks so extensive that they do not purchase transit agreements from other providers. As of 2019, there were six Tier-1 providers in the telecommunications industry: CenturyLink (Level 3), Telia Carrier, NTT, GTT, Tata Communications, and Telecom Italia.
Map of the world's submarine digital cables (2015)
Map of submarine cables, 2016
Laying the main submarine cables that serve as the physical infrastructure of the Internet
The Internet backbone consists of many networks belonging to many companies. Backbone fiber-optic lines consist of many fiber-optic cables bundled together to increase capacity or throughput. Fiber-optic communication remains the preferred medium for backbone Internet providers for several reasons. Optical fibers provide high data transfer rates and wide bandwidth, they experience relatively little attenuation, which allows them to cover long distances with few repeaters, and they are also immune to crosstalk and other forms of electromagnetic interference that disrupt power transmission. Real-time routing protocols and redundancy built into the backbone can also reroute traffic in case of failure. Data transfer speeds on backbone lines have increased over time. In 1998 all U.S. backbone networks used the lowest data rate of 45 Mbit/s. However, technological improvements by the mid-2000s allowed 41% of backbone networks to have data rates of 2488 Mbit/s or higher.
The first packet-switched computer networks, NPL and ARPANET, were interconnected in 1973 via University College London. ARPANET used a backbone of routers called Interface Message Processors. Other packet-switched computer networks quickly spread starting in the 1970s, eventually adopting TCP/IP protocols or being replaced by newer networks. The National Science Foundation created the National Science Foundation Network (NSFNET) in 1986, funding six network sites using 56 kbit/s interconnections, with peering to ARPANET. In 1987, this new network was upgraded to 1.5 Mbit/s T1 links for thirteen sites. These sites included regional networks that, in turn, connected more than 170 other networks. IBM, MCI, and Merit upgraded the backbone to 45 Mbit/s (T3) bandwidth in 1991. The combination of ARPANET and NSFNET became known as the Internet. A few years later, the dominance of the NSFNet backbone led to the decommissioning of the backup ARPANET infrastructure in 1990.
In the early days of the Internet, backbone providers exchanged their traffic at government-sponsored network access points (NAPs) until the government privatized the Internet and handed the NAPs over to commercial providers. [
Due to overlap and synergy between long-distance telephone networks and backbone networks, the largest long-distance telephone operators, such as AT&T Inc., MCI (acquired by Verizon in 2006), Sprint, and CenturyLink, also own some of the largest Internet backbone networks. These backbone providers sell their services to Internet Service Providers (ISPs).
Every Internet provider has its own network for contingencies and has external backup. These networks are interwoven and overlapping to create a redundant network. Many companies use their own backbone networks, which are interconnected at various Internet Exchange Points (IXPs) around the world. For data to travel across this network, backbone routers are needed — routers powerful enough to process information — within the Internet backbone network and capable of directing data to other routers to send it to its final destination. Without them, information would be lost. [10
The four largest IT companies — Amazon, Google, Microsoft, and Facebook — have gone so far as to invest not only directly in data centers and servers, but also in the backbone cables themselves, that is, they have entered territory that traditionally belonged to entirely different types of organizations
First, IT giants have taken up laying their own backbone networks in order to increase connectivity between elements of the companies' internal infrastructure on different continents. Precisely because of the aforementioned ping of nearly half a second between two opposite points on the globe, IT companies have to get creative in ensuring the stability of their «operations». These issues are most acute for Google and Amazon; the former began laying its own networks back in 2014, when it decided to «run» a cable between the east coast of the US and Japan to connect its data centers, which was written about on Habr at the time. Just to connect two separate data centers, the search giant was willing to spend $300 million and lay about 10,000 kilometers of cable along the bottom of the Pacific Ocean.
If anyone didn't know or has forgotten, submarine cable laying is an extremely complex undertaking, ranging from submerging reinforced structures up to half a meter in diameter in coastal zones to endless landscape surveying for laying the main part of the backbone at depths of several kilometers. When it comes to the Pacific Ocean, the complexity only increases in proportion to the depth and the number of mountain ridges on the ocean floor. Such operations require specialized vessels, a specially trained team of specialists, and, in essence, several years of intensive work, if you consider laying from the design and survey stage through to the final commissioning of the network segment. On top of that, you can add coordinating work and building relay stations on shore with local governments, working with environmentalists who monitor the preservation of the most inhabited coastal line (depth <200 m), and so on.
Perhaps new vessels have been commissioned in recent years, but even five years ago there was a solid queue of many months for the main cable-laying ships of the same Huawei (yes, the Chinese company is one of the leaders in this market). Against the backdrop of all this information, the activity of tech giants in this segment looks more and more interesting.
The official position of all major IT companies: ensuring connectivity (independence from shared networks) of their data centers. And here is what the submarine maps of different market players look like according to telegeography.com:
As can be seen on the maps, the most impressive appetite belongs not to Google or Amazon, but to Facebook, which stopped being «just a social network» a long time ago. There is also a clear interest of all major players in the Asia-Pacific region, and only Microsoft still reaches toward the Old World. If you simply count the marked backbone lines, you find that just these four companies are co-owners or full owners of 25 already built or finally planned backbone lines, most of which extend toward Japan, China, and all of Southeast Asia. Note that we are only providing statistics for the aforementioned four IT giants, while besides them, Alcatel, NEC, Huawei, and Subcom are also actively building their own networks.
We are now witnessing the birth of a new formation of the Internet, essentially a global corporate network. If we recall that Amazon, Google, Facebook, and Microsoft account for at least half of the world's traffic consumption (Amazon hosting, Google search and services, Facebook and Instagram social networks, and desktops running Microsoft Windows), then it's time to get out a second hat. Because in theory, if projects like Google Fiber (the one where Google tried itself out as a provider for the general population) start appearing in various regions, then what we are now observing is the birth of a second internet, which for now coexists alongside the one already built.
During the 2011 Egyptian revolution, the Egyptian government shut down four major providers on January 27, 2011, at approximately 17:20 EST. [12] Apparently, the networks were not physically interrupted, since transit Internet traffic through Egypt was not affected. Instead, the government disabled Border Gateway Protocol (BGP) sessions that announced local routes. BGP is responsible for routing traffic between Internet providers. [13]
Only one of Egypt's Internet providers was allowed to continue operating. The ISP Noor Group provided connectivity only to the Egyptian stock exchange, as well as to certain government ministries. [12] Other Internet providers began offering free dial-up Internet access in other countries. [14]
Europe makes a major contribution to the growth of the international backbone, as well as to the growth of Internet bandwidth. In 2003, Europe accounted for 82 percent of the world's international bandwidth. [15] Level 3 Communications began launching a line of dedicated Internet access and virtual private network services in 2011, providing large companies with direct access to the Tier-3 backbone. Connecting companies directly to the backbone network will provide businesses with faster Internet access, meeting significant market demand. [16]
Some countries in the Caucasus have very simple backbone networks; for example, in 2011 a woman in Georgia pierced a backbone fiber-optic cable with a shovel and left the neighboring country of Armenia without Internet access for 12 hours. Since then, the country has undergone significant changes in its fiber-optic backbone infrastructure, but progress is slow due to a lack of government funding. [17]
The Internet backbone network in Japan must be very efficient due to high demand for the Internet and technology in general. In 2009, Japan had more than 86 million Internet users, and by 2015 the number of Internet users was projected to increase to almost 91 million. Since there is demand for fiber-optic connectivity for the home in Japan, Japan is considering using the fiber-optic backbone of Nippon Telegraph and Telephone (NTT), the domestic backbone operator, to provide this service at lower prices. [18]
In some cases, companies that own certain sections of the Internet backbone's physical infrastructure rely on competition to maintain the profitability of the Internet market. This is especially noticeable in China. Since China Telecom and China Unicom have for some time acted as the only Internet service providers in China, smaller companies cannot compete with them in negotiating settlement prices for interconnection that ensure the profitability of the Internet market in China. Such introduction of discriminatory pricing by large companies then leads to market inefficiency and stagnation, and ultimately affects the efficiency of the Internet backbone networks serving the country
Communication between China and the rest of the world passes exclusively through «special servers». They are located at legally established traffic exchange points.
The equipment belongs to six companies. The largest are: China Telecom and China Unicom.
Full list:
All of them are affiliated with the state in one way or another.
Internet gateways are divided into several levels:
In 2015, due to annual traffic growth of 10-15%, 7 more backbone nodes had to be built in the cities of Chengdu, Wuhan, Xi'an, Shenyang, Nanjing, Chongqing, and Zhengzhou; some nodes were «upgraded» from Core Level to National, while some were built from scratch.
Global connectivity looks like this:
Chinese Internet providers use different approaches to blocking content. For example, the two largest Chinese Internet providers, China Telecom and China Unicom, place their traffic filters differently.
CNC Group (China Unicom) places most of its filters on the backbone, while ChinaNet (China Telecom), due to the colossal volume of Chinese traffic, places firewalls mainly on its own regional networks.
Other small «regional» providers and niche ISPs (Internet service providers) do not filter traffic, but can only connect to the six operators to transmit cross-border traffic.
Several mechanisms are used to block Internet content:
Backbone communication networks in Russia are divided into two segments:
The largest backbone telecom operators in Russia as of December 2011:
| No. | Telecom operator | Network length (thousand km) | Network map |
|---|---|---|---|
| 1 | «Rostelecom» | 500 | See. |
| 2 | «Mobile TeleSystems» | 243 | |
| 3 | «VympelCom» | 183 | See. |
| 4 | «MegaFon» (including «Synterra» networks) | 136 | See. |
| 5 | «TransTeleCom» | 78 | See. |
| 6 | «RetnNet» | 42 (8,562 in the Russian Federation, route miles metric, 2014) | See. |
| 7 | «Start Telecom» | 16 | See. |
| 8 | «Zummer» | 13.8 | See. |
| 9 | «Rascom» | 17 (4.4 in the Russian Federation) | See. |
| 10 | «TeliaSonera» | 2 | See. |
| 11 | JSC «Kvantum» | 1.5 |
In 1995, an international backbone network was created for exchanging multicast traffic, Mbone[en]. Based on it, a system of virtual video-conference rooms operated from 1997 to 2008.
Near the Russian segment of the internet are located the largest network hubs in the world. The geographically closest, and, coincidentally, the largest traffic exchange point in the world is DE-CIX in Frankfurt. Three of the largest Russian traffic exchange operators are connected here: MSK-IX (2 Tbit/s), Data-IX (2 Tbit/s), W-IX (1 Tbit/s), with an average load of 3.2 Gbit/s.
The map of Rostelecom's backbone networks and the map of the international backbone operator RETN show through which channels the Russian segment connects to the world's largest exchange points. Rostelecom's new fast line from Moscow to Frankfurt is also marked.
Map of Rostelecom's backbone networks
Map of RETN's backbone networks
To exchange traffic, operators can conclude agreements with each other or choose a more advanced form of peering such as W-IX. This system operates within a single city at the second level, and communication between participants takes place, as in any other peering arrangement, directly. At the same time, communication with all other traffic exchange points where W-IX is a participant is carried out via a route server.
W-IX
W-IX has its own international channels between the largest traffic exchange points.
W-IX
Experts note that in recent years there has been a certain trend toward localization of traffic, where servers are placed within the national borders of the country where the main audience is located. This localization trend is supported by the spread of CDN services and information security measures related to the threat of confidential information leaks. Now not only Russia, but also other countries are considering laws requiring confidential information (including financial and medical information) to be stored only within the country.
Fortunately, localization requirements affect only a limited number of websites, so internet companies can still choose their hosting location based on their own needs. Placing servers near global network hubs makes servers more accessible to a global audience and turns out to be much cheaper, because all the relevant network infrastructure, including data centers and hosting providers, is concentrated around the hubs.
To organize information exchange between individual local and global networks, a transport network (TN) is deployed that implements services for transporting information flows between individual subscribers, as well as providing information services (such as radio, TV, fax communication, etc.) to consumers.
A transport communication network (backhaul) is a set of resources that perform transport functions in telecommunications networks. It includes not only transmission systems, but also their associated control, operational switching, redundancy, and management facilities.
Figure 13.1 — A telecommunications network consisting of a backbone transport network and subscribers connected to it via access networks
As a rule, transport networks are deployed on a national scale. In the Russian Federation, such a transport system is the Interconnected Communication Network (ICN) of the Russian Federation (VSS).
Today, the Interconnected Communication Network of Russia is a set of networks (Fig. 13.2):
The main component of the VSS is the public communication networks, open to all individuals and legal entities on the territory of Russia.
Figure 13.2 — Structure of the VSS of the Russian Federation
Organizationally, the VSS is a set of interconnected telecommunications networks under the jurisdiction of various telecom operators as legal entities with the right to provide telecommunications services. The architecture of the VSS of the Russian Federation is shown in Fig. 13.3.
The Interconnected Communication Network, as a communication system, is a hierarchical three-level system:
Telecommunications services are provided to users via terminal equipment of telecommunications networks. Telephone communication, data transmission, telegraph communication, newspaper transmission, distribution of television and sound broadcasting programs, videophone networks — all these public telecommunications systems are part of the VSS structure as secondary networks.
In addition to public telecommunications networks, the VSS also includes secondary networks organized by various departments, corporations, and commercial companies. Such networks include:
Figure 13.3 — Architecture of the VSS
When building secondary networks, various types of telecommunications technologies are used to ensure the efficient use of channels and standard paths allocated from the primary network to a given secondary network. Telecommunications technologies for secondary networks include:
In addition to the above, in recent years new, more efficient technologies for building secondary networks have been actively introduced, which belong to integral-type telecommunications technologies. These technologies provide for the joint transmission of messages of various types of information: voice, data, fax, and video information, including the transmission of television programs, etc. The most widespread of these advanced technologies currently are: ATM, Ethernet-Gb, ISDN, and FrameRelay.
As an example, Fig. 13.4 shows the «TransTeleCom» transport network as part of the VSS, representing the backbone digital communication network of Russian Railways, as well as the MPLS IP network and access networks that rely on its resources, integrated into a single interconnected multiservice network. SDH technology was chosen as the base technology for building the backbone primary network.
Figure 13.3 — «TransTeleCom» backbone network
The main task of the VSS is transport, i.e., transmitting messages from a source to a recipient. The end result of the VSS's operation is the communication services provided to users.
Indicators characterizing the operation of the VSS:
Communication systems can provide protection of information from a number of threats to its security (blocking, unauthorized access to individual network elements, etc.). Responsibility for the overall resolution of information security issues (ensuring confidentiality, integrity, and availability properties) rests with the user (owner of the information).
Communication network resilience is its ability to remain operational under the influence of various destabilizing factors. It is determined by the network's reliability, survivability, and noise immunity.
Various measures are used to increase the resilience of VSS networks:
Comments