Basic Architecture
The Internet is hierarchical in structure. At the top are the very large national Internet service providers (ISPs), such as AT&T and Sprint, that are responsible for large Internet networks. These national ISPs, sometimes called tier 1 ISPs, connect together and exchange data at network access points (NAPs) (Figure 10.1).
In the early 1990s, when the Internet was still primarily run by the U.S. National Science Foundation (NSF), the NSF established four main NAPs in the United States to connect the major national ISPs. When the NSF stopped funding the Internet, the companies running these NAPs began charging the national ISPs for connections, so today the NAPs in the United States are all commercial enterprises run by various common carriers such as AT&T and Sprint. As the Internet has grown, so too has the number of NAPs; today there are about a dozen NAPs in the United States with many more spread around the world.
Figure 10.1 Basic Internet architecture. ISP = Internet service provider; MAE = metropolitan area exchange; NAP = network access point
NAPs were originally designed to connect only national ISPs. These national ISPs in turn provide services for their customers and also to regional ISPs (sometimes called tier 2 ISPs) such as Cogent Communications or France Telcom. These regional ISPs rely on the national ISPs to transmit their messages to national ISPs in other countries. Regional ISPs, in turn, provide services to their customers and to local ISPs (sometimes called tier 3 ISPs) who sell Internet access to individuals. As the number of ISPs grew, a new form of NAP called a metropolitan area exchange (MAE) emerged. MAEs are smaller versions of NAPs and typically link a set of regional ISPs whose networks come together in major cities (Figure 10.1). Today there are about 50 MAEs in the United States.
Because most NAPs, MAEs, and ISPs now are run by commercial firms, many of the early restrictions on who could connect to whom have been lifted. Indiana University, for example, which might be considered a local ISP because it provides Internet access for about 40,000 individuals, has a direct connection into the Chicago NAP, as do several other universities and large corporations. Regional and local ISPs often will have several connections into other national, regional, and local ISPs to provide backup connections in case one Internet connection fails. In this way, they are not dependent on just one higher-level ISP.
In general, ISPs at the same level do not charge one another for transferring messages they exchange across a NAP or MAE. That is, a national tier 1 ISP does not charge another national tier 1 ISP to transmit its messages. This is called peering. Figure 10.1 shows several examples of peering. It is peering that makes the Internet work and has led to the belief that the Internet is free. This is true to some extent, but higher-level ISPs normally charge lower-level ISPs to transmit their data (e.g., a national will charge a regional and a regional will charge a local). And of course, a local ISP will charge individuals like us for access!
In October, 2005, an argument between two national ISPs shut down 45 million Web sites for a week. The two ISPs had a peering agreement but one complained that the other was sending it more traffic than it should so it demanded payment and stopped accepting traffic, leaving large portions of the network isolated from the rest of the Internet. The dispute was resolved, and they began accepting traffic from each other and the rest of the Internet again.
In Figure 10.1, each of the ISPs are autonomous systems, as defined in next topic. Each ISP is responsible for running its own interior routing protocols and for exchanging routing information via the BGP exterior routing protocol at NAPs and MAEs and any other connection points between individual ISPs.
Connecting to an ISP
Each of the ISPs is responsible for running its own network that forms part of the Internet. ISPs make money by charging customers to connect to their part of the Internet. Local ISPs charge individuals for broadband or dial-up access whereas national and regional ISPs (and sometimes local ISPs) charge larger organizations for higher-speed access.
Each ISP has one or more points of presence (POP). A POP is simply the place at which the ISP provides services to its customers. To connect into the Internet, a customer must establish a circuit from his or her location into the ISP POP. For individuals, this is often done using a DSL modem, cable modem, or dial-up modem over a traditional telephone line (Figure 10.2). This call connects to the modem pool at the ISP and from there to a remote-access server (RAS), which checks the user ID and password to make sure the caller is a valid customer. Once logged in, the user can begin sending TCP/IP packets from his or her computer over the phone to the POP. Figure 10.2 shows a POP using a switched backbone with a layer-2 switch. The POP backbone can take many forms, as we discussed in next topic.
In the next section, we will discuss Internet access technologies such as DSL, cable modem, and Wireless Application Protocol (WAP) in more detail. Customers who need more network capacity simply lease a higher-capacity circuit. Figure 10.2 shows corporate customers with T1, T3, and OC-3 connections into the ISP POP. It is important to note that the customer must pay for both Internet access (paid to the ISP) and for the circuit connecting from their location to the POP (usually paid to the local exchange carrier [e.g., BellSouth, AT&T], but sometimes the ISP also can provide circuits). For a T1 connection, for example, a company might pay the local exchange carrier $400 per month to provide the T1 circuit from its offices to the ISP POP and also pay the ISP $300 per month to provide the Internet access.
Figure 10.2 Inside an Internet service provider (ISP) point of presence (POP). ATM = asynchronous transfer mode; CSU = channel service unit; DSU = data service unit; MAE = metropolitan area exchange; NAP = network access point
Inside the Chicago Network 10.1 Access Point
MANAGEMENT FOCUS
The Chicago network access point (NAP) is one of the busiest NAPs in the world. As we write this, it processes an average of about 4 gigabits of data per second.
More than 140 different Internet service providers (ISPs), including national ISPs (e.g., BBN Planet and Sprint), regional ISPs (e.g., Michigan’s Merit network), and local ISPs (e.g., Indiana University), as well as ISPs in other countries (e.g., Germany’s Tiscali network and the Singapore Advanced Research and Education Network), exchange traffic at the Chicago NAP. At present, most connections are asynchronous transfer mode (ATM) OC-3, or ATM OC-12, and the rest are T3. Pricing starts at about $4,000 per month for T3 and about $4,700 per month for OC-3. (Remember, this is only for Internet access; the ISPs must also lease a T3 or OC-3 circuit from their closest point-of-presence [POP] to the NAP.)
The NAP currently uses a large Cisco ATM switch that connects the more than 140 separate ISP networks (Figure 10.3). The ISP networks exchange IP packets through the NAP. They also exchange routing information through the Border Gateway Protocol (BGP) exterior routing protocol. Normally, the border router at each ISP simply generates BGP packets and sends them to the border routers at the other ISPs connected to the NAP. The Chicago NAP has so many ISPs that this is impossible. Because there are about 140 ISPs, each ISP would send messages to about 140 other ISPs, meaning a total of about 1 million BGP packets moving through the NAP every few minutes.
Instead, the Chicago NAP uses a route server in much the same way large networks based on OSPF (Open Shortest Path First) used designated routers (see ”Routing on the Internet” in next topic). The border router in each ISP sends BGP packets just to the NAP route server. The route server consolidates the routing information and then sends BGP packets back to each border router. This results in more efficient processing and only 200 messages every few minutes.
As Figure 10.2 shows, the ISP POP is connected in turn to the other POPs in the ISP’s network. Any messages destined for other customers of the same ISP would flow within the ISP’s own network. In most cases, the majority of messages entering the POP are sent outside of the ISP’s network and thus must flow through the ISP’s network to the nearest NAP/MAE, and from there, into some other ISP’s network.
This can be less efficient than one might expect. For example, suppose you are connected to the Internet via a local ISP in Minneapolis and request a Web page from another organization in Minneapolis. A short distance, right? Maybe not. If the other organization uses a different local ISP, which in turn uses a different regional ISP, the message may have to travel all the way to the Chicago NAP before it can move between the two separate parts of the Internet.
Figure 10.3 Inside the Internet’s Chicago network access point. ATM = asynchronous transfer mode; ISP = Internet service provider
The Internet Today
Sprint is one of the national ISPs in North America. Figure 10.4 shows Sprint’s North American backbone as it existed while we were writing this topic; it will have changed by the time you read this. As you can see, Sprint has a number of Internet circuits across the United States and Canada. Many interconnect in Chicago where Sprint connects into the Chicago NAP. Sprint also connects into major NAPs and MAEs in Reston, Virginia; Miami; Los Angeles; San Jose; Palo Alto; Vancouver; Calgary; Toronto; and Montreal. Most of the circuits are SONET OC-12, but a few are OC-48 and OC-192.
Today, the backbone circuits of the major U.S. national ISPs operate at SONET OC-48 and OC-192. Most of the largest national ISPs (e.g., Sprint, Cable & Wireless) plan to convert their principal backbones to OC-192 (10 Gbps). A few are now experimenting with OC-768 (80 Gbps), and several are in the planning stages with OC-3072 (160 Gbps). This is good because the amount of Internet traffic has been growing rapidly. The Internet traffic in the U.S. is expected to reach 80 Tbps (80 trillion bits per second) by 2011.
As traffic increases, ISPs can add more and faster circuits relatively easily, but where these circuits come together at NAPs and MAEs, bottlenecks are becoming more common. Network vendors such as Cisco and Juniper are making larger and larger switches capable of handling these high-capacity circuits, but it is a daunting task. When circuit capacities increase by 100 percent, switch manufacturers also must increase their capacities by 100 percent. It is simpler to go from a 622 Mbps circuit to a 10 Gbps circuit than to go from a 20 Gbps switch to a 200 Gbps switch.
Figure 10.4 Sprint’s North American Internet backbone
The Internet is constantly changing. Up-to-date maps of the major ISPs whose networks make up large portions of the Internet are available at navigators.com/isp.html.
