Strengths and Good Practices in the Region’s IXPs

17/01/2024

By Salvador Bertenbreiter, CEO – PIT Colombia, Ecuador, Guatemala and Peru IX.

Internet exchange points (IXPs) are one of the most important yet most unknown elements of the Internet’s base infrastructure. They allow ISPs, content providers, and other networks to connect locally, thus helping to reduce latency, eliminate bottlenecks, and lower Internet interconnection costs.

During the pandemic and due to the sudden surge in Internet traffic, we realized the importance for ISPs and carriers to connect to IXPs. This produced a paradigm shift, especially among the larger ISPs and carriers that had previously been somewhat reluctant to connect to IXPs, as they had not seen the value in doing so. Fortunately, this has begun to change, as end users increasingly demand lower latency and faster loading experiences, which has prompted major ISPs and carriers to view IXPs as an ally and probably the easiest way to achieve these objectives.

While the traditional alternative of establishing PNI works well for very large volumes of traffic between two networks, when seeking to reach a large number of networks, IXPs allow cost reductions and decrease the administrative burden in peering relationships. At the same time, it is advisable to maintain sufficient free capacity on the interfaces to the IXP. This is important in the event of a PNI outage, as an IXP is normally capable of absorbing this traffic and prevents it from going through IP transit.

In addition, the use of route servers offers a more secure environment, as the vast majority of IXPs implement filtering using RPKI and/or IRR. This increases security by reducing the potential impact of IP prefix hijacking, as the IXP filters out these “contaminated routes” before they reach the ISP. Moreover, the support of BGP communities in most route servers facilitates network engineering, making it easy to choose with whom and what routes are shared via peering.

The regional IXP ecosystem is quite diverse and heterogeneous, making it difficult to discuss it as a single group. In countries in the region that have well-developed interconnection ecosystems, much of the country’s traffic flows through the local IXP, which is why these have become critical infrastructure. In other countries, the ecosystems are taking longer to develop. However, all IXPs share the desire to develop the Internet in our region.

Developing IXPs and implementing good practices is one of the most important keys to establishing a robust and resilient interconnection platform. This helps reduce the number of incidents and minimizes potential problems, which is essential to having a reliable peering platform.

Based on my experience, I would like to highlight three key points that will help minimize problems when connecting to an IXP:

1. Only connect routers to IXP interfaces, never a switch. This is a key point often misunderstood by ISPs whose network topologies or lack of interfaces on their routers typically connect their IP transit carriers to switches, and then route this traffic to their routers via a VLAN. However, this poses a significant risk, not only for the member in question, but for the entire peering platform and other members of the IXP. This is why there is a policy that specifies a single MAC address per interface, meaning that members can only connect routers —not switches— to their IXP connection port.

Basically, a modern IXP is a large Layer 2 network, which can be affected by loops, excessive BUM traffic, and other issues inherent to a Layer 2 network, so connecting a member’s switch between the IXP’s interface and the member’s router introduces a considerable risk factor beyond the control of the IXP operator, which is why most IXPs do not allow this.

2. Avoid certain protocols on the interface connected to the IXP. There are three major “NOs”, i.e., parameters we must check to make sure they are NOT enabled on the ISP router interface connected to the IXP. The first NO is STP 802.11d, as IXP switches are not part of the ISP’s network. The second NO is Proxy ARP RFC1027, as the presence of this protocol can lead to the hijacking of packets destined for another member. In some devices, Proxy ARP is enabled by default, which is why disabling it is crucial. Finally, the third NO is Discovery Protocols, such as CDP, LLDP, MNDP or others.
3. Traffic engineering. To maximize the traffic exchanged at an IXP, it is important to keep in mind BGP route preference mechanisms. Thus, the following recommendations: a) send more specific prefixes to the IXP (with a /24 limit for IPv4 and /48 for IPv6, the most specific CIDR prefixes allowed in the DFZ) compared to IP transit; b) if sending more specific routes is not possible, attributes such as prepending can be used; here it is important to note that the greater the prepending, the lower the preference of that route will be, so in this case it is recommended to send the ISP prefixes without prepending to the IXP, and with greater prepending to the IP transit provider.

Have there been any IXP outages in the region similar to what recently happened at AMS-IX?

Yes. While IXP operators do everything in their power to guarantee perfect uptime, certain incidents have caused partial or total outages in some IXPs. While the infrastructure of most IXPs in the region is very robust, an ISP that connects to the Internet exchange point with a single connection may create a single point of failure. Therefore, the recommendation is to connect to the IXP via multiple connections, or at least to have sufficient IP transit capacity to handle the overflow if this link fails. It is recommended that ISPs contract their IP transit in burstable mode to ensure tolerance against outages while leveraging the advantages of the 95th percentile.