Inter-AS MPLS VPN lets two or more service providers deliver an end-to-end Layer 3 VPN to a customer whose sites span different autonomous systems. There are three standard ways to do it — Option A, Option B, and Option C — and each trades off simplicity, scalability, and security differently. This guide walks through all three options with their design considerations and a side-by-side comparison.
I explain these topics in deep detail in my Self-Paced CCDE course.
Inter-AS Option A (VRF-to-VRF)
Inter-AS Option A is the easiest, most flexible, and most secure inter-autonomous-system MPLS VPN technology. In a typical topology, VPN customers A and B are connected to two different service providers via MPLS Layer 3 VPN, and to deliver an end-to-end service the providers use special mechanisms. Option A is the most basic one, also known as the VRF-to-VRF approach, and its aim is simply to carry all of the customer routes between the service providers.
Inter-AS Option A topology
In this design there are two service providers and two customers that require inter-AS MPLS VPN service. The PE routers that connect the two providers are also known as ASBRs (Autonomous System Boundary Routers). In Option A, an ASBR router in one autonomous system attaches directly to an ASBR router in another autonomous system. The two ASBRs are connected through multiple sub-interfaces — at least one for every VPN whose routes need to pass from one AS to the other — and those sub-interfaces are associated with the VRF table. Providers could instead use separate physical connections for each customer, but that would not produce an optimal result for resource utilization. On the customer side, PE routers connected to the CE devices run MP-IBGP, either through a full mesh or through route reflectors.
Option A allows ASBR routers to keep all the VRFs for customers who require inter-AS service. The SP-A and SP-B ASBR routers maintain a VPN forwarding table in the LFIB and also keep routing information in the RIB and FIB. As a result, compared to the other options, ASBRs have high memory usage in Option A. The ASBRs can either run the same routing protocol as the customer on the VRFs or use just EBGP. For example, if customer A requires end-to-end routing information such as metric, SP-A and SP-B run the same routing protocol on the ASBRs and on the PE devices where the customer CE is attached. Because those routes appear at the ASBR as BGP routes from remote PEs, Option A also has to manage redistribution at the ASBRs: for each customer the routes must be redistributed from BGP into the PE-CE routing protocol and back. Each such sub-interface is associated with a VRF.
Design Considerations for Option A
Option A does not require MPLS at the ASBRs, unlike the other inter-AS options. However, because you need a separate VRF and sub-interface for each customer VPN, a separate routing protocol, and per-protocol redistribution, it is operationally cumbersome and therefore hard to scale. On the other hand, since only IP routing runs between the autonomous systems, Option A is considered the most secure option, and it is the easiest to implement because it does not require another control-plane mechanism between the provider ASBRs such as LDP, BGP+label, or BGP-VPNv4 — either IGP protocols or EBGP are used between the providers. The other options, B and C, do require additional control-plane protocols to advertise the customer or infrastructure prefixes between the ASBRs. Finally, because only IP traffic (not MPLS) passes between the service providers, the most granular QoS is achieved with Option A: per sub-interface and using the IP DSCP field rather than the MPLS EXP bits. As with every inter-AS option, customers have to trust the service provider for data integrity, confidentiality, and availability; MPLS itself does not encrypt packets, so a customer that needs end-to-end encryption can deploy IPSec.
Inter-AS Option B (VPNv4 Between ASBRs)
Inter-AS Option B is a highly scalable, reasonably secure, but operationally complex inter-autonomous MPLS VPN architecture. If you want the easiest, most flexible, and most secure solution, that is Inter-AS Option A, covered above.
Inter-AS Option B topology
In a typical Option B topology, AS10 and AS20 are the service providers, and there are two customers, VPN A and VPN B. Note that the customers can run different routing protocols at different sites — for example, VPN A connected to AS10 may run OSPF while VPN A connected to AS20 runs EIGRP. Inside each provider network, PE routers connect to the customer locations and run MP-BGP with route reflectors to advertise VPNv4 prefixes. The PE router that connects the two providers at the edge is the ASBR, and since the ASBR also runs MP-BGP with the RR, it learns all the VPN routes from inter-AS customers.
Unlike Option A — where separate sub-interfaces are enabled, IGP protocols run between the ASBRs, and MP-BGP is redistributed into IGP — Option B does not require separate sub-interfaces, different IGP protocols, or BGP per sub-interface between the ASBRs. Between the ASBRs, MP-BGP simply enables the VPNv4 address family, and the ASBRs advertise the customer prefixes learned from the local BGP route reflector to the other service provider over that MP-BGP VPNv4 session. You do not need to keep a VRF table for the customers on the ASBRs; you only need to keep the customer prefixes in the LFIB database. As a result, the CPU and memory requirements for Option B are much lower than those of Option A.
The PE devices have BGP VPNv4 sessions with the provider's route reflectors and redistribute customer VPN prefixes from the PE-CE routing protocol into MP-BGP and vice versa. A route-target extended community is placed on the VPNv4 update, and that route target helps place the VPN prefixes into the appropriate VRF. When a PE receives the customer IP prefixes it changes the next hop to itself, turns the customer IP prefixes into VPN prefixes by adding a route distinguisher, and sends the VPN update to the VPNv4 route reflector. The route reflector does not change the next hop; it only reflects the route to the ASBR. (This is normal BGP RR behavior, but it is different in Option C.) The ASBR does not place the MP-BGP route into a VRF, since it does not keep a VRF table but instead maintains the VPNv4 BGP table.
By changing the next hop, the ASBR in SP-A sends VPNv4 prefixes through the MP-BGP session to the SP-B ASBR, which in turn sends the customer prefixes to its local route reflector. The next hop is changed on the ASBR (normally, from EBGP to IBGP, the next hop would not change). Therefore, either the ASBR-to-ASBR link is redistributed into the global IGP table of the service providers, or next-hop-self is enabled to set the next hop to the ASBR. The route reflector in the SP-B domain reflects the prefixes as is and sends them to the PE connected to the customer A2 location, and that SP-B PE sets the next hop again and sends the prefixes to the customer A2 router. Because there are three LSPs across the provider domains and changing the BGP next hop terminates an LSP, a new VPN label is assigned on that router for all VPN prefixes.
Design Considerations for Option B
There is an important caveat. Normally an ASBR would not accept the VPN prefixes because it has no VRF for the customers, and since the route target is not associated with a VRF in Option B, the ASBR would reject the prefixes. To solve this, the no bgp route-target filter configuration knob is enabled, which allows the ASBRs to accept the VPN prefixes even though they have no route target associated with a VRF. Option B requires MP-BGP between the autonomous systems with the VPN address family enabled, but it does not require LDP or IGP protocols between the providers, so the service providers do not need to understand each other's internal addressing structure. The trade-off is that the LSP is not end-to-end; it is terminated at many points, which makes it very difficult to manage. As in Option A, you do not need to redistribute VPN prefixes at the ASBR: the route reflectors store all the VPN routing information for each customer and advertise those prefixes to the ASBR, so operators need to manage MP-BGP on the ASBRs as well as on the route reflectors.
What if, by removing those prefixes from the ASBRs, we could advertise all the VPN prefixes directly between the route reflectors? That is exactly what Inter-AS Option C does, and it is covered next.
Inter-AS Option C (VPNv4 Between Route Reflectors)
Inter AS Option C is the most complex, insecure, uncommon, but extremely scalable inter provider MPLS VPN solution. In this section, I will explain how service providers can use Inter AS Option C to assist customers to have an end-to-end MPLS VPN service.
As covered in the Inter-AS Option B section above, ASBR routers between the service providers do not keep a VRF table for the VPN customers. As shown in the figure below, in Inter AS Option B an MP-BGP VPNv4 session is set up between service providers' ASBR PEs.
Inter-AS Option B topology (recap for comparison with Option C)
As for Inter AS Option B, ASBR routers – the provider-edge devices between the service providers – maintain only the VPN prefixes of the customers in the BGP table. In fact, I have shown that a VPNv4 BGP session has been set up between the ASBRs.
The high-level operational differences between Inter AS Option C and Inter AS Option B are twofold: one is that ASBRs do not have a VRF table; the other is that, unlike Inter AS Option B, Inter AS Option C does not keep VPN prefixes for the customer on the ASBR. The ASBR is the edge router used to connect the service providers to each other.
As shown in the figure below, in Inter AS Option C the VPNv4 BGP session is set up between the route reflectors of the service providers.
Inter-AS Option C topology
In this topology, there are two service providers: service provider A and service provider B. Service provider A has two customers, customer A and customer B, and so does service provider B; A and B are companies that require inter-AS MPLS VPN service. Service provider B runs IS-IS internally (it could run other IGP protocols as well). PE-CE routing protocols are enabled on the VRF, so service provider A has two separate VRF tables. For scaling purposes, all the provider-edge routers run a VPNv4 BGP session with a VPNv4 route reflector.
Inter AS Option C runs by enabling a VPNv4 BGP session between the route reflectors of the service providers. Provider-edge routers of the service providers do not set up VPNv4 neighborship, neither within an AS nor between ASes.
In order to use the route reflectors to set up VPNv4 neighborship, they should move towards each other, i.e. BGP runs over TCP. For reachability, route reflector loopback interfaces are advertised on the global routing table of the service providers.
In Option C, the ASBR PEs of the service providers run an IPv4 BGP session between them. Over this IPv4 BGP session, the loopback interfaces of route reflectors and internal PEs are advertised. Next, I will explain why you need to advertise the internal PE loopback addresses, not just the route reflectors. The ASBR PEs know the loopback address of the route reflector through OSPF in the service provider A network and through IS-IS in the service provider B network.
Since the VPNv4 EBGP neighborship is set up between the VPNv4 RRs of the service providers, the next hop for the VPN route is the route reflector. Traffic trombones on the RR unless no bgp next-hop unchanged is configured on the route reflectors. This is one of the important caveats in Inter AS MPLS VPN Option C. Once this feature is enabled, the route reflector does not change the next hop to itself even though the peering is EBGP VPNv4 between the RRs. (Whenever there is an EBGP session between two BGP speakers, the next hop is changed since they are not clients of each other but regular EBGP neighbors; otherwise the RR doesn't change the next hop in IBGP.)
In this case, because the internal PEs continue to be the next hop for the customer prefixes, the internal PE loopback is advertised on the IPv4 BGP session between the ASBRs. As for the VPN next hop, a transport label should be assigned for the MPLS VPN to work efficiently, and an MPLS label is assigned to the IPv4 unicast routes, which are the loopbacks of the RRs and internal PEs of the ASBR.
Inter-AS Option C Design Objectives
- Inter AS Option C is unique, since it requires internal addressing advertisement between the service providers.
- Those addresses are leaked into the global routing table of the providers, a process that providers dislike.
- It can still be used in large-scale designs if the company has multiple autonomous systems (not between the service providers).
- This is just a network design option; please note that it is not the only option.
- Inter AS Option C is very difficult to implement because it requires meticulous redistribution at the edge of the networks, VPNv4 neighborship between the route reflectors, IPv4 BGP neighborship between the ASBRs, label advertisement, route target agreement between the service providers, and so on.
- It is insecure because the internal addresses leak between the providers.
Which Inter-AS Option Should You Choose?
The right choice depends on how you weigh simplicity, scalability, and security. Option A is the easiest and most secure and gives the most granular QoS, but because it needs a VRF, sub-interface, and redistribution per customer, it consumes the most memory on the ASBRs and scales the worst. Option B scales far better and keeps no VRF on the ASBRs, at the cost of an operationally complex, non-end-to-end LSP and the no bgp route-target filter workaround. Option C scales the most but is the most complex and the least secure, because it leaks internal addressing between the providers. The table below compares all three from a design point of view, which is useful both in real deployments and for design exams.
Inter-AS MPLS VPN options comparison
You might wonder whether inter-AS designs are common between service providers in real life. In practice, I have helped design inter-AS MPLS VPNs for service providers twice, and both times the design concluded with Inter AS Option A.
To gain a deeper understanding of service provider networks, you can check my published Service Provider Networks Design and Architecture Perspective book, which covers SP network technologies and explains a fictitious SP network in detail.
