Pseudowire Headend Termination - in 8 steps - PART1

PROBLEM: Swansea, Newport and Cardiff subscribers have been, temporarily, terminated on vACX hardware at those sites. The vACX routers are typically used for mobile backhaul and generally don't have the 'grunt' to terminate subscribers. They don't support per-unit-scheduling, for example. We need to get them off these routers.

GOAL: Terminate all subscribers from the Access Provider at Cardiff, on to the new vMX960 at Llanelli.

Juniper's Configuring the Broadband Edge as a Service Node Within Seamless MPLS Network Designs article was the main inspiration for this post.

 

 

Let's log on to Llanelli and get cracking! Before we begin, let's check reachability to the loopbacks:

root@Llanelli> ping 172.16.99.1 count 1    
PING 172.16.99.1 (172.16.99.1): 56 data bytes
64 bytes from 172.16.99.1: icmp_seq=0 ttl=64 time=0.066 ms

--- 172.16.99.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.066/0.066/0.066/0.000 ms

root@Llanelli> ping 172.16.99.2 count 1    
PING 172.16.99.2 (172.16.99.2): 56 data bytes
64 bytes from 172.16.99.2: icmp_seq=0 ttl=64 time=93.739 ms

--- 172.16.99.2 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 93.739/93.739/93.739/0.000 ms

root@Llanelli> ping 172.16.99.3 count 1    
PING 172.16.99.3 (172.16.99.3): 56 data bytes
64 bytes from 172.16.99.3: icmp_seq=0 ttl=64 time=21.710 ms

--- 172.16.99.3 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 21.710/21.710/21.710/0.000 ms

root@Llanelli> ping 172.16.99.4 count 1    
PING 172.16.99.4 (172.16.99.4): 56 data bytes
64 bytes from 172.16.99.4: icmp_seq=0 ttl=63 time=281.251 ms

--- 172.16.99.4 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 281.251/281.251/281.251/0.000 ms

 

  1. Add secondary IP to lo0.0:
    set interfaces lo0 unit 0 family inet address 203.0.113.1/32
  2. Configure the dynamic VLAN profile with versioning:
    set system dynamic-profile-options versioning
    set dynamic-profiles DYNINTF-DHCP-INET interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" proxy-arp restricted
    set dynamic-profiles DYNINTF-DHCP-INET interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" vlan-tags outer "$junos-stacked-vlan-id"
    set dynamic-profiles DYNINTF-DHCP-INET interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" vlan-tags inner "$junos-vlan-id"
    set dynamic-profiles DYNINTF-DHCP-INET interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" family inet unnumbered-address lo0.0
    set dynamic-profiles DYNINTF-DHCP-INET interfaces "$junos-interface-ifd-name" unit "$junos-interface-unit" family inet unnumbered-address preferred-source-address 203.0.113.1
  3. Enable tunnel-services on the PFE:
    set chassis fpc 0 pic 0 tunnel-services bandwidth 1g
    set chassis fpc 0 pic 0 traffic-manager egress-shaping-overhead 0
    set chassis network-services enhanced-ip
    set chassis pseudowire-service device-count 4
  4. Configure the pseudowire (ps0) interface for single and double tagged traffic:
    set interfaces ps0 anchor-point lt-0/0/10
    set interfaces ps0 flexible-vlan-tagging
    set interfaces ps0 auto-configure stacked-vlan-ranges dynamic-profile DYNINTF-DHCP-INET accept any
    set interfaces ps0 auto-configure stacked-vlan-ranges dynamic-profile DYNINTF-DHCP-INET ranges any,any
    set interfaces ps0 auto-configure vlan-ranges dynamic-profile DYNINTF-DHCP-INET accept any
    set interfaces ps0 auto-configure vlan-ranges dynamic-profile DYNINTF-DHCP-INET ranges any
    set interfaces ps0 auto-configure remove-when-no-subscribers
    set interfaces ps0 no-gratuitous-arp-request
    set interfaces ps0 unit 0 encapsulation ethernet-ccc
  5. Configure DHCP and bind it to the transport logical (ps0.0) interface:
    set system services dhcp-local-server pool-match-order ip-address-first
    set system services dhcp-local-server authentication username-include interface-name
    set system services dhcp-local-server group local interface ps0.0
  6. Enable subscriber management:
    set system services subscriber-management enable
    set system configuration-database max-db-size 104857600
  7. Configure the access profile and DHCP scope. Apply the access-profile:
    set access profile local authentication-order none
    set access address-assignment pool TEST-NET-3 family inet network 203.0.113.0/24
    set access address-assignment pool TEST-NET-3 family inet range 1 low 203.0.113.2
    set access address-assignment pool TEST-NET-3 family inet range 1 high 203.0.113.250
    set access address-assignment pool TEST-NET-3 family inet dhcp-attributes maximum-lease-time 3600
    set access address-assignment pool TEST-NET-3 family inet dhcp-attributes domain-name msbnet.co.uk
    set access address-assignment pool TEST-NET-3 family inet dhcp-attributes router 203.0.113.1
    set access-profile local
  8. Configure the l2circuit at both sites:
    # Llanelli
    set protocols l2circuit neighbor 172.16.99.4 interface ps0.0 virtual-circuit-id 1
    set protocols l2circuit neighbor 172.16.99.4 interface ps0.0 ignore-mtu-mismatch
    # Cardiff
    set protocols l2circuit neighbor 172.16.99.1 interface ge-0/0/3.1 virtual-circuit-id 1
    set protocols l2circuit neighbor 172.16.99.1 interface ge-0/0/3.1 encapsulation-type ethernet
    set protocols l2circuit neighbor 172.16.99.1 interface ge-0/0/3.1 ignore-mtu-mismatch
    set interfaces ge-0/0/3 unit 1 encapsulation vlan-ccc
    set interfaces ge-0/0/3 unit 1 vlan-id-range 2-10

 

Finally, commit the configuration at both sites. Llanelli displays the follow message upon commit:

root@Llanelli# commit and-quit 
[edit system services subscriber-management]
  'enable'
    warning: Chassis configuration for subscriber-management has been changed. A system reboot is mandatory.  Please reboot the system NOW. Continuing without a reboot might result in unexpected system behavior.

Message from syslogd@Llanelli at Sep 20 17:18:05  ...
Llanelli fpc0 CMLC: Going disconnected; Routing engine chassis socket closed abruptly 
commit complete
Exiting configuration mode

If we take a sneaky peak at the PFE directly after the commit, we see the following:

root@Llanelli> show chassis fpc 
                     Temp  CPU Utilization (%)   CPU Utilization (%)  Memory    Utilization (%)
Slot State            (C)  Total  Interrupt      1min   5min   15min  DRAM (MB) Heap     Buffer
  0  Offline         ---Restarted by cli command---
  1  Empty           
  2  Empty           
  3  Empty           
  4  Empty           
  5  Empty           
  6  Empty           
  7  Empty           
  8  Empty           
  9  Empty           
 10  Empty           
 11  Empty           

Enabling tunnel services caused the PFE to restart. It is at this point it gives birth to the logical tunnel interface, lt-0/0/10, amongst others. Periodically check the chassis to see if the PFE has come back up:

root@Llanelli> show chassis fpc    
                     Temp  CPU Utilization (%)   CPU Utilization (%)  Memory    Utilization (%)
Slot State            (C)  Total  Interrupt      1min   5min   15min  DRAM (MB) Heap     Buffer
  0  Online           Testing  20         0       30     33     32    2047        7          0
  1  Empty           
  2  Empty           
  3  Empty           
  4  Empty           
  5  Empty           
  6  Empty           
  7  Empty           
  8  Empty           
  9  Empty           
 10  Empty           
 11  Empty           

Excellent! Now reboot the routing engine as indicated after the commit:

root@Llanelli> request system reboot 
Reboot the system ? [yes,no] (no) yes 

                                                                               
*** FINAL System shutdown message from root@Llanelli ***                     

System going down IMMEDIATELY                                                  

                                                                               
Waiting (max 60 seconds) for system process `vnlru' to stop... done
Waiting (max 60 seconds) for system process `bufdaemon' to stop... done
Waiting (max 60 seconds) for system process `syncer' to stop... 
Syncing disks, vnodes remaining... 0 0 0 done
All buffers synced.
Uptime: 1h4m47s
Khelp module "jsocket" can't unload until its refcount drops from 4 to 0.
Rebooting...
cpu_reset: Stopping other CPUs

Once the routing engine has come back up, it whistles to the packet forwarding engine to come hither. Once they're back in sync, we can proceed:

root@Llanelli> show chassis fpc    
                     Temp  CPU Utilization (%)   CPU Utilization (%)  Memory    Utilization (%)
Slot State            (C)  Total  Interrupt      1min   5min   15min  DRAM (MB) Heap     Buffer
  0  Online           Absent   0          0        0      0      0      0         0          0
  1  Empty           
  2  Empty           
  3  Empty           
  4  Empty           
  5  Empty           
  6  Empty           
  7  Empty           
  8  Empty           
  9  Empty           
 10  Empty           
 11  Empty                

Not yet.

root@Llanelli> show chassis fpc    
                     Temp  CPU Utilization (%)   CPU Utilization (%)  Memory    Utilization (%)
Slot State            (C)  Total  Interrupt      1min   5min   15min  DRAM (MB) Heap     Buffer
  0  Online           Testing  40         0       17      4      1    2047        7          0
  1  Empty           
  2  Empty           
  3  Empty           
  4  Empty           
  5  Empty           
  6  Empty           
  7  Empty           
  8  Empty           
  9  Empty           
 10  Empty           
 11  Empty

Job done.

 

Now let's check the pseudowire has come up:

root@Llanelli> show l2circuit connections status    
Layer-2 Circuit Connections:

Legend for connection status (St)   
EI -- encapsulation invalid      NP -- interface h/w not present   
MM -- mtu mismatch               Dn -- down                       
EM -- encapsulation mismatch     VC-Dn -- Virtual circuit Down    
CM -- control-word mismatch      Up -- operational                
VM -- vlan id mismatch           CF -- Call admission control failure
OL -- no outgoing label          IB -- TDM incompatible bitrate 
NC -- intf encaps not CCC/TCC    TM -- TDM misconfiguration 
BK -- Backup Connection          ST -- Standby Connection
CB -- rcvd cell-bundle size bad  SP -- Static Pseudowire
LD -- local site signaled down   RS -- remote site standby
RD -- remote site signaled down  HS -- Hot-standby Connection
XX -- unknown

Legend for interface status  
Up -- operational            
Dn -- down                   
Neighbor: 172.16.99.4 
    Interface                 Type  St     Time last up          # Up trans
    ps0.0(vc 1)               rmt   Up     Sep 20 17:29:48 2019           1
      Remote PE: 172.16.99.4, Negotiated control-word: Yes (Null)
      Incoming label: 16, Outgoing label: 299872
      Negotiated PW status TLV: No
      Local interface: ps0.0, Status: Up, Encapsulation: ETHERNET
      Flow Label Transmit: No, Flow Label Receive: No

It has!

 

Now, let's check that our Cardiff subscribers have found their way over to us:

root@Llanelli> show subscribers 
Total subscribers: 0, Active Subscribers: 0

Oh dear. What have I missed?

 

A quick glance at Cardiff's access port highlights the error of my ways. I've enabled the CVLANs on the pseudowire instead of the SVLAN. Easily corrected:

root@Cardiff> show configuration interfaces ge-0/0/3 | display set 
set interfaces ge-0/0/3 description "Access Provider 1"
set interfaces ge-0/0/3 flexible-vlan-tagging
set interfaces ge-0/0/3 encapsulation flexible-ethernet-services
set interfaces ge-0/0/3 unit 1 encapsulation vlan-ccc
set interfaces ge-0/0/3 unit 1 vlan-id-range 2-10	<--------- CVLANs


root@Cardiff> edit 
Entering configuration mode
root@Cardiff# set interfaces ge-0/0/3 unit 1 vlan-id 101          
root@Cardiff# commit and-quit 
commit complete
Exiting configuration mode

 

Let's check if that worked:

root@Llanelli> show subscribers 
Interface             IP Address/VLAN ID                      User Name                      LS:RI
ps0.3221225472        0x8100.101 0x8100.2                                               default:default      
ps0.3221225473        0x8100.101 0x8100.3                                               default:default      
ps0.3221225474        0x8100.101 0x8100.4                                               default:default      
ps0.3221225475        0x8100.101 0x8100.5                                               default:default      
ps0.3221225476        0x8100.101 0x8100.6                                               default:default      
ps0.3221225477        0x8100.101 0x8100.7                                               default:default      
ps0.3221225478        0x8100.101 0x8100.8                                               default:default      
ps0.3221225479        0x8100.101 0x8100.9                                               default:default      
ps0.3221225480        0x8100.101 0x8100.10                                              default:default      
ps0.3221225472        203.0.113.2                             ps0:101-2                 default:default      
ps0.3221225473        203.0.113.3                             ps0:101-3                 default:default      
ps0.3221225474        203.0.113.4                             ps0:101-4                 default:default      
ps0.3221225475        203.0.113.5                             ps0:101-5                 default:default      
ps0.3221225476        203.0.113.6                             ps0:101-6                 default:default      
ps0.3221225477        203.0.113.7                             ps0:101-7                 default:default      
ps0.3221225478        203.0.113.8                             ps0:101-8                 default:default      
ps0.3221225479        203.0.113.9                             ps0:101-9                 default:default      
ps0.3221225480        203.0.113.10                            ps0:101-10                default:default

Can we ping a subscriber?

root@Llanelli> ping 203.0.113.2 count 3    
PING 203.0.113.2 (203.0.113.2): 56 data bytes
64 bytes from 203.0.113.2: icmp_seq=0 ttl=255 time=34.714 ms
64 bytes from 203.0.113.2: icmp_seq=1 ttl=255 time=40.958 ms
64 bytes from 203.0.113.2: icmp_seq=2 ttl=255 time=41.267 ms

--- 203.0.113.2 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 34.714/38.980/41.267/3.019 ms

Job done!

We've covered the basics of pseudowire headend termination.
In part 2, we'll look at RADIUS and CoS profiles, assuming I can coax vMX to comply :)

 

Versions
vMX: 18.2R1.9
IOS: 15.2(4)M7

Configs
2_Llanelli
2_Swansea
2_Newport
2_Cardiff
2_AccessProvider1