diff --git a/content/articles/2016-10-07-fiber7-litexchange.md b/content/articles/2016-10-07-fiber7-litexchange.md new file mode 100644 index 0000000..8d313a6 --- /dev/null +++ b/content/articles/2016-10-07-fiber7-litexchange.md @@ -0,0 +1,390 @@ +--- +date: "2016-10-13" +title: Fiber7 on LiteXchange +aliases: +- /s/articles/2016/10/13/fiber7-litexchange.html +--- + + +* Author: Pim van Pelt <[pim@ipng.nl](mailto:pim@ipng.nl)> +* Reviewed: Fredy Kuenzler <[kuenzler@init7.net](mailto:kuenzler@init7.net)> +* Status: Draft - Review - **Approved** + + +## Introduction + +{{< image width="15em" float="right" src="/assets/fiber7-litexchange/init7-logo-rgb.jpg" alt="Init7 logo" >}} + +In a pilot of the Fiber7 product on the LiteXchange platform, the author took service to vet the +product stability and quality. The pilot ran from 2016-09-25 to 2016-10-12, in which the Fiber7 +connection was used exclusively by the author in their home internet connection, both for IPv4/IPv6 +service as well as IP Television (via Init7) and IP Telephony (via a third party provider). + +## Executive Summary + +Fiber7 via ‘direct connect’ on the LiteXchange platform works as expected and very satisfactory, +including native IPv6, which was made available for this pilot. Throughput, latency and jitter are +superior due to the direct fiber connection, and significantly better than existing connections, +exceeding expectations compared to competing FTTH offerings that use customer premise equipment. +IPTV worked correctly with multiple STB devices. + +## Detailed findings + +### Architecture + +The author currently has a subscription via EasyZone, an Init7 subsidiary, with gigabit ethernet +symmetric connectivity. The EasyZone product delivers service via the LiteXchange platform, which is +an L2 broker, offering end users a choice of multiple internet providers +[[site](http://litexchange.ch/services/default.aspx?servicecategory=isp)]. An ONT is supplied, an +ISP managed device that takes the fiber connection and exposes service via one of four gigabit +ethernet copper ports. The Fiber7 product delivers via LiteXchange a direct fiber connection without +the ONT. Fiber7 offers a range of termination options, including plugging the fiber into a provided +CPE (AVM Fritz!Box [[vendor](https://en.avm.de/products/fritzbox/fritzbox-5490/)] or alternatively +MikroTik RB2011UiAS [[vendor](https://www.mikrotik-store.eu/en/MikroTik-RB2011UiAS-2HnD-IN)]), a +media converter (TP-Link +[[vendor](http://www.tp-link.com/en/products/details/cat-4792_MC220L.html)]), or simply an SFP +(Flexoptix +[[vendor](https://www.flexoptix.net/en/sfp-bidi-transceiver-1-gigabit-sm-tx1310nm-rx1550nm-10km-12db-ddm-dom.html)]) +to use in customer provided switch/router infrastructure. + +#### Architecture (details) + +For this pilot, we chose a barebones connection type, consisting of a bidirectional SFP (Flexoptix +[[vendor](https://www.flexoptix.net/en/sfp-bidi-transceiver-1-gigabit-sm-tx1310nm-rx1550nm-10km-12db-ddm-dom.html)]) +directly terminating the FTTH connection from OTO position 1 into our own managed switch (Unifi +US-24-500W [[vendor](https://www.ubnt.com/unifi-switching/unifi-switch/)]). The L3 routers used are +a pair of PC routers (PC Engines APU2 [[vendor](http://pcengines.ch/apu2b4.htm)]), running Linux. +They are configured in CARP failover on egress (to Fiber7) and ingress (to local network). + +Configuring IPv4 address on egress interface is done via DHCP - initially, DHCPv6 was not active on +LiteXchange, so a local tunnelbroker (SixXS, hosted at Init7) was used. Within one week, the +engineers at Init7 informed me that DHCPv6 was ready, and it worked spotlessly after configuring it +to request an NA and a /48 PD, and bumping `accept_ra=2` on the egress interface (note: this allows +forwarding while at the same time accepting router advertisements). + +Additional details of the L3 connection: + +1. The routers operate an L2VPN to a third party provider (IP-Max, AS25091) which routes + `194.1.163.32/27` via eBGP using GRE. The MSS on this tunnel is clamped to 1436 (from 1460) to allow + for encapsulating IPv4 and GRE. AS13030 and AS25091 meet at CIXP in Geneva, with a round trip time + of 4.2ms. +1. The routers operate an IPv6 tunnel to a common tunnel provider (SixXS, AS13030), which routes + `2001:1620:fb6::/48` via AICCU using SIT to the active router. The MTU is set to 1440 bytes to allow + encapsulating IPv6 in IPv4. Note that the Fiber7 connection via LiteXchange provides native IPv6 as + well, so this tunnel is used only via a secondary IPv4 uplink. +1. The routers operate native IPv6 -- with DHCPv6, a /128 address and a /48 delegated prefix are + obtained. This prefix is stable due to the use of DUID client identification. The default gateway is + obtained via RS/RA. For IPv6, reversed DNS delegation for fixed DUID/PD delegation is provided. + +It is worth pointing out the very low technical entry barrier to both IPv4 and IPv6. The termination +is principally plug and play. An end user can use standard issue DHCP for IPv4 and RA/RS for IPv6. +DHCPv6 is not widely used - but similarly the /48 prefix acquisition is hasslefree. + +Failover between the routers is managed by a script that swaps the CARP +[[source](https://ucarp.wordpress.com/)] master to the standby PC router (automatically in case of +CARP heartbeat timeouts; or manually in case of maintenance), ensuring the L2VPN, DHCP client, and +IPv6 tunnels are running on the active machine. + +Policy based routing [[source](http://tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.rpdb.simple.html)] is +used to separate Fiber7/SixXS and L2VPN/IP-Max routing domains. Routing tables are maintained with a +popular open source routing platform called BiRD [[source](http://bird.network.cz/)], OSPF between +the PC routers, and eBGP with the third party provider. + +### IP Television + +In this pilot the author was sent an IPTV device (Amino Aminet A140 +[[vendor](https://www.aminocom.com/products/amino-view/client-devices/a140)]), which operates with +IPv4. The device acquires video streams using IPv4 multicast. Setting this up was straightforward, +using an IGMP Proxy [[github](https://github.com/pali/igmpproxy)] also used in commercial CPEs. The +IGMP Proxy was configured on the PC routers. + +With two such Amino IPTV devices, tuning in to SRF1 and SRF2 (both HD channels), a stream of UDP +from multicast servers within the Init7 network was started. At the time of writing, SRF1 is on +multicast address `239.44.0.77` port 5000; SRF2 is on multicast address `239.44.0.78` port 5000; +both coming from source `109.202.223.18` port 5000. Average bandwidth was 13.0Mbit/s with a peak of +17.1Mbit/s per HD stream, and 4.2Mbit/s with a peak of 5.3Mbit/s per SD stream. + +Multiple Amino IPTV devices in multiple backend VLANs can be used at the same time: + +``` +$ ip mroute | grep 239.44.0 +(109.202.223.18, 239.44.0.77) Iif: eth0.9 Oifs: eth0 +(109.202.223.18, 239.44.0.78) Iif: eth0.9 Oifs: eth0.2 +``` + +A list of channels available on the EasyZone IPTV provider (a subsidiary of Init7) can be found on +their website [[source](https://www.easyzone.ch/tv/sender)]. + +#### Netflix: IPv6 + +Worth noting during the pilot is that Netflix, a popular online television streaming service +[[website](http://www.netflix.com/)], was served from within the Init7 network as well. Connections +were observed from host `netflix-cache-1.init7.net` (AS13030) via IPv6, which is impressive. + +UHD (4K) streaming is also available with Netflix - the device used to test this (Samsung JU7080 +Series 7 [[vendor](http://www.samsung.com/ch/consumer/tv-av/tv/uhd-tv/UE65JU7080TXZG)]) has a native +client but it does not support IPv6, as such the traffic was observed from host +`ipv4_1.cxl0.c117.ams001.ix.nflxvideo.net` in AS2906 located in the Netherlands. + +In both cases (local within Init7 and remote to AS2906), Netflix streaming was free of interruptions +and great quality. + +### Test Results +#### Throughput + +A throughput test was started on September 27, lasting 12 hours, from the active PC router to a +machine in the Init7 network [[caveat](#caveats)]: + +``` +$ traceroute to chzrh02.sixxs.net (213.144.148.74), 30 hops max, 60 byte packets + 1 77.109.172.1.easyzone.ch (77.109.172.1) 0.755 ms 0.813 ms 0.803 ms + 2 r1zrh2.core.init7.net (77.109.183.61) 0.379 ms 0.373 ms 0.377 ms + 3 r1zrh1.core.init7.net (77.109.128.241) 0.477 ms 0.429 ms 0.397 ms + 4 r1zlz1.core.init7.net (77.109.128.210) 8.810 ms 8.783 ms 8.738 ms + 5 chzrh02.sixxs.net (213.144.148.74) 0.545 ms 0.490 ms 0.469 ms +``` + +Using a popular network bandwidth tool (iperf [[source](https://iperf.fr/)]), IPv4 bandwidth was +measured for 10 minutes each, both upstream (from the PC router to a machine in the init7 network: +891Mbit), and downstream (from the init7 machine to the PC router: 895Mbit). In IPv6, the results +were similar (771Mbit upstream, and 831Mbit downstream). + +A standard internet test was performed (Speedtest.net, using Init7) +[[link](http://beta.speedtest.net); [results](http://beta.speedtest.net/result/5667723165)], +yielding 925Mbit downstream and 893Mbit upstream. In addition to the direct link, the author’s +L2VPN connection to a third party provider was tested (Speedtest.net, using Init7) +[[link](http://beta.speedtest.net/); [results](http://beta.speedtest.net/result/5668135633)], +yielding 609Mbit downstream and 578Mbit upstream. The L2VPN throughput regression is explained by +tunneling en/decapsulation. + +#### Latency + +Latency to Google was tested -- Init7 AS13030 and Google AS15169 meet in Zurich, with very low +latency. IPv6 was tested twice (once via SixXS tunnelbroker tunnel, and once natively when it was +available). Tunneled IPv6 reports slightly elevated latency due tunneling to an on-net IPv6 +tunnelbroker[[caveat](#caveats)]. Native IPv6 reports equivalent latency to IPv4. + +``` +IPv4 google.com ping statistics: +10 packets transmitted, 10 received, 0% packet loss, time 9002ms +rtt min/avg/max/mdev = 0.566/0.579/0.594/0.025 ms + +Native IPv6 google.com ping6 statistics: +10 packets transmitted, 10 received, 0% packet loss, time 9015ms +rtt min/avg/max/mdev = 0.705/0.771/0.828/0.043 ms + +Tunneled IPv6 google.com ping6 statistics: +10 packets transmitted, 10 received, 0% packet loss, time 9011ms +rtt min/avg/max/mdev = 1.154/1.451/2.206/0.276 ms +``` +### Caveats + +IPv6 was initially not natively available on this connection. IPv6 was tunneled via +chzrh02.sixxs.net (on-net at AS13030). The IPv6 server endpoint runs on a virtualized platform, with +slightly less than bare-bones throughput. Shortly thereafter, native IPv6 was configured on the +Fiber7 product via the LiteXchange platform. + +Each OTO delivered by the city of Wangen-Brüttisellen +[[site](http://www.werkewb.ch/cms/?page_id=52)] holds four simplex single mode fibers. The first +position of the OTO is typically used to connect the ONT and subsequently the enduser internet +connection (in the author’s case an EasyZone connection). The other three positions on the OTO are +reserved for future use. For some reason unknown to the author, the Fiber7 connection was installed +on a second OTO, again with four simplex single mode fibers. The first position of the second OTO +was used to provide the Fiber7 internet connection. + +## Appendix + +### Appendix 1 - Terminology + +**Term** | **Description** +-------- | --------------- +ONT | **optical network terminal** - The ONT converts fiber-optic light signals to copper based electric signals, usually Ethernet. +OTO | **optical telecommunication outlet** - The OTO is a fiber optic outlet that allows easy termination of cables in an office and home environment. Installed OTOs are referred to by their OTO-ID. +CARP | **common address redundancy protocol** - Its purpose is to allow multiple hosts on the same network segment to share an IP address. CARP is a secure, free alternative to the Virtual Router Redundancy Protocol (VRRP) and the Hot Standby Router Protocol (HSRP). +SIT | **simple internet transition** - Its purpose is to interconnect isolated IPv6 networks, located in global IPv4 Internet via tunnels. +STB | **set top box** - a device that enables a television set to become a user interface to the Internet and also enables a television set to receive and decode digital television (DTV) broadcasts. +GRE | **generic routing encapsulation** - a tunneling protocol developed by Cisco Systems that can encapsulate a wide variety of network layer protocols inside virtual point-to-point links over an Internet Protocol network. +L2VPN | **layer2 virtual private network** - a service that emulates a switched Ethernet (V)LAN across a pseudo-wire (typically an IP tunnel) +DHCP | **dynamic host configuration protocol** - an IPv4 network protocol that enables a server to automatically assign an IP address to a computer from a defined range of numbers. +DHCP6 | **Dynamic host configuration protocol: prefix delegation** - an IPv6 network protocol that enables a server to automatically assign network prefixes to a customer from a defined range of numbers. +NDP NS/NA | **neighbor discovery protocol: neighbor solicitation / advertisement** - an ipv6 specific protocol to discover and judge reachability of other nodes on a shared link. +NDP RS/RA | **neighbor discovery protocol: router solicitation / advertisement** - an ipv6 specific protocol to discover and install local address and gateway information. + +### Appendix 2 - Supporting data + +#### Bandwidth with Speedtest + +Directly on Fiber7: [speedtest](http://beta.speedtest.net/result/5667723165) + +{{< image width="17em" src="/assets/fiber7-litexchange/image0.png" alt="Speedtest Fiber7" >}} + +GRE via IP-Max: [speedtest](http://beta.speedtest.net/result/5668135633) + +{{< image width="17em" src="/assets/fiber7-litexchange/image1.png" alt="Speedtest IP-Max" >}} + +#### Bandwidth with Iperf upstream + +``` +(AS13030 IPv4) $ iperf -t 600 -P 4 -i 60 -l 1M -m -c chzrh02.sixxs.net +------------------------------------------------------------ +Client connecting to chzrh02.sixxs.net, TCP port 5001 +TCP window size: 85.0 KByte (default) +------------------------------------------------------------ +[ 3] local 77.109.173.198 port 41199 connected with 213.144.148.74 port 5001 +[ ID] Interval Transfer Bandwidth +[ 3] 0.0-60.0 sec 6.23 GBytes 892 Mbits/sec +[ 3] 60.0-120.0 sec 6.21 GBytes 889 Mbits/sec +[ 3] 120.0-180.0 sec 6.22 GBytes 891 Mbits/sec +[ 3] 180.0-240.0 sec 6.25 GBytes 894 Mbits/sec +[ 3] 240.0-300.0 sec 6.25 GBytes 894 Mbits/sec +[ 3] 300.0-360.0 sec 6.23 GBytes 892 Mbits/sec +[ 3] 360.0-420.0 sec 6.22 GBytes 890 Mbits/sec +[ 3] 420.0-480.0 sec 6.20 GBytes 888 Mbits/sec +[ 3] 480.0-540.0 sec 6.21 GBytes 889 Mbits/sec +[ 3] 540.0-600.0 sec 6.18 GBytes 885 Mbits/sec +[ 3] 0.0-600.0 sec 62.2 GBytes 891 Mbits/sec +[ 3] MSS size 1448 bytes (MTU 1500 bytes, ethernet) +``` + +``` +(AS25091 IPv6) $ iperf -V -t 600 -P 4 -i 60 -l 1M -m -c charb02.paphosting.net +------------------------------------------------------------ +Client connecting to charb02.paphosting.net, TCP port 5001 +TCP window size: 85.0 KByte (default) +------------------------------------------------------------ +[ 3] local 2a02:168:2000:4b:469:a025:5293:84ad port 45044 connected with 2a02:2528:503:1::83 port 5001 +[ ID] Interval Transfer Bandwidth +[ 3] 0.0-60.0 sec 5.22 GBytes 748 Mbits/sec +[ 3] 60.0-120.0 sec 5.52 GBytes 791 Mbits/sec +[ 3] 120.0-180.0 sec 5.67 GBytes 811 Mbits/sec +[ 3] 180.0-240.0 sec 4.86 GBytes 696 Mbits/sec +[ 3] 240.0-300.0 sec 4.85 GBytes 695 Mbits/sec +[ 3] 300.0-360.0 sec 5.44 GBytes 779 Mbits/sec +[ 3] 360.0-420.0 sec 5.97 GBytes 855 Mbits/sec +[ 3] 420.0-480.0 sec 5.54 GBytes 792 Mbits/sec +[ 3] 480.0-540.0 sec 5.17 GBytes 739 Mbits/sec +[ 3] 540.0-600.0 sec 5.63 GBytes 806 Mbits/sec +[ 3] 0.0-600.0 sec 53.9 GBytes 771 Mbits/sec +[ 3] MSS size 1428 bytes (MTU 1500 bytes, ethernet) +``` + +#### Bandwidth with Iperf downstream + +``` +(AS13030 IPv4) $ iperf -t 600 -P 4 -i 60 -l 1M -m -c 77.109.173.198 +------------------------------------------------------------ +Client connecting to 77.109.173.198, TCP port 5001 +TCP window size: 85.0 KByte (default) +------------------------------------------------------------ +[ 3] local 213.144.148.74 port 56642 connected with 77.109.173.198 port 5001 +[ ID] Interval Transfer Bandwidth +[ 3] 0.0-60.0 sec 6.22 GBytes 891 Mbits/sec +[ 3] 60.0-120.0 sec 6.25 GBytes 895 Mbits/sec +[ 3] 120.0-180.0 sec 6.24 GBytes 894 Mbits/sec +[ 3] 180.0-240.0 sec 6.23 GBytes 891 Mbits/sec +[ 3] 240.0-300.0 sec 6.21 GBytes 889 Mbits/sec +[ 3] 300.0-360.0 sec 6.23 GBytes 892 Mbits/sec +[ 3] 360.0-420.0 sec 6.27 GBytes 898 Mbits/sec +[ 3] 420.0-480.0 sec 6.25 GBytes 895 Mbits/sec +[ 3] 480.0-540.0 sec 6.27 GBytes 897 Mbits/sec +[ 3] 540.0-600.0 sec 6.26 GBytes 896 Mbits/sec +[ 3] 0.0-600.0 sec 62.4 GBytes 894 Mbits/sec +[ 3] MSS size 1448 bytes (MTU 1500 bytes, ethernet) +``` + +``` +(AS25091 IPv6) $ iperf -V -t 600 -P 4 -i 60 -l 1M -m -c 2a02:168:2000:4b:20d:b9ff:fe41:94c +------------------------------------------------------------ +Client connecting to 2a02:168:2000:4b:20d:b9ff:fe41:94c, TCP port 5001 +TCP window size: 85.0 KByte (default) +------------------------------------------------------------ +[ 3] local 2a02:2528:503:1::83 port 43499 connected with 2a02:168:2000:4b:20d:b9ff:fe41:94c port 5001 +[ ID] Interval Transfer Bandwidth +[ 3] 0.0-60.0 sec 5.68 GBytes 813 Mbits/sec +[ 3] 60.0-120.0 sec 5.50 GBytes 787 Mbits/sec +[ 3] 120.0-180.0 sec 5.75 GBytes 823 Mbits/sec +[ 3] 180.0-240.0 sec 6.06 GBytes 868 Mbits/sec +[ 3] 240.0-300.0 sec 5.96 GBytes 853 Mbits/sec +[ 3] 300.0-360.0 sec 5.95 GBytes 852 Mbits/sec +[ 3] 360.0-420.0 sec 5.99 GBytes 858 Mbits/sec +[ 3] 420.0-480.0 sec 5.56 GBytes 796 Mbits/sec +[ 3] 480.0-540.0 sec 6.10 GBytes 874 Mbits/sec +[ 3] 540.0-600.0 sec 6.21 GBytes 889 Mbits/sec +[ 3] 0.0-600.0 sec 58.8 GBytes 841 Mbits/sec +[ 3] MSS size 1428 bytes (MTU 1500 bytes, ethernet) +``` + +### Appendix 3 - Configuration files + +#### DHCPv6 Configuration + +Two IPv6 access mechanisms were used. Firstly, IPv6 was acquired via SixXS +[[site](https://www.sixxs.net/)] who are present at Init7. After it was made available +(approximately one week into the pilot), standard issue WIDE DHCPv6 client was used with the +following configuration file: + +``` +$ cat /etc/wide-dhcpv6/dhcpc.conf + +interface eth0.9 { # interface VLAN9 - Fiber7 + send ia-na 1; + send ia-pd 1; + script "/etc/wide-dhcpv6/dhcp6c-script"; +}; + +id-assoc pd 1 { + prefix ::/48 infinity; + + prefix-interface lo { + sla-id 0; + ifid 1; + sla-len 16; + }; + + # Test interface + prefix-interface eth1 { + sla-id 4096; + ifid 1; + sla-len 16; + }; +}; + +id-assoc na 1 { + # id-assoc for eth0.9 +}; +``` + +#### IGMP Proxy Configuration + +Taking IGMPProxy from [github](https://github.com/pali/igmpproxy) and the following configuration +file, IPTV worked reliably throughout the pilot: + +``` +$ cat /etc/igmpproxy.conf +##------------------------------------------------------ +## Enable Quickleave mode (Sends Leave instantly) +##------------------------------------------------------ +quickleave + +##------------------------------------------------------ +## Configuration for Upstream Interface +##------------------------------------------------------ +phyint eth0.9 upstream ratelimit 0 threshold 1 + altnet 109.202.223.0/24 + altnet 192.168.2.0/23 + altnet 239.44.0.0/16 + +##------------------------------------------------------ +## Configuration for Downstream Interface +##------------------------------------------------------ +phyint eth0 downstream ratelimit 0 threshold 1 +phyint eth0.2 downstream ratelimit 0 threshold 1 + +##------------------------------------------------------ +## Configuration for Disabled Interface +##------------------------------------------------------ +phyint eth0.3 disabled # Guest +phyint eth0.4 disabled # IPCam +phyint eth0.5 disabled # BIT +phyint eth0.6 disabled # IP-Max +``` diff --git a/content/articles/_index.md b/content/articles/_index.md new file mode 100644 index 0000000..003de6e --- /dev/null +++ b/content/articles/_index.md @@ -0,0 +1,9 @@ +--- +title: "IPng Networks Articles" +date: 2024-07-28 +menu: + main: + name: "Articles" + weight: 50 +url: "/s/articles" +--- diff --git a/static/assets/fiber7-litexchange/init7-logo-rgb.jpg b/static/assets/fiber7-litexchange/init7-logo-rgb.jpg new file mode 100644 index 0000000..e6c64cd --- /dev/null +++ b/static/assets/fiber7-litexchange/init7-logo-rgb.jpg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:edfc100f8ac3d9f46562a9660d3e5c67d8d512692c8557793947e71213cd0cf4 +size 20212