Rewrite all images to Hugo format
This commit is contained in:
156
content/articles/2021-02-27-network.md
Normal file
156
content/articles/2021-02-27-network.md
Normal file
@ -0,0 +1,156 @@
|
||||
---
|
||||
date: "2021-02-27T23:46:12Z"
|
||||
title: IPng Network
|
||||
---
|
||||
|
||||
# Introduction to IPng Networks
|
||||
|
||||
At IPng Networks, we run a modest network with European reach. With our home base
|
||||
in Zurich, Switzerland, we are pretty well connected into the Swiss internet scene.
|
||||
We operate four sites in Zurich, and an additional set of sites in European cities,
|
||||
each of which are described on this post. If you're curious as to how the network
|
||||
runs, you can find two main pieces here: Firstly, the physical parts, where exactly
|
||||
are IPng's routers and switches, what types of kit does the ISP use, and so on.
|
||||
Secondly, the logical parts, what operating systems and configurations are in use.
|
||||
|
||||
## Physical
|
||||
|
||||
### Zurich Metropolitan Area
|
||||
|
||||
The Canton of Zurich, Switzerland is our home-base, and it's where IPng
|
||||
Networks GmbH is registered. The local commercial datacenter scene is dominated
|
||||
by Interxion, NTT and Equinix. The small town of Brüttisellen (zipcode
|
||||
CH-8306), is where our founder lives and, due to the ongoing Corona pandemic,
|
||||
where he works from home.
|
||||
|
||||
{{< image width="400px" float="left" src="/assets/network/zurich-ring.png" alt="Zurich Metro" >}}
|
||||
|
||||
In Brüttisellen, marked with **C**, we have our first two routers,
|
||||
`chbtl0.ipng.ch` and `chbtl1.ipng.ch`, racked in our office. There are only two
|
||||
fiber operators in this town - UPC and Swisscom. The orange trace (**C** to **D**)
|
||||
is a leased line from UPC, which we rent from [Openfactory](https://openfactory.net/)
|
||||
and it gets terminated at Interxion Glattbrugg, where our first router
|
||||
called `chgtg0.ipng.ch` is located. From there, Openfactory rents darkfiber
|
||||
to multiple locations - but notably the dark purple trace (**D** to **E**)
|
||||
that connects from Interxion Glattbrugg to NTT Rümlang, where our second
|
||||
router called `chrma0.ipng.ch` is located.
|
||||
|
||||
We rent a 10G CWDM wave between these two datacenters, directly connecting these
|
||||
two routers. Now, Equinix also has a sizable footprint in Zürich, and
|
||||
operating ZH04 (**B** where we only have passive optical presence) in the
|
||||
Industriekwartier (our local internet exchange [SwissIX](https://swissix.net/)
|
||||
was born in the now defunct Equinix ZH01 office building). From the neighboring
|
||||
building Equinix ZH04, our partner [IP-Max](https://ip-max.net/) rents dark fiber
|
||||
to Equinix ZH05 in the Zurich Allmend area (the light purple trace **B** to **F**),
|
||||
and from there, IP-Max rents dark fiber to NTT Rümlang again (**F** to **E**),
|
||||
completing the ring. We rent a 10G circuit on that path, to redundantly connect our
|
||||
routers `chgtg0` and `chrma0`. If at any time we'd need to connect partners
|
||||
or customers, we can do so at a moment's notice, as rackspace is available in
|
||||
all Equinix sites for IPng Networks.
|
||||
|
||||
The green link (**D** to **B**) is a 10G carrier ethernet circuit between Interxion,
|
||||
over the light purple path (**B** to **A**) on its last mile to Albisrieden, where
|
||||
we built a very small colocation site, which you can read about in more detail in our
|
||||
[informational post]({% post_url 2022-02-24-colo %}) - the colo is open for private
|
||||
individuals and small businesses ([contact](/s/contact/) us for details!).
|
||||
|
||||
### European Ring
|
||||
|
||||
At IPng, we are strong believers in a free and open Internet. Having seen
|
||||
the shakeout of internet backbone providers over the last two decades, it
|
||||
seems to be a race to the bottom, with mergers, acquisitions and takeovers
|
||||
of datacenters and network carriers. Prices are going lower, and small fish
|
||||
traffic (let's be honest, IPng Networks is definitely a small provider), to
|
||||
the point that purchasing IP transit is cheaper than connecting to local
|
||||
Internet exchange points. We've decided specifically to go the extra mile,
|
||||
quite literally, and plot a path to several continental european internet
|
||||
hubs.
|
||||
|
||||
{{< image width="400px" float="left" src="/assets/network/european-ring.png" alt="European Ring" >}}
|
||||
|
||||
***Frankfurt*** - Connected from NTT's datacenter at Rümlang (Zurich) with
|
||||
a first 10G circuit, and from Interxion's datacenter at Glattbrugg (Zurich)
|
||||
with a second 10G circuit, this is our first hop into the world. Here, we
|
||||
connect to [DE-CIX](https://de-cix.net/) from Equinix FR5 at the Kleyerstrasse.
|
||||
More details in our post [IPng Arrives in Frankfurt]({% post_url 2021-05-17-frankfurt %}).
|
||||
|
||||
***Amsterdam*** - The Amsterdam Science Park is where European Internet was born.
|
||||
[NIKHEF](https://www.nikhef.nl/) is where we rent rackspace that connects with a 10G
|
||||
circuit to Frankfurt, and a 10G circuit onwards towards Lille. We connect to
|
||||
[Speed-IX](https://speed-ix.net/), [LSIX](https://lsix.net/), [NL-IX](https://nl-ix.net),
|
||||
and an exchange point we help run called [FrysIX](https://www.frys-ix.net/).
|
||||
More details in our post [IPng Arrives in Amsterdam]({% post_url 2021-05-26-amsterdam %}).
|
||||
|
||||
***Lille*** - [IP-Max](https://ip-max.net/) does lots of business in this
|
||||
region, with presence in both local datacenters here, one in Lille and one in
|
||||
Anzin. IPng has a point of presence here too, at the [CIV1](https://www.civ.fr/)
|
||||
facility, with a northbound 10G circuit to Amsterdam, and a southbound 10G
|
||||
circuit to Paris. Here, we connect to [LillIX](https://lillix.fr/).
|
||||
More details in our post [IPng Arrives in Lille]({% post_url 2021-05-28-lille %}).
|
||||
|
||||
***Paris*** - Where two large facilities are placed back-to-back in the middle
|
||||
of the city, originally Telehouse TH2, with a new facility at Léon Frot,
|
||||
where we pick up a 10G circuit from Lille and further on the ring with a 10G
|
||||
circuit to Geneva. Here, we connect to [FranceIX](https://franceix.net).
|
||||
More details in our post [IPng Arrives in Paris]({% post_url 2021-06-01-paris %}).
|
||||
|
||||
***Geneva*** - The home-base of [IP-Max](https://ip-max.net) is where we close
|
||||
our ring. From Paris, IP-Max has two redundant paths back to Switzerland, the first
|
||||
being a DWDM link from to Zurich, and the second being a DWDM link to Lyon and
|
||||
then into Geneva. Here, at [SafeHost](https://safehost.com/) in Plan les Ouates,
|
||||
is where we have our fourth Swiss point of presence, with a connection to our very
|
||||
own [Free-IX](https://free-ix.net/) and a 10G circuit to Interxion at Glattbrugg
|
||||
(Zurich), and of course to Paris.
|
||||
More details in our post [IPng Arrives in Geneva]({% post_url 2021-07-03-geneva %}).
|
||||
|
||||
## Logical
|
||||
|
||||
As a small operator, we'd love to be able to boast the newest Juniper [PTX10016](https://www.juniper.net/us/en/products/routers/ptx-series.html)
|
||||
routers but we neither have the rack space, the power budget, and to be
|
||||
perfectly honest, the monetary budget to run these at IPng Networks. But it
|
||||
turns out, we know a fair bit about hardware silicon, architecture and the
|
||||
controlplane software running on commercial routers.
|
||||
|
||||
We've decided to go a different route. In our opinion, at speeds under 100Gbit,
|
||||
it's perfectly viable to use software routers on off-the-shelf hardware, notably
|
||||
Intel network cards and CPUs, notably those that have support for the
|
||||
[Dataplane Development Kit](https://dpdk.org/) (aka DPDK), which offers libraries
|
||||
to accelerate packet processing workloads, which turn ordinary servers into very
|
||||
performant routers. Two notable applications are [VPP](https://fd.io/) and
|
||||
[Danos](https://danosproject.org).
|
||||
|
||||
### VPP
|
||||
|
||||
VPP originally comes from the house of Cisco [[ref](https://www.cisco.com/c/dam/m/en_us/service-provider/ciscoknowledgenetwork/files/592_05_25-16-fdio_is_the_future_of_software_dataplanes-v2.pdf)] and looks quite a bit like
|
||||
the commercial ASR9k platform. In development since 2002, VPP is production
|
||||
code currently running in shipping products. It runs in user space on multiple
|
||||
architectures including x86, ARM, and Power architectures on both x86 servers
|
||||
and embedded devices. The design of VPP is hardware, kernel, and deployment
|
||||
(bare metal, VM, container) agnostic. It runs completely in userspace.
|
||||
|
||||
We've contributed a little bit to the Control Plane abstraction [[ref](https://docs.fd.io/vpp/21.06/dc/d2e/clicmd_src_plugins_linux-cp.html)],
|
||||
which allows users to combine the throughput of a dataplane with usual routing
|
||||
software like [Bird](https://bird.network.cz/) or [FRR](https://frrouting.org/).
|
||||
We've been running it in production since December 2020 on `chbtl1.ipng.ch`.
|
||||
It's our ultimate goal to run VPP and Linux Control Plane on the entire network,
|
||||
as the design and architecture really resonates with us as software and systems
|
||||
engineers.
|
||||
|
||||
### DANOS
|
||||
|
||||
The Disaggregated Network Operating System (DANOS) project originally comes
|
||||
from AT&T’s “dNOS” software framework and provides an open, cost-effective and
|
||||
flexible alternative to traditional networking equipment. As part of The Linux
|
||||
Foundation, it now incorporates contributions from complementary open source
|
||||
communities in building a standardized distributed Network Operating System (NOS)
|
||||
to speed the adoption and use of white boxes in a service provider’s
|
||||
infrastructure.
|
||||
|
||||
We've been using DANOS since its first release in August 2019, and it's
|
||||
currently our routing platform of choice -- it combines the sheer speed of
|
||||
DPDK with a [Vyatta](https://en.wikipedia.org/wiki/Vyatta) command line
|
||||
interface. As an appliance, care was taken to complete the _whole package_,
|
||||
with SNMP, YANG interface, image and upgrade management, interface monitoring
|
||||
with wireshark semantics, et cetera. Performing easily at wire speed 10G
|
||||
workloads (including 64byte ethernet frames), and being completely open source,
|
||||
it fits very well with our philosophy of an open and free internet.
|
||||
Reference in New Issue
Block a user