318 lines
20 KiB
Markdown
318 lines
20 KiB
Markdown
---
|
|
date: "2021-07-26T11:16:44Z"
|
|
title: A story of a Bucketlist
|
|
aliases:
|
|
- /s/articles/2021/07/26/bucketlist.html
|
|
---
|
|
|
|
## Introduction
|
|
|
|
Many people maintain what is called a Bucketlist, a list of things they
|
|
wish to do before they _kick the bucket_. I have one also, and although
|
|
most of the items on that list are earthly and more on the emotional
|
|
realm, and private, there is one specific thing that I have wanted to
|
|
do ever since I first started working in IT in 1998: Peer at the
|
|
Amsterdam Internet Exchange.
|
|
|
|
This post details striking this particular item off my bucketlist. It's
|
|
both indulgent, humblebraggy and incredibly nerdy and it talks a bit about
|
|
mental health. If those are trigger words for you, skip ahead to another
|
|
post, like my series on VPP ;-)
|
|
|
|
|
|
## 1998 - Netherlands
|
|
|
|
{{< image width="300px" float="right" src="/assets/bucketlist/bucketlist-ede.png" alt="The Kelvinstraat" >}}
|
|
|
|
I started working when I was still at the TU/Eindhoven, and after a great
|
|
sysadmin job at Radar Internet, which became Track and was sold to Wegener
|
|
Arcade, I turned towards networking. After building Freeler (the first
|
|
_free_ ISP in the Netherlands) with Adrianus and co, and a small stint at
|
|
their primary uplink Intouch with Rager (rest in peace, Brother), I joined
|
|
BIT (AS12859) from 2000 to 2006, and it was here where I developed a true
|
|
passion for that which makes the internet 'tick': routing protocols.
|
|
|
|
I was secretly jealous that BIT could afford Junipers, F5 loadbalancers and
|
|
large Cisco switches, and I loved working with and on those machines. BIT
|
|
had a reseller relationship with BBNed, and were able to directly connect
|
|
ADSL modems into their own infrastructure, and as such I could afford to get
|
|
myself a subnet from 213.154.224.0/19 routed to my house in Wageningen. It
|
|
was where I had a half-19" rack in a clothing closet in our guest bedroom,
|
|
and it was there that I decided: I want to eventually participate in the
|
|
BGP world and peer at AMS-IX (the only exchange at the time, NLIX was just
|
|
starting up, thanks again, Jan!).
|
|
|
|
Pictured to the right was my first contribution to AS12859 - deploying a
|
|
CWDM ring from Ede to Amsterdam and upgrading our backbone from an ATM E3
|
|
(34Mbit) and POS STM1 (155Mbit) leased line to Gigabit Ethernet on Juniper
|
|
M5 routers, this was in 2001, 20 years ago almost to the month.
|
|
|
|
## 2008 - Switzerland
|
|
|
|
{{< image width="300px" float="right" src="/assets/bucketlist/bucketlist-dk2.png" alt="The Cavern" >}}
|
|
|
|
Fast forward to 2006, I moved to Switzerland and while I remained friendly
|
|
with NLNOG and SWINOG (and a few other network operator groups), I did not
|
|
pursue the whole internet exchange thing. I had operated networks for the
|
|
greater part of a decade, and with my full time job, I spent a lot of time
|
|
learning how to be a good _Site Reliability Engineer_. I still had three /24
|
|
PI space blocks, used for different purposes in the past, but I was much
|
|
more comfortable letting the "real" ISPs announce them - in my case AS25091
|
|
[IP-Max](https://ip-max.net/) (thanks, Fred!) and AS13030 [Init7](https://init7.net/)
|
|
(thanks, Fredy!) and AS12859 [BIT](https://bit.nl/) (thanks, Michel!). I
|
|
cannot remember any meaningful downtime in any of those operators, of course
|
|
there is always some, but due to the N+2 nature of my network deployment, I
|
|
don't think any global downtime for my internet presence has ever occured.
|
|
|
|
It's not a coincidence that even Google for the longest time used my website
|
|
at [SixXS](https://sixxs.net/) for their own monitoring, now _that_ is
|
|
cool. Although Jeroen and I did decide to retire the SixXS project (see my
|
|
[Sunset]({{< ref "2017-03-01-sixxs-sunset" >}}) article on why), the website
|
|
is still up and served off of three distinct networks, because I have to stay
|
|
true to the SRE life.
|
|
|
|
Pictured to the right was one of the two racks at Deltalis DK2, a datacenter
|
|
built into a mountain in the heart of the swiss Alps. Classic edge/core/border
|
|
approach with (at the time) state of the art Cisco 7600 routers. One of these
|
|
is destined to become my nightstand at some point, this was in 2013, which
|
|
is now (almost) 10 years ago.
|
|
|
|
### Corona Motivation
|
|
|
|
My buddy Fred from IP-Max would regularly ask me "why don't you just announce
|
|
your /24 yourself?" It'd be fun, he said. In 2007, we registered a /24 PI for
|
|
SixXS, and I was always quite content to let _him_ handle the routing. But it
|
|
started to itch and a neighbor of mine inadvertently reminded me of this itch
|
|
(thanks, Max) by asking me if I was interested to share an L2 ethernet link
|
|
with him from our place in Brüttisellen to one of the datacenters in
|
|
Zürich, a distance of about 7km as the photons fly.
|
|
|
|
{{< image width="110px" float="left" src="/assets/bucketlist/bucketlist-corona.png" alt="The Virus" >}}
|
|
|
|
I could not resist any longer. I was working long(er) than average hours due
|
|
to the work-from-home situation: you easily chop off 45-60min of commute each
|
|
day, but I noticed myself spending it in more meetings instead of in the train.
|
|
I was slowly getting into a bad state, and my motivation was very low. I wanted
|
|
to do something other than sleep-eat-work-sleep and even my jogging went to an
|
|
all time minimum. I had very low emotional energy.
|
|
|
|
To put my mind off of things, I decided to reattach to my networking roots in
|
|
a few ways: one was to build an AS and operate it for a while (maybe a few years
|
|
until I get bored of it, and then re-parent my IP space to some friendly ISP,
|
|
or who knows, cash in rich and sell my IP space to the highest bidder!), and
|
|
the other was to continue my desire to have a competent replacement for silicon
|
|
now that CPUs-of-now are just as fast as ASICs-of-then, and contribute to DANOS
|
|
and VPP.
|
|
|
|
#### Step 1. Build a basement ISP
|
|
|
|
So getting a PC with Bird, or in my case, an appliance called [DANOS](https://danosproject.org/)
|
|
which uses [DPDK](https://dpdk.org/) to implement wirespeed routing on commodity
|
|
x86/64 hardware. So I happily announced my /24 and /48 from NTT's datacenter,
|
|
connected to the local internet exchange [Swissix](https://swissix.ch/) and
|
|
rented an L2 circuit to my house via [Openfactory](https://openfactory.net/). Also,
|
|
I showed that a simple Supermicro (for example [SYS-5018D-FN8T](https://www.supermicro.com/products/system/1u/5018/SYS-5018D-FN8T.cfm))
|
|
could easily handle line rate 64 byte frames in both directions on its TenGigabit
|
|
interfaces, that's 29Mpps, and still have a responsive IPMI serial console. It
|
|
reminded me of the early days of Juniper martini class routers, where Jean would
|
|
say ".. and the chassis doesn't even get warm". That's certainly correct today,
|
|
cuz that Supermicro draws 35W, which is one microwatt per packet routed!
|
|
|
|
#### Step 2. Build a European Ring
|
|
|
|
{{< image width="350px" float="right" src="/assets/bucketlist/bucketlist-staging-ams.png" alt="Staging Amsterdam" >}}
|
|
|
|
Of course, I cannot end there, as I have a bucketlist item to work towards. I always
|
|
wanted to peer in Amsterdam, ever since 2001 when I joined BIT. So I worked out a
|
|
plan with Fred, who has also been wanting to go to Amsterdam with his Swiss ISP
|
|
[IP-Max](https://ip-max.net/).
|
|
|
|
So, in a really epic roadtrip full of nerd, Fred and I went into total geek-mode
|
|
as we traveled to several European cities to deploy AS50869 on a european ring. I
|
|
wrote about my experience extensively in these blog posts:
|
|
|
|
* [Frankfurt]({{< ref "2021-05-17-frankfurt" >}}): May 17th 2021.
|
|
* [Amsterdam]({{< ref "2021-05-26-amsterdam" >}}): May 26th 2021.
|
|
* [Lille]({{< ref "2021-05-28-lille" >}}): May 28th 2021.
|
|
* [Paris]({{< ref "2021-06-01-paris" >}}): June 1st 2021.
|
|
* [Geneva]({{< ref "2021-07-03-geneva" >}}): July 3rd 2021.
|
|
|
|
I think we can now say that I'm _peering on the FLAP_. It's not that this AS50869
|
|
carries that much traffic, but it's a very welcome relief of daily worklife to be
|
|
able to do something _fun_ and _immediately rewarding_ like turn up a BGP session
|
|
and see the traffic go from Zurich to any one of these cities at 10Gbit in any
|
|
direction. No congestion, no _packetlo_, just pure horsepower performance.
|
|
|
|
#### Step 3. Build Linux CP in VPP
|
|
|
|
Next month, I plan to take [VPP](https://fd.io/) out for an elaborate spin. I've been
|
|
running DANOS on my routers for a while now, and I'm pretty happy with it, but there
|
|
are a few quirks that are annoying me more and more. Notably, the conversion of Vyatta
|
|
style commands in the configuration into an FRR config, are often lossy. There's a few
|
|
key features (such as RPKI or LDP signalling for MPLS paths) that I'm missing, and
|
|
the dataplane, although pretty stable, has crashed maybe three or four times over the
|
|
last year. Note: One of IP-Max's many Cisco ASR9k also had a few line card reboots in
|
|
the last year so maybe these crashes are par for the course.
|
|
|
|
Ever since seeing Netgate and Cisco started work on the Linux Control Plane plugin, which
|
|
takes interfaces in the VPP dataplane and exposes those as TAP interfaces in Linux, I've
|
|
wanted to contribute to that. I've been determined to make use of VPP+LinuxCP in my own
|
|
network. However, development has completely stalled on the plugin; the one that ships with
|
|
VPP 21.06 is rudimentary at best: doesn't do QinQ/QinAD; doesn't apply any changes from the
|
|
dataplane into the Linux network interface; and the plugin that mirrors netlink message has
|
|
been stuck in limbo for a few months. So I reached out to the authors in May and offered to
|
|
complete / rewrite the plugins. I find that writing code, compiling and testing it, and
|
|
being able to immediately see the improvements in a live network incredibly motivating
|
|
and energizing.
|
|
|
|
Expect to see a few posts in August/September about this work!
|
|
|
|
## 2021 - Switzerland
|
|
|
|
{{< image width="400px" float="right" src="/assets/bucketlist/bucketlist-mentalhealth.png" alt="Alpine Health" >}}
|
|
|
|
I can say that after making a few small tweaks and adjustments, and breaking the WFH
|
|
regime into "work" from home and "play" from home, helps a lot. I now have a HDMI
|
|
switch that flips my desk from my work Mac into my personal OpenBSD machine, and a
|
|
19" rack in my basement with equipment to loadtest and develop VPP, and I often do
|
|
some small chores like establish a peering session and happily traceroute from my
|
|
basement to Amsterdam.
|
|
|
|
I've spent some time in the mountains, in a family commitment to go to a new swiss
|
|
canton every month. The picture on the right was taken from First in Grindelwald,
|
|
looking south towards Eiger and Mönch. I live in an absolutely beautiful country.
|
|
Thanks, Switzerland ;-)
|
|
|
|
On the Bucketlist front, I have the following to report. I waited a few months before
|
|
writing the post, but I can confidently say that accomplishing this L2/L3 path from
|
|
my workstation in Brüttisellen where I'm typing this blogpost, all the way over
|
|
Frankfurt to Amsterdam and being able to reach my original colocation machine at AS8283
|
|
[Coloclue](https://coloclue.net/) using only switches, routers and IP addresses I own
|
|
is a continual joy. Seeing that my work now affords me a straight gigabit bandwidth
|
|
in each direction, makes me just fill with engineering pride and happiness.
|
|
|
|
```
|
|
pim@chumbucket:~$ traceroute ghoul.ipng.nl
|
|
traceroute to gripe.ipng.nl (94.142.241.186), 30 hops max, 60 byte packets
|
|
1 chbtl0.ipng.ch (194.1.163.66) 0.236 ms 0.178 ms 0.143 ms
|
|
2 chrma0.ipng.ch (194.1.163.17) 1.394 ms 1.363 ms 1.332 ms
|
|
3 defra0.ipng.ch (194.1.163.25) 7.275 ms 7.362 ms 7.213 ms
|
|
4 nlams0.ipng.ch (194.1.163.27) 12.905 ms 12.843 ms 12.844 ms
|
|
5 ghoul.ipng.nl (94.142.244.54) 13.120 ms 13.181 ms 13.044 ms
|
|
```
|
|
|
|
And as far as the _actual_ bucketlist item goes, although I made a bit harder on myself
|
|
because I moved to Switzerland, IP-Max also made it easier by giving me a great price
|
|
on the backhaul connectivity to Amsterdam, so I can report that the bucket list item
|
|
is indeed checked off the list:
|
|
|
|
```
|
|
pim@nlams0:~$ show protocols bgp address-family ipv6 unicast summary
|
|
|
|
IPv6 Unicast Summary:
|
|
BGP table version 689670802
|
|
RIB entries 251402, using 46 MiB of memory
|
|
Peers 67, using 1427 KiB of memory
|
|
Peer groups 32, using 2048 bytes of memory
|
|
|
|
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt
|
|
2a02:1668:a2b:5:869::1 4 51088 1561576 216485 0 0 0 08w4d03h 126136 5
|
|
2a02:1668:a2b:5:869::2 4 51088 1546990 216485 0 0 0 08w4d03h 126127 5
|
|
2a02:898::d1 4 8283 812846953 127814 0 0 0 08w6d20h 130590 6
|
|
2a02:898::d2 4 8283 828908332 127814 0 0 0 08w0d16h 130590 6
|
|
2a02:898:146::2 4 112 101560 228562328 0 0 0 06w2d15h 2 132437
|
|
2a07:cd40:1::4 4 212855 105513 238069267 0 0 0 2d14h12m 1 132437
|
|
2602:fed2:fff:ffff::1 4 137933 4180058 124978 0 0 0 04w4d10h 551 7
|
|
2602:fed2:fff:ffff::253 4 209762 2034724 125048 0 0 0 1d00h14m 618 7
|
|
2001:7f8:10f::205b:140 4 8283 137242 121460 0 0 0 08w5d17h 34 7
|
|
2001:7f8:10f::207b:145 4 8315 278651 274793 0 0 0 06w0d12h 34 7
|
|
2001:7f8:10f::500f:139 4 20495 117590 107877 0 0 0 04w3d00h 208 7
|
|
2001:7f8:10f::ac47:131 4 44103 152949 55010 0 0 0 05w1d13h 24 7
|
|
2001:7f8:10f::af36:129 4 44854 134969 146240 0 0 0 09w2d16h 1 7
|
|
2001:7f8:10f::afd1:133 4 45009 35438 35477 0 0 0 01w0d02h 3 7
|
|
2001:7f8:10f::e20a:148 4 57866 302505 280603 0 0 0 05w5d18h 161 7
|
|
2001:7f8:10f::e3bb:137 4 58299 1419455 104321 0 0 0 04w0d13h 531 7
|
|
2001:7f8:10f::ec8d:132 4 60557 120509 108071 0 0 0 01w4d20h 7 7
|
|
2001:7f8:10f::3:259e:143 4 206238 278960 272776 0 0 0 04w4d18h 2 7
|
|
2001:7f8:10f::3:3e9b:134 4 212635 823944 140075 0 0 0 08w5d17h 1 7
|
|
2001:7f8:10f::dc49:253 4 56393 5693179 157171 0 0 0 02w6d22h 26680 7
|
|
2001:7f8:10f::dc49:254 4 56393 5698910 162197 0 0 0 08w5d17h 26680 7
|
|
2a02:2528:1902::1 4 25091 9964126 137696 0 0 0 09w1d22h 113020 5
|
|
2001:7f8:8f::a500:6939:1 4 6939 8496149 138188 0 0 0 01w2d20h 48079 7
|
|
2001:7f8:8f::a500:8283:1 4 8283 23251 52823 0 0 0 03w3d02h Active 0
|
|
2001:7f8:8f::a501:3335:1 4 13335 3279 3199 0 0 0 1d02h35m 102 7
|
|
2001:7f8:8f::a502:495:1 4 20495 117248 107466 0 0 0 04w3d00h 208 7
|
|
2001:7f8:8f::a503:2934:1 4 32934 194428 193990 0 0 0 01w3d08h 30 7
|
|
2001:7f8:8f::a503:2934:2 4 32934 194035 194002 0 0 0 03w3d11h 30 7
|
|
2001:7f8:8f::a504:4854:1 4 44854 0 9052 0 0 0 never Idle (Admin) 0
|
|
2001:7f8:8f::a504:5009:1 4 45009 35433 35467 0 0 0 01w0d02h 3 7
|
|
2001:7f8:8f::a505:7866:1 4 57866 302602 276459 0 0 0 04w4d01h 161 7
|
|
2001:7f8:8f::a505:8299:1 4 58299 912125 141718 0 0 0 04w0d13h 531 7
|
|
2001:7f8:8f::a506:557:1 4 60557 120482 108067 0 0 0 01w4d20h 7 7
|
|
2001:7f8:8f::a521:2635:1 4 212635 622475 85332 0 0 0 02w5d10h 1 7
|
|
2001:7f8:8f::a504:9917:1 4 49917 8370930 158851 0 0 0 03w4d13h 25257 7
|
|
2001:7f8:8f::a504:9917:2 4 49917 8397150 160118 0 0 0 04w4d01h 25011 7
|
|
2001:7f8:13::a500:714:1 4 714 67722 66645 0 0 0 03w2d03h 146 7
|
|
2001:7f8:13::a500:714:2 4 714 68208 66645 0 0 0 03w2d03h 146 7
|
|
2001:7f8:13::a500:6939:1 4 6939 10980475 98099 0 0 0 07w0d10h 48079 7
|
|
2001:7f8:13::a502:495:1 4 20495 117773 107873 0 0 0 04w0d14h 208 7
|
|
2001:7f8:13::a503:4307:1 4 34307 10709086 100814 0 0 0 09w4d23h 23339 7
|
|
2001:7f8:13::a503:4307:2 4 34307 10694266 100814 0 0 0 09w4d23h 22137 7
|
|
2001:7f8:8f::a504:4103:1 4 44103 152932 55010 0 0 0 05w1d13h 24 7
|
|
2001:7f8:b7::a500:8283:1 4 8283 126035 98846 0 0 0 06w4d22h 34 7
|
|
2001:7f8:b7::a501:3335:1 4 13335 4277 4157 0 0 0 1d10h34m 102 7
|
|
2001:7f8:b7::a502:495:1 4 20495 117588 107871 0 0 0 04w3d00h 208 7
|
|
2001:7f8:b7::a504:5009:1 4 45009 35441 35504 0 0 0 01w0d02h 3 7
|
|
2001:7f8:b7::a506:557:1 4 60557 120546 108067 0 0 0 01w4d20h 7 7
|
|
2001:7f8:b7::a521:2635:1 4 212635 716031 94458 0 0 0 08w5d17h 1 7
|
|
2001:7f8:b7::a504:1441:1 4 41441 12911969 107363 0 0 0 08w2d12h 50606 7
|
|
2001:7f8:b7::a504:1441:2 4 41441 12733337 107304 0 0 0 08w2d12h 50606 7
|
|
|
|
Total number of neighbors 67
|
|
|
|
pim@nlams0:~$ show protocols ospfv3 neighbor
|
|
Neighbor ID Pri DeadTime State/IfState Duration I/F[State]
|
|
194.1.163.7 1 00:00:32 Full/PointToPoint 62d21:41:24 dp0p6s0f3.100[PointToPoint]
|
|
194.1.163.34 1 00:00:39 Full/PointToPoint 27d22:28:30 dp0p6s0f3.200[PointToPoint]
|
|
```
|
|
|
|
There are three full IPv4 and IPv6 transit providers: AS51088 ([A2B Internet](https://a2b-internet.com/),
|
|
thanks Erik!), AS8283 ([Coloclue](https://coloclue.net/)) and AS25091 ([IP-Max](https://ip-max.net/),
|
|
thanks Fred!). Also, the router is connected directly to Speed-IX, LSIX, FrysIX and NL-IX. Along with
|
|
the many other internet exchanges I've connected to, it puts my humble AS50869 as #5
|
|
[best connected](https://bgp.he.net/country/CH) ISP in Switzerland!
|
|
|
|
I mean, just look at that stability, BGP sessions often times up as long as the machine
|
|
has been there (remember, I deployed `nlams0.ipng.ch` only in May, so 9 weeks is all we can ask for!).
|
|
OSPF uptime (helpfully shown with duration with OSPFv3 on FRR) is impeccable as well. The link with 27d
|
|
of uptime is because I took out that router for maintenance 27 days ago to upgrade it to a preliminary
|
|
version of DANOS + Bird2, as I prepare the move to VPP + Bird2 later this year.
|
|
|
|
#### A note on mental health
|
|
|
|
Mental health includes our emotional, psychological, and social well-being. It
|
|
affects how we think, feel, and act. It also helps determine how we handle stress,
|
|
relate to others, and make choices. Mental health is important at every stage of
|
|
life, from childhood and adolescence through adulthood.
|
|
|
|
If you've read so far, thanks! I can imagine that some find this story a mixture of
|
|
nerd and brag, and that's OK. I am writing these stories because ***I find happiness in writing***
|
|
about the small and large technical things that I perceive as important to my
|
|
feelings of accomplishment and therefor my wellbeing.
|
|
|
|
I do many non-nerd and non-technical things, but I try to make it a habit of keeping my personal
|
|
life off the internet (I'm not on social media and not often on digital messaging boards or chat
|
|
apps). I could tell you equally enthusiastically about those hikes I took in Grindelwald, or
|
|
those Bürli I baked, but that would have to be in person.
|
|
|
|
Well-being is a positive outcome that is meaningful for people and for many sectors
|
|
of society, because it tells us that people perceive that their lives are going
|
|
well. However, many indicators that measure living conditions fail to measure what
|
|
people think and feel about their lives, such as the quality of their relationships,
|
|
their positive emotions and resilience, the realization of their potential, or their
|
|
overall satisfaction with life.
|
|
|
|
I find satisfaction in my modest dabbles with IPng Networks, both the software and
|
|
the hardware and physical aspects of it. I encourage everybody to have a safe/fun place
|
|
where they spend some meaningful time doing things that _spark joy_. To your health!
|