Halloween and more pics

content/blog/wk14day2.md

@@ -1,7 +1,69 @@
 ---
-title: "Week 14, Tuesday:"
+title: "Week 14, Tuesday: Patches Galore"
 date: 2024-10-29T21:55:00+02:00
-draft: true
 ---
 
-{{< image frame="true" width="17em" float="right" src="/img/headline/todo.png" alt="Credit: " >}}
+{{< image frame="true" width="17em" float="right" src="/img/headline/dreamscape-03.png" alt="Credit: Dreamscape, Kush Sessions, YouTube" >}}
+
+This morning my buddy Arend sends me a message on Telegram - and asks me if I can check the port in
+Qupra. Oh my deity, it's finally happening! After the general assembly of Coloclue approved a
+member's petition to allow members to install cross connects at Qupra, a few months of "kastje, muur"
+happened, and the networking committee and association board reached an agreement on how this would
+happen.
+
+A few months after that, Arend tried to install a crossconnect but we weren't ready paper-wise. I
+brought it back to the attention of the networking committee and we identified the missing pieces: a
+patch panel with keystones needed to be installed in each rack (and it was only available in two of
+the racks at the time), and a change to the administrative database needed to be added to document
+which members had which crossconnects.
+
+Tim took care of the first thing - he ordered the panels and keystones and went to the datacenter to
+install them. I offered to take careo f the second thing - but since the administrative database has
+need-to-know information, our treasurer Arjan preferred to add the records himself. Once these two
+things were taken care of, all I had to do is wait for a practical moment :) I had planned to deploy
+the fiber myself last week, but I had to cancel my trip to Amsterdam due to the COVID situation.
+
+So I was surprised and delighted that Arend pinged me. The Qupra FrysIX switch was pre-configured,
+and all that was left was to plug things in. Arend made quick work of it, and as well put in the
+cross connect for a few other members at Coloclue, he's such a sweetheart! For me, this link will be
+used to alleviate the hypervisor at Equinix AM3, as it is running low on disk throughput due to me
+using Samsung consumer SSDs. I shipped Arend a few enterprise SAS SSDs before, but he hasn't gotten
+around to deploying them yet. More importantly, the AM3 hypervisor runs FrysIX routeserver, LibreNMS
+and IXPManager.
+
+After the Qupra gig, Arend made his way to NIKHEF where he installed the FrysIX patch for FreeIX
+Remote, directly into the VPP router `nlams0.net.free-ix.net`. That router now has LSIX, SpeedIX,
+and FrysIX connected. I spend some time bringing FreeIX Remote AS50869 into quarantine and then into
+the production VLAN. That's a benefit of running the IXP: I get to expedite my own connections :)
+
+Now that the FreeIX Remote router is connected to FrysIX, I allocate a private VLAN between it and
+IPng's infrastructure. This allows me to create a VPWS (L2VPN, Ethernet over MPLS) on IPng's MPLS
+switches `msw0.nlams0` and `msw1.chrma0` from this router in Amsterdam to the one I already installed
+in Zurich. iBGP comes up, and there are now three routers in play (`nlams0`, `chrma0`, and
+`grskg0`), and amongst them, they know about 207K IPv4 prefixes and 64.7K IPv6 prefixes, and all of
+them can be reached via direct or routeserver peering. How cool is that?
+
+
+```
+pim@nlams0:~$ birdc show route count
+BIRD v2.15.1-4-g280daed5-x ready.
+800934 of 800934 routes for 207438 networks in table master4
+449754 of 449754 routes for 64696 networks in table master6
+1501107 of 1501107 routes for 500369 networks in table t_roa4
+364077 of 364077 routes for 121359 networks in table t_roa6
+Total: 3115872 of 3115872 routes for 893862 networks in 4 tables
+```
+
+In the evening I send a maintenance announcement out to FrysIX members: in the night of Wednesday to
+Thursday of this week, I will move the routeserver RS2 and the IXPManager over to the hypervisor
+at Qupra, which now sports a 10G connection to the FrysIX peering switch there. I have plumbed the
+management VLAN 264 and the Quarantine 2605 and the Peering LAN 2604 through to the hypervisor.
+
+I practice by moving `nms.frys-ix.net` over - this is a non-intrusive change. Using ZFS block device
+replication, I can pump over the boot disk with about 110MB/s, because the hypervisor itself has
+"only" a one gigabit connection. I boot the VM, and it comes up cleanly. Nice. I spend a few hours
+preparing the move of the other two machines (RS2 and IXPManager), which are service impacting. But
+I can start by making a snapshot of the block devices, copy their data over ahead of time, and then
+copy a final snapshot incrementally.
+
+Today was a good day for FrysIX :)

content/blog/wk14day3.md

@@ -1,7 +1,77 @@
 ---
-title: "Week 14, Wednesday:"
+title: "Week 14, Wednesday: Candy and FrysIX Moves"
 date: 2024-10-30T21:55:00+02:00
-draft: true
 ---
 
-{{< image frame="true" width="17em" float="right" src="/img/headline/todo.png" alt="Credit: " >}}
+{{< image frame="true" width="17em" float="right" src="/img/headline/aurora.png" alt="Credit: Aurora, YouTube" >}}
+
+Today I did pretty much nothing :) I spent a bunch of time in the flight simulator flying circuits
+and trying to keep my altitude. Also I did about twenty (flyby) approaches where I would try to have
+a sink rate of 400ft/min and 60kts with flaps at 10 degrees. The airplane is sluggish at this speed,
+so there's a bit of a delay before the correct attitude is reached. Think of it as a really slow PID
+controller. I'm still pretty terrible at it, but at least I am putting the airplane down in one
+piece. Doing visual approach is really difficult by the way, because looking left and right is a bit
+awkward. Seeing as my spiffy video card has four monitor outputs, I think I'll try to get my hands
+on some cheap screens that could serve as left and right windows.
+
+After lunch I take another look at the `sFlow` plugin - for some reason it is emitting large
+interface IDs (1e9 + hw_if_index), but Neil has written it such that if a Linux Control Plane pair
+exists, it'll use the Linux vif_index. Clearly that isn't happening, and I have a pending call with
+him tonight to try to get to the bottom of it.
+
+In the late afternoon, Quinn Marina and I draw smiley faces on our halloween candy baggies, and fill
+them with an inordinate amount of candy. We make 120 baggies, as in previous years, we've often been
+able to offload 100+ of these things to the neighborhood kids, the weather is going to be good
+tomorrow, and Quinn decorated our front porch in a very inviting way: I'm sure the kids will enjoy
+it, but not as much as we will, methinks :)
+
+After dinner, Marina and I watch some news, and then I retire to the basement for my call. I futz a
+bit with the audio, but eventually am able to greet Peter and Neil. We talk about a few operational
+details, notably:
+
+1.   the ability of the `hsflowd` to send its update traffic from a network namespace different to
+the main one (because VPP wants to run in a `dataplane` netns). The tool already supports this,
+yaay!
+1.   if samples are received before the interface counters (with their respective interface IDs
+mapped to the Linux Control Plane vif_index'es, then those samples will arrive at the collector with
+these if_index values of 1000000001 and so on. This will make tools like Akvorado try to retrieve
+SNMP indexes with that - but they won't exist and this is a bug. Neil agrees, and will inhibit
+sending of samples until after the first counter for an interface is found
+1.   a propose the large numbers, they are repeatably and permanemently `1e9+idx` for me, while they
+should really be the LCP vif_index. We add some debug logging, but it doesn't trigger. Then we run
+out of time and we promise each other we'll both look into it offline.
+
+All in all, a super productive meeting, and once again I learned a tonne.
+
+At 23:00 I need to complete the move of the FrysIX virtual machines from my hypervisor at Equinix
+AM3 to another one at Qupra, which Arend connected to FrysIX infrastructure yesterday. This move is
+important, because the SSDs in the AM3 hypervisor are crappy, while the ones in Qupra are enterprise
+SAS-12 SSDs from NetApp: much faster, must higher write rate, much longer durability (famous last
+words).
+
+I take a snapshot copy of `rs2.frys-ix.net` first, and copy that VM to its new spot, but disconnect
+the FrysIX port, by moving the virtio network card from the FrysIX bridge group to an empty bridge
+group. I boot it, and upgrade it from Debian Bullseye to Debian Bookworm while I'm here. I have it
+download its config and at 23:00 exactly, I shutdown the old `rs2` and connect the virtio network
+interface to FrysIX in the new spot. Finally, I add VLAN 2604 (production) and 2605 (quarantine) to
+the port in Qupra, and a few seconds later we're fully online. The migration of `rs2` took about 150
+seconds of downtime. Not bad, eh??
+
+The IXPManager virtual machine is a bit larger, but thanks to ZFS block device snapshots I can make
+an incremental ZFS transfer, halt the old `ixpmanager` VM, quickly `zfs send | zfs recv`  the last
+bits from the penultimate snapshot to HEAD, and boot the VM on its new spot. The IXPManager comes up
+within 10 minutes or so. All in all, the maintenance window lasted from 23:00-23:35 and I was quite
+happy with the results.
+
+## Pictures of the Day
+
+{{< gallery-category >}}
+  {{< gallery-photo fn="2024-10-30/IMG_2249.JPG" caption="The MS Flight Simulator with an ad-hoc righthand side window, easier to see the airport when doing righthand patterns. Need another screen :)" >}}
+  {{< gallery-photo fn="2024-10-30/IMG_2250.JPG" caption="We are filling candy baggies for the trick-or-treat'ers tomorrow. We expect one hundred or so kids." >}}
+  {{< gallery-photo fn="2024-10-30/IMG_2253.JPG" caption="The amount of sugar we are pumping into the youth in Bruttisellen is .. significant." >}}
+  {{< gallery-photo fn="2024-10-30/IMG_2254.JPG" caption="The empty distribution-sized bags from Aligro. That's ... a lot of sugar." >}}
+{{< /gallery-category >}}
+
+{{< gallery-modal >}}
+{{< gallery-script >}}
+

content/blog/wk14day4.md

@@ -1,7 +1,103 @@
 ---
-title: "Week 14, Thursday:"
+title: "Week 14, Thursday: Hallowe'en"
 date: 2024-10-31T21:55:00+02:00
-draft: true
 ---
 
-{{< image frame="true" width="17em" float="right" src="/img/headline/todo.png" alt="Credit: " >}}
+{{< image frame="true" width="17em" float="right" src="/img/headline/nicky-havey.png" alt="Credit: Nicky Havey, YouTube" >}}
+
+Today is Hallowe'en! When we moved to Br&uuml;ttisellen in 2006, it was _not_ a thing. Over the
+years, a few of us would decorate our houses, but most of the kids were not really with the program:
+no trick or treating, no dressing up in scary costumes, nothing. But when we bought our place in
+2015, we we decided to really decorate in the week of Hallowe'en. These days, kids will come round,
+sometimes in small groups, sometimes in very large ones, parents and all. Many of them still don't
+really know what they are doing: ring doorbell --> get candy. The German way to say _trick or treat_
+by the way is _s&uuml;sses oder saures_ (English: sweet or sour) which I find hilarious because if
+you say "well I choose sour!", the kid will look at you as if you're crazy: "Sir, this is where you
+hand over the candy?!".
+
+> You have much to learn, Switzerland, but I love you!
+
+We have lots of candy in our baggies, we have Elza (our door-ghoul), spiders and lots of spider
+webs, and some UV lights to give them a spooky glow-in-the-dark feeling. I will be dressing up in a
+black hoodie with a _zipperface_, which takes a while to prepare for, but really hits the spot with
+freaking out the children.
+
+In the morning, the doorbell rings at 09:00 and oh look, a package! It's a rather huge package, the
+size of a small refridgerator... in it are two 32" curved screen displays, which I bought for the
+low low price of CHF 120,- apiece! It's amazing how affordable this stuff is! I unpack them, which
+takes actually quite a long time, and take care of the cardboard and styrofoam carnage that ensues.
+After my hard work, two screens are available for the flight sim.
+
+OK, this thing will need to move, because hanging off a screen on the left and right hand side of
+the main screen will make the simulator a lot wider. Luckily, I have a spot, sort of, in the server
+room. There's an unused desk there which I move out of the way. I then move the simulator into the
+room and Marina helps me measure how high the left/right screen need to be. It turns out, 96cm is
+the sweet spot.
+
+I have a few hours left before I need to change my appearance, so after a quick lunch I go to the
+Ikea, as they happen to have a 60cm x 96cm x 30cm cabinet called _Spiksmed_. Did you know that
+almost all the IKEA product names are fake Swedish words? My IKEA in Dietlikon has exactly two of
+these _Spiksmed_ units, so I grab them! The trip takes me only twenty minutes, because I don't have
+to go through the whole maze of the store, rather I can just purchase these items and move to the
+_Warenausgabe_ (English: Goods Distribution), and these little ones are quickly in hand.
+
+At home, the cabinets are pretty quickly assembled, and while I don't have time to play with the
+simulator, I do manage to connect it all and project a left- and righthand door + window on these
+two screens. It looks ... absolutely stunning. Very, very immersive!
+
+Marina interrupts my fun in the basement and suggests that I shave before dressing up, because the
+zipper-face uses latex to glue onto the skin, and I don't want to feel the burn of a brazilian
+face-wax later when removing it. So I shower and shave and put some skin lotion on my face. The plan
+is:
+
+*   15:00-15:45 Marina gives Pim a Zipper Face
+*   15:45-16:15 Pim goes to get Quinn. Marina converts into Sally (from the Nightmare before Christmas)
+*   16:15-17:00 Pim and Quinn return.
+*   17:00-18:00 Marina turns Quinn into Tate Langdon (from American Horror Story)
+*   17:30-18:00 Pim makes dinner
+*   18:00-18:15 Dinner had
+*   18:15-22:30 Freak out the neighborhood kids.
+
+For dinner I make baked beans, potatoes and a chicken schnitzel, it's a simple meal but we don't
+have much time to eat. Most of the younger kids will want to be in bed at 19:00 or so, as it's a
+school night (remember those? School nights, heh). I apply some extra bloody gore after dinner, and
+drink water through a straw. Marina puts on her Sally-wig and grabs Zero, the floating ghost-dog. 
+
+It's fair to say we freak the kids out. It's funny to me because I've seen grownups, teens and
+toddlers all keep their distance and kind of refuse to approach the door. They don't know what's
+going to happen. One set of teenage girls stood a good three meters (ten feet) from the door, and
+when Marina said they had to come get their candy, one of them said - I kid you not - "Oh, no I'm
+good over here, you can throw us the bag of candy, kthx". But we made them approach, anyway :)
+
+Marina and I take an evening stroll through the neighborhood. There's about ten houses or so that
+really put themselves out there; and about twenty houses that are approachable (although, honestly:
+if you want to participate in trick or treating, you really should decorate your house!). It's
+pretty clear to find our house, and many kids do. Some even twice. Or three times.
+
+Quinn and I also take a walk and I often ask him: "Hey Quinn, can I eat that kid over there?", and
+Quinn plays his part too, walking with a cane but then lunging at kids screaming at them. So many
+heart attacks were had. I love it.
+
+At about 22:00 we call it a night. A cute thing about Switzerland is the so called _Nachtruhe_
+(English: night rest), which is a federal law to avoid street noises after 22:00. So we turn off the
+lights and UV, and finish watching the _Adam's Family_ movie that Marina put on while Quinn and I
+were out taking a walk and scaring the neighborhood kids.
+
+We had so much fun today - I'm glad.
+
+## Pictures of the Day
+
+{{< gallery-category >}}
+  {{< gallery-photo fn="2024-10-31/IMG_2257.JPG" caption="All the baggies of candy for tonight's trick-or-treating. We expect many kids." >}}
+  {{< gallery-photo fn="2024-10-31/IMG_2258.JPG" caption="The Digitec package is big enough to hold a fridge. In it: two 32 inch monitors for my flight simulator." >}}
+  {{< gallery-photo fn="2024-10-31/IMG_2262.JPG" caption="I've set up these monitors as left- and right facing doors + wings in the cockpit. I line up the horizon so that it's a smooth transition from left to center to right." >}}
+  {{< gallery-photo fn="2024-10-31/IMG_2265.JPG" caption="Marina is starting to apply the zipper to my face. The latex stinks (really bad, of ammonia, and it burns the skin We suffer for our art)." >}}
+  {{< gallery-photo fn="2024-10-31/IMG_2285.JPG" caption="Zero is ready to float around with Marina, who is going as Sally." >}}
+  {{< gallery-photo fn="2024-10-31/IMG_2286.JPG" caption="My Zipper-face and Quinn's Tate" >}}
+  {{< gallery-photo fn="2024-10-31/IMG_2287.JPG" caption="Marina's Sally and my Zipper-face" >}}
+  {{< gallery-photo fn="2024-10-31/IMG_2292.JPG" caption="Our front porch has spider webs and lots of creepy crawlies. The webs are illumated with UV light which makes them glow in the dark" >}}
+{{< /gallery-category >}}
+
+{{< gallery-modal >}}
+{{< gallery-script >}}
+