From 55c09a884adc4a3ee609b088ef5d8712ba0e9ddb Mon Sep 17 00:00:00 2001 From: Andreas Rammhold Date: Thu, 28 May 2020 01:07:30 +0200 Subject: [PATCH] nixos/modules/system/boot/networkd: enable socket activation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Since cd1dedac67d4b077a556a660ef5724c909da8006 systemd-networkd has it's netlink socket created via a systemd.socket unit. One might think that this doesn't make much sense since networkd is just going to create it's own socket on startup anyway. The difference here is that we have configuration-time control over things like socket buffer sizes vs compile-time constants. For larger setups where networkd has to create a lot of (virtual) devices the default buffer size of currently 128MB is not enough. A good example is a machine with >100 virtual interfaces (e.g., wireguard tunnels, VLANs, …) that all have to be brought up during startup. The receive buffer size will spike due to all the generated message from the new interfaces. Eventually some of the message will be dropped since there is not enough (permitted) buffer space available. By having networkd start through / with a netlink socket created by systemd we can configure the `ReceiveBufferSize` parameter in the socket options without recompiling networkd. Since the actual memory requirements depend on hardware, timing, exact configurations etc. it isn't currently possible to infer a good default from within the NixOS module system. Administrators are advised to monitor the logs of systemd-networkd for `rtnl: kernel receive buffer overrun` spam and increase the memory as required. Note: Increasing the ReceiveBufferSize doesn't allocate any memory. It just increases the upper bound on the kernel side. The memory allocation depends on the amount of messages that are queued on the kernel side of the netlink socket. --- nixos/doc/manual/release-notes/rl-2009.xml | 32 ++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/nixos/doc/manual/release-notes/rl-2009.xml b/nixos/doc/manual/release-notes/rl-2009.xml index a8a16bcc14f..43661125338 100644 --- a/nixos/doc/manual/release-notes/rl-2009.xml +++ b/nixos/doc/manual/release-notes/rl-2009.xml @@ -566,6 +566,38 @@ systemd.services.nginx.serviceConfig.ReadWritePaths = [ "/var/www" ]; was removed, as udev gained native support to handle FIDO security tokens. + + + With this release systemd-networkd (when enabled through ) + has it's netlink socket created through a systemd.socket unit. This gives us control over + socket buffer sizes and other parameters. For larger setups where networkd has to create a lot of (virtual) + devices the default buffer size (currently 128MB) is not enough. + + + On a machine with >100 virtual interfaces (e.g., wireguard tunnels, VLANs, …), that all have to + be brought up during system startup, the receive buffer size will spike for a brief period. + Eventually some of the message will be dropped since there is not enough (permitted) buffer + space available. + + + By having systemd-networkd start with a netlink socket created by + systemd we can configure the ReceiveBufferSize= parameter + in the socket options (i.e. systemd.sockets.systemd-networkd.socketOptions.ReceiveBufferSize) + without recompiling systemd-networkd. + + + Since the actual memory requirements depend on hardware, timing, exact + configurations etc. it isn't currently possible to infer a good default + from within the NixOS module system. Administrators are advised to + monitor the logs of systemd-networkd for rtnl: kernel receive buffer + overrun spam and increase the memory limit as they see fit. + + + Note: Increasing the ReceiveBufferSize= doesn't allocate any memory. It just increases + the upper bound on the kernel side. The memory allocation depends on the amount of messages that are + queued on the kernel side of the netlink socket. + +