From 8d62d7972f9ca3179619b0b7ddc7f2e45c57d000 Mon Sep 17 00:00:00 2001 From: Johan Thomsen Date: Mon, 17 Sep 2018 14:12:44 +0200 Subject: [PATCH] nixos/kubernetes: adding manual section for kubernetes and writing release note for NixOS 19.03 --- .../manual/configuration/configuration.xml | 1 + nixos/doc/manual/configuration/kubernetes.xml | 127 ++++++++++++++++++ nixos/doc/manual/release-notes/rl-1903.xml | 41 ++++++ 3 files changed, 169 insertions(+) create mode 100644 nixos/doc/manual/configuration/kubernetes.xml diff --git a/nixos/doc/manual/configuration/configuration.xml b/nixos/doc/manual/configuration/configuration.xml index cebc4122c6c..138d1d86d7f 100644 --- a/nixos/doc/manual/configuration/configuration.xml +++ b/nixos/doc/manual/configuration/configuration.xml @@ -23,5 +23,6 @@ + diff --git a/nixos/doc/manual/configuration/kubernetes.xml b/nixos/doc/manual/configuration/kubernetes.xml new file mode 100644 index 00000000000..ddc026c0c01 --- /dev/null +++ b/nixos/doc/manual/configuration/kubernetes.xml @@ -0,0 +1,127 @@ + + Kubernetes + + + The NixOS Kubernetes module is a collective term for a handful of + individual submodules implementing the Kubernetes cluster components. + + + + There are generally two ways of enabling Kubernetes on NixOS. + One way is to enable and configure cluster components appropriately by hand: + +services.kubernetes = { + apiserver.enable = true; + controllerManager.enable = true; + scheduler.enable = true; + addonManager.enable = true; + proxy.enable = true; + flannel.enable = true; +}; + + Another way is to assign cluster roles ("master" and/or "node") to the host. + This enables apiserver, controllerManager, scheduler, addonManager, + kube-proxy and etcd: + + = [ "master" ]; + + While this will enable the kubelet and kube-proxy only: + + = [ "node" ]; + + Assigning both the master and node roles is usable if you want a single + node Kubernetes cluster for dev or testing purposes: + + = [ "master" "node" ]; + + Note: Assigning either role will also default both + and + to true. + This sets up flannel as CNI and activates automatic PKI bootstrapping. + + + + As of kubernetes 1.10.X it has been deprecated to open + non-tls-enabled ports on kubernetes components. Thus, from NixOS 19.03 all + plain HTTP ports have been disabled by default. + While opening insecure ports is still possible, it is recommended not to + bind these to other interfaces than loopback. + + To re-enable the insecure port on the apiserver, see options: + + and + + + + + + As of NixOS 19.03, it is mandatory to configure: + . + The masterAddress must be resolveable and routeable by all cluster nodes. + In single node clusters, this can be set to localhost. + + + + + Role-based access control (RBAC) authorization mode is enabled by default. + This means that anonymous requests to the apiserver secure port will + expectedly cause a permission denied error. All cluster components must + therefore be configured with x509 certificates for two-way tls communication. + The x509 certificate subject section determines the roles and permissions + granted by the apiserver to perform clusterwide or namespaced operations. + See also: + + Using RBAC Authorization. + + + + The NixOS kubernetes module provides an option for automatic certificate + bootstrapping and configuration, + . + The PKI bootstrapping process involves setting up a certificate authority + (CA) daemon (cfssl) on the kubernetes master node. cfssl generates a CA-cert + for the cluster, and uses the CA-cert for signing subordinate certs issued to + each of the cluster components. Subsequently, the certmgr daemon monitors + active certificates and renews them when needed. For single node Kubernetes + clusters, setting = true + is sufficient and no further action is required. For joining extra node + machines to an existing cluster on the other hand, establishing initial trust + is mandatory. + + + + To add new nodes to the cluster: + On any (non-master) cluster node where + is enabled, the helper + script nixos-kubernetes-node-join is available on PATH. + Given a token on stdin, it will copy the token to the kubernetes + secrets directory and restart the certmgr service. As requested + certificates are issued, the script will restart kubernetes cluster + components as needed for them to pick up new keypairs. + + + + + Multi-master (HA) clusters are not supported by the easyCerts module. + + + + + In order to interact with an RBAC-enabled cluster as an administrator, one + needs to have cluster-admin privileges. By default, when easyCerts is + enabled, a cluster-admin kubeconfig file is generated and linked into + /etc/kubernetes/cluster-admin.kubeconfig as determined by + . + export KUBECONFIG=/etc/kubernetes/cluster-admin.kubeconfig + will make kubectl use this kubeconfig to access and authenticate the cluster. + The cluster-admin kubeconfig references an auto-generated keypair owned by + root. Thus, only root on the kubernetes master may obtain cluster-admin + rights by means of this file. + + + diff --git a/nixos/doc/manual/release-notes/rl-1903.xml b/nixos/doc/manual/release-notes/rl-1903.xml index 6f78983d482..269f27f74fb 100644 --- a/nixos/doc/manual/release-notes/rl-1903.xml +++ b/nixos/doc/manual/release-notes/rl-1903.xml @@ -54,6 +54,13 @@ to false and enable your preferred display manager. + + A major refactoring of the Kubernetes module has been completed. + Refactorings primarily focus on decoupling components and enhancing + security. Two-way TLS and RBAC has been enabled by default for all + components, which slightly changes the way the module is configured. + See: for details. + @@ -564,6 +571,40 @@ provisioning. + + + The use of insecure ports on kubernetes has been deprecated. + Thus options: + services.kubernetes.apiserver.port and + services.kubernetes.controllerManager.port + has been renamed to .insecurePort, + and default of both options has changed to 0 (disabled). + + + + + Note that the default value of + services.kubernetes.apiserver.bindAddress + has changed from 127.0.0.1 to 0.0.0.0, allowing the apiserver to be + accessible from outside the master node itself. + If the apiserver insecurePort is enabled, + it is strongly recommended to only bind on the loopback interface. See: + services.kubernetes.apiserver.insecurebindAddress. + + + + + The option services.kubernetes.apiserver.allowPrivileged + and services.kubernetes.kubelet.allowPrivileged now + defaults to false. Disallowing privileged containers on the cluster. + + + + + The kubernetes module does no longer add the kubernetes package to + environment.systemPackages implicitly. + +