diff --git a/nixos/doc/manual/configuration/configuration.xml b/nixos/doc/manual/configuration/configuration.xml
index cebc4122c6c..138d1d86d7f 100644
--- a/nixos/doc/manual/configuration/configuration.xml
+++ b/nixos/doc/manual/configuration/configuration.xml
@@ -23,5 +23,6 @@
+
diff --git a/nixos/doc/manual/configuration/kubernetes.xml b/nixos/doc/manual/configuration/kubernetes.xml
new file mode 100644
index 00000000000..ddc026c0c01
--- /dev/null
+++ b/nixos/doc/manual/configuration/kubernetes.xml
@@ -0,0 +1,127 @@
+
+ Kubernetes
+
+
+ The NixOS Kubernetes module is a collective term for a handful of
+ individual submodules implementing the Kubernetes cluster components.
+
+
+
+ There are generally two ways of enabling Kubernetes on NixOS.
+ One way is to enable and configure cluster components appropriately by hand:
+
+services.kubernetes = {
+ apiserver.enable = true;
+ controllerManager.enable = true;
+ scheduler.enable = true;
+ addonManager.enable = true;
+ proxy.enable = true;
+ flannel.enable = true;
+};
+
+ Another way is to assign cluster roles ("master" and/or "node") to the host.
+ This enables apiserver, controllerManager, scheduler, addonManager,
+ kube-proxy and etcd:
+
+ = [ "master" ];
+
+ While this will enable the kubelet and kube-proxy only:
+
+ = [ "node" ];
+
+ Assigning both the master and node roles is usable if you want a single
+ node Kubernetes cluster for dev or testing purposes:
+
+ = [ "master" "node" ];
+
+ Note: Assigning either role will also default both
+ and
+ to true.
+ This sets up flannel as CNI and activates automatic PKI bootstrapping.
+
+
+
+ As of kubernetes 1.10.X it has been deprecated to open
+ non-tls-enabled ports on kubernetes components. Thus, from NixOS 19.03 all
+ plain HTTP ports have been disabled by default.
+ While opening insecure ports is still possible, it is recommended not to
+ bind these to other interfaces than loopback.
+
+ To re-enable the insecure port on the apiserver, see options:
+
+ and
+
+
+
+
+
+ As of NixOS 19.03, it is mandatory to configure:
+ .
+ The masterAddress must be resolveable and routeable by all cluster nodes.
+ In single node clusters, this can be set to localhost.
+
+
+
+
+ Role-based access control (RBAC) authorization mode is enabled by default.
+ This means that anonymous requests to the apiserver secure port will
+ expectedly cause a permission denied error. All cluster components must
+ therefore be configured with x509 certificates for two-way tls communication.
+ The x509 certificate subject section determines the roles and permissions
+ granted by the apiserver to perform clusterwide or namespaced operations.
+ See also:
+
+ Using RBAC Authorization.
+
+
+
+ The NixOS kubernetes module provides an option for automatic certificate
+ bootstrapping and configuration,
+ .
+ The PKI bootstrapping process involves setting up a certificate authority
+ (CA) daemon (cfssl) on the kubernetes master node. cfssl generates a CA-cert
+ for the cluster, and uses the CA-cert for signing subordinate certs issued to
+ each of the cluster components. Subsequently, the certmgr daemon monitors
+ active certificates and renews them when needed. For single node Kubernetes
+ clusters, setting = true
+ is sufficient and no further action is required. For joining extra node
+ machines to an existing cluster on the other hand, establishing initial trust
+ is mandatory.
+
+
+
+ To add new nodes to the cluster:
+ On any (non-master) cluster node where
+ is enabled, the helper
+ script nixos-kubernetes-node-join is available on PATH.
+ Given a token on stdin, it will copy the token to the kubernetes
+ secrets directory and restart the certmgr service. As requested
+ certificates are issued, the script will restart kubernetes cluster
+ components as needed for them to pick up new keypairs.
+
+
+
+
+ Multi-master (HA) clusters are not supported by the easyCerts module.
+
+
+
+
+ In order to interact with an RBAC-enabled cluster as an administrator, one
+ needs to have cluster-admin privileges. By default, when easyCerts is
+ enabled, a cluster-admin kubeconfig file is generated and linked into
+ /etc/kubernetes/cluster-admin.kubeconfig as determined by
+ .
+ export KUBECONFIG=/etc/kubernetes/cluster-admin.kubeconfig
+ will make kubectl use this kubeconfig to access and authenticate the cluster.
+ The cluster-admin kubeconfig references an auto-generated keypair owned by
+ root. Thus, only root on the kubernetes master may obtain cluster-admin
+ rights by means of this file.
+
+
+
diff --git a/nixos/doc/manual/release-notes/rl-1903.xml b/nixos/doc/manual/release-notes/rl-1903.xml
index 6f78983d482..269f27f74fb 100644
--- a/nixos/doc/manual/release-notes/rl-1903.xml
+++ b/nixos/doc/manual/release-notes/rl-1903.xml
@@ -54,6 +54,13 @@
to false and enable your preferred display manager.
+
+ A major refactoring of the Kubernetes module has been completed.
+ Refactorings primarily focus on decoupling components and enhancing
+ security. Two-way TLS and RBAC has been enabled by default for all
+ components, which slightly changes the way the module is configured.
+ See: for details.
+
@@ -564,6 +571,40 @@
provisioning.
+
+
+ The use of insecure ports on kubernetes has been deprecated.
+ Thus options:
+ services.kubernetes.apiserver.port and
+ services.kubernetes.controllerManager.port
+ has been renamed to .insecurePort,
+ and default of both options has changed to 0 (disabled).
+
+
+
+
+ Note that the default value of
+ services.kubernetes.apiserver.bindAddress
+ has changed from 127.0.0.1 to 0.0.0.0, allowing the apiserver to be
+ accessible from outside the master node itself.
+ If the apiserver insecurePort is enabled,
+ it is strongly recommended to only bind on the loopback interface. See:
+ services.kubernetes.apiserver.insecurebindAddress.
+
+
+
+
+ The option services.kubernetes.apiserver.allowPrivileged
+ and services.kubernetes.kubelet.allowPrivileged now
+ defaults to false. Disallowing privileged containers on the cluster.
+
+
+
+
+ The kubernetes module does no longer add the kubernetes package to
+ environment.systemPackages implicitly.
+
+