Merge branch 'master' into staging-next

.github/CONTRIBUTING.md

@@ -53,10 +53,10 @@ For package version upgrades and such a one-line commit message is usually suffi
 Follow these steps to backport a change into a release branch in compliance with the [commit policy](https://nixos.org/nixpkgs/manual/#submitting-changes-stable-release-branches).
 
 1. Take note of the commits in which the change was introduced into `master` branch.
-2. Check out the target _release branch_, e.g. `release-20.03`. Do not use a _channel branch_ like `nixos-20.03` or `nixpkgs-20.03`.
+2. Check out the target _release branch_, e.g. `release-20.09`. Do not use a _channel branch_ like `nixos-20.09` or `nixpkgs-20.09`.
 3. Create a branch for your change, e.g. `git checkout -b backport`.
 4. When the reason to backport is not obvious from the original commit message, use `git cherry-pick -xe <original commit>` and add a reason. Otherwise use `git cherry-pick -x <original commit>`. That's fine for minor version updates that only include security and bug fixes, commits that fixes an otherwise broken package or similar. Please also ensure the commits exists on the master branch; in the case of squashed or rebased merges, the commit hash will change and the new commits can be found in the merge message at the bottom of the master pull request.
-5. Push to GitHub and open a backport pull request. Make sure to select the release branch (e.g. `release-20.03`) as the target branch of the pull request, and link to the pull request in which the original change was comitted to `master`. The pull request title should be the commit title with the release version as prefix, e.g. `[20.03]`.
+5. Push to GitHub and open a backport pull request. Make sure to select the release branch (e.g. `release-20.09`) as the target branch of the pull request, and link to the pull request in which the original change was comitted to `master`. The pull request title should be the commit title with the release version as prefix, e.g. `[20.09]`.
 
 ## Reviewing contributions
 

.github/STALE-BOT.md

@@ -0,0 +1,34 @@
+# Stale bot information
+
+- Thanks for your contribution!
+- To remove the stale label, just leave a new comment.
+- _How to find the right people to ping?_ → [`git blame`](https://git-scm.com/docs/git-blame) to the rescue! (or GitHub's history and blame buttons.)
+- You can always ask for help on [our Discourse Forum](https://discourse.nixos.org/) or on the [#nixos IRC channel](https://webchat.freenode.net/#nixos).
+
+## Suggestions for PRs
+
+1. If it is unfinished but you plan to finish it, please mark it as a draft.
+2. If you don't expect to work on it any time soon, closing it with a short comment may encourage someone else to pick up your work.
+3. To get things rolling again, rebase the PR against the target branch and address valid comments.
+4. If you need a review to move forward, ask in [the Discourse thread for PRs that need help](https://discourse.nixos.org/t/prs-in-distress/3604).
+5. If all you need is a merge, check the git history to find and [request reviews](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/requesting-a-pull-request-review) from people who usually merge related contributions.
+
+## Suggestions for issues
+
+1. If it is resolved (either for you personally, or in general), please consider closing it.
+2. If this might still be an issue, but you are not interested in promoting its resolution, please consider closing it while encouraging others to take over and reopen an issue if they care enough.
+3. If you still have interest in resolving it, try to ping somebody who you believe might have an interest in the topic. Consider discussing the problem in [our Discourse Forum](https://discourse.nixos.org/).
+4. As with all open source projects, your best option is to submit a Pull Request that addresses this issue. We :heart: this attitude!
+
+**Memorandum on closing issues**
+
+Don't be afraid to close an issue that holds valuable information. Closed issues stay in the system for people to search, read, cross-reference, or even reopen--nothing is lost! Closing obsolete issues is an important way to help maintainers focus their time and effort.
+
+## Useful GitHub search queries
+
+- [Open PRs with any stale-bot interaction](https://github.com/NixOS/nixpkgs/pulls?q=is%3Apr+is%3Aopen+commenter%3Aapp%2Fstale+)
+- [Open PRs with any stale-bot interaction and `2.status: stale`](https://github.com/NixOS/nixpkgs/pulls?q=is%3Apr+is%3Aopen+commenter%3Aapp%2Fstale+label%3A%222.status%3A+stale%22)
+- [Open PRs with any stale-bot interaction and NOT `2.status: stale`](https://github.com/NixOS/nixpkgs/pulls?q=is%3Apr+is%3Aopen+commenter%3Aapp%2Fstale+-label%3A%222.status%3A+stale%22+)
+- [Open Issues with any stale-bot interaction](https://github.com/NixOS/nixpkgs/issues?q=is%3Aissue+is%3Aopen+commenter%3Aapp%2Fstale+)
+- [Open Issues with any stale-bot interaction and `2.status: stale`](https://github.com/NixOS/nixpkgs/issues?q=is%3Aissue+is%3Aopen+commenter%3Aapp%2Fstale+label%3A%222.status%3A+stale%22+)
+- [Open Issues with any stale-bot interaction and NOT `2.status: stale`](https://github.com/NixOS/nixpkgs/issues?q=is%3Aissue+is%3Aopen+commenter%3Aapp%2Fstale+-label%3A%222.status%3A+stale%22+)

.github/stale.yml

@@ -1,41 +1,10 @@
 # Configuration for probot-stale - https://github.com/probot/stale
-# Number of days of inactivity before an issue becomes stale
 daysUntilStale: 180
-# Number of days of inactivity before a stale issue is closed
 daysUntilClose: false
-# Issues with these labels will never be considered stale
 exemptLabels:
   - "1.severity: security"
-# Label to use when marking an issue as stale
+  - "2.status: never-stale"
 staleLabel: "2.status: stale"
-# Comment to post when marking an issue as stale. Set to `false` to disable
+markComment: |
-pulls:
+  I marked this as stale due to inactivity. → [More info](https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md)
-  markComment: |
-    Hello, I'm a bot and I thank you in the name of the community for your contributions.
-
-    Nixpkgs is a busy repository, and unfortunately sometimes PRs get left behind for too long. Nevertheless, we'd like to help committers reach the PRs that are still important. This PR has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.
-
-    If this is still important to you and you'd like to remove the stale label, we ask that you leave a comment. Your comment can be as simple as "still important to me".  But there's a bit more you can do:
-
-    If you received an approval by an unprivileged maintainer and you are just waiting for a merge, you can @ mention someone with merge permissions and ask them to help. You might be able to find someone relevant by using [Git blame](https://git-scm.com/docs/git-blame) on the relevant files, or via [GitHub's web interface](https://docs.github.com/en/github/managing-files-in-a-repository/tracking-changes-in-a-file). You can see if someone's a member of the [nixpkgs-committers](https://github.com/orgs/NixOS/teams/nixpkgs-committers) team, by hovering with the mouse over their username on the web interface, or by searching them directly on [the list](https://github.com/orgs/NixOS/teams/nixpkgs-committers).
-
-    If your PR wasn't reviewed at all, it might help to find someone who's perhaps a user of the package or module you are changing, or alternatively, ask once more for a review by the maintainer of the package/module this is about. If you don't know any, you can use [Git blame](https://git-scm.com/docs/git-blame) on the relevant files, or [GitHub's web interface](https://docs.github.com/en/github/managing-files-in-a-repository/tracking-changes-in-a-file) to find someone who touched the relevant files in the past.
-
-    If your PR has had reviews and nevertheless got stale, make sure you've responded to all of the reviewer's requests / questions. Usually when PR authors show responsibility and dedication, reviewers (privileged or not) show dedication as well. If you've pushed a change, it's possible the reviewer wasn't notified about your push via email, so you can always [officially request them for a review](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/requesting-a-pull-request-review), or just @ mention them and say you've addressed their comments.
-
-    Lastly, you can always ask for help at [our Discourse Forum](https://discourse.nixos.org/), or more specifically, [at this thread](https://discourse.nixos.org/t/prs-in-distress/3604) or at [#nixos' IRC channel](https://webchat.freenode.net/#nixos).
-
-issues:
-  markComment: |
-    Hello, I'm a bot and I thank you in the name of the community for opening this issue.
-
-    To help our human contributors focus on the most-relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.
-
-    The community would appreciate your effort in checking if the issue is still valid. If it isn't, please close it.
-
-    If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me". If you'd like it to get more attention, you can ask for help by searching for maintainers and people that previously touched related code and @ mention them in a comment. You can use [Git blame](https://git-scm.com/docs/git-blame) or [GitHub's web interface](https://docs.github.com/en/github/managing-files-in-a-repository/tracking-changes-in-a-file) on the relevant files to find them.
-
-    Lastly, you can always ask for help at [our Discourse Forum](https://discourse.nixos.org/) or at [#nixos' IRC channel](https://webchat.freenode.net/#nixos).
-
-# Comment to post when closing a stale issue. Set to `false` to disable
 closeComment: false

README.md

@@ -46,9 +46,9 @@ Nixpkgs and NixOS are built and tested by our continuous integration
 system, [Hydra](https://hydra.nixos.org/).
 
 * [Continuous package builds for unstable/master](https://hydra.nixos.org/jobset/nixos/trunk-combined)
-* [Continuous package builds for the NixOS 20.03 release](https://hydra.nixos.org/jobset/nixos/release-20.03)
+* [Continuous package builds for the NixOS 20.09 release](https://hydra.nixos.org/jobset/nixos/release-20.09)
 * [Tests for unstable/master](https://hydra.nixos.org/job/nixos/trunk-combined/tested#tabs-constituents)
-* [Tests for the NixOS 20.03 release](https://hydra.nixos.org/job/nixos/release-20.03/tested#tabs-constituents)
+* [Tests for the NixOS 20.09 release](https://hydra.nixos.org/job/nixos/release-20.09/tested#tabs-constituents)
 
 Artifacts successfully built with Hydra are published to cache at
 https://cache.nixos.org/. When successful build and test criteria are

doc/contributing/coding-conventions.xml

@@ -522,6 +522,16 @@ args.stdenv.mkDerivation (args // {
          </para>
         </listitem>
        </varlistentry>
+       <varlistentry>
+        <term>
+         If it’s a <emphasis>terminal emulator</emphasis>:
+        </term>
+        <listitem>
+         <para>
+          <filename>applications/terminal-emulators</filename> (e.g. <filename>alacritty</filename> or <filename>rxvt</filename> or <filename>termite</filename>)
+         </para>
+        </listitem>
+       </varlistentry>
        <varlistentry>
         <term>
          If it’s for <emphasis>video playback / editing</emphasis>:

doc/languages-frameworks/vim.section.md

@@ -265,6 +265,20 @@ To add a new plugin, run `./update.py --add "[owner]/[name]"`. **NOTE**: This sc
 
 Finally, there are some plugins that are also packaged in nodePackages because they have Javascript-related build steps, such as running webpack. Those plugins are not listed in `vim-plugin-names` or managed by `update.py` at all, and are included separately in `overrides.nix`. Currently, all these plugins are related to the `coc.nvim` ecosystem of Language Server Protocol integration with vim/neovim.
 
+## Updating plugins in nixpkgs
+
+Run the update script with a GitHub API token that has at least `public_repo` access. Running the script without the token is likely to result in rate-limiting (429 errors). For steps on creating an API token, please refer to [GitHub's token documentation](https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/creating-a-personal-access-token).
+
+```sh
+GITHUB_API_TOKEN=my_token ./pkgs/misc/vim-plugins/update.py
+```
+
+Alternatively, set the number of processes to a lower count to avoid rate-limiting.
+
+```sh
+./pkgs/misc/vim-plugins/update.py --proc 1
+```
+
 ## Important repositories
 
 - [vim-pi](https://bitbucket.org/vimcommunity/vim-pi) is a plugin repository

lib/debug.nix

@@ -14,9 +14,25 @@
 */
 { lib }:
 let
-  inherit (builtins) trace isAttrs isList isInt
+  inherit (lib)
-          head substring attrNames;
+    isInt
-  inherit (lib) id elem isFunction;
+    attrNames
+    isList
+    isAttrs
+    substring
+    addErrorContext
+    attrValues
+    concatLists
+    concatStringsSep
+    const
+    elem
+    generators
+    head
+    id
+    isDerivation
+    isFunction
+    mapAttrs
+    trace;
 in
 
 rec {
@@ -94,7 +110,7 @@ rec {
        trace: { a = { b = {…}; }; }
        => null
    */
-  traceSeqN = depth: x: y: with lib;
+  traceSeqN = depth: x: y:
     let snip = v: if      isList  v then noQuotes "[…]" v
                   else if isAttrs v then noQuotes "{…}" v
                   else v;
@@ -149,7 +165,7 @@ rec {
   */
   runTests =
     # Tests to run
-    tests: lib.concatLists (lib.attrValues (lib.mapAttrs (name: test:
+    tests: concatLists (attrValues (mapAttrs (name: test:
     let testsToRun = if tests ? tests then tests.tests else [];
     in if (substring 0 4 name == "test" ||  elem name testsToRun)
        && ((testsToRun == []) || elem name tests.tests)
@@ -176,9 +192,9 @@ rec {
           + "and will be removed in the next release. "
           + "Please use more specific concatenation "
           + "for your uses (`lib.concat(Map)StringsSep`)." )
-    (lib.concatStringsSep "; " (map (x: "${x}=") (attrNames a)));
+    (concatStringsSep "; " (map (x: "${x}=") (attrNames a)));
 
-  showVal = with lib;
+  showVal =
     trace ( "Warning: `showVal` is deprecated "
           + "and will be removed in the next release, "
           + "please use `traceSeqN`" )
@@ -226,7 +242,7 @@ rec {
     trace ( "Warning: `addErrorContextToAttrs` is deprecated "
           + "and will be removed in the next release. "
           + "Please use `builtins.addErrorContext` directly." )
-    (lib.mapAttrs (a: v: lib.addErrorContext "while evaluating ${a}" v) attrs);
+    (mapAttrs (a: v: addErrorContext "while evaluating ${a}" v) attrs);
 
   # example: (traceCallXml "myfun" id 3) will output something like
   # calling myfun arg 1: 3 result: 3

lib/default.nix

@@ -9,7 +9,7 @@ let
 
   lib = makeExtensible (self: let
     callLibs = file: import file { lib = self; };
-  in with self; {
+  in {
 
     # often used, or depending on very little
     trivial = callLibs ./trivial.nix;
@@ -54,7 +54,7 @@ let
     filesystem = callLibs ./filesystem.nix;
 
     # back-compat aliases
-    platforms = systems.doubles;
+    platforms = self.systems.doubles;
 
     # linux kernel configuration
     kernel = callLibs ./kernel.nix;
@@ -63,14 +63,14 @@ let
       deepSeq elem elemAt filter genericClosure genList getAttr
       hasAttr head isAttrs isBool isInt isList isString length
       lessThan listToAttrs pathExists readFile replaceStrings seq
-      stringLength sub substring tail;
+      stringLength sub substring tail trace;
-    inherit (trivial) id const pipe concat or and bitAnd bitOr bitXor
+    inherit (self.trivial) id const pipe concat or and bitAnd bitOr bitXor
-      bitNot boolToString mergeAttrs flip mapNullable inNixShell min max
+      bitNot boolToString mergeAttrs flip mapNullable inNixShell isFloat min max
       importJSON importTOML warn info showWarnings nixpkgsVersion version mod compare
       splitByAndCompare functionArgs setFunctionArgs isFunction toHexString toBaseDigits;
-    inherit (fixedPoints) fix fix' converge extends composeExtensions
+    inherit (self.fixedPoints) fix fix' converge extends composeExtensions
       makeExtensible makeExtensibleWithCustomName;
-    inherit (attrsets) attrByPath hasAttrByPath setAttrByPath
+    inherit (self.attrsets) attrByPath hasAttrByPath setAttrByPath
       getAttrFromPath attrVals attrValues getAttrs catAttrs filterAttrs
       filterAttrsRecursive foldAttrs collect nameValuePair mapAttrs
       mapAttrs' mapAttrsToList mapAttrsRecursive mapAttrsRecursiveCond
@@ -79,13 +79,13 @@ let
       recursiveUpdate matchAttrs overrideExisting getOutput getBin
       getLib getDev getMan chooseDevOutputs zipWithNames zip
       recurseIntoAttrs dontRecurseIntoAttrs;
-    inherit (lists) singleton forEach foldr fold foldl foldl' imap0 imap1
+    inherit (self.lists) singleton forEach foldr fold foldl foldl' imap0 imap1
       concatMap flatten remove findSingle findFirst any all count
       optional optionals toList range partition zipListsWith zipLists
       reverseList listDfs toposort sort naturalSort compareLists take
       drop sublist last init crossLists unique intersectLists
       subtractLists mutuallyExclusive groupBy groupBy';
-    inherit (strings) concatStrings concatMapStrings concatImapStrings
+    inherit (self.strings) concatStrings concatMapStrings concatImapStrings
       intersperse concatStringsSep concatMapStringsSep
       concatImapStringsSep makeSearchPath makeSearchPathOutput
       makeLibraryPath makeBinPath optionalString
@@ -97,19 +97,19 @@ let
       nameFromURL enableFeature enableFeatureAs withFeature
       withFeatureAs fixedWidthString fixedWidthNumber isStorePath
       toInt readPathsFromFile fileContents;
-    inherit (stringsWithDeps) textClosureList textClosureMap
+    inherit (self.stringsWithDeps) textClosureList textClosureMap
       noDepEntry fullDepEntry packEntry stringAfter;
-    inherit (customisation) overrideDerivation makeOverridable
+    inherit (self.customisation) overrideDerivation makeOverridable
       callPackageWith callPackagesWith extendDerivation hydraJob
       makeScope;
-    inherit (meta) addMetaAttrs dontDistribute setName updateName
+    inherit (self.meta) addMetaAttrs dontDistribute setName updateName
       appendToName mapDerivationAttrset setPrio lowPrio lowPrioSet hiPrio
       hiPrioSet;
-    inherit (sources) pathType pathIsDirectory cleanSourceFilter
+    inherit (self.sources) pathType pathIsDirectory cleanSourceFilter
       cleanSource sourceByRegex sourceFilesBySuffices
       commitIdFromGitRepo cleanSourceWith pathHasContext
       canCleanSource pathIsRegularFile pathIsGitRepo;
-    inherit (modules) evalModules unifyModuleSyntax
+    inherit (self.modules) evalModules unifyModuleSyntax
       applyIfFunction mergeModules
       mergeModules' mergeOptionDecls evalOptionValue mergeDefinitions
       pushDownProperties dischargeProperties filterOverrides
@@ -119,21 +119,21 @@ let
       mkAliasAndWrapDefinitions fixMergeModules mkRemovedOptionModule
       mkRenamedOptionModule mkMergedOptionModule mkChangedOptionModule
       mkAliasOptionModule doRename;
-    inherit (options) isOption mkEnableOption mkSinkUndeclaredOptions
+    inherit (self.options) isOption mkEnableOption mkSinkUndeclaredOptions
       mergeDefaultOption mergeOneOption mergeEqualOption getValues
       getFiles optionAttrSetToDocList optionAttrSetToDocList'
       scrubOptionValue literalExample showOption showFiles
       unknownModule mkOption;
-    inherit (types) isType setType defaultTypeMerge defaultFunctor
+    inherit (self.types) isType setType defaultTypeMerge defaultFunctor
       isOptionType mkOptionType;
-    inherit (asserts)
+    inherit (self.asserts)
       assertMsg assertOneOf;
-    inherit (debug) addErrorContextToAttrs traceIf traceVal traceValFn
+    inherit (self.debug) addErrorContextToAttrs traceIf traceVal traceValFn
       traceXMLVal traceXMLValMarked traceSeq traceSeqN traceValSeq
       traceValSeqFn traceValSeqN traceValSeqNFn traceShowVal
       traceShowValMarked showVal traceCall traceCall2 traceCall3
       traceValIfNot runTests testAllTrue traceCallXml attrNamesToStr;
-    inherit (misc) maybeEnv defaultMergeArg defaultMerge foldArgs
+    inherit (self.misc) maybeEnv defaultMergeArg defaultMerge foldArgs
       maybeAttrNullable maybeAttr ifEnable checkFlag getValue
       checkReqs uniqList uniqListExt condConcat lazyGenericClosure
       innerModifySumArgs modifySumArgs innerClosePropagation
@@ -143,7 +143,7 @@ let
       mergeAttrsByFuncDefaultsClean mergeAttrBy
       fakeHash fakeSha256 fakeSha512
       nixType imap;
-    inherit (versions)
+    inherit (self.versions)
       splitVersion;
   });
 in lib

lib/lists.nix

@@ -1,9 +1,9 @@
 # General list operations.
 
 { lib }:
-with lib.trivial;
 let
   inherit (lib.strings) toInt;
+  inherit (lib.trivial) compare min;
 in
 rec {
 

lib/modules.nix

@@ -1,12 +1,53 @@
 { lib }:
 
-with lib.lists;
+let
-with lib.strings;
+  inherit (lib)
-with lib.trivial;
+    all
-with lib.attrsets;
+    any
-with lib.options;
+    attrByPath
-with lib.debug;
+    attrNames
-with lib.types;
+    catAttrs
+    concatLists
+    concatMap
+    count
+    elem
+    filter
+    findFirst
+    flip
+    foldl
+    foldl'
+    getAttrFromPath
+    head
+    id
+    imap1
+    isAttrs
+    isBool
+    isFunction
+    isString
+    length
+    mapAttrs
+    mapAttrsToList
+    mapAttrsRecursiveCond
+    min
+    optional
+    optionalAttrs
+    optionalString
+    recursiveUpdate
+    reverseList sort
+    setAttrByPath
+    toList
+    types
+    warn
+    ;
+  inherit (lib.options)
+    isOption
+    mkOption
+    showDefs
+    showFiles
+    showOption
+    unknownModule
+    ;
+in
 
 rec {
 
@@ -616,7 +657,7 @@ rec {
   fixupOptionType = loc: opt:
     let
       options = opt.options or
-        (throw "Option `${showOption loc'}' has type optionSet but has no option attribute, in ${showFiles opt.declarations}.");
+        (throw "Option `${showOption loc}' has type optionSet but has no option attribute, in ${showFiles opt.declarations}.");
       f = tp:
         let optionSetIn = type: (tp.name == type) && (tp.functor.wrapped.name == "optionSet");
         in

lib/options.nix

@@ -1,11 +1,40 @@
 # Nixpkgs/NixOS option handling.
 { lib }:
 
-with lib.trivial;
+let
-with lib.lists;
+  inherit (lib)
-with lib.attrsets;
+    all
-with lib.strings;
+    collect
-
+    concatLists
+    concatMap
+    elemAt
+    filter
+    foldl'
+    head
+    isAttrs
+    isBool
+    isDerivation
+    isFunction
+    isInt
+    isList
+    isString
+    length
+    mapAttrs
+    optional
+    optionals
+    take
+    ;
+  inherit (lib.attrsets)
+    optionalAttrs
+    ;
+  inherit (lib.strings)
+    concatMapStrings
+    concatStringsSep
+    ;
+  inherit (lib.types)
+    mkOptionType
+    ;
+in
 rec {
 
   /* Returns true when the given argument is an option
@@ -110,7 +139,7 @@ rec {
     # Return early if we only have one element
     # This also makes it work for functions, because the foldl' below would try
     # to compare the first element with itself, which is false for functions
-    else if length defs == 1 then (elemAt defs 0).value
+    else if length defs == 1 then (head defs).value
     else (foldl' (first: def:
       if def.value != first.value then
         throw "The option `${showOption loc}' has conflicting definition values:${showDefs [ first def ]}"

lib/sources.nix

@@ -1,16 +1,33 @@
 # Functions for copying sources to the Nix store.
 { lib }:
 
+let
+  inherit (builtins)
+    hasContext
+    match
+    readDir
+    split
+    storeDir
+    tryEval
+    ;
+  inherit (lib)
+    filter
+    getAttr
+    isString
+    pathExists
+    readFile
+    ;
+in
 rec {
 
   # Returns the type of a path: regular (for file), symlink, or directory
-  pathType = p: with builtins; getAttr (baseNameOf p) (readDir (dirOf p));
+  pathType = p: getAttr (baseNameOf p) (readDir (dirOf p));
 
   # Returns true if the path exists and is a directory, false otherwise
-  pathIsDirectory = p: if builtins.pathExists p then (pathType p) == "directory" else false;
+  pathIsDirectory = p: if pathExists p then (pathType p) == "directory" else false;
 
   # Returns true if the path exists and is a regular file, false otherwise
-  pathIsRegularFile = p: if builtins.pathExists p then (pathType p) == "regular" else false;
+  pathIsRegularFile = p: if pathExists p then (pathType p) == "regular" else false;
 
   # Bring in a path as a source, filtering out all Subversion and CVS
   # directories, as well as backup files (*~).
@@ -19,8 +36,8 @@ rec {
     (baseName == ".git" || type == "directory" && (baseName == ".svn" || baseName == "CVS" || baseName == ".hg")) ||
     # Filter out editor backup / swap files.
     lib.hasSuffix "~" baseName ||
-    builtins.match "^\\.sw[a-z]$" baseName != null ||
+    match "^\\.sw[a-z]$" baseName != null ||
-    builtins.match "^\\..*\\.sw[a-z]$" baseName != null ||
+    match "^\\..*\\.sw[a-z]$" baseName != null ||
 
     # Filter out generates files.
     lib.hasSuffix ".o" baseName ||
@@ -89,7 +106,7 @@ rec {
     in lib.cleanSourceWith {
       filter = (path: type:
         let relPath = lib.removePrefix (toString origSrc + "/") (toString path);
-        in lib.any (re: builtins.match re relPath != null) regexes);
+        in lib.any (re: match re relPath != null) regexes);
       inherit src;
     };
 
@@ -102,13 +119,12 @@ rec {
       in type == "directory" || lib.any (ext: lib.hasSuffix ext base) exts;
     in cleanSourceWith { inherit filter; src = path; };
 
-  pathIsGitRepo = path: (builtins.tryEval (commitIdFromGitRepo path)).success;
+  pathIsGitRepo = path: (tryEval (commitIdFromGitRepo path)).success;
 
   # Get the commit id of a git repo
   # Example: commitIdFromGitRepo <nixpkgs/.git>
   commitIdFromGitRepo =
     let readCommitFromFile = file: path:
-      with builtins;
         let fileName       = toString path + "/" + file;
             packedRefsName = toString path + "/packed-refs";
             absolutePath   = base: path:
@@ -145,11 +161,11 @@ rec {
            # packed-refs file, so we have to grep through it:
            then
              let fileContent = readFile packedRefsName;
-                 matchRef = builtins.match "([a-z0-9]+) ${file}";
+                 matchRef = match "([a-z0-9]+) ${file}";
-                 isRef = s: builtins.isString s && (matchRef s) != null;
+                 isRef = s: isString s && (matchRef s) != null;
                  # there is a bug in libstdc++ leading to stackoverflow for long strings:
                  # https://github.com/NixOS/nix/issues/2147#issuecomment-659868795
-                 refs = builtins.filter isRef (builtins.split "\n" fileContent);
+                 refs = filter isRef (split "\n" fileContent);
              in if refs == []
                 then throw ("Could not find " + file + " in " + packedRefsName)
                 else lib.head (matchRef (lib.head refs))
@@ -157,7 +173,7 @@ rec {
            else throw ("Not a .git directory: " + path);
     in readCommitFromFile "HEAD";
 
-  pathHasContext = builtins.hasContext or (lib.hasPrefix builtins.storeDir);
+  pathHasContext = builtins.hasContext or (lib.hasPrefix storeDir);
 
   canCleanSource = src: src ? _isLibCleanSourceWith || !(pathHasContext (toString src));
 }

lib/strings-with-deps.nix

@@ -41,10 +41,15 @@ Usage:
   [1] maybe this behaviour should be removed to keep things simple (?)
 */
 
-with lib.lists;
+let
-with lib.attrsets;
+  inherit (lib)
-with lib.strings;
+    concatStringsSep
-
+    head
+    isAttrs
+    listToAttrs
+    tail
+    ;
+in
 rec {
 
   /* !!! The interface of this function is kind of messed up, since

lib/strings.nix

@@ -8,7 +8,29 @@ in
 
 rec {
 
-  inherit (builtins) stringLength substring head tail isString replaceStrings;
+  inherit (builtins)
+    compareVersions
+    elem
+    elemAt
+    filter
+    fromJSON
+    head
+    isInt
+    isList
+    isString
+    match
+    parseDrvName
+    readFile
+    replaceStrings
+    split
+    storeDir
+    stringLength
+    substring
+    tail
+    toJSON
+    typeOf
+    unsafeDiscardStringContext
+    ;
 
   /* Concatenate a list of strings.
 
@@ -120,7 +142,7 @@ rec {
     subDir:
     # List of base paths
     paths:
-    concatStringsSep ":" (map (path: path + "/" + subDir) (builtins.filter (x: x != null) paths));
+    concatStringsSep ":" (map (path: path + "/" + subDir) (filter (x: x != null) paths));
 
   /* Construct a Unix-style search path by appending the given
      `subDir` to the specified `output` of each of the packages. If no
@@ -313,7 +335,7 @@ rec {
        escapeNixString "hello\${}\n"
        => "\"hello\\\${}\\n\""
   */
-  escapeNixString = s: escape ["$"] (builtins.toJSON s);
+  escapeNixString = s: escape ["$"] (toJSON s);
 
   /* Turn a string into an exact regular expression
 
@@ -337,7 +359,7 @@ rec {
   */
   escapeNixIdentifier = s:
     # Regex from https://github.com/NixOS/nix/blob/d048577909e383439c2549e849c5c2f2016c997e/src/libexpr/lexer.l#L91
-    if builtins.match "[a-zA-Z_][a-zA-Z0-9_'-]*" s != null
+    if match "[a-zA-Z_][a-zA-Z0-9_'-]*" s != null
     then s else escapeNixString s;
 
   # Obsolete - use replaceStrings instead.
@@ -466,7 +488,7 @@ rec {
        versionOlder "1.1" "1.1"
        => false
   */
-  versionOlder = v1: v2: builtins.compareVersions v2 v1 == 1;
+  versionOlder = v1: v2: compareVersions v2 v1 == 1;
 
   /* Return true if string v1 denotes a version equal to or newer than v2.
 
@@ -492,7 +514,7 @@ rec {
   */
   getName = x:
    let
-     parse = drv: (builtins.parseDrvName drv).name;
+     parse = drv: (parseDrvName drv).name;
    in if isString x
       then parse x
       else x.pname or (parse x.name);
@@ -509,7 +531,7 @@ rec {
   */
   getVersion = x:
    let
-     parse = drv: (builtins.parseDrvName drv).version;
+     parse = drv: (parseDrvName drv).version;
    in if isString x
       then parse x
       else x.version or (parse x.name);
@@ -527,7 +549,7 @@ rec {
     let
       components = splitString "/" url;
       filename = lib.last components;
-      name = builtins.head (splitString sep filename);
+      name = head (splitString sep filename);
     in assert name != filename; name;
 
   /* Create an --{enable,disable}-<feat> string that can be passed to
@@ -617,14 +639,14 @@ rec {
   */
   floatToString = float: let
     result = toString float;
-    precise = float == builtins.fromJSON result;
+    precise = float == fromJSON result;
   in if precise then result
     else lib.warn "Imprecise conversion from float to string ${result}" result;
 
   /* Check whether a value can be coerced to a string */
   isCoercibleToString = x:
-    builtins.elem (builtins.typeOf x) [ "path" "string" "null" "int" "float" "bool" ] ||
+    elem (typeOf x) [ "path" "string" "null" "int" "float" "bool" ] ||
-    (builtins.isList x && lib.all isCoercibleToString x) ||
+    (isList x && lib.all isCoercibleToString x) ||
     x ? outPath ||
     x ? __toString;
 
@@ -643,8 +665,8 @@ rec {
   isStorePath = x:
     if isCoercibleToString x then
       let str = toString x; in
-      builtins.substring 0 1 str == "/"
+      substring 0 1 str == "/"
-      && dirOf str == builtins.storeDir
+      && dirOf str == storeDir
     else
       false;
 
@@ -662,8 +684,8 @@ rec {
   */
   # Obviously, it is a bit hacky to use fromJSON this way.
   toInt = str:
-    let may_be_int = builtins.fromJSON str; in
+    let may_be_int = fromJSON str; in
-    if builtins.isInt may_be_int
+    if isInt may_be_int
     then may_be_int
     else throw "Could not convert ${str} to int.";
 
@@ -685,10 +707,10 @@ rec {
   readPathsFromFile = lib.warn "lib.readPathsFromFile is deprecated, use a list instead"
     (rootPath: file:
       let
-        lines = lib.splitString "\n" (builtins.readFile file);
+        lines = lib.splitString "\n" (readFile file);
         removeComments = lib.filter (line: line != "" && !(lib.hasPrefix "#" line));
         relativePaths = removeComments lines;
-        absolutePaths = builtins.map (path: rootPath + "/${path}") relativePaths;
+        absolutePaths = map (path: rootPath + "/${path}") relativePaths;
       in
         absolutePaths);
 
@@ -702,7 +724,7 @@ rec {
        fileContents ./version
        => "1.0"
   */
-  fileContents = file: removeSuffix "\n" (builtins.readFile file);
+  fileContents = file: removeSuffix "\n" (readFile file);
 
 
   /* Creates a valid derivation name from a potentially invalid one.
@@ -720,13 +742,13 @@ rec {
   sanitizeDerivationName = string: lib.pipe string [
     # Get rid of string context. This is safe under the assumption that the
     # resulting string is only used as a derivation name
-    builtins.unsafeDiscardStringContext
+    unsafeDiscardStringContext
     # Strip all leading "."
-    (x: builtins.elemAt (builtins.match "\\.*(.*)" x) 0)
+    (x: elemAt (match "\\.*(.*)" x) 0)
     # Split out all invalid characters
     # https://github.com/NixOS/nix/blob/2.3.2/src/libstore/store-api.cc#L85-L112
     # https://github.com/NixOS/nix/blob/2242be83c61788b9c0736a92bb0b5c7bbfc40803/nix-rust/src/store/path.rs#L100-L125
-    (builtins.split "[^[:alnum:]+._?=-]+")
+    (split "[^[:alnum:]+._?=-]+")
     # Replace invalid character ranges with a "-"
     (concatMapStrings (s: if lib.isList s then "-" else s))
     # Limit to 211 characters (minus 4 chars for ".drv")

lib/types.nix

@@ -1,12 +1,65 @@
 # Definitions related to run-time type checking.  Used in particular
 # to type-check NixOS configurations.
 { lib }:
-with lib.lists;
+
-with lib.attrsets;
-with lib.options;
-with lib.trivial;
-with lib.strings;
 let
+  inherit (lib)
+    elem
+    flip
+    functionArgs
+    isAttrs
+    isBool
+    isDerivation
+    isFloat
+    isFunction
+    isInt
+    isList
+    isString
+    isStorePath
+    setFunctionArgs
+    toDerivation
+    toList
+    ;
+  inherit (lib.lists)
+    all
+    concatLists
+    count
+    elemAt
+    filter
+    foldl'
+    head
+    imap1
+    last
+    length
+    tail
+    unique
+    ;
+  inherit (lib.attrsets)
+    attrNames
+    filterAttrs
+    hasAttr
+    mapAttrs
+    optionalAttrs
+    zipAttrsWith
+    ;
+  inherit (lib.options)
+    getFiles
+    getValues
+    mergeDefaultOption
+    mergeEqualOption
+    mergeOneOption
+    showFiles
+    showOption
+    ;
+  inherit (lib.strings)
+    concatMapStringsSep
+    concatStringsSep
+    escapeNixString
+    isCoercibleToString
+    ;
+  inherit (lib.trivial)
+    boolToString
+    ;
 
   inherit (lib.modules) mergeDefinitions;
   outer_types =
@@ -270,7 +323,7 @@ rec {
       name = "attrs";
       description = "attribute set";
       check = isAttrs;
-      merge = loc: foldl' (res: def: mergeAttrs res def.value) {};
+      merge = loc: foldl' (res: def: res // def.value) {};
       emptyValue = { value = {}; };
     };
 

maintainers/maintainer-list.nix

@@ -1303,10 +1303,12 @@
     name = "Vladimir Serov";
     keys = [
       # compare with https://keybase.io/cab404
-      { longkeyid = "1BB96810926F4E715DEF567E6BA7C26C3FDF7BB3";
+      {
+        longkeyid = "1BB96810926F4E715DEF567E6BA7C26C3FDF7BB3";
         fingerprint = "rsa3072/0xCBDECF658C38079E";
       }
-      { longkeyid = "1EBC648C64D6045463013B3EB7EFFC271D55DB8A";
+      {
+        longkeyid = "1EBC648C64D6045463013B3EB7EFFC271D55DB8A";
         fingerprint = "ed25519/0xB7EFFC271D55DB8A";
       }
     ];
@@ -1431,10 +1433,12 @@
     githubId = 2054509;
     name = "Constantine Evans";
     keys = [
-      { longkeyid = "rsa4096/0xB67DB1D20A93A9F9";
+      {
+        longkeyid = "rsa4096/0xB67DB1D20A93A9F9";
         fingerprint = "32B1 6EE7 DBA5 16DE 526E  4C5A B67D B1D2 0A93 A9F9";
       }
-      { longkeyid = "rsa4096/0x1A1D58B86AE2AABD";
+      {
+        longkeyid = "rsa4096/0x1A1D58B86AE2AABD";
         fingerprint = "669C 1D24 5A87 DB34 6BE4  3216 1A1D 58B8 6AE2 AABD";
       }
     ];
@@ -2169,6 +2173,12 @@
     githubId = 8852888;
     name = "David Izquierdo";
   };
+  djanatyn = {
+    email = "djanatyn@gmail.com";
+    github = "djanatyn";
+    githubId = 523628;
+    name = "Jonathan Strickland";
+  };
   Dje4321 = {
     email = "dje4321@gmail.com";
     github = "dje4321";
@@ -3437,6 +3447,12 @@
     githubId = 2405974;
     name = "Sébastian Méric de Bellefon";
   };
+  henrikolsson = {
+    email = "henrik@fixme.se";
+    github = "henrikolsson";
+    githubId = 982322;
+    name = "Henrik Olsson";
+  };
   henrytill = {
     email = "henrytill@gmail.com";
     github = "henrytill";
@@ -5673,12 +5689,6 @@
     githubId = 1387206;
     name = "Mike Sperber";
   };
-  mildlyincompetent = {
-    email = "nix@kch.dev";
-    github = "mildlyincompetent";
-    githubId = 19479662;
-    name = "Kajetan Champlewski";
-  };
   millerjason = {
     email = "mailings-github@millerjason.com";
     github = "millerjason";
@@ -7263,10 +7273,12 @@
     github = "rissson";
     githubId = 18313093;
     keys = [
-      { longkeyid = "rsa4096/0xF6FD87B15C263EC9";
+      {
+        longkeyid = "rsa4096/0xF6FD87B15C263EC9";
         fingerprint = "8A0E 6A7C 08AB B9DE 67DE  2A13 F6FD 87B1 5C26 3EC9";
       }
-      { longkeyid = "ed25519/0xBBB7A6801DF1E03F";
+      {
+        longkeyid = "ed25519/0xBBB7A6801DF1E03F";
         fingerprint = "C0A7 A9BB 115B C857 4D75  EA99 BBB7 A680 1DF1 E03F";
       }
     ];
@@ -8346,7 +8358,7 @@
     name = "Szczyp";
   };
   szlend = {
-    email = "pub+nix@zlender.si";
+    email = "pub.nix@zlender.si";
     github = "szlend";
     githubId = 7301807;
     name = "Simon Žlender";
@@ -9232,6 +9244,12 @@
     email = "kirill.wedens@gmail.com";
     name = "wedens";
   };
+  wheelsandmetal = {
+    email = "jakob@schmutz.co.uk";
+    github = "wheelsandmetal";
+    githubId = 13031455;
+    name = "Jakob Schmutz";
+  };
   WhittlesJr = {
     email = "alex.joseph.whitt@gmail.com";
     github = "WhittlesJr";
@@ -9316,7 +9334,7 @@
     email = "worldofpeace@protonmail.ch";
     github = "worldofpeace";
     githubId = 28888242;
-    name = "worldofpeace";
+    name = "WORLDofPEACE";
   };
   wscott = {
     email = "wsc9tt@gmail.com";

maintainers/scripts/hydra-eval-failures.py

@@ -1,5 +1,5 @@
 #!/usr/bin/env nix-shell
-#!nix-shell -i python3 -p 'python3.withPackages(ps: with ps; [ requests pyquery click ])'
+#!nix-shell -i python3 -p "python3.withPackages(ps: with ps; [ requests pyquery click ])"
 
 # To use, just execute this script with --help to display help.
 

nixos/doc/manual/configuration/user-mgmt.xml

@@ -38,7 +38,7 @@
   assigned by setting the user's
   <link linkend="opt-users.users._name_.hashedPassword">hashedPassword</link>
   option. A hashed password can be generated using <command>mkpasswd -m
-  sha-512</command> after installing the <literal>mkpasswd</literal> package.
+  sha-512</command>.
  </para>
  <para>
   A user ID (uid) is assigned automatically. You can also specify a uid

nixos/doc/manual/installation/upgrading.xml

@@ -14,7 +14,7 @@
     <para>
      <emphasis>Stable channels</emphasis>, such as
      <literal
-    xlink:href="https://nixos.org/channels/nixos-20.03">nixos-20.03</literal>.
+    xlink:href="https://nixos.org/channels/nixos-20.09">nixos-20.09</literal>.
      These only get conservative bug fixes and package upgrades. For instance,
      a channel update may cause the Linux kernel on your system to be upgraded
      from 4.19.34 to 4.19.38 (a minor bug fix), but not from
@@ -38,7 +38,7 @@
     <para>
      <emphasis>Small channels</emphasis>, such as
      <literal
-    xlink:href="https://nixos.org/channels/nixos-20.03-small">nixos-20.03-small</literal>
+    xlink:href="https://nixos.org/channels/nixos-20.09-small">nixos-20.09-small</literal>
      or
      <literal
     xlink:href="https://nixos.org/channels/nixos-unstable-small">nixos-unstable-small</literal>.
@@ -63,8 +63,8 @@
  <para>
   When you first install NixOS, you’re automatically subscribed to the NixOS
   channel that corresponds to your installation source. For instance, if you
-  installed from a 20.03 ISO, you will be subscribed to the
+  installed from a 20.09 ISO, you will be subscribed to the
-  <literal>nixos-20.03</literal> channel. To see which NixOS channel you’re
+  <literal>nixos-20.09</literal> channel. To see which NixOS channel you’re
   subscribed to, run the following as root:
 <screen>
 <prompt># </prompt>nix-channel --list | grep nixos
@@ -75,13 +75,13 @@ nixos https://nixos.org/channels/nixos-unstable
 <prompt># </prompt>nix-channel --add https://nixos.org/channels/<replaceable>channel-name</replaceable> nixos
 </screen>
   (Be sure to include the <literal>nixos</literal> parameter at the end.) For
-  instance, to use the NixOS 20.03 stable channel:
+  instance, to use the NixOS 20.09 stable channel:
 <screen>
-<prompt># </prompt>nix-channel --add https://nixos.org/channels/nixos-20.03 nixos
+<prompt># </prompt>nix-channel --add https://nixos.org/channels/nixos-20.09 nixos
 </screen>
   If you have a server, you may want to use the “small” channel instead:
 <screen>
-<prompt># </prompt>nix-channel --add https://nixos.org/channels/nixos-20.03-small nixos
+<prompt># </prompt>nix-channel --add https://nixos.org/channels/nixos-20.09-small nixos
 </screen>
   And if you want to live on the bleeding edge:
 <screen>
@@ -132,7 +132,7 @@ nixos https://nixos.org/channels/nixos-unstable
    kernel, initrd or kernel modules.
    You can also specify a channel explicitly, e.g.
 <programlisting>
-<xref linkend="opt-system.autoUpgrade.channel"/> = https://nixos.org/channels/nixos-20.03;
+<xref linkend="opt-system.autoUpgrade.channel"/> = https://nixos.org/channels/nixos-20.09;
 </programlisting>
   </para>
  </section>

nixos/doc/manual/man-nixos-install.xml

@@ -49,6 +49,12 @@
     <option>--flake</option> <replaceable>flake-uri</replaceable>
    </arg>
 
+   <arg>
+    <group choice='req'>
+     <arg choice='plain'><option>--impure</option></arg>
+    </group>
+   </arg>
+
    <arg>
      <arg choice='plain'>
        <option>--channel</option>

nixos/doc/manual/release-notes/rl-2009.xml

@@ -3,8 +3,11 @@
          xmlns:xi="http://www.w3.org/2001/XInclude"
          version="5.0"
          xml:id="sec-release-20.09">
- <title>Release 20.09 (“Nightingale”, 2020.09/??)</title>
+ <title>Release 20.09 (“Nightingale”, 2020.10/27)</title>
 
+  <para>
+   Support is planned until the end of April 2021, handing over to 21.03.
+  </para>
  <section xmlns="http://docbook.org/ns/docbook"
          xmlns:xlink="http://www.w3.org/1999/xlink"
          xmlns:xi="http://www.w3.org/2001/XInclude"
@@ -13,63 +16,135 @@
   <title>Highlights</title>
 
   <para>
-   In addition to numerous new and upgraded packages, this release has the
+   In addition to 7349 new, 14442 updated, and 8181 removed packages, this release has the
    following highlights:
   </para>
 
   <itemizedlist>
    <listitem>
     <para>
-     Support is planned until the end of April 2021, handing over to 21.03.
+     Core version changes:
     </para>
-   </listitem>
+    <itemizedlist>
      <listitem>
-    <para>GNOME desktop environment was upgraded to 3.36, see its <link xlink:href="https://help.gnome.org/misc/release-notes/3.36/">release notes</link>.</para>
+      <para>
+       gcc: 9.2.0 -> 9.3.0
+      </para>
      </listitem>
      <listitem>
       <para>
-     The Cinnamon desktop environment (v4.6) has been added. <varname>services.xserver.desktopManager.cinnamon.enable = true;</varname> to try it out!
+       glibc: 2.30 -> 2.31
-     Remember that, with any new feature it's possible you could run into issues, so please send all support requests to <link xlink:href="https://github.com/NixOS/nixpkgs/issues">github.com/NixOS/nixpkgs</link> to notify the maintainers.
       </para>
      </listitem>
+     <listitem>
+      <para>
+       linux: still defaults to 5.4.x, all supported kernels available
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       mesa: 19.3.5 -> 20.1.7
+      </para>
+     </listitem>
+    </itemizedlist>
+   </listitem>
+   <listitem>
+    <para>
+     Desktop Environments:
+    </para>
+    <itemizedlist>
+     <listitem>
+      <para>
+       plasma5: 5.17.5 -> 5.18.5
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       kdeApplications: 19.12.3 -> 20.08.1
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       gnome3: 3.34 -> 3.36, see its <link xlink:href="https://help.gnome.org/misc/release-notes/3.36/">release notes</link>
+      </para>
+     </listitem>
+      <listitem>
+       <para>
+        cinnamon: added at 4.6
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       NixOS now distributes an official <link xlink:href="https://nixos.org/download.html#nixos-iso">GNOME ISO</link>
+      </para>
+     </listitem>
+    </itemizedlist>
+   </listitem>
+
+   <listitem>
+    <para>
+     Programming Languages and Frameworks:
+    </para>
+    <itemizedlist>
+
+     <listitem>
+      <para>
+       Agda ecosystem was heavily reworked (see more details below)
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       PHP now defaults to PHP 7.4, updated from 7.3
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       PHP 7.2 is no longer supported due to upstream not supporting this version for the entire lifecycle of the 20.09 release
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       Python 3 now defaults to Python 3.8 instead of 3.7
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       Python 3.5 reached its upstream EOL at the end of September 2020: it
+       has been removed from the list of available packages
+      </para>
+     </listitem>
+    </itemizedlist>
+   </listitem>
+
+   <listitem>
+    <para>
+     Databases and Service Monitoring:
+    </para>
+    <itemizedlist>
+     <listitem>
+      <para>
+       MariaDB has been updated to 10.4, MariaDB Galera to 26.4. Please read the related upgrade instructions under <link linkend="sec-release-20.09-incompatibilities">backwards incompatibilities</link> before upgrading.
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+        Zabbix now defaults to 5.0, updated from 4.4. Please read related sections under <link linkend="sec-release-20.09-incompatibilities">backwards compatibilities</link> before upgrading.
+      </para>
+     </listitem>
+    </itemizedlist>
+   </listitem>
+
+   <listitem>
+    <para>
+     Major module changes:
+    </para>
+    <itemizedlist>
      <listitem>
       <para>
        Quickly configure a complete, private, self-hosted video
        conferencing solution with the new Jitsi Meet module.
       </para>
      </listitem>
-   <listitem>
-    <para>
-    <package>maxx</package> package removed along with <varname>services.xserver.desktopManager.maxx</varname> module.
-    Please migrate to <package>cdesktopenv</package> and <varname>services.xserver.desktopManager.cde</varname> module.
-    </para>
-   </listitem>
-   <listitem>
-    <para>
-     We now distribute a GNOME ISO.
-    </para>
-   </listitem>
-   <listitem>
-    <para>
-     PHP now defaults to PHP 7.4, updated from 7.3.
-    </para>
-   </listitem>
-   <listitem>
-    <para>
-     PHP 7.2 is no longer supported due to upstream not supporting this version for the entire lifecycle of the 20.09 release.
-    </para>
-   </listitem>
-   <listitem>
-    <para>
-     Python 3 now defaults to Python 3.8 instead of 3.7.
-    </para>
-   </listitem>
-   <listitem>
-    <para>
-     Python 3.5 has reached its upstream EOL at the end of September 2020: it
-     has been removed from the list of available packages.
-    </para>
-   </listitem>
      <listitem>
       <para>
        Two new options, <link linkend="opt-services.openssh.authorizedKeysCommand">authorizedKeysCommand</link>
@@ -81,7 +156,7 @@
      </listitem>
      <listitem>
       <para>
-     There is a new module for Podman(<varname>virtualisation.podman</varname>), a drop-in replacement for the Docker command line.
+       There is a new module for Podman (<varname>virtualisation.podman</varname>), a drop-in replacement for the Docker command line.
       </para>
      </listitem>
      <listitem>
@@ -95,6 +170,447 @@
         This is to make it possible to use <literal>podman</literal> instead of <literal>docker</literal>.
       </para>
      </listitem>
+     <listitem>
+      <para>
+        The new option <link linkend="opt-documentation.man.generateCaches">documentation.man.generateCaches</link>
+        has been added to automatically generate the <literal>man-db</literal> caches, which are needed by utilities
+        like <command>whatis</command> and <command>apropos</command>. The caches are generated during the build of
+        the NixOS configuration: since this can be expensive when a large number of packages are installed, the
+        feature is disabled by default.
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <varname>services.postfix.sslCACert</varname> was replaced by <varname>services.postfix.tlsTrustedAuthorities</varname> which now defaults to system certificate authorities.
+      </para>
+     </listitem>
+     <listitem>
+       <para>
+         The various documented workarounds to use steam have been converted to a module. <varname>programs.steam.enable</varname> enables steam, controller support and the workarounds.
+       </para>
+     </listitem>
+     <listitem>
+       <para>
+         Support for built-in LCDs in various pieces of Logitech hardware (keyboards and USB speakers). <varname>hardware.logitech.lcd.enable</varname> enables support for all hardware supported by the <link xlink:href="https://sourceforge.net/projects/g15daemon/">g15daemon project</link>.
+       </para>
+     </listitem>
+     <listitem>
+       <para>
+         The GRUB module gained support for basic password protection, which
+         allows to restrict non-default entries in the boot menu to one or more
+         users. The users and passwords are defined via the option
+         <option>boot.loader.grub.users</option>.
+         Note: Password support is only available in GRUB version 2.
+       </para>
+     </listitem>
+    </itemizedlist>
+   </listitem>
+
+   <listitem>
+    <para>
+     NixOS module changes:
+    </para>
+    <itemizedlist>
+     <listitem>
+       <para>
+        The NixOS module system now supports freeform modules as a mix between <literal>types.attrsOf</literal> and <literal>types.submodule</literal>. These allow you to explicitly declare a subset of options while still permitting definitions without an associated option. See <xref linkend='sec-freeform-modules'/> for how to use them.
+       </para>
+     </listitem>
+     <listitem>
+      <para>
+       Following its deprecation in 20.03, the Perl NixOS test driver has been removed.
+       All remaining tests have been ported to the Python test framework.
+       Code outside nixpkgs using <filename>make-test.nix</filename> or
+       <filename>testing.nix</filename> needs to be ported to
+       <filename>make-test-python.nix</filename> and
+       <filename>testing-python.nix</filename> respectively.
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+        Subordinate GID and UID mappings are now set up automatically for all normal users.
+        This will make container tools like Podman work as non-root users out of the box.
+      </para>
+     </listitem>
+    </itemizedlist>
+   </listitem>
+
+  </itemizedlist>
+ </section>
+
+ <section xmlns="http://docbook.org/ns/docbook"
+         xmlns:xlink="http://www.w3.org/1999/xlink"
+         xmlns:xi="http://www.w3.org/2001/XInclude"
+         version="5.0"
+         xml:id="sec-release-20.09-new-services">
+  <title>New Services</title>
+
+  <para>
+   In addition to 1119 new, 118 updated, and 476 removed options; 61 new modules were added since the last release:
+  </para>
+
+  <itemizedlist>
+   <listitem>
+    <para>
+       Hardware:
+    </para>
+    <itemizedlist>
+     <listitem>
+      <para>
+       <xref linkend="opt-hardware.system76.firmware-daemon.enable" /> adds easy support of system76 firmware
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-hardware.uinput.enable" /> loads uinput kernel module
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-hardware.video.hidpi.enable" /> enable good defaults for HiDPI displays
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-hardware.wooting.enable" /> support for Wooting keyboards
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-hardware.xpadneo.enable" /> xpadneo driver for Xbox One wireless controllers
+      </para>
+     </listitem>
+    </itemizedlist>
+   </listitem>
+   <listitem>
+    <para>
+       Programs:
+    </para>
+    <itemizedlist>
+     <listitem>
+      <para>
+       <xref linkend="opt-programs.hamster.enable" /> enable hamster time tracking
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-programs.steam.enable" /> adds easy enablement of steam and related system configuration
+      </para>
+     </listitem>
+    </itemizedlist>
+   </listitem>
+   <listitem>
+    <para>
+       Security:
+    </para>
+    <itemizedlist>
+     <listitem>
+      <para>
+       <xref linkend="opt-security.doas.enable" /> alternative to sudo, allows non-root users to execute commands as root
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-security.tpm2.enable" /> add Trusted Platform Module 2 support
+      </para>
+     </listitem>
+    </itemizedlist>
+   </listitem>
+   <listitem>
+    <para>
+       System:
+    </para>
+    <itemizedlist>
+     <listitem>
+      <para>
+       <xref linkend="opt-boot.initrd.network.openvpn.enable" /> start an OpenVPN client during initrd boot
+      </para>
+     </listitem>
+    </itemizedlist>
+   </listitem>
+   <listitem>
+    <para>
+       Virtualization:
+    </para>
+    <itemizedlist>
+     <listitem>
+      <para>
+       <xref linkend="opt-boot.enableContainers" /> use nixos-containers
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-virtualisation.oci-containers.containers" /> run OCI (Docker) containers
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-virtualisation.podman.enable" /> daemonless container engine
+      </para>
+     </listitem>
+    </itemizedlist>
+   </listitem>
+
+   <listitem>
+    <para>
+       Services:
+    </para>
+    <itemizedlist>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.ankisyncd.enable" /> Anki sync server
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.bazarr.enable" /> Subtitle manager for Sonarr and Radarr
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.biboumi.enable" /> Biboumi XMPP gateway to IRC
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.blockbook-frontend" /> Blockbook-frontend, a service for the Trezor wallet
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.cage.enable" /> Wayland cage service
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.convos.enable" /> IRC daemon, which can be accessed throught the browser
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.engelsystem.enable" /> Tool for coordinating volunteers and shifts on large events
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.espanso.enable" /> text-expander written in rust
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.foldingathome.enable" /> Folding@home client
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.gerrit.enable" /> Web-based team code collaboration tool
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.go-neb.enable" /> Matrix bot
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.hardware.xow.enable" /> xow as a systemd service
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.hercules-ci-agent.enable" /> Hercules CI build agent
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.jicofo.enable" /> Jitsi Conference Focus, component of Jitsi Meet
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.jirafeau.enable" /> A web file repository
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.jitsi-meet.enable" /> Secure, simple and scalable video conferences
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.jitsi-videobridge.enable" /> Jitsi Videobridge, a WebRTC compatible router
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.jupyterhub.enable" /> Jupyterhub development server
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.k3s.enable" /> Lightweight Kubernetes distribution
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.magic-wormhole-mailbox-server.enable" /> Magic Wormhole Mailbox Server
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.malcontent.enable" /> Parental Control support
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.matrix-appservice-discord.enable" /> Matrix and Discord bridge
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.mautrix-telegram.enable" /> Matrix-Telegram puppeting/relaybot bridge
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.mirakurun.enable" /> Japanese DTV Tuner Server Service
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.molly-brown.enable" /> Molly-Brown Gemini server
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.mullvad-vpn.enable" /> Mullvad VPN daemon
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.ncdns.enable" /> Namecoin to DNS bridge
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.nextdns.enable" /> NextDNS to DoH Proxy service
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.nix-store-gcs-proxy" /> Google storage bucket to be used as a nix store
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.onedrive.enable" /> OneDrive sync service
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.pinnwand.enable" /> Pastebin-like service
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.pixiecore.enable" /> Manage network booting of machines
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.privacyidea.enable" /> Privacy authentication server
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.quorum.enable" /> Quorum blockchain daemon
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.robustirc-bridge.enable" /> RobustIRC bridge
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.rss-bridge.enable" /> Generate RSS and Atom feeds
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.rtorrent.enable" /> rTorrent service
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.smartdns.enable" /> SmartDNS DNS server
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.sogo.enable" /> SOGo groupware
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.teeworlds.enable" /> Teeworlds game server
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.torque.mom.enable" /> torque computing node
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.torque.server.enable" /> torque server
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.tuptime.enable" /> A total uptime service
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.urserver.enable" /> X11 remote server
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.wasabibackend.enable" /> Wasabi backend service
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.yubikey-agent.enable" /> Yubikey agent
+      </para>
+     </listitem>
+     <listitem>
+      <para>
+       <xref linkend="opt-services.zigbee2mqtt.enable" /> Zigbee to MQTT bridge
+      </para>
+     </listitem>
+    </itemizedlist>
+   </listitem>
+
+  </itemizedlist>
+
+ </section>
+
+ <section xmlns="http://docbook.org/ns/docbook"
+         xmlns:xlink="http://www.w3.org/1999/xlink"
+         xmlns:xi="http://www.w3.org/2001/XInclude"
+         version="5.0"
+         xml:id="sec-release-20.09-incompatibilities">
+  <title>Backward Incompatibilities</title>
+
+  <para>
+   When upgrading from a previous release, please be aware of the following
+   incompatible changes:
+  </para>
+
+  <itemizedlist>
    <listitem>
     <para>
       MariaDB has been updated to 10.4, MariaDB Galera to 26.4.
@@ -144,36 +660,7 @@ GRANT ALL PRIVILEGES ON *.* TO 'mysql'@'localhost' WITH GRANT OPTION;
       from the default of <literal>mysql</literal> to a different user please change <literal>'mysql'@'localhost'</literal> to the corresponding user instead.
     </para>
    </listitem>
-   <listitem>
+
-    <para>
-      The new option <link linkend="opt-documentation.man.generateCaches">documentation.man.generateCaches</link>
-      has been added to automatically generate the <literal>man-db</literal> caches, which are needed by utilities
-      like <command>whatis</command> and <command>apropos</command>. The caches are generated during the build of
-      the NixOS configuration: since this can be expensive when a large number of packages are installed, the
-      feature is disabled by default.
-    </para>
-   </listitem>
-   <listitem>
-    <para>
-     <varname>services.postfix.sslCACert</varname> was replaced by <varname>services.postfix.tlsTrustedAuthorities</varname> which now defaults to system certificate authorities.
-    </para>
-   </listitem>
-   <listitem>
-    <para>
-      Subordinate GID and UID mappings are now set up automatically for all normal users.
-      This will make container tools like Podman work as non-root users out of the box.
-    </para>
-   </listitem>
-   <listitem>
-     <para>
-       The various documented workarounds to use steam have been converted to a module. <varname>programs.steam.enable</varname> enables steam, controller support and the workarounds.
-     </para>
-   </listitem>
-   <listitem>
-     <para>
-       Support for built-in LCDs in various pieces of Logitech hardware (keyboards and USB speakers). <varname>hardware.logitech.lcd.enable</varname> enables support for all hardware supported by the g15daemon project.
-     </para>
-   </listitem>
    <listitem>
     <para>
       Zabbix now defaults to 5.0, updated from 4.4. Please carefully read through
@@ -208,72 +695,13 @@ GRANT ALL PRIVILEGES ON *.* TO 'mysql'@'localhost' WITH GRANT OPTION;
 </programlisting>
     </para>
    </listitem>
+
    <listitem>
     <para>
-      The NixOS module system now supports freeform modules as a mix between <literal>types.attrsOf</literal> and <literal>types.submodule</literal>. These allow you to explicitly declare a subset of options while still permitting definitions without an associated option. See <xref linkend='sec-freeform-modules'/> for how to use them.
+    <package>maxx</package> package removed along with <varname>services.xserver.desktopManager.maxx</varname> module.
+    Please migrate to <package>cdesktopenv</package> and <varname>services.xserver.desktopManager.cde</varname> module.
     </para>
    </listitem>
-   <listitem>
-     <para>
-       The GRUB module gained support for basic password protection, which
-       allows to restrict non-default entries in the boot menu to one or more
-       users. The users and passwords are defined via the option
-       <option>boot.loader.grub.users</option>.
-       Note: Password support is only avaiable in GRUB version 2.
-     </para>
-   </listitem>
-   <listitem>
-     <para>
-       Following its deprecation in 20.03, the Perl NixOS test driver has been removed.
-       All remaining tests have been ported to the Python test framework.
-       Code outside nixpkgs using <filename>make-test.nix</filename> or
-       <filename>testing.nix</filename> needs to be ported to
-       <filename>make-test-python.nix</filename> and
-       <filename>testing-python.nix</filename> respectively.
-     </para>
-   </listitem>
-  </itemizedlist>
- </section>
-
- <section xmlns="http://docbook.org/ns/docbook"
-         xmlns:xlink="http://www.w3.org/1999/xlink"
-         xmlns:xi="http://www.w3.org/2001/XInclude"
-         version="5.0"
-         xml:id="sec-release-20.09-new-services">
-  <title>New Services</title>
-
-  <para>
-   The following new services were added since the last release:
-  </para>
-
-  <itemizedlist>
-   <listitem>
-    <para>
-      There is a new <xref linkend="opt-security.doas.enable"/> module that provides <command>doas</command>, a lighter alternative to <command>sudo</command> with many of the same features.
-    </para>
-  </listitem>
-  <listitem>
-    <para>
-      <link xlink:href="https://hercules-ci.com">Hercules CI</link> Agent is a specialized build agent for projects built with Nix. See the <link xlink:href="https://nixos.org/nixos/options.html#services.hercules-ci-agent">options</link> and <link xlink:href="https://docs.hercules-ci.com/hercules-ci/getting-started/#deploy-agent">setup</link>.
-    </para>
-   </listitem>
-  </itemizedlist>
-
- </section>
-
- <section xmlns="http://docbook.org/ns/docbook"
-         xmlns:xlink="http://www.w3.org/1999/xlink"
-         xmlns:xi="http://www.w3.org/2001/XInclude"
-         version="5.0"
-         xml:id="sec-release-20.09-incompatibilities">
-  <title>Backward Incompatibilities</title>
-
-  <para>
-   When upgrading from a previous release, please be aware of the following
-   incompatible changes:
-  </para>
-
-  <itemizedlist>
    <listitem>
     <para>
      The <link linkend="opt-services.matrix-synapse.enable">matrix-synapse</link> module no longer includes optional dependencies by default, they have to be added through the <link linkend="opt-services.matrix-synapse.plugins">plugins</link> option.
@@ -300,7 +728,7 @@ GRANT ALL PRIVILEGES ON *.* TO 'mysql'@'localhost' WITH GRANT OPTION;
      It can still be enabled by providing <literal>phantomJsSupport = true</literal> to the package instantiation:
 <programlisting>{
   services.grafana.package = pkgs.grafana.overrideAttrs (oldAttrs: rec {
-    phantomJsSupport = false;
+    phantomJsSupport = true;
   });
 }</programlisting>
     </para>
@@ -1190,7 +1618,7 @@ services.transmission.settings.rpc-bind-address = "0.0.0.0";
       nixpkgs. It was a work in progress to package the
       <link xlink:href="https://www.deepin.org/en/dde/">Deepin Desktop Environment (DDE)</link>,
       including libraries, tools and applications, and it was still
-      missing a service to lauch the desktop environment. It has shown
+      missing a service to launch the desktop environment. It has shown
       to no longer be a feasible goal due to reasons discussed in
       <link xlink:href="https://github.com/NixOS/nixpkgs/issues/94870">issue #94870</link>.
       The package <literal>netease-cloud-music</literal> has also been
@@ -1235,4 +1663,131 @@ services.transmission.settings.rpc-bind-address = "0.0.0.0";
     </listitem>
   </itemizedlist>
  </section>
+
+ <section xmlns="http://docbook.org/ns/docbook"
+         xmlns:xlink="http://www.w3.org/1999/xlink"
+         xmlns:xi="http://www.w3.org/2001/XInclude"
+         version="5.0"
+         xml:id="sec-release-20.09-contributions">
+  <title>Contributions</title>
+  <para>
+        I, Jonathan Ringer, would like to thank the following individuals for their work on nixpkgs. This release could not be done without the hard work of the NixOS community. There were 31282 contributions across 1313 contributors.
+  </para>
+  <orderedlist>
+   <para>
+    Top contributors to NixOS/Nixpkgs from the 20.03 release to the 20.09 release:
+   </para>
+   <listitem>
+    <para>
+  2288  Mario Rodas
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+  1837  Frederik Rietdijk
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+   946  Jörg Thalheim
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+   925  Maximilian Bosch
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+   687  Jonathan Ringer
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+   651  Jan Tojnar
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+   622  Daniël de Kok
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+   605  WORLDofPEACE
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+   597  Florian Klink
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+   528  José Romildo Malaquias
+    </para>
+   </listitem>
+  </orderedlist>
+
+  <orderedlist>
+   <para>
+    Top contributors to stabilizing this release (Zero Hydra Failures period):
+   </para>
+   <listitem>
+    <para>
+      281  volth
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+   101  Robert Scott
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+    86  Tim Steinbach
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+    76  WORLDofPEACE
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+    49  Maximilian Bosch
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+    42  Thomas Tuegel
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+    37  Doron Behar
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+    36  Vladimír Čunát
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+    27  Jonathan Ringer
+    </para>
+   </listitem>
+   <listitem>
+    <para>
+    27  Maciej Krüger
+    </para>
+   </listitem>
+  </orderedlist>
+
+  <para>
+   I, Jonathan Ringer, would also like to personally thank @WORLDofPEACE for their help in mentoring me on the release process. Special thanks also goes to Thomas Tuegel for helping immensely with stabilizing Qt, KDE, and Plasma5; I would also like to thank Robert Scott for his numerous fixes and pull request reviews.
+   </para>
+
+ </section>
 </section>

nixos/lib/make-disk-image.nix

@@ -28,6 +28,9 @@
   #   partition of reasonable size is created in addition to the root partition.
   # For "legacy", the msdos partition table is used and a single large root
   #   partition is created.
+  # For "legacy+gpt", the GPT partition table is used, a 1MiB no-fs partition for
+  #   use by the bootloader is created, and a single large root partition is
+  #   created.
   # For "hybrid", the GPT partition table is used and a mandatory ESP
   #   partition of reasonable size is created in addition to the root partition.
   #   Also a legacy MBR will be present.
@@ -54,7 +57,7 @@
   format ? "raw"
 }:
 
-assert partitionTableType == "legacy" || partitionTableType == "efi" || partitionTableType == "hybrid" || partitionTableType == "none";
+assert partitionTableType == "legacy" || partitionTableType == "legacy+gpt" || partitionTableType == "efi" || partitionTableType == "hybrid" || partitionTableType == "none";
 # We use -E offset=X below, which is only supported by e2fsprogs
 assert partitionTableType != "none" -> fsType == "ext4";
 
@@ -75,6 +78,7 @@ let format' = format; in let
 
   rootPartition = { # switch-case
     legacy = "1";
+    "legacy+gpt" = "2";
     efi = "2";
     hybrid = "3";
   }.${partitionTableType};
@@ -85,6 +89,16 @@ let format' = format; in let
         mklabel msdos \
         mkpart primary ext4 1MiB -1
     '';
+    "legacy+gpt" = ''
+      parted --script $diskImage -- \
+        mklabel gpt \
+        mkpart no-fs 1MB 2MB \
+        set 1 bios_grub on \
+        align-check optimal 1 \
+        mkpart primary ext4 2MB -1 \
+        align-check optimal 2 \
+        print
+    '';
     efi = ''
       parted --script $diskImage -- \
         mklabel gpt \

nixos/lib/test-driver/test-driver.py

@@ -110,7 +110,6 @@ def create_vlan(vlan_nr: str) -> Tuple[str, str, "subprocess.Popen[bytes]", Any]
     pty_master, pty_slave = pty.openpty()
     vde_process = subprocess.Popen(
         ["vde_switch", "-s", vde_socket, "--dirmode", "0700"],
-        bufsize=1,
         stdin=pty_slave,
         stdout=subprocess.PIPE,
         stderr=subprocess.PIPE,
@@ -748,7 +747,6 @@ class Machine:
 
         self.process = subprocess.Popen(
             self.script,
-            bufsize=1,
             stdin=subprocess.DEVNULL,
             stdout=subprocess.PIPE,
             stderr=subprocess.STDOUT,

nixos/lib/testing-python.nix

@@ -3,13 +3,13 @@
   # Use a minimal kernel?
 , minimal ? false
   # Ignored
-, config ? {}
+, config ? { }
   # !!! See comment about args in lib/modules.nix
-, specialArgs ? {}
+, specialArgs ? { }
   # Modules to add to each VM
-, extraConfigurations ? [] }:
+, extraConfigurations ? [ ]
+}:
 
-with import ./build-vms.nix { inherit system pkgs minimal specialArgs extraConfigurations; };
 with pkgs;
 
 rec {
@@ -17,9 +17,11 @@ rec {
   inherit pkgs;
 
 
-  mkTestDriver = let
+  mkTestDriver =
+    let
       testDriverScript = ./test-driver/test-driver.py;
-  in qemu_pkg: stdenv.mkDerivation {
+    in
+    qemu_pkg: stdenv.mkDerivation {
       name = "nixos-test-driver";
 
       nativeBuildInputs = [ makeWrapper ];
@@ -51,9 +53,6 @@ rec {
         '';
     };
 
-  testDriver = mkTestDriver qemu_test;
-  testDriverInteractive = mkTestDriver qemu_kvm;
-
   # Run an automated test suite in the given virtual network.
   # `driver' is the script that runs the network.
   runTests = driver:
@@ -79,7 +78,6 @@ rec {
     , skipLint ? false
     , ...
     } @ t:
-
     let
       # A standard store path to the vm monitor is built like this:
       #   /tmp/nix-build-vm-test-run-$name.drv-0/vm-state-machine/monitor
@@ -88,25 +86,7 @@ rec {
       maxTestNameLen = 50;
       testNameLen = builtins.stringLength name;
 
-      testDriverName = with builtins;
-        if testNameLen > maxTestNameLen then
-          abort ("The name of the test '${name}' must not be longer than ${toString maxTestNameLen} " +
-            "it's currently ${toString testNameLen} characters long.")
-        else
-          "nixos-test-driver-${name}";
 
-      nodes = buildVirtualNetwork (
-        t.nodes or (if t ? machine then { machine = t.machine; } else { }));
-
-      testScript' =
-        # Call the test script with the computed nodes.
-        if lib.isFunction testScript
-        then testScript { inherit nodes; }
-        else testScript;
-
-      vlans = map (m: m.config.virtualisation.vlans) (lib.attrValues nodes);
-
-      vms = map (m: m.config.system.build.vm) (lib.attrValues nodes);
 
       ocrProg = tesseract4.override { enableLanguages = [ "eng" ]; };
 
@@ -115,15 +95,51 @@ rec {
       # Generate convenience wrappers for running the test driver
       # interactively with the specified network, and for starting the
       # VMs from the command line.
-      driver = testDriver:
+      mkDriver = qemu_pkg:
         let
+          build-vms = import ./build-vms.nix {
+            inherit system pkgs minimal specialArgs;
+            extraConfigurations = extraConfigurations ++ (pkgs.lib.optional (qemu_pkg != null)
+              {
+                virtualisation.qemu.package = qemu_pkg;
+              }
+            );
+          };
+
+          # FIXME: get this pkg from the module system
+          testDriver = mkTestDriver (if qemu_pkg == null then pkgs.qemu_test else qemu_pkg);
+
+          nodes = build-vms.buildVirtualNetwork (
+            t.nodes or (if t ? machine then { machine = t.machine; } else { })
+          );
+          vlans = map (m: m.config.virtualisation.vlans) (lib.attrValues nodes);
+          vms = map (m: m.config.system.build.vm) (lib.attrValues nodes);
+
+          testScript' =
+            # Call the test script with the computed nodes.
+            if lib.isFunction testScript
+            then testScript { inherit nodes; }
+            else testScript;
+
+          testDriverName = with builtins;
+            if testNameLen > maxTestNameLen then
+              abort
+                ("The name of the test '${name}' must not be longer than ${toString maxTestNameLen} " +
+                  "it's currently ${toString testNameLen} characters long.")
+            else
+              "nixos-test-driver-${name}";
+
           warn = if skipLint then lib.warn "Linting is disabled!" else lib.id;
         in
         warn (runCommand testDriverName
-        { buildInputs = [ makeWrapper];
+          {
+            buildInputs = [ makeWrapper ];
             testScript = testScript';
             preferLocalBuild = true;
             testName = name;
+            passthru = {
+              inherit nodes;
+            };
           }
           ''
             mkdir -p $out/bin
@@ -151,14 +167,18 @@ rec {
           ''); # "
 
       passMeta = drv: drv // lib.optionalAttrs (t ? meta) {
-        meta = (drv.meta or {}) // t.meta;
+        meta = (drv.meta or { }) // t.meta;
       };
 
-      test = passMeta (runTests (driver testDriver));
+      driver = mkDriver null;
+      driverInteractive = mkDriver pkgs.qemu;
 
-      nodeNames = builtins.attrNames nodes;
+      test = passMeta (runTests driver);
+
+      nodeNames = builtins.attrNames driver.nodes;
       invalidNodeNames = lib.filter
-        (node: builtins.match "^[A-z_]([A-z0-9_]+)?$" node == null) nodeNames;
+        (node: builtins.match "^[A-z_]([A-z0-9_]+)?$" node == null)
+        nodeNames;
 
     in
     if lib.length invalidNodeNames > 0 then
@@ -171,9 +191,8 @@ rec {
       ''
     else
       test // {
-          inherit nodes test;
+        inherit test driver driverInteractive;
-          driver = driver testDriver;
+        inherit (driver) nodes;
-          driverInteractive = driver testDriverInteractive;
       };
 
   runInMachine =
@@ -181,12 +200,19 @@ rec {
     , machine
     , preBuild ? ""
     , postBuild ? ""
+    , qemu ? pkgs.qemu_test
     , ... # ???
     }:
     let
-      vm = buildVM { }
+      build-vms = import ./build-vms.nix {
-        [ machine
+        inherit system pkgs minimal specialArgs extraConfigurations;
-          { key = "run-in-machine";
+      };
+
+      vm = build-vms.buildVM { }
+        [
+          machine
+          {
+            key = "run-in-machine";
             networking.hostName = "client";
             nix.readOnlyStore = false;
             virtualisation.writableStore = false;
@@ -229,20 +255,20 @@ rec {
         unset xchg
 
         export tests='${testScript}'
-        ${testDriver}/bin/nixos-test-driver ${vm.config.system.build.vm}/bin/run-*-vm
+        ${mkTestDriver qemu}/bin/nixos-test-driver --keep-vm-state ${vm.config.system.build.vm}/bin/run-*-vm
       ''; # */
 
     in
     lib.overrideDerivation drv (attrs: {
       requiredSystemFeatures = [ "kvm" ];
       builder = "${bash}/bin/sh";
-        args = ["-e" vmRunCommand];
+      args = [ "-e" vmRunCommand ];
       origArgs = attrs.args;
       origBuilder = attrs.builder;
     });
 
 
-  runInMachineWithX = { require ? [], ... } @ args:
+  runInMachineWithX = { require ? [ ], ... } @ args:
     let
       client =
         { ... }:

nixos/maintainers/scripts/ec2/amazon-image.nix

@@ -57,7 +57,7 @@ in {
     inherit (cfg) contents format name;
     pkgs = import ../../../.. { inherit (pkgs) system; }; # ensure we use the regular qemu-kvm package
     partitionTableType = if config.ec2.efi then "efi"
-                         else if config.ec2.hvm then "legacy"
+                         else if config.ec2.hvm then "legacy+gpt"
                          else "none";
     diskSize = cfg.sizeMB;
     fsType = "ext4";

nixos/maintainers/scripts/ec2/create-amis.sh

@@ -1,13 +1,15 @@
 #!/usr/bin/env nix-shell
 #!nix-shell -p awscli -p jq -p qemu -i bash
+# shellcheck shell=bash
 
 # Uploads and registers NixOS images built from the
 # <nixos/release.nix> amazonImage attribute. Images are uploaded and
 # registered via a home region, and then copied to other regions.
 
-# The home region requires an s3 bucket, and a "vmimport" IAM role
+# The home region requires an s3 bucket, and an IAM role named "vmimport"
-# with access to the S3 bucket.  Configuration of the vmimport role is
+# (by default) with access to the S3 bucket. The name can be
-# documented in
+# configured with the "service_role_name" variable. Configuration of the
+# vmimport role is documented in
 # https://docs.aws.amazon.com/vm-import/latest/userguide/vmimport-image-import.html
 
 # set -x
@@ -17,6 +19,7 @@ set -euo pipefail
 state_dir=$HOME/amis/ec2-images
 home_region=eu-west-1
 bucket=nixos-amis
+service_role_name=vmimport
 
 regions=(eu-west-1 eu-west-2 eu-west-3 eu-central-1 eu-north-1
          us-east-1 us-east-2 us-west-1 us-west-2
@@ -64,7 +67,7 @@ image_logical_bytes=$(read_image_info .logical_bytes)
 
 # Derived attributes
 
-image_logical_gigabytes=$((($image_logical_bytes-1)/1024/1024/1024+1)) # Round to the next GB
+image_logical_gigabytes=$(((image_logical_bytes-1)/1024/1024/1024+1)) # Round to the next GB
 
 case "$image_system" in
     aarch64-linux)
@@ -100,7 +103,7 @@ write_state() {
     local type=$2
     local val=$3
 
-    mkdir -p $state_dir
+    mkdir -p "$state_dir"
     echo "$val" > "$state_dir/$state_key.$type"
 }
 
@@ -110,8 +113,8 @@ wait_for_import() {
     local state snapshot_id
     log "Waiting for import task $task_id to be completed"
     while true; do
-        read state progress snapshot_id < <(
+        read -r state progress snapshot_id < <(
-            aws ec2 describe-import-snapshot-tasks --region $region --import-task-ids "$task_id" | \
+            aws ec2 describe-import-snapshot-tasks --region "$region" --import-task-ids "$task_id" | \
                 jq -r '.ImportSnapshotTasks[].SnapshotTaskDetail | "\(.Status) \(.Progress) \(.SnapshotId)"'
         )
         log " ... state=$state progress=$progress snapshot_id=$snapshot_id"
@@ -125,6 +128,8 @@ wait_for_import() {
                 ;;
             *)
                 log "Unexpected snapshot import state: '${state}'"
+                log "Full response: "
+                aws ec2 describe-import-snapshot-tasks --region "$region" --import-task-ids "$task_id" >&2
                 exit 1
                 ;;
         esac
@@ -138,8 +143,8 @@ wait_for_image() {
     log "Waiting for image $ami_id to be available"
 
     while true; do
-        read state < <(
+        read -r state < <(
-            aws ec2 describe-images --image-ids "$ami_id" --region $region | \
+            aws ec2 describe-images --image-ids "$ami_id" --region "$region" | \
                 jq -r ".Images[].State"
         )
         log " ... state=$state"
@@ -163,7 +168,7 @@ make_image_public() {
     local region=$1
     local ami_id=$2
 
-    wait_for_image $region "$ami_id"
+    wait_for_image "$region" "$ami_id"
 
     log "Making image $ami_id public"
 
@@ -177,27 +182,30 @@ upload_image() {
     local aws_path=${image_file#/}
 
     local state_key="$region.$image_label.$image_system"
-    local task_id=$(read_state "$state_key" task_id)
+    local task_id
-    local snapshot_id=$(read_state "$state_key" snapshot_id)
+    task_id=$(read_state "$state_key" task_id)
-    local ami_id=$(read_state "$state_key" ami_id)
+    local snapshot_id
+    snapshot_id=$(read_state "$state_key" snapshot_id)
+    local ami_id
+    ami_id=$(read_state "$state_key" ami_id)
 
     if [ -z "$task_id" ]; then
         log "Checking for image on S3"
         if ! aws s3 ls --region "$region" "s3://${bucket}/${aws_path}" >&2; then
             log "Image missing from aws, uploading"
-            aws s3 cp --region $region "$image_file" "s3://${bucket}/${aws_path}" >&2
+            aws s3 cp --region "$region" "$image_file" "s3://${bucket}/${aws_path}" >&2
         fi
 
         log "Importing image from S3 path s3://$bucket/$aws_path"
 
-        task_id=$(aws ec2 import-snapshot --disk-container "{
+        task_id=$(aws ec2 import-snapshot --role-name "$service_role_name" --disk-container "{
           \"Description\": \"nixos-image-${image_label}-${image_system}\",
           \"Format\": \"vhd\",
           \"UserBucket\": {
               \"S3Bucket\": \"$bucket\",
               \"S3Key\": \"$aws_path\"
           }
-        }" --region $region | jq -r '.ImportTaskId')
+        }" --region "$region" | jq -r '.ImportTaskId')
 
         write_state "$state_key" task_id "$task_id"
     fi
@@ -221,16 +229,16 @@ upload_image() {
             --virtualization-type hvm
         )
 
-        block_device_mappings+=(DeviceName=/dev/sdb,VirtualName=ephemeral0)
+        block_device_mappings+=("DeviceName=/dev/sdb,VirtualName=ephemeral0")
-        block_device_mappings+=(DeviceName=/dev/sdc,VirtualName=ephemeral1)
+        block_device_mappings+=("DeviceName=/dev/sdc,VirtualName=ephemeral1")
-        block_device_mappings+=(DeviceName=/dev/sdd,VirtualName=ephemeral2)
+        block_device_mappings+=("DeviceName=/dev/sdd,VirtualName=ephemeral2")
-        block_device_mappings+=(DeviceName=/dev/sde,VirtualName=ephemeral3)
+        block_device_mappings+=("DeviceName=/dev/sde,VirtualName=ephemeral3")
 
         ami_id=$(
             aws ec2 register-image \
                 --name "$image_name" \
                 --description "$image_description" \
-                --region $region \
+                --region "$region" \
                 --architecture $amazon_arch \
                 --block-device-mappings "${block_device_mappings[@]}" \
                 "${extra_flags[@]}" \
@@ -240,7 +248,7 @@ upload_image() {
         write_state "$state_key" ami_id "$ami_id"
     fi
 
-    make_image_public $region "$ami_id"
+    make_image_public "$region" "$ami_id"
 
     echo "$ami_id"
 }
@@ -268,7 +276,7 @@ copy_to_region() {
         write_state "$state_key" ami_id "$ami_id"
     fi
 
-    make_image_public $region "$ami_id"
+    make_image_public "$region" "$ami_id"
 
     echo "$ami_id"
 }

nixos/modules/config/fonts/fontdir.nix

@@ -8,7 +8,7 @@ let
 
   x11Fonts = pkgs.runCommand "X11-fonts" { preferLocalBuild = true; } ''
     mkdir -p "$out/share/X11/fonts"
-    font_regexp='.*\.\(ttf\|otf\|pcf\|pfa\|pfb\|bdf\)\(\.gz\)?'
+    font_regexp='.*\.\(ttf\|ttc\|otf\|pcf\|pfa\|pfb\|bdf\)\(\.gz\)?'
     find ${toString config.fonts.fonts} -regex "$font_regexp" \
       -exec ln -sf -t "$out/share/X11/fonts" '{}' \;
     cd "$out/share/X11/fonts"

nixos/modules/config/system-path.nix

@@ -33,6 +33,7 @@ let
       pkgs.ncurses
       pkgs.netcat
       config.programs.ssh.package
+      pkgs.mkpasswd
       pkgs.procps
       pkgs.su
       pkgs.time

nixos/modules/config/users-groups.nix

@@ -35,8 +35,7 @@ let
   '';
 
   hashedPasswordDescription = ''
-    To generate a hashed password install the <literal>mkpasswd</literal>
+    To generate a hashed password run <literal>mkpasswd -m sha-512</literal>.
-    package and run <literal>mkpasswd -m sha-512</literal>.
 
     If set to an empty string (<literal>""</literal>), this user will
     be able to log in without being asked for a password (but not via remote

nixos/modules/installer/cd-dvd/installation-cd-graphical-gnome.nix

@@ -9,7 +9,14 @@ with lib;
 
   isoImage.edition = "gnome";
 
-  services.xserver.desktopManager.gnome3.enable = true;
+  services.xserver.desktopManager.gnome3 = {
+    # Add firefox to favorite-apps
+    favoriteAppsOverride = ''
+      [org.gnome.shell]
+      favorite-apps=[ 'firefox.desktop', 'org.gnome.Geary.desktop', 'org.gnome.Calendar.desktop', 'org.gnome.Music.desktop', 'org.gnome.Photos.desktop', 'org.gnome.Nautilus.desktop' ]
+    '';
+    enable = true;
+  };
 
   services.xserver.displayManager = {
     gdm = {

nixos/modules/installer/tools/nixos-build-vms/build-vms.nix

@@ -6,12 +6,7 @@
 let
   nodes = builtins.mapAttrs (vm: module: {
     _file = "${networkExpr}@node-${vm}";
-    imports = [
+    imports = [ module ];
-      module
-      ({ pkgs, ... }: {
-        virtualisation.qemu.package = pkgs.qemu;
-      })
-    ];
   }) (import networkExpr);
 in
 
@@ -20,4 +15,4 @@ with import ../../../../lib/testing-python.nix {
   pkgs = import ../../../../.. { inherit system config; };
 };
 
-(makeTest { inherit nodes; testScript = ""; }).driver
+(makeTest { inherit nodes; testScript = ""; }).driverInteractive

nixos/modules/installer/tools/nixos-install.sh

@@ -64,7 +64,7 @@ while [ "$#" -gt 0 ]; do
         --no-bootloader)
             noBootLoader=1
             ;;
-        --show-trace)
+        --show-trace|--impure)
             extraBuildFlags+=("$i")
             ;;
         --help)

nixos/modules/misc/ids.nix

@@ -468,7 +468,7 @@ in
       #minecraft = 114; # unused
       vault = 115;
       #ripped = 116; # unused
-      #murmur = 117; # unused
+      murmur = 117;
       foundationdb = 118;
       newrelic = 119;
       starbound = 120;

nixos/modules/module-list.nix

@@ -173,6 +173,7 @@
   ./programs/wavemon.nix
   ./programs/waybar.nix
   ./programs/wireshark.nix
+  ./programs/wshowkeys.nix
   ./programs/x2goserver.nix
   ./programs/xfs_quota.nix
   ./programs/xonsh.nix

nixos/modules/programs/wshowkeys.nix

@@ -0,0 +1,22 @@
+{ config, lib, pkgs, ... }:
+
+with lib;
+
+let
+  cfg = config.programs.wshowkeys;
+in {
+  meta.maintainers = with maintainers; [ primeos ];
+
+  options = {
+    programs.wshowkeys = {
+      enable = mkEnableOption ''
+        wshowkeys (displays keypresses on screen on supported Wayland
+        compositors). It requires root permissions to read input events, but
+        these permissions are dropped after startup'';
+    };
+  };
+
+  config = mkIf cfg.enable {
+    security.wrappers.wshowkeys.source = "${pkgs.wshowkeys}/bin/wshowkeys";
+  };
+}

nixos/modules/security/acme.nix

@@ -63,7 +63,7 @@ let
     script = with builtins; concatStringsSep "\n" (mapAttrsToList (cert: data: ''
       for fixpath in /var/lib/acme/${escapeShellArg cert} /var/lib/acme/.lego/${escapeShellArg cert}; do
         if [ -d "$fixpath" ]; then
-          chmod -R 750 "$fixpath"
+          chmod -R u=rwX,g=rX,o= "$fixpath"
           chown -R acme:${data.group} "$fixpath"
         fi
       done
@@ -271,7 +271,7 @@ let
 
         mv domainhash.txt certificates/
         chmod 640 certificates/*
-        chmod -R 700 accounts/*
+        chmod -R u=rwX,g=,o= accounts/*
 
         # Group might change between runs, re-apply it
         chown 'acme:${data.group}' certificates/*

nixos/modules/security/pam.nix

@@ -318,6 +318,42 @@ let
         '';
       };
 
+      gnupg = {
+        enable = mkOption {
+          type = types.bool;
+          default = false;
+          description = ''
+            If enabled, pam_gnupg will attempt to automatically unlock the
+            user's GPG keys with the login password via
+            <command>gpg-agent</command>. The keygrips of all keys to be
+            unlocked should be written to <filename>~/.pam-gnupg</filename>,
+            and can be queried with <command>gpg -K --with-keygrip</command>.
+            Presetting passphrases must be enabled by adding
+            <literal>allow-preset-passphrase</literal> in
+            <filename>~/.gnupg/gpg-agent.conf</filename>.
+          '';
+        };
+
+        noAutostart = mkOption {
+          type = types.bool;
+          default = false;
+          description = ''
+            Don't start <command>gpg-agent</command> if it is not running.
+            Useful in conjunction with starting <command>gpg-agent</command> as
+            a systemd user service.
+          '';
+        };
+
+        storeOnly = mkOption {
+          type = types.bool;
+          default = false;
+          description = ''
+            Don't send the password immediately after login, but store for PAM
+            <literal>session</literal>.
+          '';
+        };
+      };
+
       text = mkOption {
         type = types.nullOr types.lines;
         description = "Contents of the PAM service file.";
@@ -386,6 +422,7 @@ let
             || cfg.enableKwallet
             || cfg.enableGnomeKeyring
             || cfg.googleAuthenticator.enable
+            || cfg.gnupg.enable
             || cfg.duoSecurity.enable)) ''
               auth required pam_unix.so ${optionalString cfg.allowNullPassword "nullok"} ${optionalString cfg.nodelay "nodelay"} likeauth
               ${optionalString config.security.pam.enableEcryptfs
@@ -397,6 +434,10 @@ let
                  " kwalletd=${pkgs.kdeFrameworks.kwallet.bin}/bin/kwalletd5")}
               ${optionalString cfg.enableGnomeKeyring
                 "auth optional ${pkgs.gnome3.gnome-keyring}/lib/security/pam_gnome_keyring.so"}
+              ${optionalString cfg.gnupg.enable
+                "auth optional ${pkgs.pam_gnupg}/lib/security/pam_gnupg.so"
+                + optionalString cfg.gnupg.storeOnly " store-only"
+               }
               ${optionalString cfg.googleAuthenticator.enable
                 "auth required ${pkgs.googleAuthenticator}/lib/security/pam_google_authenticator.so no_increment_hotp"}
               ${optionalString cfg.duoSecurity.enable
@@ -472,6 +513,10 @@ let
                " kwalletd=${pkgs.kdeFrameworks.kwallet.bin}/bin/kwalletd5")}
           ${optionalString (cfg.enableGnomeKeyring)
               "session optional ${pkgs.gnome3.gnome-keyring}/lib/security/pam_gnome_keyring.so auto_start"}
+          ${optionalString cfg.gnupg.enable
+              "session optional ${pkgs.pam_gnupg}/lib/security/pam_gnupg.so"
+              + optionalString cfg.gnupg.noAutostart " no-autostart"
+           }
           ${optionalString (config.virtualisation.lxc.lxcfs.enable)
                "session optional ${pkgs.lxc}/lib/security/pam_cgfs.so -c all"}
         '');

nixos/modules/services/backup/syncoid.nix

@@ -4,6 +4,15 @@ with lib;
 
 let
   cfg = config.services.syncoid;
+
+  # Extract pool names of local datasets (ones that don't contain "@") that
+  # have the specified type (either "source" or "target")
+  getPools = type: unique (map (d: head (builtins.match "([^/]+).*" d)) (
+    # Filter local datasets
+    filter (d: !hasInfix "@" d)
+    # Get datasets of the specified type
+    (catAttrs type (attrValues cfg.commands))
+  ));
 in {
 
     # Interface
@@ -26,14 +35,25 @@ in {
 
       user = mkOption {
         type = types.str;
-        default = "root";
+        default = "syncoid";
         example = "backup";
         description = ''
-          The user for the service. Sudo or ZFS privilege delegation must be
+          The user for the service. ZFS privilege delegation will be
-          configured to use a user other than root.
+          automatically configured for any local pools used by syncoid if this
+          option is set to a user other than root. The user will be given the
+          "hold" and "send" privileges on any pool that has datasets being sent
+          and the "create", "mount", "receive", and "rollback" privileges on
+          any pool that has datasets being received.
         '';
       };
 
+      group = mkOption {
+        type = types.str;
+        default = "syncoid";
+        example = "backup";
+        description = "The group for the service.";
+      };
+
       sshKey = mkOption {
         type = types.nullOr types.path;
         # Prevent key from being copied to store
@@ -150,6 +170,18 @@ in {
     # Implementation
 
     config = mkIf cfg.enable {
+      users =  {
+        users = mkIf (cfg.user == "syncoid") {
+          syncoid = {
+            group = cfg.group;
+            isSystemUser = true;
+          };
+        };
+        groups = mkIf (cfg.group == "syncoid") {
+          syncoid = {};
+        };
+      };
+
       systemd.services.syncoid = {
         description = "Syncoid ZFS synchronization service";
         script = concatMapStringsSep "\n" (c: lib.escapeShellArgs
@@ -160,10 +192,22 @@ in {
             ++ c.extraArgs
             ++ [ "--sendoptions" c.sendOptions
                  "--recvoptions" c.recvOptions
+                 "--no-privilege-elevation"
                  c.source c.target
                ])) (attrValues cfg.commands);
         after = [ "zfs.target" ];
-        serviceConfig.User = cfg.user;
+        serviceConfig = {
+          ExecStartPre = (map (pool: lib.escapeShellArgs [
+            "+/run/booted-system/sw/bin/zfs" "allow"
+            cfg.user "hold,send" pool
+          ]) (getPools "source")) ++
+          (map (pool: lib.escapeShellArgs [
+            "+/run/booted-system/sw/bin/zfs" "allow"
+            cfg.user "create,mount,receive,rollback" pool
+          ]) (getPools "target"));
+          User = cfg.user;
+          Group = cfg.group;
+        };
         startAt = cfg.interval;
       };
     };

nixos/modules/services/databases/postgresql.nix

@@ -69,11 +69,16 @@ in
         type = types.lines;
         default = "";
         description = ''
-          Defines how users authenticate themselves to the server. By
+          Defines how users authenticate themselves to the server. See the
-          default, "trust" access to local users will always be granted
+          <link xlink:href="https://www.postgresql.org/docs/current/auth-pg-hba-conf.html">
-          along with any other custom options. If you do not want this,
+          PostgreSQL documentation for pg_hba.conf</link>
-          set this option using "lib.mkForce" to override this
+          for details on the expected format of this option. By default,
-          behaviour.
+          peer based authentication will be used for users connecting
+          via the Unix socket, and md5 password authentication will be
+          used for users connecting via TCP. Any added rules will be
+          inserted above the default rules. If you'd like to replace the
+          default rules entirely, you can use <function>lib.mkForce</function> in your
+          module.
         '';
       };
 

nixos/modules/services/desktops/flatpak.nix

@@ -15,6 +15,18 @@ in {
   options = {
     services.flatpak = {
       enable = mkEnableOption "flatpak";
+
+      guiPackages = mkOption {
+        internal = true;
+        type = types.listOf types.package;
+        default = [];
+        example = literalExample "[ pkgs.gnome3.gnome-software ]";
+        description = ''
+          Packages that provide an interface for flatpak
+          (like gnome-software) that will be automatically available
+          to all users when flatpak is enabled.
+        '';
+      };
     };
   };
 
@@ -28,7 +40,7 @@ in {
       }
     ];
 
-    environment.systemPackages = [ pkgs.flatpak ];
+    environment.systemPackages = [ pkgs.flatpak ] ++ cfg.guiPackages;
 
     services.dbus.packages = [ pkgs.flatpak ];
 

nixos/modules/services/hardware/undervolt.nix

@@ -3,7 +3,12 @@
 with lib;
 let
   cfg = config.services.undervolt;
-  cliArgs = lib.cli.toGNUCommandLineShell {} {
+
+  mkPLimit = limit: window:
+    if (isNull limit && isNull window) then null
+    else assert asserts.assertMsg (!isNull limit && !isNull window) "Both power limit and window must be set";
+      "${toString limit} ${toString window}";
+  cliArgs = lib.cli.toGNUCommandLine {} {
     inherit (cfg)
       verbose
       temp
@@ -21,6 +26,9 @@ let
 
     temp-bat = cfg.tempBat;
     temp-ac = cfg.tempAc;
+
+    power-limit-long = mkPLimit cfg.p1.limit cfg.p1.window;
+    power-limit-short = mkPLimit cfg.p2.limit cfg.p2.window;
   };
 in
 {
@@ -104,6 +112,40 @@ in
       '';
     };
 
+    p1.limit = mkOption {
+      type = with types; nullOr int;
+      default = null;
+      description = ''
+        The P1 Power Limit in Watts.
+        Both limit and window must be set.
+      '';
+    };
+    p1.window = mkOption {
+      type = with types; nullOr (oneOf [ float int ]);
+      default = null;
+      description = ''
+        The P1 Time Window in seconds.
+        Both limit and window must be set.
+      '';
+    };
+
+    p2.limit = mkOption {
+      type = with types; nullOr int;
+      default = null;
+      description = ''
+        The P2 Power Limit in Watts.
+        Both limit and window must be set.
+      '';
+    };
+    p2.window = mkOption {
+      type = with types; nullOr (oneOf [ float int ]);
+      default = null;
+      description = ''
+        The P2 Time Window in seconds.
+        Both limit and window must be set.
+      '';
+    };
+
     useTimer = mkOption {
       type = types.bool;
       default = false;
@@ -133,7 +175,7 @@ in
       serviceConfig = {
         Type = "oneshot";
         Restart = "no";
-        ExecStart = "${pkgs.undervolt}/bin/undervolt ${cliArgs}";
+        ExecStart = "${pkgs.undervolt}/bin/undervolt ${toString cliArgs}";
       };
     };
 

nixos/modules/services/networking/avahi-daemon.nix

@@ -86,7 +86,8 @@ in
 
     ipv6 = mkOption {
       type = types.bool;
-      default = false;
+      default = config.networking.enableIPv6;
+      defaultText = "config.networking.enableIPv6";
       description = "Whether to use IPv6.";
     };
 

nixos/modules/services/networking/babeld.nix

@@ -87,9 +87,37 @@ in
       description = "Babel routing daemon";
       after = [ "network.target" ];
       wantedBy = [ "multi-user.target" ];
-      serviceConfig.ExecStart = "${pkgs.babeld}/bin/babeld -c ${configFile}";
+      serviceConfig = {
+        ExecStart = "${pkgs.babeld}/bin/babeld -c ${configFile} -I /run/babeld/babeld.pid -S /var/lib/babeld/state";
+        CapabilityBoundingSet = [ "CAP_NET_ADMIN" ];
+        IPAddressAllow = [ "fe80::/64" "ff00::/8" "::1/128" "127.0.0.0/8" ];
+        IPAddressDeny = "any";
+        LockPersonality = true;
+        NoNewPrivileges = true;
+        MemoryDenyWriteExecute = true;
+        ProtectSystem = "strict";
+        ProtectClock = true;
+        ProtectKernelTunables = false; # Couldn't write sysctl: Read-only file system
+        ProtectKernelModules = true;
+        ProtectKernelLogs = true;
+        ProtectControlGroups = true;
+        RestrictAddressFamilies = [ "AF_NETLINK" "AF_INET6" ];
+        RestrictNamespaces = true;
+        RestrictRealtime = true;
+        RestrictSUIDSGID = true;
+        RemoveIPC = true;
+        ProtectHome = true;
+        ProtectHostname = true;
+        PrivateMounts = true;
+        PrivateTmp = true;
+        PrivateDevices = true;
+        PrivateUsers = false; # kernel_route(ADD): Operation not permitted
+        SystemCallArchitectures = "native";
+        SystemCallFilter = [ "@system-service" ];
+        UMask = "0177";
+        RuntimeDirectory = "babeld";
+        StateDirectory = "babeld";
+      };
     };
-
   };
-
 }

nixos/modules/services/networking/blockbook-frontend.nix

@@ -158,15 +158,21 @@ let
         type = types.attrs;
         default = {};
         example = literalExample '' {
-          alternative_estimate_fee = "whatthefee-disabled";
+          "alternative_estimate_fee" = "whatthefee-disabled";
-          alternative_estimate_fee_params = "{\"url\": \"https://whatthefee.io/data.json\", \"periodSeconds\": 60}";
+          "alternative_estimate_fee_params" = "{\"url\": \"https://whatthefee.io/data.json\", \"periodSeconds\": 60}";
-          fiat_rates = "coingecko";
+          "fiat_rates" = "coingecko";
-          fiat_rates_params = "{\"url\": \"https://api.coingecko.com/api/v3\", \"coin\": \"bitcoin\", \"periodSeconds\": 60}";
+          "fiat_rates_params" = "{\"url\": \"https://api.coingecko.com/api/v3\", \"coin\": \"bitcoin\", \"periodSeconds\": 60}";
-          coin_shortcut = "BTC";
+          "coin_shortcut" = "BTC";
-          coin_label = "Bitcoin";
+          "coin_label" = "Bitcoin";
-          xpub_magic = 76067358;
+          "parse" = true;
-          xpub_magic_segwit_p2sh = 77429938;
+          "subversion" = "";
-          xpub_magic_segwit_native = 78792518;
+          "address_format" = "";
+          "xpub_magic" = 76067358;
+          "xpub_magic_segwit_p2sh" = 77429938;
+          "xpub_magic_segwit_native" = 78792518;
+          "mempool_workers" = 8;
+          "mempool_sub_workers" = 2;
+          "block_addresses_to_keep" = 300;
         }'';
         description = ''
           Additional configurations to be appended to <filename>coin.conf</filename>.

nixos/modules/services/networking/dnscrypt-wrapper.nix

@@ -55,7 +55,10 @@ let
   rotateKeys = ''
     # check if keys are not expired
     keyValid() {
-      fingerprint=$(dnscrypt-wrapper --show-provider-publickey | awk '{print $(NF)}')
+      fingerprint=$(dnscrypt-wrapper \
+        --show-provider-publickey \
+        --provider-publickey-file=${publicKey} \
+        | awk '{print $(NF)}')
       dnscrypt-proxy --test=${toString (cfg.keys.checkInterval + 1)} \
         --resolver-address=127.0.0.1:${toString cfg.port} \
         --provider-name=${cfg.providerName} \

nixos/modules/services/networking/mosquitto.nix

@@ -123,12 +123,33 @@ in
               '';
             };
 
+            passwordFile = mkOption {
+              type = with types; uniq (nullOr str);
+              example = "/path/to/file";
+              default = null;
+              description = ''
+                Specifies the path to a file containing the
+                clear text password for the MQTT user.
+              '';
+            };
+
             hashedPassword = mkOption {
               type = with types; uniq (nullOr str);
               default = null;
               description = ''
                 Specifies the hashed password for the MQTT User.
-                <option>hashedPassword</option> overrides <option>password</option>.
+                To generate hashed password install <literal>mosquitto</literal>
+                package and use <literal>mosquitto_passwd</literal>.
+              '';
+            };
+
+            hashedPasswordFile = mkOption {
+              type = with types; uniq (nullOr str);
+              example = "/path/to/file";
+              default = null;
+              description = ''
+                Specifies the path to a file containing the
+                hashed password for the MQTT user.
                 To generate hashed password install <literal>mosquitto</literal>
                 package and use <literal>mosquitto_passwd</literal>.
               '';
@@ -190,6 +211,13 @@ in
 
   config = mkIf cfg.enable {
 
+    assertions = mapAttrsToList (name: cfg: {
+      assertion = length (filter (s: s != null) (with cfg; [
+        password passwordFile hashedPassword hashedPasswordFile
+      ])) <= 1;
+      message = "Cannot set more than one password option";
+    }) cfg.users;
+
     systemd.services.mosquitto = {
       description = "Mosquitto MQTT Broker Daemon";
       wantedBy = [ "multi-user.target" ];
@@ -210,7 +238,11 @@ in
         touch ${cfg.dataDir}/passwd
       '' + concatStringsSep "\n" (
         mapAttrsToList (n: c:
-          if c.hashedPassword != null then
+          if c.hashedPasswordFile != null then
+            "echo '${n}:'$(cat '${c.hashedPasswordFile}') >> ${cfg.dataDir}/passwd"
+          else if c.passwordFile != null then
+            "${pkgs.mosquitto}/bin/mosquitto_passwd -b ${cfg.dataDir}/passwd ${n} $(cat '${c.passwordFile}')"
+          else if c.hashedPassword != null then
             "echo '${n}:${c.hashedPassword}' >> ${cfg.dataDir}/passwd"
           else optionalString (c.password != null)
             "${pkgs.mosquitto}/bin/mosquitto_passwd -b ${cfg.dataDir}/passwd ${n} '${c.password}'"

nixos/modules/services/networking/murmur.nix

@@ -278,6 +278,10 @@ in
       home            = "/var/lib/murmur";
       createHome      = true;
       uid             = config.ids.uids.murmur;
+      group           = "murmur";
+    };
+    users.groups.murmur = {
+      gid             = config.ids.gids.murmur;
     };
 
     systemd.services.murmur = {
@@ -300,6 +304,7 @@ in
         RuntimeDirectory = "murmur";
         RuntimeDirectoryMode = "0700";
         User = "murmur";
+        Group = "murmur";
       };
     };
   };

nixos/modules/services/networking/tailscale.nix

@@ -17,6 +17,7 @@ in {
   };
 
   config = mkIf cfg.enable {
+    environment.systemPackages = [ pkgs.tailscale ]; # for the CLI
     systemd.services.tailscale = {
       description = "Tailscale client daemon";
 

nixos/modules/services/web-apps/engelsystem.nix

@@ -10,7 +10,7 @@ in {
         default = false;
         example = true;
         description = ''
-          Whether to enable engelsystem, an online tool for coordinating helpers
+          Whether to enable engelsystem, an online tool for coordinating volunteers
           and shifts on large events.
         '';
         type = lib.types.bool;

nixos/modules/services/web-apps/shiori.nix

@@ -37,11 +37,60 @@ in {
       description = "Shiori simple bookmarks manager";
       wantedBy = [ "multi-user.target" ];
 
+      environment.SHIORI_DIR = "/var/lib/shiori";
+
       serviceConfig = {
         ExecStart = "${package}/bin/shiori serve --address '${address}' --port '${toString port}'";
+
         DynamicUser = true;
-        Environment = "SHIORI_DIR=/var/lib/shiori";
         StateDirectory = "shiori";
+        # As the RootDirectory
+        RuntimeDirectory = "shiori";
+
+        # Security options
+
+        BindReadOnlyPaths = [
+          "/nix/store"
+
+          # For SSL certificates, and the resolv.conf
+          "/etc"
+        ];
+
+        CapabilityBoundingSet = "";
+
+        DeviceAllow = "";
+
+        LockPersonality = true;
+
+        MemoryDenyWriteExecute = true;
+
+        PrivateDevices = true;
+        PrivateUsers = true;
+
+        ProtectClock = true;
+        ProtectControlGroups = true;
+        ProtectHome = true;
+        ProtectHostname = true;
+        ProtectKernelLogs = true;
+        ProtectKernelModules = true;
+        ProtectKernelTunables = true;
+
+        RestrictNamespaces = true;
+        RestrictAddressFamilies = [ "AF_INET" "AF_INET6" ];
+        RestrictRealtime = true;
+        RestrictSUIDSGID = true;
+
+        RootDirectory = "/run/shiori";
+
+        SystemCallArchitectures = "native";
+        SystemCallErrorNumber = "EPERM";
+        SystemCallFilter = [
+          "@system-service"
+
+          "~@chown" "~@cpu-emulation" "~@debug" "~@ipc" "~@keyring" "~@memlock"
+          "~@module" "~@obsolete" "~@privileged" "~@process" "~@raw-io"
+          "~@resources" "~@setuid"
+        ];
       };
     };
   };

nixos/modules/services/web-servers/caddy.nix

@@ -24,6 +24,10 @@ let
     ${pkgs.jq}/bin/jq -s '.[0] * .[1]' ${adaptedConfig} ${tlsJSON} > $out
   '';
 in {
+  imports = [
+    (mkRemovedOptionModule [ "services" "caddy" "agree" ] "this option is no longer necessary for Caddy 2")
+  ];
+
   options.services.caddy = {
     enable = mkEnableOption "Caddy web server";
 
@@ -66,12 +70,6 @@ in {
       description = "Email address (for Let's Encrypt certificate)";
     };
 
-    agree = mkOption {
-      default = false;
-      type = types.bool;
-      description = "Agree to Let's Encrypt Subscriber Agreement";
-    };
-
     dataDir = mkOption {
       default = "/var/lib/caddy";
       type = types.path;

nixos/modules/services/web-servers/nginx/default.nix

@@ -86,7 +86,7 @@ let
       ''}
 
       ssl_protocols ${cfg.sslProtocols};
-      ssl_ciphers ${cfg.sslCiphers};
+      ${optionalString (cfg.sslCiphers != null) "ssl_ciphers ${cfg.sslCiphers};"}
       ${optionalString (cfg.sslDhparam != null) "ssl_dhparam ${cfg.sslDhparam};"}
 
       ${optionalString (cfg.recommendedTlsSettings) ''
@@ -487,7 +487,7 @@ in
       };
 
       sslCiphers = mkOption {
-        type = types.str;
+        type = types.nullOr types.str;
         # Keep in sync with https://ssl-config.mozilla.org/#server=nginx&config=intermediate
         default = "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384";
         description = "Ciphers to choose from when negotiating TLS handshakes.";

nixos/modules/services/x11/desktop-managers/gnome3.nix

@@ -17,6 +17,11 @@ let
     '';
   };
 
+  defaultFavoriteAppsOverride = ''
+    [org.gnome.shell]
+    favorite-apps=[ 'org.gnome.Geary.desktop', 'org.gnome.Calendar.desktop', 'org.gnome.Music.desktop', 'org.gnome.Photos.desktop', 'org.gnome.Nautilus.desktop' ]
+  '';
+
   nixos-gsettings-desktop-schemas = let
     defaultPackages = with pkgs; [ gsettings-desktop-schemas gnome3.gnome-shell ];
   in
@@ -42,8 +47,7 @@ let
        [org.gnome.desktop.screensaver]
        picture-uri='file://${pkgs.nixos-artwork.wallpapers.simple-dark-gray-bottom.gnomeFilePath}'
 
-       [org.gnome.shell]
+       ${cfg.favoriteAppsOverride}
-       favorite-apps=[ 'org.gnome.Epiphany.desktop', 'org.gnome.Geary.desktop', 'org.gnome.Music.desktop', 'org.gnome.Photos.desktop', 'org.gnome.Nautilus.desktop', 'org.gnome.Software.desktop' ]
 
        ${cfg.extraGSettingsOverrides}
      EOF
@@ -123,6 +127,17 @@ in
         apply = list: list ++ [ pkgs.gnome3.gnome-shell pkgs.gnome3.gnome-shell-extensions ];
       };
 
+      favoriteAppsOverride = mkOption {
+        internal = true; # this is messy
+        default = defaultFavoriteAppsOverride;
+        type = types.lines;
+        example = literalExample ''
+          [org.gnome.shell]
+          favorite-apps=[ 'firefox.desktop', 'org.gnome.Calendar.desktop' ]
+        '';
+        description = "List of desktop files to put as favorite apps into gnome-shell. These need to be installed somehow globally.";
+      };
+
       extraGSettingsOverrides = mkOption {
         default = "";
         type = types.lines;
@@ -215,6 +230,11 @@ in
 
        # If gnome3 is installed, build vim for gtk3 too.
       nixpkgs.config.vim.gui = "gtk3";
+
+      # Install gnome-software if flatpak is enabled
+      services.flatpak.guiPackages = [
+        pkgs.gnome3.gnome-software
+      ];
     })
 
     (mkIf flashbackEnabled {
@@ -397,7 +417,6 @@ in
         gnome-music
         gnome-photos
         gnome-screenshot
-        gnome-software
         gnome-system-monitor
         gnome-weather
         nautilus

nixos/modules/services/x11/desktop-managers/pantheon.nix

@@ -180,7 +180,6 @@ in
         gtk3.out
         hicolor-icon-theme
         lightlocker
-        nixos-artwork.wallpapers.simple-dark-gray
         onboard
         qgnomeplatform
         shared-mime-info

nixos/modules/system/boot/networkd.nix

@@ -648,11 +648,13 @@ let
           "RapidCommit"
           "ForceDHCPv6PDOtherInformation"
           "PrefixDelegationHint"
+          "RouteMetric"
         ])
         (assertValueOneOf "UseDNS" boolValues)
         (assertValueOneOf "UseNTP" boolValues)
         (assertValueOneOf "RapidCommit" boolValues)
         (assertValueOneOf "ForceDHCPv6PDOtherInformation" boolValues)
+        (assertInt "RouteMetric")
       ];
 
       sectionDHCPServer = checkUnitConfig "DHCPServer" [

nixos/modules/system/boot/stage-1-init.sh

@@ -120,7 +120,7 @@ eval "exec $logOutFd>&1 $logErrFd>&2"
 if test -w /dev/kmsg; then
     tee -i < /tmp/stage-1-init.log.fifo /proc/self/fd/"$logOutFd" | while read -r line; do
         if test -n "$line"; then
-            echo "<7>stage-1-init: $line" > /dev/kmsg
+            echo "<7>stage-1-init: [$(date)] $line" > /dev/kmsg
         fi
     done &
 else
@@ -356,6 +356,7 @@ mountFS() {
     case $options in
         *x-nixos.autoresize*)
             if [ "$fsType" = ext2 -o "$fsType" = ext3 -o "$fsType" = ext4 ]; then
+                modprobe "$fsType"
                 echo "resizing $device..."
                 e2fsck -fp "$device"
                 resize2fs "$device"

nixos/modules/testing/test-instrumentation.nix

@@ -45,16 +45,22 @@ with import ../../lib/qemu-flags.nix { inherit pkgs; };
     systemd.services."serial-getty@${qemuSerialDevice}".enable = false;
     systemd.services."serial-getty@hvc0".enable = false;
 
+    # Only set these settings when the options exist. Some tests (e.g. those
+    # that do not specify any nodes, or an empty attr set as nodes) will not
+    # have the QEMU module loaded and thuse these options can't and should not
+    # be set.
+    virtualisation = lib.optionalAttrs (options ? virtualisation.qemu) {
+      qemu = {
         # Only use a serial console, no TTY.
         # NOTE: optionalAttrs
         #       test-instrumentation.nix appears to be used without qemu-vm.nix, so
         #       we avoid defining consoles if not possible.
         # TODO: refactor such that test-instrumentation can import qemu-vm
         #       or declare virtualisation.qemu.console option in a module that's always imported
-    virtualisation.qemu = {
+        consoles = [ qemuSerialDevice ];
-      consoles = lib.optional (options ? virtualisation.qemu.consoles) qemuSerialDevice;
         package  = lib.mkDefault pkgs.qemu_test;
       };
+    };
 
     boot.initrd.preDeviceCommands =
       ''

nixos/modules/virtualisation/amazon-image.nix

@@ -48,7 +48,7 @@ in
     ];
     boot.initrd.kernelModules = [ "xen-blkfront" "xen-netfront" ];
     boot.initrd.availableKernelModules = [ "ixgbevf" "ena" "nvme" ];
-    boot.kernelParams = mkIf cfg.hvm [ "console=ttyS0" ];
+    boot.kernelParams = mkIf cfg.hvm [ "console=ttyS0" "random.trust_cpu=on" ];
 
     # Prevent the nouveau kernel module from being loaded, as it
     # interferes with the nvidia/nvidia-uvm modules needed for CUDA.

nixos/modules/virtualisation/ec2-amis.nix

@@ -329,5 +329,24 @@ let self = {
   "20.03".ap-east-1.hvm-ebs = "ami-0d18fdd309cdefa86";
   "20.03".sa-east-1.hvm-ebs = "ami-09859378158ae971d";
 
-  latest = self."20.03";
+  # 20.09.1465.9a0b14b097d
+  "20.09".eu-west-1.hvm-ebs = "ami-0d90f16418e3c364c";
+  "20.09".eu-west-2.hvm-ebs = "ami-0635ec0780ea57cfe";
+  "20.09".eu-west-3.hvm-ebs = "ami-0714e94352f2eabb9";
+  "20.09".eu-central-1.hvm-ebs = "ami-0979d39762a4d2a02";
+  "20.09".eu-north-1.hvm-ebs = "ami-0b14e273185c66e9b";
+  "20.09".us-east-1.hvm-ebs = "ami-0f8b063ac3f2d9645";
+  "20.09".us-east-2.hvm-ebs = "ami-0959202a0393fdd0c";
+  "20.09".us-west-1.hvm-ebs = "ami-096d50833b785478b";
+  "20.09".us-west-2.hvm-ebs = "ami-0fc31031df0df6104";
+  "20.09".ca-central-1.hvm-ebs = "ami-0787786a38cde3905";
+  "20.09".ap-southeast-1.hvm-ebs = "ami-0b3f693d3a2a0b9ae";
+  "20.09".ap-southeast-2.hvm-ebs = "ami-02471872bc876b610";
+  "20.09".ap-northeast-1.hvm-ebs = "ami-06505fd2bf44a59a7";
+  "20.09".ap-northeast-2.hvm-ebs = "ami-0754b4c014eea1e8a";
+  "20.09".ap-south-1.hvm-ebs = "ami-05100e32242ae65a6";
+  "20.09".ap-east-1.hvm-ebs = "ami-045288859a39de009";
+  "20.09".sa-east-1.hvm-ebs = "ami-0a937748db48fb00d";
+
+  latest = self."20.09";
 }; in self

nixos/tests/bitcoind.nix

@@ -31,16 +31,16 @@ import ./make-test-python.nix ({ pkgs, ... }: {
     machine.wait_for_unit("bitcoind-testnet.service")
 
     machine.wait_until_succeeds(
-        'curl --user rpc:rpc --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:8332 |  grep \'"chain":"main"\' '
+        'curl --fail --user rpc:rpc --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:8332 |  grep \'"chain":"main"\' '
     )
     machine.wait_until_succeeds(
-        'curl --user rpc2:rpc2 --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:8332 |  grep \'"chain":"main"\' '
+        'curl --fail --user rpc2:rpc2 --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:8332 |  grep \'"chain":"main"\' '
     )
     machine.wait_until_succeeds(
-        'curl --user rpc:rpc --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:18332 |  grep \'"chain":"test"\' '
+        'curl --fail --user rpc:rpc --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:18332 |  grep \'"chain":"test"\' '
     )
     machine.wait_until_succeeds(
-        'curl --user rpc2:rpc2 --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:18332 |  grep \'"chain":"test"\' '
+        'curl --fail --user rpc2:rpc2 --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:18332 |  grep \'"chain":"test"\' '
     )
   '';
 })

nixos/tests/caddy.nix

@@ -57,11 +57,13 @@ import ./make-test-python.nix ({ pkgs, ... }: {
 
     def check_etag(url):
         etag = webserver.succeed(
-            "curl -v '{}' 2>&1 | sed -n -e \"s/^< [Ee][Tt][Aa][Gg]: *//p\"".format(url)
+            "curl --fail -v '{}' 2>&1 | sed -n -e \"s/^< [Ee][Tt][Aa][Gg]: *//p\"".format(
+                url
+            )
         )
         etag = etag.replace("\r\n", " ")
         http_code = webserver.succeed(
-            "curl --silent --show-error -o /dev/null -w \"%{{http_code}}\" --head -H 'If-None-Match: {}' {}".format(
+            "curl --fail --silent --show-error -o /dev/null -w \"%{{http_code}}\" --head -H 'If-None-Match: {}' {}".format(
                 etag, url
             )
         )

nixos/tests/cadvisor.nix

@@ -19,16 +19,16 @@ import ./make-test-python.nix ({ pkgs, ... } : {
   testScript =  ''
       start_all()
       machine.wait_for_unit("cadvisor.service")
-      machine.succeed("curl http://localhost:8080/containers/")
+      machine.succeed("curl -f http://localhost:8080/containers/")
 
       influxdb.wait_for_unit("influxdb.service")
 
       # create influxdb database
       influxdb.succeed(
-          'curl -XPOST http://localhost:8086/query --data-urlencode "q=CREATE DATABASE root"'
+          'curl -f -XPOST http://localhost:8086/query --data-urlencode "q=CREATE DATABASE root"'
       )
 
       influxdb.wait_for_unit("cadvisor.service")
-      influxdb.succeed("curl http://localhost:8080/containers/")
+      influxdb.succeed("curl -f http://localhost:8080/containers/")
     '';
 })

nixos/tests/cfssl.nix

@@ -38,7 +38,7 @@ import ./make-test-python.nix ({ pkgs, ...} : {
   testScript =
   let
     cfsslrequest = with pkgs; writeScript "cfsslrequest" ''
-      curl -X POST -H "Content-Type: application/json" -d @${csr} \
+      curl -f -X POST -H "Content-Type: application/json" -d @${csr} \
         http://localhost:8888/api/v1/cfssl/newkey | ${cfssl}/bin/cfssljson /tmp/certificate
     '';
     csr = pkgs.writeText "csr.json" (builtins.toJSON {

nixos/tests/common/acme/server/README.md

@@ -0,0 +1,21 @@
+# Fake Certificate Authority for ACME testing
+
+This will set up a test node running [pebble](https://github.com/letsencrypt/pebble)
+to serve ACME certificate requests.
+
+## "Snake oil" certs
+
+The snake oil certs are hard coded into the repo for reasons explained [here](https://github.com/NixOS/nixpkgs/pull/91121#discussion_r505410235).
+The root of the issue is that Nix will hash the derivation based on the arguments
+to mkDerivation, not the output. [Minica](https://github.com/jsha/minica) will
+always generate a random certificate even if the arguments are unchanged. As a
+result, it's possible to end up in a situation where the cached and local
+generated certs mismatch and cause issues with testing.
+
+To generate new certificates, run the following commands:
+
+```bash
+nix-build generate-certs.nix
+cp result/* .
+rm result
+```

nixos/tests/common/acme/server/acme.test.cert.pem

@@ -0,0 +1,19 @@
+-----BEGIN CERTIFICATE-----
+MIIDLDCCAhSgAwIBAgIIRDAN3FHH//IwDQYJKoZIhvcNAQELBQAwIDEeMBwGA1UE
+AxMVbWluaWNhIHJvb3QgY2EgNzg3NDZmMB4XDTIwMTAyMTEzMjgzNloXDTIyMTEy
+MDEzMjgzNlowFDESMBAGA1UEAxMJYWNtZS50ZXN0MIIBIjANBgkqhkiG9w0BAQEF
+AAOCAQ8AMIIBCgKCAQEAo8XjMVUaljcaqQ5MFhfPuQgSwdyXEUbpSHz+5yPkE0h9
+Z4Xu5BJF1Oq7h5ggCtadVsIspiY6Jm6aWDOjlh4myzW5UNBNUG3OPEk50vmmHFeH
+pImHO/d8yb33QoF9VRcTZs4tuJYg7l9bSs4jNG72vYvv2YiGAcmjJcsmAZIfniCN
+Xf/LjIm+Cxykn+Vo3UuzO1w5/iuofdgWO/aZxMezmXUivlL3ih4cNzCJei8WlB/l
+EnHrkcy3ogRmmynP5zcz7vmGIJX2ji6dhCa4Got5B7eZK76o2QglhQXqPatG0AOY
+H+RfQfzKemqPG5om9MgJtwFtTOU1LoaiBw//jXKESQIDAQABo3YwdDAOBgNVHQ8B
+Af8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwGA1UdEwEB
+/wQCMAAwHwYDVR0jBBgwFoAU+8IZlLV/Qp5CXqpXMLvtxWlxcJwwFAYDVR0RBA0w
+C4IJYWNtZS50ZXN0MA0GCSqGSIb3DQEBCwUAA4IBAQB0pe8I5/VDkB5VMgQB2GJV
+GKzyigfWbVez9uLmqMj9PPP/zzYKSYeq+91aMuOZrnH7NqBxSTwanULkmqAmhbJJ
+YkXw+FlFekf9FyxcuArzwzzNZDSGcjcdXpN8S2K1qkBd00iSJF9kU7pdZYCIKR20
+QirdBrELEfsJ3GU62a6N3a2YsrisZUvq5TbjGJDcytAtt+WG3gmV7RInLdFfPwbw
+bEHPCnx0uiV0nxLjd/aVT+RceVrFQVt4hR99jLoMlBitSKluZ1ljsrpIyroBhQT0
+pp/pVi6HJdijG0fsPrC325NEGAwcpotLUhczoeM/rffKJd54wLhDkfYxOyRZXivs
+-----END CERTIFICATE-----

nixos/tests/common/acme/server/acme.test.key.pem

@@ -0,0 +1,27 @@
+-----BEGIN RSA PRIVATE KEY-----
+MIIEowIBAAKCAQEAo8XjMVUaljcaqQ5MFhfPuQgSwdyXEUbpSHz+5yPkE0h9Z4Xu
+5BJF1Oq7h5ggCtadVsIspiY6Jm6aWDOjlh4myzW5UNBNUG3OPEk50vmmHFeHpImH
+O/d8yb33QoF9VRcTZs4tuJYg7l9bSs4jNG72vYvv2YiGAcmjJcsmAZIfniCNXf/L
+jIm+Cxykn+Vo3UuzO1w5/iuofdgWO/aZxMezmXUivlL3ih4cNzCJei8WlB/lEnHr
+kcy3ogRmmynP5zcz7vmGIJX2ji6dhCa4Got5B7eZK76o2QglhQXqPatG0AOYH+Rf
+QfzKemqPG5om9MgJtwFtTOU1LoaiBw//jXKESQIDAQABAoIBADox/2FwVFo8ioS4
+R+Ex5OZjMAcjU6sX/516jTmlT05q2+UFerYgqB/YqXqtW/V9/brulN8VhmRRuRbO
+grq9TBu5o3hMDK0f18EkZB/MBnLbx594H033y6gEkPBZAyhRYtuNOEH3VwxdZhtW
+1Lu1EoiYSUqLcNMBy6+KWJ8GRaXyacMYBlj2lMHmyzkA/t1+2mwTGC3lT6zN0F5Y
+E5umXOxsn6Tb6q3KM9O5IvtmMMKpgj4HIHZLZ6j40nNgHwGRaAv4Sha/vx0DeBw3
+6VlNiTTPdShEkhESlM5/ocqTfI92VHJpM5gkqTYOWBi2aKIPfAopXoqoJdWl4pQ/
+NCFIu2ECgYEAzntNKIcQtf0ewe0/POo07SIFirvz6jVtYNMTzeQfL6CoEjYArJeu
+Vzc4wEQfA4ZFVerBb1/O6M449gI3zex1PH4AX0h8q8DSjrppK1Jt2TnpVh97k7Gg
+Tnat/M/yW3lWYkcMVJJ3AYurXLFTT1dYP0HvBwZN04yInrEcPNXKfmcCgYEAywyJ
+51d4AE94PrANathKqSI/gk8sP+L1gzylZCcUEAiGk/1r45iYB4HN2gvWbS+CvSdp
+F7ShlDWrTaNh2Bm1dgTjc4pWb4J+CPy/KN2sgLwIuM4+ZWIZmEDcio6khrM/gNqK
+aR7xUsvWsqU26O84woY/xR8IHjSNF7cFWE1H2c8CgYEAt6SSi2kVQ8dMg84uYE8t
+o3qO00U3OycpkOQqyQQLeKC62veMwfRl6swCfX4Y11mkcTXJtPTRYd2Ia8StPUkB
+PDwUuKoPt/JXUvoYb59wc7M+BIsbrdBdc2u6cw+/zfutCNuH6/AYSBeg4WAVaIuW
+wSwzG1xP+8cR+5IqOzEqWCECgYATweeVTCyQEyuHJghYMi2poXx+iIesu7/aAkex
+pB/Oo5W8xrb90XZRnK7UHbzCqRHWqAQQ23Gxgztk9ZXqui2vCzC6qGZauV7cLwPG
+zTMg36sVmHP314DYEM+k59ZYiQ6P0jQPoIQo407D2VGrfsOOIhQIcUmP7tsfyJ5L
+hlGMfwKBgGq4VNnnuX8I5kl03NpaKfG+M8jEHmVwtI9RkPTCCX9bMjeG0cDxqPTF
+TRkf3r8UWQTZ5QfAfAXYAOlZvmGhHjSembRbXMrMdi3rGsYRSrQL6n5NHnORUaMy
+FCWo4gyAnniry7tx9dVNgmHmbjEHuQnf8AC1r3dibRCjvJWUiQ8H
+-----END RSA PRIVATE KEY-----

nixos/tests/common/acme/server/ca.cert.pem

@@ -0,0 +1,20 @@
+-----BEGIN CERTIFICATE-----
+MIIDSzCCAjOgAwIBAgIIeHRvRrNvbGQwDQYJKoZIhvcNAQELBQAwIDEeMBwGA1UE
+AxMVbWluaWNhIHJvb3QgY2EgNzg3NDZmMCAXDTIwMTAyMTEzMjgzNloYDzIxMjAx
+MDIxMTMyODM2WjAgMR4wHAYDVQQDExVtaW5pY2Egcm9vdCBjYSA3ODc0NmYwggEi
+MA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCrNTzVLDJOKtGYGLU98EEcLKps
+tXHCLC6G54LKbEcU80fn+ArX8qsPSHyhdXQkcYjq6Vh/EDJ1TctyRSnvAjwyG4Aa
+1Zy1QFc/JnjMjvzimCkUc9lQ+wkLwHSM/KGwR1cGjmtQ/EMClZTA0NwulJsXMKVz
+bd5asXbq/yJTQ5Ww25HtdNjwRQXTvB7r3IKcY+DsED9CvFvC9oG/ZhtZqZuyyRdC
+kFUrrv8WNUDkWSN+lMR6xMx8v0583IN6f11IhX0b+svK98G81B2eswBdkzvVyv9M
+unZBO0JuJG8sdM502KhWLmzBC1ZbvgUBF9BumDRpMFH4DCj7+qQ2taWeGyc7AgMB
+AAGjgYYwgYMwDgYDVR0PAQH/BAQDAgKEMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggr
+BgEFBQcDAjASBgNVHRMBAf8ECDAGAQH/AgEAMB0GA1UdDgQWBBT7whmUtX9CnkJe
+qlcwu+3FaXFwnDAfBgNVHSMEGDAWgBT7whmUtX9CnkJeqlcwu+3FaXFwnDANBgkq
+hkiG9w0BAQsFAAOCAQEARMe1wKmF33GjEoLLw0oDDS4EdAv26BzCwtrlljsEtwQN
+95oSzUNd6o4Js7WCG2o543OX6cxzM+yju8TES3+vJKDgsbNMU0bWCv//tdrb0/G8
+OkU3Kfi5q4fOauZ1pqGv/pXdfYhZ5ieB/zwis3ykANe5JfB0XqwCb1Vd0C3UCIS2
+NPKngRwNSzphIsbzfvxGDkdM1enuGl5CVyDhrwTMqGaJGDSOv6U5jKFxKRvigqTN
+Ls9lPmT5NXYETduWLBR3yUIdH6kZXrcozZ02B9vjOB2Cv4RMDc+9eM30CLIWpf1I
+097e7JkhzxFhfC/bMMt3P1FeQc+fwH91wdBmNi7tQw==
+-----END CERTIFICATE-----

nixos/tests/common/acme/server/ca.key.pem

@@ -0,0 +1,27 @@
+-----BEGIN RSA PRIVATE KEY-----
+MIIEowIBAAKCAQEAqzU81SwyTirRmBi1PfBBHCyqbLVxwiwuhueCymxHFPNH5/gK
+1/KrD0h8oXV0JHGI6ulYfxAydU3LckUp7wI8MhuAGtWctUBXPyZ4zI784pgpFHPZ
+UPsJC8B0jPyhsEdXBo5rUPxDApWUwNDcLpSbFzClc23eWrF26v8iU0OVsNuR7XTY
+8EUF07we69yCnGPg7BA/QrxbwvaBv2YbWambsskXQpBVK67/FjVA5FkjfpTEesTM
+fL9OfNyDen9dSIV9G/rLyvfBvNQdnrMAXZM71cr/TLp2QTtCbiRvLHTOdNioVi5s
+wQtWW74FARfQbpg0aTBR+Awo+/qkNrWlnhsnOwIDAQABAoIBAA3ykVkgd5ysmlSU
+trcsCnHcJaojgff6l3PACoSpG4VWaGY6a8+54julgRm6MtMBONFCX0ZCsImj484U
+Wl0xRmwil2YYPuL5MeJgJPktMObY1IfpBCw3tz3w2M3fiuCMf0d2dMGtO1xLiUnH
++hgFXTkfamsj6ThkOrbcQBSebeRxbKM5hqyCaQoieV+0IJnyxUVq/apib8N50VsH
+SHd4oqLUuEZgg6N70+l5DpzedJUb4nrwS/KhUHUBgnoPItYBCiGPmrwLk7fUhPs6
+kTDqJDtc/xW/JbjmzhWEpVvtumcC/OEKULss7HLdeQqwVBrRQkznb0M9AnSra3d0
+X11/Y4ECgYEA3FC8SquLPFb2lHK4+YbJ4Ac6QVWeYFEHiZ0Rj+CmONmjcAvOGLPE
+SblRLm3Nbrkxbm8FF6/AfXa/rviAKEVPs5xqGfSDw/3n1uInPcmShiBCLwM/jHH5
+NeVG+R5mTg5zyQ/pQMLWRcs+Ail+ZAnZuoGpW3Cdc8OtCUYFQ7XB6nsCgYEAxvBJ
+zFxcTtsDzWbMWXejugQiUqJcEbKWwEfkRbf3J2rAVO2+EFr7LxdRfN2VwPiTQcWc
+LnN2QN+ouOjqBMTh3qm5oQY+TLLHy86k9g1k0gXWkMRQgP2ZdfWH1HyrwjLUgLe1
+VezFN7N1azgy6xFkInAAvuA4loxElZNvkGBgekECgYA/Xw26ILvNIGqO6qzgQXAh
++5I7JsiGheg4IjDiBMlrQtbrLMoceuD0H9UFGNplhel9DXwWgxxIOncKejpK2x0A
+2fX+/0FDh+4+9hA5ipiV8gN3iGSoHkSDxy5yC9d7jlapt+TtFt4Rd1OfxZWwatDw
+/8jaH3t6yAcmyrhK8KYVrwKBgAE5KwsBqmOlvyE9N5Z5QN189wUREIXfVkP6bTHs
+jq2EX4hmKdwJ4y+H8i1VY31bSfSGlY5HkXuWpH/2lrHO0CDBZG3UDwADvWzIaYVF
+0c/kz0v2mRQh+xaZmus4lQnNrDbaalgL666LAPbW0qFVaws3KxoBYPe0BxvwWyhF
+H3LBAoGBAKRRNsq2pWQ8Gqxc0rVoH0FlexU9U2ci3lsLmgEB0A/o/kQkSyAxaRM+
+VdKp3sWfO8o8lX5CVQslCNBSjDTNcat3Co4NEBLg6Xv1yKN/WN1GhusnchP9szsP
+oU47gC89QhUyWSd6vvr2z2NG9C3cACxe4dhDSHQcE4nHSldzCKv2
+-----END RSA PRIVATE KEY-----

nixos/tests/common/acme/server/default.nix

@@ -51,10 +51,7 @@
 # that it has to be started _before_ the ACME service.
 { config, pkgs, lib, ... }:
 let
-  testCerts = import ./snakeoil-certs.nix {
+  testCerts = import ./snakeoil-certs.nix;
-    minica = pkgs.minica;
-    mkDerivation = pkgs.stdenv.mkDerivation;
-  };
   domain = testCerts.domain;
 
   resolver = let

nixos/tests/common/acme/server/generate-certs.nix

@@ -0,0 +1,29 @@
+# Minica can provide a CA key and cert, plus a key
+# and cert for our fake CA server's Web Front End (WFE).
+{
+  pkgs ? import <nixpkgs> {},
+  minica ? pkgs.minica,
+  mkDerivation ? pkgs.stdenv.mkDerivation
+}:
+let
+  conf = import ./snakeoil-certs.nix;
+  domain = conf.domain;
+in mkDerivation {
+  name = "test-certs";
+  buildInputs = [ minica ];
+  phases = [ "buildPhase" "installPhase" ];
+
+  buildPhase = ''
+    minica \
+      --ca-key ca.key.pem \
+      --ca-cert ca.cert.pem \
+      --domains ${domain}
+  '';
+
+  installPhase = ''
+    mkdir -p $out
+    mv ca.*.pem $out/
+    mv ${domain}/key.pem $out/${domain}.key.pem
+    mv ${domain}/cert.pem $out/${domain}.cert.pem
+  '';
+}

nixos/tests/common/acme/server/snakeoil-certs.nix

@@ -1,37 +1,13 @@
-# Minica can provide a CA key and cert, plus a key
-# and cert for our fake CA server's Web Front End (WFE).
-{ minica, mkDerivation }:
 let
   domain = "acme.test";
-
-  selfSignedCertData = mkDerivation {
-    name = "test-certs";
-    buildInputs = [ minica ];
-    phases = [ "buildPhase" "installPhase" ];
-
-    buildPhase = ''
-      mkdir ca
-      minica \
-        --ca-key ca/key.pem \
-        --ca-cert ca/cert.pem \
-        --domains ${domain}
-      chmod 600 ca/*
-      chmod 640 ${domain}/*.pem
-    '';
-
-    installPhase = ''
-      mkdir -p $out
-      mv ${domain} ca $out/
-    '';
-  };
 in {
   inherit domain;
   ca = {
-    cert = "${selfSignedCertData}/ca/cert.pem";
+    cert = ./ca.cert.pem;
-    key = "${selfSignedCertData}/ca/key.pem";
+    key = ./ca.key.pem;
   };
   "${domain}" = {
-    cert = "${selfSignedCertData}/${domain}/cert.pem";
+    cert = ./. + "/${domain}.cert.pem";
-    key = "${selfSignedCertData}/${domain}/key.pem";
+    key = ./. + "/${domain}.key.pem";
   };
 }

nixos/tests/convos.nix

@@ -25,6 +25,6 @@ in
     machine.wait_for_unit("convos")
     machine.wait_for_open_port("${toString port}")
     machine.succeed("journalctl -u convos | grep -q 'Listening at.*${toString port}'")
-    machine.succeed("curl http://localhost:${toString port}/")
+    machine.succeed("curl -f http://localhost:${toString port}/")
   '';
 })

nixos/tests/corerad.nix

@@ -80,7 +80,7 @@ import ./make-test-python.nix (
           ), "SLAAC temporary address was not configured on client after router advertisement"
 
       with subtest("Verify HTTP debug server is configured"):
-          out = router.succeed("curl localhost:9430/metrics")
+          out = router.succeed("curl -f localhost:9430/metrics")
 
           assert (
               "corerad_build_info" in out

nixos/tests/docker-edge.nix

@@ -43,7 +43,7 @@ import ./make-test-python.nix ({ pkgs, ...} : {
     docker.fail("sudo -u noprivs docker ps")
     docker.succeed("docker stop sleeping")
 
-    # Must match version twice to ensure client and server versions are correct
+    # Must match version 4 times to ensure client and server git commits and versions are correct
-    docker.succeed('[ $(docker version | grep ${pkgs.docker-edge.version} | wc -l) = "2" ]')
+    docker.succeed('[ $(docker version | grep ${pkgs.docker-edge.version} | wc -l) = "4" ]')
   '';
 })

nixos/tests/docker-tools.nix

@@ -115,7 +115,7 @@ import ./make-test-python.nix ({ pkgs, ... }: {
             "docker load --input='${examples.nginx}'",
             "docker run --name nginx -d -p 8000:80 ${examples.nginx.imageName}",
         )
-        docker.wait_until_succeeds("curl http://localhost:8000/")
+        docker.wait_until_succeeds("curl -f http://localhost:8000/")
         docker.succeed(
             "docker rm --force nginx", "docker rmi '${examples.nginx.imageName}'",
         )

nixos/tests/go-neb.nix

@@ -34,7 +34,7 @@ import ./make-test-python.nix ({ pkgs, ... }:
     start_all()
     server.wait_for_unit("go-neb.service")
     server.wait_until_succeeds(
-        "curl -L http://localhost:4050/services/hooks/d2lraXBlZGlhX3NlcnZpY2U"
+        "curl -fL http://localhost:4050/services/hooks/d2lraXBlZGlhX3NlcnZpY2U"
     )
     server.wait_until_succeeds(
         "journalctl -eu go-neb -o cat | grep -q service_id=wikipedia_service"

nixos/tests/hadoop/hdfs.nix

@@ -48,7 +48,7 @@ import ../make-test-python.nix ({...}: {
     datanode.wait_for_open_port(9866)
     datanode.wait_for_open_port(9867)
 
-    namenode.succeed("curl http://namenode:9870")
+    namenode.succeed("curl -f http://namenode:9870")
-    datanode.succeed("curl http://datanode:9864")
+    datanode.succeed("curl -f http://datanode:9864")
   '';
 })

nixos/tests/hadoop/yarn.nix

@@ -40,7 +40,7 @@ import ../make-test-python.nix ({...}: {
     nodemanager.wait_for_open_port(8042)
     nodemanager.wait_for_open_port(8041)
 
-    resourcemanager.succeed("curl http://localhost:8088")
+    resourcemanager.succeed("curl -f http://localhost:8088")
-    nodemanager.succeed("curl http://localhost:8042")
+    nodemanager.succeed("curl -f http://localhost:8042")
   '';
 })

nixos/tests/haproxy.nix

@@ -39,9 +39,9 @@ import ./make-test-python.nix ({ pkgs, ...}: {
     machine.wait_for_unit("multi-user.target")
     machine.wait_for_unit("haproxy.service")
     machine.wait_for_unit("httpd.service")
-    assert "We are all good!" in machine.succeed("curl -k http://localhost:80/index.txt")
+    assert "We are all good!" in machine.succeed("curl -fk http://localhost:80/index.txt")
     assert "haproxy_process_pool_allocated_bytes" in machine.succeed(
-        "curl -k http://localhost:80/metrics"
+        "curl -fk http://localhost:80/metrics"
     )
 
     with subtest("reload"):
@@ -49,7 +49,7 @@ import ./make-test-python.nix ({ pkgs, ...}: {
         # wait some time to ensure the following request hits the reloaded haproxy
         machine.sleep(5)
         assert "We are all good!" in machine.succeed(
-            "curl -k http://localhost:80/index.txt"
+            "curl -fk http://localhost:80/index.txt"
         )
   '';
 })

nixos/tests/hitch/default.nix

@@ -28,6 +28,6 @@ import ../make-test-python.nix ({ pkgs, ... }:
       machine.wait_for_unit("multi-user.target")
       machine.wait_for_unit("hitch.service")
       machine.wait_for_open_port(443)
-      assert "We are all good!" in machine.succeed("curl -k https://localhost:443/index.txt")
+      assert "We are all good!" in machine.succeed("curl -fk https://localhost:443/index.txt")
     '';
 })

nixos/tests/hound.nix

@@ -53,7 +53,7 @@ import ./make-test-python.nix ({ pkgs, ... } : {
     machine.wait_for_unit("hound.service")
     machine.wait_for_open_port(6080)
     machine.wait_until_succeeds(
-        "curl http://127.0.0.1:6080/api/v1/search\?stats\=fosho\&repos\=\*\&rng=%3A20\&q\=hi\&files\=\&i=nope | grep 'Filename' | grep 'hello'"
+        "curl -f http://127.0.0.1:6080/api/v1/search\?stats\=fosho\&repos\=\*\&rng=%3A20\&q\=hi\&files\=\&i=nope | grep 'Filename' | grep 'hello'"
     )
   '';
 })

nixos/tests/leaps.nix

@@ -26,7 +26,7 @@ import ./make-test-python.nix ({ pkgs,  ... }:
       server.wait_for_open_port(6666)
       client.wait_for_unit("network.target")
       assert "leaps" in client.succeed(
-          "${pkgs.curl}/bin/curl http://server:6666/leaps/"
+          "${pkgs.curl}/bin/curl -f http://server:6666/leaps/"
       )
     '';
 })

nixos/tests/limesurvey.nix

@@ -20,7 +20,7 @@ import ./make-test-python.nix ({ pkgs, ... }: {
 
     machine.wait_for_unit("phpfpm-limesurvey.service")
     assert "The following surveys are available" in machine.succeed(
-        "curl http://example.local/"
+        "curl -f http://example.local/"
     )
   '';
 })

nixos/tests/mailcatcher.nix

@@ -24,7 +24,7 @@ import ./make-test-python.nix ({ lib, ... }:
         'echo "this is the body of the email" | mail -s "subject" root@example.org'
     )
     assert "this is the body of the email" in machine.succeed(
-        "curl http://localhost:1080/messages/1.source"
+        "curl -f http://localhost:1080/messages/1.source"
     )
   '';
 })

nixos/tests/matrix-synapse.nix

@@ -77,12 +77,12 @@ in {
     start_all()
     serverpostgres.wait_for_unit("matrix-synapse.service")
     serverpostgres.wait_until_succeeds(
-        "curl -L --cacert ${ca_pem} https://localhost:8448/"
+        "curl --fail -L --cacert ${ca_pem} https://localhost:8448/"
     )
     serverpostgres.require_unit_state("postgresql.service")
     serversqlite.wait_for_unit("matrix-synapse.service")
     serversqlite.wait_until_succeeds(
-        "curl -L --cacert ${ca_pem} https://localhost:8448/"
+        "curl --fail -L --cacert ${ca_pem} https://localhost:8448/"
     )
     serversqlite.succeed("[ -e /var/lib/matrix-synapse/homeserver.db ]")
   '';

nixos/tests/mediawiki.nix

@@ -22,7 +22,7 @@ import ./make-test-python.nix ({ pkgs, lib, ... }: {
 
     machine.wait_for_unit("phpfpm-mediawiki.service")
 
-    page = machine.succeed("curl -L http://localhost/")
+    page = machine.succeed("curl -fL http://localhost/")
     assert "MediaWiki has been installed" in page
   '';
 })

nixos/tests/metabase.nix

@@ -15,6 +15,6 @@ import ./make-test-python.nix ({ pkgs, ... }: {
     start_all()
     machine.wait_for_unit("metabase.service")
     machine.wait_for_open_port(3000)
-    machine.wait_until_succeeds("curl -L http://localhost:3000/setup | grep Metabase")
+    machine.wait_until_succeeds("curl -fL http://localhost:3000/setup | grep Metabase")
   '';
 })

nixos/tests/morty.nix

@@ -24,7 +24,7 @@ import ./make-test-python.nix ({ pkgs, ... }:
     ''
       mortyProxyWithKey.wait_for_unit("default.target")
       mortyProxyWithKey.wait_for_open_port(3001)
-      mortyProxyWithKey.succeed("curl -L 127.0.0.1:3001 | grep MortyProxy")
+      mortyProxyWithKey.succeed("curl -fL 127.0.0.1:3001 | grep MortyProxy")
     '';
 
 })

nixos/tests/neo4j.nix

@@ -15,6 +15,6 @@ import ./make-test-python.nix {
 
     master.wait_for_unit("neo4j")
     master.wait_for_open_port(7474)
-    master.succeed("curl http://localhost:7474/")
+    master.succeed("curl -f http://localhost:7474/")
   '';
 }

nixos/tests/nzbget.nix

@@ -21,7 +21,7 @@ import ./make-test-python.nix ({ pkgs, ...} : {
     server.wait_for_unit("network.target")
     server.wait_for_open_port(6789)
     assert "This file is part of nzbget" in server.succeed(
-        "curl -s -u nzbget:tegbzn6789 http://127.0.0.1:6789"
+        "curl -f -s -u nzbget:tegbzn6789 http://127.0.0.1:6789"
     )
     server.succeed(
         "${pkgs.nzbget}/bin/nzbget -n -o Control_iP=127.0.0.1 -o Control_port=6789 -o Control_password=tegbzn6789 -V"

nixos/tests/oci-containers.nix

@@ -32,7 +32,7 @@ let
       start_all()
       ${backend}.wait_for_unit("${backend}-nginx.service")
       ${backend}.wait_for_open_port(8181)
-      ${backend}.wait_until_succeeds("curl http://localhost:8181 | grep Hello")
+      ${backend}.wait_until_succeeds("curl -f http://localhost:8181 | grep Hello")
     '';
   };
 

nixos/tests/osrm-backend.nix

@@ -48,10 +48,10 @@ in {
     machine.wait_for_unit("osrm.service")
     machine.wait_for_open_port(${toString port})
     assert "Boulevard Rainier III" in machine.succeed(
-        "curl --silent '${query}' | jq .waypoints[0].name"
+        "curl --fail --silent '${query}' | jq .waypoints[0].name"
     )
     assert "Avenue de la Costa" in machine.succeed(
-        "curl --silent '${query}' | jq .waypoints[1].name"
+        "curl --fail --silent '${query}' | jq .waypoints[1].name"
     )
   '';
 })

nixos/tests/paperless.nix

@@ -23,14 +23,14 @@ import ./make-test-python.nix ({ lib, ... } : {
     with subtest("Service gets ready"):
         machine.wait_for_unit("paperless-server.service")
         # Wait until server accepts connections
-        machine.wait_until_succeeds("curl -s localhost:28981")
+        machine.wait_until_succeeds("curl -fs localhost:28981")
 
     with subtest("Test document is consumed"):
         machine.wait_until_succeeds(
-            "(($(curl -s localhost:28981/api/documents/ | jq .count) == 1))"
+            "(($(curl -fs localhost:28981/api/documents/ | jq .count) == 1))"
         )
         assert "2005-10-16" in machine.succeed(
-            "curl -s localhost:28981/api/documents/ | jq '.results | .[0] | .created'"
+            "curl -fs localhost:28981/api/documents/ | jq '.results | .[0] | .created'"
         )
   '';
 })

nixos/tests/peerflix.nix

@@ -18,6 +18,6 @@ import ./make-test-python.nix ({ pkgs, ...} : {
     start_all()
 
     peerflix.wait_for_unit("peerflix.service")
-    peerflix.wait_until_succeeds("curl localhost:9000")
+    peerflix.wait_until_succeeds("curl -f localhost:9000")
   '';
 })

nixos/tests/php/fpm.nix

@@ -43,7 +43,7 @@ import ../make-test-python.nix ({pkgs, lib, ...}: {
     machine.wait_for_unit("phpfpm-foobar.service")
 
     # Check so we get an evaluated PHP back
-    response = machine.succeed("curl -vvv -s http://127.0.0.1:80/")
+    response = machine.succeed("curl -fvvv -s http://127.0.0.1:80/")
     assert "PHP Version ${pkgs.php.version}" in response, "PHP version not detected"
 
     # Check so we have database and some other extensions loaded

nixos/tests/php/httpd.nix

@@ -21,7 +21,7 @@ import ../make-test-python.nix ({pkgs, lib, ...}: {
     machine.wait_for_unit("httpd.service")
 
     # Check so we get an evaluated PHP back
-    response = machine.succeed("curl -vvv -s http://127.0.0.1:80/")
+    response = machine.succeed("curl -fvvv -s http://127.0.0.1:80/")
     assert "PHP Version ${pkgs.php.version}" in response, "PHP version not detected"
 
     # Check so we have database and some other extensions loaded

nixos/tests/php/pcre.nix

@@ -32,7 +32,7 @@ in import ../make-test-python.nix ({lib, ...}: {
     ''
       machine.wait_for_unit("httpd.service")
       # Ensure php evaluation by matching on the var_dump syntax
-      response = machine.succeed("curl -vvv -s http://127.0.0.1:80/index.php")
+      response = machine.succeed("curl -fvvv -s http://127.0.0.1:80/index.php")
       expected = 'string(${toString (builtins.stringLength testString)}) "${testString}"'
       assert expected in response, "Does not appear to be able to use subgroups."
     '';

nixos/tests/prometheus.nix

@@ -193,13 +193,13 @@ in import ./make-test-python.nix {
     # Check if prometheus responds to requests:
     prometheus.wait_for_unit("prometheus.service")
     prometheus.wait_for_open_port(${toString queryPort})
-    prometheus.succeed("curl -s http://127.0.0.1:${toString queryPort}/metrics")
+    prometheus.succeed("curl -sf http://127.0.0.1:${toString queryPort}/metrics")
 
     # Let's test if pushing a metric to the pushgateway succeeds:
     prometheus.wait_for_unit("pushgateway.service")
     prometheus.succeed(
         "echo 'some_metric 3.14' | "
-        + "curl --data-binary \@- "
+        + "curl -f --data-binary \@- "
         + "http://127.0.0.1:${toString pushgwPort}/metrics/job/some_job"
     )
 

nixos/tests/sanoid.nix

@@ -38,7 +38,7 @@ in {
 
       services.syncoid = {
         enable = true;
-        sshKey = "/root/.ssh/id_ecdsa";
+        sshKey = "/var/lib/syncoid/id_ecdsa";
         commonArgs = [ "--no-sync-snap" ];
         commands."pool/test".target = "root@target:pool/test";
       };
@@ -69,11 +69,12 @@ in {
         "udevadm settle",
     )
 
-    source.succeed("mkdir -m 700 /root/.ssh")
     source.succeed(
-        "cat '${snakeOilPrivateKey}' > /root/.ssh/id_ecdsa"
+        "mkdir -m 700 -p /var/lib/syncoid",
+        "cat '${snakeOilPrivateKey}' > /var/lib/syncoid/id_ecdsa",
+        "chmod 600 /var/lib/syncoid/id_ecdsa",
+        "chown -R syncoid:syncoid /var/lib/syncoid/",
     )
-    source.succeed("chmod 600 /root/.ssh/id_ecdsa")
 
     source.succeed("touch /tmp/mnt/test.txt")
     source.systemctl("start --wait sanoid.service")

nixos/tests/service-runner.nix

@@ -29,7 +29,7 @@ import ./make-test-python.nix ({ pkgs, ... }: {
             """
         )
         machine.wait_for_open_port(80)
-        machine.succeed(f"curl {url}")
+        machine.succeed(f"curl -f {url}")
         machine.succeed("kill -INT $(cat my-nginx.pid)")
         machine.wait_for_closed_port(80)
   '';