| 
									
										
										
										
											2017-01-22 15:52:35 -05:00
										 |  |  | <chapter xmlns="http://docbook.org/ns/docbook" | 
					
						
							|  |  |  |          xmlns:xlink="http://www.w3.org/1999/xlink" | 
					
						
							|  |  |  |          xml:id="chap-cross"> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |  <title>Cross-compilation</title> | 
					
						
							|  |  |  |  <section xml:id="sec-cross-intro"> | 
					
						
							| 
									
										
										
										
											2017-01-22 15:52:35 -05:00
										 |  |  |   <title>Introduction</title> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2017-01-22 15:52:35 -05:00
										 |  |  |   <para> | 
					
						
							| 
									
										
										
										
											2019-03-08 21:07:11 -08:00
										 |  |  |    "Cross-compilation" means compiling a program on one machine for another | 
					
						
							|  |  |  |    type of machine. For example, a typical use of cross-compilation is to | 
					
						
							|  |  |  |    compile programs for embedded devices. These devices often don't have the | 
					
						
							|  |  |  |    computing power and memory to compile their own programs. One might think | 
					
						
							|  |  |  |    that cross-compilation is a fairly niche concern. However, there are | 
					
						
							|  |  |  |    significant advantages to rigorously distinguishing between build-time and | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |    run-time environments! Significant, because the benefits apply even when one | 
					
						
							|  |  |  |    is developing and deploying on the same machine. Nixpkgs is increasingly | 
					
						
							|  |  |  |    adopting the opinion that packages should be written with cross-compilation | 
					
						
							|  |  |  |    in mind, and nixpkgs should evaluate in a similar way (by minimizing | 
					
						
							|  |  |  |    cross-compilation-specific special cases) whether or not one is | 
					
						
							|  |  |  |    cross-compiling. | 
					
						
							| 
									
										
										
										
											2017-01-22 15:52:35 -05:00
										 |  |  |   </para> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |   <para> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    This chapter will be organized in three parts. First, it will describe the | 
					
						
							|  |  |  |    basics of how to package software in a way that supports cross-compilation. | 
					
						
							|  |  |  |    Second, it will describe how to use Nixpkgs when cross-compiling. Third, it | 
					
						
							|  |  |  |    will describe the internal infrastructure supporting cross-compilation. | 
					
						
							| 
									
										
										
										
											2017-01-22 15:52:35 -05:00
										 |  |  |   </para> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |  </section> | 
					
						
							| 
									
										
										
										
											2017-01-22 15:52:35 -05:00
										 |  |  | <!--============================================================--> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |  <section xml:id="sec-cross-packaging"> | 
					
						
							| 
									
										
										
										
											2017-02-08 16:43:52 -05:00
										 |  |  |   <title>Packaging in a cross-friendly manner</title> | 
					
						
							| 
									
										
										
										
											2017-01-22 15:52:35 -05:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |   <section xml:id="ssec-cross-platform-parameters"> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    <title>Platform parameters</title> | 
					
						
							| 
									
										
										
										
											2017-09-15 14:16:22 +02:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    <para> | 
					
						
							| 
									
										
										
										
											2019-03-08 21:07:11 -08:00
										 |  |  |     Nixpkgs follows the | 
					
						
							|  |  |  |     <link | 
					
						
							| 
									
										
										
										
											2018-11-18 23:23:22 -06:00
										 |  |  |      xlink:href="https://gcc.gnu.org/onlinedocs/gccint/Configure-Terms.html">conventions | 
					
						
							| 
									
										
										
										
											2019-03-08 21:07:11 -08:00
										 |  |  |     of GNU autoconf</link>. We distinguish between 3 types of platforms when | 
					
						
							|  |  |  |     building a derivation: <wordasword>build</wordasword>, | 
					
						
							|  |  |  |     <wordasword>host</wordasword>, and <wordasword>target</wordasword>. In | 
					
						
							|  |  |  |     summary, <wordasword>build</wordasword> is the platform on which a package | 
					
						
							|  |  |  |     is being built, <wordasword>host</wordasword> is the platform on which it | 
					
						
							|  |  |  |     will run. The third attribute, <wordasword>target</wordasword>, is relevant | 
					
						
							|  |  |  |     only for certain specific compilers and build tools. | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    </para> | 
					
						
							| 
									
										
										
										
											2017-09-15 14:16:22 +02:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    <para> | 
					
						
							|  |  |  |     In Nixpkgs, these three platforms are defined as attribute sets under the | 
					
						
							| 
									
										
										
										
											2018-10-02 15:59:59 -04:00
										 |  |  |     names <literal>buildPlatform</literal>, <literal>hostPlatform</literal>, | 
					
						
							|  |  |  |     and <literal>targetPlatform</literal>. They are always defined as | 
					
						
							|  |  |  |     attributes in the standard environment. That means one can access them | 
					
						
							|  |  |  |     like: | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  | <programlisting>{ stdenv, fooDep, barDep, .. }: ...stdenv.buildPlatform...</programlisting> | 
					
						
							|  |  |  |     . | 
					
						
							|  |  |  |    </para> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |    <variablelist> | 
					
						
							|  |  |  |     <varlistentry> | 
					
						
							| 
									
										
										
										
											2018-05-31 21:03:37 -04:00
										 |  |  |      <term> | 
					
						
							|  |  |  |       <varname>buildPlatform</varname> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |      </term> | 
					
						
							|  |  |  |      <listitem> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  |        The "build platform" is the platform on which a package is built. Once | 
					
						
							|  |  |  |        someone has a built package, or pre-built binary package, the build | 
					
						
							| 
									
										
										
										
											2018-11-18 23:23:22 -06:00
										 |  |  |        platform should not matter and can be ignored. | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |       </para> | 
					
						
							|  |  |  |      </listitem> | 
					
						
							|  |  |  |     </varlistentry> | 
					
						
							|  |  |  |     <varlistentry> | 
					
						
							| 
									
										
										
										
											2018-05-31 21:03:37 -04:00
										 |  |  |      <term> | 
					
						
							|  |  |  |       <varname>hostPlatform</varname> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |      </term> | 
					
						
							|  |  |  |      <listitem> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  |        The "host platform" is the platform on which a package will be run. This | 
					
						
							|  |  |  |        is the simplest platform to understand, but also the one with the worst | 
					
						
							|  |  |  |        name. | 
					
						
							|  |  |  |       </para> | 
					
						
							|  |  |  |      </listitem> | 
					
						
							|  |  |  |     </varlistentry> | 
					
						
							|  |  |  |     <varlistentry> | 
					
						
							| 
									
										
										
										
											2018-05-31 21:03:37 -04:00
										 |  |  |      <term> | 
					
						
							|  |  |  |       <varname>targetPlatform</varname> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |      </term> | 
					
						
							|  |  |  |      <listitem> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  |        The "target platform" attribute is, unlike the other two attributes, not | 
					
						
							|  |  |  |        actually fundamental to the process of building software. Instead, it is | 
					
						
							|  |  |  |        only relevant for compatibility with building certain specific compilers | 
					
						
							|  |  |  |        and build tools. It can be safely ignored for all other packages. | 
					
						
							|  |  |  |       </para> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  |        The build process of certain compilers is written in such a way that the | 
					
						
							|  |  |  |        compiler resulting from a single build can itself only produce binaries | 
					
						
							| 
									
										
										
										
											2018-11-18 23:23:22 -06:00
										 |  |  |        for a single platform. The task of specifying this single "target | 
					
						
							| 
									
										
										
										
											2019-03-08 21:07:11 -08:00
										 |  |  |        platform" is thus pushed to build time of the compiler. The root cause | 
					
						
							|  |  |  |        of this is that the compiler (which will be run on the host) and the | 
					
						
							|  |  |  |        standard library/runtime (which will be run on the target) are built by | 
					
						
							|  |  |  |        a single build process. | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |       </para> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  |        There is no fundamental need to think about a single target ahead of | 
					
						
							|  |  |  |        time like this. If the tool supports modular or pluggable backends, both | 
					
						
							|  |  |  |        the need to specify the target at build time and the constraint of | 
					
						
							|  |  |  |        having only a single target disappear. An example of such a tool is | 
					
						
							|  |  |  |        LLVM. | 
					
						
							|  |  |  |       </para> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  |        Although the existence of a "target platfom" is arguably a historical | 
					
						
							|  |  |  |        mistake, it is a common one: examples of tools that suffer from it are | 
					
						
							|  |  |  |        GCC, Binutils, GHC and Autoconf. Nixpkgs tries to avoid sharing in the | 
					
						
							|  |  |  |        mistake where possible. Still, because the concept of a target platform | 
					
						
							|  |  |  |        is so ingrained, it is best to support it as is. | 
					
						
							|  |  |  |       </para> | 
					
						
							|  |  |  |      </listitem> | 
					
						
							|  |  |  |     </varlistentry> | 
					
						
							|  |  |  |    </variablelist> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |    <para> | 
					
						
							|  |  |  |     The exact schema these fields follow is a bit ill-defined due to a long and | 
					
						
							|  |  |  |     convoluted evolution, but this is slowly being cleaned up. You can see | 
					
						
							|  |  |  |     examples of ones used in practice in | 
					
						
							|  |  |  |     <literal>lib.systems.examples</literal>; note how they are not all very | 
					
						
							|  |  |  |     consistent. For now, here are few fields can count on them containing: | 
					
						
							|  |  |  |    </para> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |    <variablelist> | 
					
						
							|  |  |  |     <varlistentry> | 
					
						
							| 
									
										
										
										
											2018-05-31 21:03:37 -04:00
										 |  |  |      <term> | 
					
						
							|  |  |  |       <varname>system</varname> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |      </term> | 
					
						
							|  |  |  |      <listitem> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  |        This is a two-component shorthand for the platform. Examples of this | 
					
						
							|  |  |  |        would be "x86_64-darwin" and "i686-linux"; see | 
					
						
							| 
									
										
										
										
											2018-11-18 23:23:22 -06:00
										 |  |  |        <literal>lib.systems.doubles</literal> for more. The first component | 
					
						
							| 
									
										
										
										
											2019-03-08 21:07:11 -08:00
										 |  |  |        corresponds to the CPU architecture of the platform and the second to | 
					
						
							|  |  |  |        the operating system of the platform (<literal>[cpu]-[os]</literal>). | 
					
						
							|  |  |  |        This format has built-in support in Nix, such as the | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |        <varname>builtins.currentSystem</varname> impure string. | 
					
						
							|  |  |  |       </para> | 
					
						
							|  |  |  |      </listitem> | 
					
						
							|  |  |  |     </varlistentry> | 
					
						
							|  |  |  |     <varlistentry> | 
					
						
							| 
									
										
										
										
											2018-05-31 21:03:37 -04:00
										 |  |  |      <term> | 
					
						
							|  |  |  |       <varname>config</varname> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |      </term> | 
					
						
							|  |  |  |      <listitem> | 
					
						
							|  |  |  |       <para> | 
					
						
							| 
									
										
										
										
											2019-03-08 21:07:11 -08:00
										 |  |  |        This is a 3- or 4- component shorthand for the platform. Examples of | 
					
						
							|  |  |  |        this would be <literal>x86_64-unknown-linux-gnu</literal> and | 
					
						
							| 
									
										
										
										
											2018-11-18 23:23:22 -06:00
										 |  |  |        <literal>aarch64-apple-darwin14</literal>. This is a standard format | 
					
						
							|  |  |  |        called the "LLVM target triple", as they are pioneered by LLVM. In the | 
					
						
							|  |  |  |        4-part form, this corresponds to | 
					
						
							|  |  |  |        <literal>[cpu]-[vendor]-[os]-[abi]</literal>. This format is strictly | 
					
						
							| 
									
										
										
										
											2019-03-08 21:07:11 -08:00
										 |  |  |        more informative than the "Nix host double", as the previous format | 
					
						
							|  |  |  |        could analogously be termed. This needs a better name than | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |        <varname>config</varname>! | 
					
						
							|  |  |  |       </para> | 
					
						
							|  |  |  |      </listitem> | 
					
						
							|  |  |  |     </varlistentry> | 
					
						
							|  |  |  |     <varlistentry> | 
					
						
							| 
									
										
										
										
											2018-05-31 21:03:37 -04:00
										 |  |  |      <term> | 
					
						
							|  |  |  |       <varname>parsed</varname> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |      </term> | 
					
						
							|  |  |  |      <listitem> | 
					
						
							|  |  |  |       <para> | 
					
						
							| 
									
										
										
										
											2019-03-08 21:07:11 -08:00
										 |  |  |        This is a Nix representation of a parsed LLVM target triple with | 
					
						
							|  |  |  |        white-listed components. This can be specified directly, or actually | 
					
						
							|  |  |  |        parsed from the <varname>config</varname>. See | 
					
						
							|  |  |  |        <literal>lib.systems.parse</literal> for the exact representation. | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |       </para> | 
					
						
							|  |  |  |      </listitem> | 
					
						
							|  |  |  |     </varlistentry> | 
					
						
							|  |  |  |     <varlistentry> | 
					
						
							| 
									
										
										
										
											2018-05-31 21:03:37 -04:00
										 |  |  |      <term> | 
					
						
							|  |  |  |       <varname>libc</varname> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |      </term> | 
					
						
							|  |  |  |      <listitem> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  |        This is a string identifying the standard C library used. Valid | 
					
						
							|  |  |  |        identifiers include "glibc" for GNU libc, "libSystem" for Darwin's | 
					
						
							|  |  |  |        Libsystem, and "uclibc" for µClibc. It should probably be refactored to | 
					
						
							|  |  |  |        use the module system, like <varname>parse</varname>. | 
					
						
							|  |  |  |       </para> | 
					
						
							|  |  |  |      </listitem> | 
					
						
							|  |  |  |     </varlistentry> | 
					
						
							|  |  |  |     <varlistentry> | 
					
						
							| 
									
										
										
										
											2018-05-31 21:03:37 -04:00
										 |  |  |      <term> | 
					
						
							|  |  |  |       <varname>is*</varname> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |      </term> | 
					
						
							|  |  |  |      <listitem> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  |        These predicates are defined in <literal>lib.systems.inspect</literal>, | 
					
						
							| 
									
										
										
										
											2018-11-18 23:10:39 -06:00
										 |  |  |        and slapped onto every platform. They are superior to the ones in | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |        <varname>stdenv</varname> as they force the user to be explicit about | 
					
						
							|  |  |  |        which platform they are inspecting. Please use these instead of those. | 
					
						
							|  |  |  |       </para> | 
					
						
							|  |  |  |      </listitem> | 
					
						
							|  |  |  |     </varlistentry> | 
					
						
							|  |  |  |     <varlistentry> | 
					
						
							| 
									
										
										
										
											2018-05-31 21:03:37 -04:00
										 |  |  |      <term> | 
					
						
							|  |  |  |       <varname>platform</varname> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |      </term> | 
					
						
							|  |  |  |      <listitem> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  |        This is, quite frankly, a dumping ground of ad-hoc settings (it's an | 
					
						
							|  |  |  |        attribute set). See <literal>lib.systems.platforms</literal> for | 
					
						
							|  |  |  |        examples—there's hopefully one in there that will work verbatim for | 
					
						
							|  |  |  |        each platform that is working. Please help us triage these flags and | 
					
						
							|  |  |  |        give them better homes! | 
					
						
							|  |  |  |       </para> | 
					
						
							|  |  |  |      </listitem> | 
					
						
							|  |  |  |     </varlistentry> | 
					
						
							|  |  |  |    </variablelist> | 
					
						
							| 
									
										
										
										
											2017-01-22 15:52:35 -05:00
										 |  |  |   </section> | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |   <section xml:id="ssec-cross-dependency-categorization"> | 
					
						
							|  |  |  |    <title>Theory of dependency categorization</title> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |    <note> | 
					
						
							|  |  |  |     <para> | 
					
						
							|  |  |  |      This is a rather philosophical description that isn't very | 
					
						
							|  |  |  |      Nixpkgs-specific. For an overview of all the relevant attributes given to | 
					
						
							|  |  |  |      <varname>mkDerivation</varname>, see | 
					
						
							|  |  |  |      <xref | 
					
						
							|  |  |  |      linkend="ssec-stdenv-dependencies"/>. For a description of how | 
					
						
							|  |  |  |      everything is implemented, see | 
					
						
							|  |  |  |      <xref linkend="ssec-cross-dependency-implementation" />. | 
					
						
							|  |  |  |     </para> | 
					
						
							|  |  |  |    </note> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  | 
 | 
					
						
							|  |  |  |    <para> | 
					
						
							|  |  |  |     In this section we explore the relationship between both runtime and | 
					
						
							| 
									
										
										
										
											2018-11-18 23:10:39 -06:00
										 |  |  |     build-time dependencies and the 3 Autoconf platforms. | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    </para> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |    <para> | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |     A run time dependency between two packages requires that their host | 
					
						
							|  |  |  |     platforms match. This is directly implied by the meaning of "host platform" | 
					
						
							|  |  |  |     and "runtime dependency": The package dependency exists while both packages | 
					
						
							|  |  |  |     are running on a single host platform. | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    </para> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |    <para> | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |     A build time dependency, however, has a shift in platforms between the | 
					
						
							|  |  |  |     depending package and the depended-on package. "build time dependency" | 
					
						
							|  |  |  |     means that to build the depending package we need to be able to run the | 
					
						
							|  |  |  |     depended-on's package. The depending package's build platform is therefore | 
					
						
							|  |  |  |     equal to the depended-on package's host platform. | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    </para> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |    <para> | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |     If both the dependency and depending packages aren't compilers or other | 
					
						
							|  |  |  |     machine-code-producing tools, we're done. And indeed | 
					
						
							|  |  |  |     <varname>buildInputs</varname> and <varname>nativeBuildInputs</varname> | 
					
						
							|  |  |  |     have covered these simpler build-time and run-time (respectively) changes | 
					
						
							| 
									
										
										
										
											2019-03-25 17:50:13 -04:00
										 |  |  |     for many years. But if the dependency does produce machine code, we might | 
					
						
							|  |  |  |     need to worry about its target platform too. In principle, that target | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |     platform might be any of the depending package's build, host, or target | 
					
						
							|  |  |  |     platforms, but we prohibit dependencies from a "later" platform to an | 
					
						
							|  |  |  |     earlier platform to limit confusion because we've never seen a legitimate | 
					
						
							|  |  |  |     use for them. | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    </para> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |    <para> | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |     Finally, if the depending package is a compiler or other | 
					
						
							|  |  |  |     machine-code-producing tool, it might need dependencies that run at "emit | 
					
						
							| 
									
										
										
										
											2019-03-25 17:50:13 -04:00
										 |  |  |     time". This is for compilers that (regrettably) insist on being built | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |     together with their source langauges' standard libraries. Assuming build != | 
					
						
							|  |  |  |     host != target, a run-time dependency of the standard library cannot be run | 
					
						
							|  |  |  |     at the compiler's build time or run time, but only at the run time of code | 
					
						
							|  |  |  |     emitted by the compiler. | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    </para> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |    <para> | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |     Putting this all together, that means we have dependencies in the form | 
					
						
							|  |  |  |     "host → target", in at most the following six combinations: | 
					
						
							|  |  |  |     <table> | 
					
						
							|  |  |  |      <caption>Possible dependency types</caption> | 
					
						
							|  |  |  |      <thead> | 
					
						
							|  |  |  |       <tr> | 
					
						
							|  |  |  |        <th>Dependency's host platform</th> | 
					
						
							|  |  |  |        <th>Dependency's target platform</th> | 
					
						
							|  |  |  |       </tr> | 
					
						
							|  |  |  |      </thead> | 
					
						
							|  |  |  |      <tbody> | 
					
						
							|  |  |  |       <tr> | 
					
						
							|  |  |  |        <td>build</td> | 
					
						
							|  |  |  |        <td>build</td> | 
					
						
							|  |  |  |       </tr> | 
					
						
							|  |  |  |       <tr> | 
					
						
							|  |  |  |        <td>build</td> | 
					
						
							|  |  |  |        <td>host</td> | 
					
						
							|  |  |  |       </tr> | 
					
						
							|  |  |  |       <tr> | 
					
						
							|  |  |  |        <td>build</td> | 
					
						
							|  |  |  |        <td>target</td> | 
					
						
							|  |  |  |       </tr> | 
					
						
							|  |  |  |       <tr> | 
					
						
							|  |  |  |        <td>host</td> | 
					
						
							|  |  |  |        <td>host</td> | 
					
						
							|  |  |  |       </tr> | 
					
						
							|  |  |  |       <tr> | 
					
						
							|  |  |  |        <td>host</td> | 
					
						
							|  |  |  |        <td>target</td> | 
					
						
							|  |  |  |       </tr> | 
					
						
							|  |  |  |       <tr> | 
					
						
							|  |  |  |        <td>target</td> | 
					
						
							|  |  |  |        <td>target</td> | 
					
						
							|  |  |  |       </tr> | 
					
						
							|  |  |  |      </tbody> | 
					
						
							|  |  |  |     </table> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    </para> | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |    <para> | 
					
						
							|  |  |  |     Some examples will make this table clearer. Suppose there's some package | 
					
						
							|  |  |  |     that is being built with a <literal>(build, host, target)</literal> | 
					
						
							|  |  |  |     platform triple of <literal>(foo, bar, baz)</literal>. If it has a | 
					
						
							|  |  |  |     build-time library dependency, that would be a "host → build" dependency | 
					
						
							|  |  |  |     with a triple of <literal>(foo, foo, *)</literal> (the target platform is | 
					
						
							|  |  |  |     irrelevant). If it needs a compiler to be built, that would be a "build → | 
					
						
							|  |  |  |     host" dependency with a triple of <literal>(foo, foo, *)</literal> (the | 
					
						
							|  |  |  |     target platform is irrelevant). That compiler, would be built with another | 
					
						
							|  |  |  |     compiler, also "build → host" dependency, with a triple of <literal>(foo, | 
					
						
							|  |  |  |     foo, foo)</literal>. | 
					
						
							|  |  |  |    </para> | 
					
						
							| 
									
										
										
										
											2017-01-22 15:52:35 -05:00
										 |  |  |   </section> | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |   <section xml:id="ssec-cross-cookbook"> | 
					
						
							| 
									
										
										
										
											2018-07-06 15:52:19 +00:00
										 |  |  |    <title>Cross packaging cookbook</title> | 
					
						
							| 
									
										
										
										
											2017-01-22 15:52:35 -05:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    <para> | 
					
						
							| 
									
										
										
										
											2018-11-18 23:23:22 -06:00
										 |  |  |     Some frequently encountered problems when packaging for cross-compilation | 
					
						
							|  |  |  |     should be answered here. Ideally, the information above is exhaustive, so | 
					
						
							|  |  |  |     this section cannot provide any new information, but it is ludicrous and | 
					
						
							|  |  |  |     cruel to expect everyone to spend effort working through the interaction of | 
					
						
							| 
									
										
										
										
											2019-03-08 21:07:11 -08:00
										 |  |  |     many features just to figure out the same answer to the same common | 
					
						
							|  |  |  |     problem. Feel free to add to this list! | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    </para> | 
					
						
							| 
									
										
										
										
											2017-01-22 15:52:35 -05:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    <qandaset> | 
					
						
							| 
									
										
										
										
											2018-09-01 15:20:57 -04:00
										 |  |  |     <qandaentry xml:id="cross-qa-build-c-program-in-build-environment"> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |      <question> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  |        What if my package's build system needs to build a C program to be run | 
					
						
							|  |  |  |        under the build environment? | 
					
						
							|  |  |  |       </para> | 
					
						
							|  |  |  |      </question> | 
					
						
							|  |  |  |      <answer> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  | <programlisting>depsBuildBuild = [ buildPackages.stdenv.cc ];</programlisting> | 
					
						
							|  |  |  |        Add it to your <function>mkDerivation</function> invocation. | 
					
						
							|  |  |  |       </para> | 
					
						
							|  |  |  |      </answer> | 
					
						
							|  |  |  |     </qandaentry> | 
					
						
							| 
									
										
										
										
											2018-09-01 15:20:57 -04:00
										 |  |  |     <qandaentry xml:id="cross-qa-fails-to-find-ar"> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |      <question> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  |        My package fails to find <command>ar</command>. | 
					
						
							|  |  |  |       </para> | 
					
						
							|  |  |  |      </question> | 
					
						
							|  |  |  |      <answer> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  |        Many packages assume that an unprefixed <command>ar</command> is | 
					
						
							|  |  |  |        available, but Nix doesn't provide one. It only provides a prefixed one, | 
					
						
							|  |  |  |        just as it only does for all the other binutils programs. It may be | 
					
						
							|  |  |  |        necessary to patch the package to fix the build system to use a prefixed | 
					
						
							|  |  |  |        `ar`. | 
					
						
							|  |  |  |       </para> | 
					
						
							|  |  |  |      </answer> | 
					
						
							|  |  |  |     </qandaentry> | 
					
						
							| 
									
										
										
										
											2018-09-01 15:20:57 -04:00
										 |  |  |     <qandaentry xml:id="cross-testsuite-runs-host-code"> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |      <question> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  |        My package's testsuite needs to run host platform code. | 
					
						
							|  |  |  |       </para> | 
					
						
							|  |  |  |      </question> | 
					
						
							|  |  |  |      <answer> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  | <programlisting>doCheck = stdenv.hostPlatform != stdenv.buildPlatfrom;</programlisting> | 
					
						
							|  |  |  |        Add it to your <function>mkDerivation</function> invocation. | 
					
						
							|  |  |  |       </para> | 
					
						
							|  |  |  |      </answer> | 
					
						
							|  |  |  |     </qandaentry> | 
					
						
							|  |  |  |    </qandaset> | 
					
						
							|  |  |  |   </section> | 
					
						
							|  |  |  |  </section> | 
					
						
							|  |  |  | <!--============================================================--> | 
					
						
							|  |  |  |  <section xml:id="sec-cross-usage"> | 
					
						
							| 
									
										
										
										
											2017-01-22 15:52:35 -05:00
										 |  |  |   <title>Cross-building packages</title> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2017-02-08 16:43:52 -05:00
										 |  |  |   <para> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    Nixpkgs can be instantiated with <varname>localSystem</varname> alone, in | 
					
						
							| 
									
										
										
										
											2018-11-18 23:23:22 -06:00
										 |  |  |    which case there is no cross-compiling and everything is built by and for | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    that system, or also with <varname>crossSystem</varname>, in which case | 
					
						
							|  |  |  |    packages run on the latter, but all building happens on the former. Both | 
					
						
							|  |  |  |    parameters take the same schema as the 3 (build, host, and target) platforms | 
					
						
							|  |  |  |    defined in the previous section. As mentioned above, | 
					
						
							|  |  |  |    <literal>lib.systems.examples</literal> has some platforms which are used as | 
					
						
							|  |  |  |    arguments for these parameters in practice. You can use them | 
					
						
							|  |  |  |    programmatically, or on the command line: | 
					
						
							|  |  |  | <programlisting> | 
					
						
							| 
									
										
										
										
											2018-01-25 18:32:18 -05:00
										 |  |  | nix-build <nixpkgs> --arg crossSystem '(import <nixpkgs/lib>).systems.examples.fooBarBaz' -A whatever</programlisting> | 
					
						
							| 
									
										
										
										
											2017-02-08 16:43:52 -05:00
										 |  |  |   </para> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2018-01-26 12:30:05 -05:00
										 |  |  |   <note> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    <para> | 
					
						
							|  |  |  |     Eventually we would like to make these platform examples an unnecessary | 
					
						
							|  |  |  |     convenience so that | 
					
						
							|  |  |  | <programlisting> | 
					
						
							| 
									
										
										
										
											2018-11-28 16:00:43 -06:00
										 |  |  | nix-build <nixpkgs> --arg crossSystem '{ config = "<arch>-<os>-<vendor>-<abi>"; }' -A whatever</programlisting> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |     works in the vast majority of cases. The problem today is dependencies on | 
					
						
							|  |  |  |     other sorts of configuration which aren't given proper defaults. We rely on | 
					
						
							|  |  |  |     the examples to crudely to set those configuration parameters in some | 
					
						
							|  |  |  |     vaguely sane manner on the users behalf. Issue | 
					
						
							|  |  |  |     <link xlink:href="https://github.com/NixOS/nixpkgs/issues/34274">#34274</link> | 
					
						
							|  |  |  |     tracks this inconvenience along with its root cause in crufty configuration | 
					
						
							|  |  |  |     options. | 
					
						
							|  |  |  |    </para> | 
					
						
							| 
									
										
										
										
											2018-01-26 12:30:05 -05:00
										 |  |  |   </note> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2017-05-23 18:04:15 -04:00
										 |  |  |   <para> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    While one is free to pass both parameters in full, there's a lot of logic to | 
					
						
							|  |  |  |    fill in missing fields. As discussed in the previous section, only one of | 
					
						
							|  |  |  |    <varname>system</varname>, <varname>config</varname>, and | 
					
						
							|  |  |  |    <varname>parsed</varname> is needed to infer the other two. Additionally, | 
					
						
							|  |  |  |    <varname>libc</varname> will be inferred from <varname>parse</varname>. | 
					
						
							|  |  |  |    Finally, <literal>localSystem.system</literal> is also | 
					
						
							|  |  |  |    <emphasis>impurely</emphasis> inferred based on the platform evaluation | 
					
						
							|  |  |  |    occurs. This means it is often not necessary to pass | 
					
						
							|  |  |  |    <varname>localSystem</varname> at all, as in the command-line example in the | 
					
						
							|  |  |  |    previous paragraph. | 
					
						
							| 
									
										
										
										
											2017-05-23 18:04:15 -04:00
										 |  |  |   </para> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2017-05-23 18:04:15 -04:00
										 |  |  |   <note> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    <para> | 
					
						
							|  |  |  |     Many sources (manual, wiki, etc) probably mention passing | 
					
						
							|  |  |  |     <varname>system</varname>, <varname>platform</varname>, along with the | 
					
						
							|  |  |  |     optional <varname>crossSystem</varname> to nixpkgs: <literal>import | 
					
						
							|  |  |  |     <nixpkgs> { system = ..; platform = ..; crossSystem = ..; | 
					
						
							|  |  |  |     }</literal>. Passing those two instead of <varname>localSystem</varname> is | 
					
						
							|  |  |  |     still supported for compatibility, but is discouraged. Indeed, much of the | 
					
						
							|  |  |  |     inference we do for these parameters is motivated by compatibility as much | 
					
						
							|  |  |  |     as convenience. | 
					
						
							|  |  |  |    </para> | 
					
						
							| 
									
										
										
										
											2017-05-23 18:04:15 -04:00
										 |  |  |   </note> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2017-01-22 15:52:35 -05:00
										 |  |  |   <para> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    One would think that <varname>localSystem</varname> and | 
					
						
							|  |  |  |    <varname>crossSystem</varname> overlap horribly with the three | 
					
						
							|  |  |  |    <varname>*Platforms</varname> (<varname>buildPlatform</varname>, | 
					
						
							|  |  |  |    <varname>hostPlatform,</varname> and <varname>targetPlatform</varname>; see | 
					
						
							|  |  |  |    <varname>stage.nix</varname> or the manual). Actually, those identifiers are | 
					
						
							|  |  |  |    purposefully not used here to draw a subtle but important distinction: While | 
					
						
							|  |  |  |    the granularity of having 3 platforms is necessary to properly *build* | 
					
						
							|  |  |  |    packages, it is overkill for specifying the user's *intent* when making a | 
					
						
							|  |  |  |    build plan or package set. A simple "build vs deploy" dichotomy is adequate: | 
					
						
							|  |  |  |    the sliding window principle described in the previous section shows how to | 
					
						
							|  |  |  |    interpolate between the these two "end points" to get the 3 platform triple | 
					
						
							| 
									
										
										
										
											2019-03-08 21:07:11 -08:00
										 |  |  |    for each bootstrapping stage. That means for any package a given package | 
					
						
							|  |  |  |    set, even those not bound on the top level but only reachable via | 
					
						
							|  |  |  |    dependencies or <varname>buildPackages</varname>, the three platforms will | 
					
						
							|  |  |  |    be defined as one of <varname>localSystem</varname> or | 
					
						
							|  |  |  |    <varname>crossSystem</varname>, with the former replacing the latter as one | 
					
						
							|  |  |  |    traverses build-time dependencies. A last simple difference is that | 
					
						
							|  |  |  |    <varname>crossSystem</varname> should be null when one doesn't want to | 
					
						
							|  |  |  |    cross-compile, while the <varname>*Platform</varname>s are always non-null. | 
					
						
							|  |  |  |    <varname>localSystem</varname> is always non-null. | 
					
						
							| 
									
										
										
										
											2017-01-22 15:52:35 -05:00
										 |  |  |   </para> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |  </section> | 
					
						
							| 
									
										
										
										
											2017-01-22 15:52:35 -05:00
										 |  |  | <!--============================================================--> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |  <section xml:id="sec-cross-infra"> | 
					
						
							| 
									
										
										
										
											2017-01-22 15:52:35 -05:00
										 |  |  |   <title>Cross-compilation infrastructure</title> | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |   <section xml:id="ssec-cross-dependency-implementation"> | 
					
						
							|  |  |  |    <title>Implementation of dependencies</title> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  | 
 | 
					
						
							|  |  |  |    <para> | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |     The categorizes of dependencies developed in | 
					
						
							|  |  |  |     <xref | 
					
						
							|  |  |  |     linkend="ssec-cross-dependency-categorization"/> are specified as | 
					
						
							|  |  |  |     lists of derivations given to <varname>mkDerivation</varname>, as | 
					
						
							| 
									
										
										
										
											2019-03-25 17:50:13 -04:00
										 |  |  |     documented in <xref linkend="ssec-stdenv-dependencies"/>. In short, | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |     each list of dependencies for "host → target" of "foo → bar" is called | 
					
						
							| 
									
										
										
										
											2019-03-25 17:50:13 -04:00
										 |  |  |     <varname>depsFooBar</varname>, with exceptions for backwards | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |     compatibility that <varname>depsBuildHost</varname> is instead called | 
					
						
							|  |  |  |     <varname>nativeBuildInputs</varname> and <varname>depsHostTarget</varname> | 
					
						
							|  |  |  |     is instead called <varname>buildInputs</varname>. Nixpkgs is now structured | 
					
						
							|  |  |  |     so that each <varname>depsFooBar</varname> is automatically taken from | 
					
						
							|  |  |  |     <varname>pkgsFooBar</varname>. (These <varname>pkgsFooBar</varname>s are | 
					
						
							|  |  |  |     quite new, so there is no special case for | 
					
						
							|  |  |  |     <varname>nativeBuildInputs</varname> and <varname>buildInputs</varname>.) | 
					
						
							|  |  |  |     For example, <varname>pkgsBuildHost.gcc</varname> should be used at | 
					
						
							|  |  |  |     build-time, while <varname>pkgsHostTarget.gcc</varname> should be used at | 
					
						
							|  |  |  |     run-time. | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |    </para> | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  | 
 | 
					
						
							|  |  |  |    <para> | 
					
						
							| 
									
										
										
										
											2019-03-25 17:50:13 -04:00
										 |  |  |     Now, for most of Nixpkgs's history, there were no | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |     <varname>pkgsFooBar</varname> attributes, and most packages have not been | 
					
						
							|  |  |  |     refactored to use it explicitly. Prior to those, there were just | 
					
						
							|  |  |  |     <varname>buildPackages</varname>, <varname>pkgs</varname>, and | 
					
						
							|  |  |  |     <varname>targetPackages</varname>. Those are now redefined as aliases to | 
					
						
							|  |  |  |     <varname>pkgsBuildHost</varname>, <varname>pkgsHostTarget</varname>, and | 
					
						
							| 
									
										
										
										
											2019-03-25 17:50:13 -04:00
										 |  |  |     <varname>pkgsTargetTarget</varname>. It is acceptable, even | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |     recommended, to use them for libraries to show that the host platform is | 
					
						
							|  |  |  |     irrelevant. | 
					
						
							|  |  |  |    </para> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |    <para> | 
					
						
							|  |  |  |     But before that, there was just <varname>pkgs</varname>, even though both | 
					
						
							|  |  |  |     <varname>buildInputs</varname> and <varname>nativeBuildInputs</varname> | 
					
						
							|  |  |  |     existed. [Cross barely worked, and those were implemented with some hacks | 
					
						
							|  |  |  |     on <varname>mkDerivation</varname> to override dependencies.] What this | 
					
						
							|  |  |  |     means is the vast majority of packages do not use any explicit package set | 
					
						
							|  |  |  |     to populate their dependencies, just using whatever | 
					
						
							|  |  |  |     <varname>callPackage</varname> gives them even if they do correctly sort | 
					
						
							|  |  |  |     their dependencies into the multiple lists described above. And indeed, | 
					
						
							|  |  |  |     asking that users both sort their dependencies, <emphasis>and</emphasis> | 
					
						
							|  |  |  |     take them from the right attribute set, is both too onerous and redundant, | 
					
						
							| 
									
										
										
										
											2019-03-25 17:50:13 -04:00
										 |  |  |     so the recommended approach (for now) is to continue just categorizing by | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |     list and not using an explicit package set. | 
					
						
							|  |  |  |    </para> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |    <para> | 
					
						
							| 
									
										
										
										
											2019-03-25 17:50:13 -04:00
										 |  |  |     To make this work, we "splice" together the six | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |     <varname>pkgsFooBar</varname> package sets and have | 
					
						
							|  |  |  |     <varname>callPackage</varname> actually take its arguments from that. This | 
					
						
							|  |  |  |     is currently implemented in <filename>pkgs/top-level/splice.nix</filename>. | 
					
						
							|  |  |  |     <varname>mkDerivation</varname> then, for each dependency attribute, pulls | 
					
						
							|  |  |  |     the right derivation out from the splice. This splicing can be skipped when | 
					
						
							|  |  |  |     not cross-compiling as the package sets are the same, but still is a bit | 
					
						
							|  |  |  |     slow for cross-compiling. We'd like to do something better, but haven't | 
					
						
							|  |  |  |     come up with anything yet. | 
					
						
							|  |  |  |    </para> | 
					
						
							|  |  |  |   </section> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |   <section xml:id="ssec-bootstrapping"> | 
					
						
							|  |  |  |    <title>Bootstrapping</title> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |    <para> | 
					
						
							|  |  |  |     Each of the package sets described above come from a single bootstrapping | 
					
						
							|  |  |  |     stage. While <filename>pkgs/top-level/default.nix</filename>, coordinates | 
					
						
							|  |  |  |     the composition of stages at a high level, | 
					
						
							|  |  |  |     <filename>pkgs/top-level/stage.nix</filename> "ties the knot" (creates the | 
					
						
							|  |  |  |     fixed point) of each stage. The package sets are defined per-stage however, | 
					
						
							|  |  |  |     so they can be thought of as edges between stages (the nodes) in a graph. | 
					
						
							| 
									
										
										
										
											2019-03-25 17:50:13 -04:00
										 |  |  |     Compositions like <literal>pkgsBuildTarget.targetPackages</literal> can be | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |     thought of as paths to this graph. | 
					
						
							|  |  |  |    </para> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |    <para> | 
					
						
							|  |  |  |     While there are many package sets, and thus many edges, the stages can also | 
					
						
							|  |  |  |     be arranged in a linear chain. In other words, many of the edges are | 
					
						
							|  |  |  |     redundant as far as connectivity is concerned. This hinges on the type of | 
					
						
							|  |  |  |     bootstrapping we do. Currently for cross it is: | 
					
						
							|  |  |  |     <orderedlist> | 
					
						
							|  |  |  |      <listitem> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  |        <literal>(native, native, native)</literal> | 
					
						
							|  |  |  |       </para> | 
					
						
							|  |  |  |      </listitem> | 
					
						
							|  |  |  |      <listitem> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  |        <literal>(native, native, foreign)</literal> | 
					
						
							|  |  |  |       </para> | 
					
						
							|  |  |  |      </listitem> | 
					
						
							|  |  |  |      <listitem> | 
					
						
							|  |  |  |       <para> | 
					
						
							|  |  |  |        <literal>(native, foreign, foreign)</literal> | 
					
						
							|  |  |  |       </para> | 
					
						
							|  |  |  |      </listitem> | 
					
						
							|  |  |  |     </orderedlist> | 
					
						
							|  |  |  |     In each stage, <varname>pkgsBuildHost</varname> refers the the previous | 
					
						
							|  |  |  |     stage, <varname>pkgsBuildBuild</varname> refers to the one before that, and | 
					
						
							|  |  |  |     <varname>pkgsHostTarget</varname> refers to the current one, and | 
					
						
							|  |  |  |     <varname>pkgsTargetTarget</varname> refers to the next one. When there is | 
					
						
							|  |  |  |     no previous or next stage, they instead refer to the current stage. Note | 
					
						
							| 
									
										
										
										
											2019-03-25 17:50:13 -04:00
										 |  |  |     how all the invariants regarding the mapping between dependency and depending | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |     packages' build host and target platforms are preserved. | 
					
						
							|  |  |  |     <varname>pkgsBuildTarget</varname> and <varname>pkgsHostHost</varname> are | 
					
						
							|  |  |  |     more complex in that the stage fitting the requirements isn't always a | 
					
						
							|  |  |  |     fixed chain of "prevs" and "nexts" away (modulo the "saturating" | 
					
						
							| 
									
										
										
										
											2019-03-25 17:50:13 -04:00
										 |  |  |     self-references at the ends). We just special case each instead. All the primary | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |     edges are implemented is in <filename>pkgs/stdenv/booter.nix</filename>, | 
					
						
							|  |  |  |     and secondarily aliases in <filename>pkgs/top-level/stage.nix</filename>. | 
					
						
							|  |  |  |    </para> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |    <note> | 
					
						
							|  |  |  |     <para> | 
					
						
							|  |  |  |      Note the native stages are bootstrapped in legacy ways that predate the | 
					
						
							|  |  |  |      current cross implementation. This is why the the bootstrapping stages | 
					
						
							|  |  |  |      leading up to the final stages are ignored inthe previous paragraph. | 
					
						
							|  |  |  |     </para> | 
					
						
							|  |  |  |    </note> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |    <para> | 
					
						
							|  |  |  |     If one looks at the 3 platform triples, one can see that they overlap such | 
					
						
							|  |  |  |     that one could put them together into a chain like: | 
					
						
							|  |  |  | <programlisting> | 
					
						
							|  |  |  | (native, native, native, foreign, foreign) | 
					
						
							|  |  |  | </programlisting> | 
					
						
							|  |  |  |     If one imagines the saturating self references at the end being replaced | 
					
						
							|  |  |  |     with infinite stages, and then overlays those platform triples, one ends up | 
					
						
							|  |  |  |     with the infinite tuple: | 
					
						
							|  |  |  | <programlisting> | 
					
						
							|  |  |  | (native..., native, native, native, foreign, foreign, foreign...) | 
					
						
							|  |  |  | </programlisting> | 
					
						
							|  |  |  |     On can then imagine any sequence of platforms such that there are bootstrap | 
					
						
							|  |  |  |     stages with their 3 platforms determined by "sliding a window" that is the | 
					
						
							|  |  |  |     3 tuple through the sequence. This was the original model for | 
					
						
							|  |  |  |     bootstrapping. Without a target platform (assume a better world where all | 
					
						
							|  |  |  |     compilers are multi-target and all standard libraries are built in their | 
					
						
							|  |  |  |     own derivation), this is sufficient. Conversely if one wishes to cross | 
					
						
							|  |  |  |     compile "faster", with a "Canadian Cross" bootstraping stage where | 
					
						
							|  |  |  |     <literal>build != host != target</literal>, more bootstrapping stages are | 
					
						
							|  |  |  |     needed since no sliding window providess the pesky | 
					
						
							|  |  |  |     <varname>pkgsBuildTarget</varname> package set since it skips the Canadian | 
					
						
							|  |  |  |     cross stage's "host". | 
					
						
							|  |  |  |    </para> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |    <note> | 
					
						
							|  |  |  |     <para> | 
					
						
							|  |  |  |      It is much better to refer to <varname>buildPackages</varname> than | 
					
						
							|  |  |  |      <varname>targetPackages</varname>, or more broadly package sets that do | 
					
						
							|  |  |  |      not mention "target". There are three reasons for this. | 
					
						
							|  |  |  |     </para> | 
					
						
							|  |  |  |     <para> | 
					
						
							|  |  |  |      First, it is because bootstrapping stages do not have a unique | 
					
						
							|  |  |  |      <varname>targetPackages</varname>. For example a <literal>(x86-linux, | 
					
						
							|  |  |  |      x86-linux, arm-linux)</literal> and <literal>(x86-linux, x86-linux, | 
					
						
							|  |  |  |      x86-windows)</literal> package set both have a <literal>(x86-linux, | 
					
						
							|  |  |  |      x86-linux, x86-linux)</literal> package set. Because there is no canonical | 
					
						
							|  |  |  |      <varname>targetPackages</varname> for such a native (<literal>build == | 
					
						
							|  |  |  |      host == target</literal>) package set, we set their | 
					
						
							|  |  |  |      <varname>targetPackages</varname> | 
					
						
							|  |  |  |     </para> | 
					
						
							|  |  |  |     <para> | 
					
						
							|  |  |  |      Second, it is because this is a frequent source of hard-to-follow | 
					
						
							| 
									
										
										
										
											2019-03-25 17:50:13 -04:00
										 |  |  |      "infinite recursions" / cycles. When only package sets that don't mention | 
					
						
							|  |  |  |      target are used, the package set forms a directed acyclic graph. This | 
					
						
							|  |  |  |      means that all cycles that exist are confined to one stage. This means | 
					
						
							|  |  |  |      they are a lot smaller, and easier to follow in the code or a backtrace. It | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |      also means they are present in native and cross builds alike, and so more | 
					
						
							|  |  |  |      likely to be caught by CI and other users. | 
					
						
							|  |  |  |     </para> | 
					
						
							|  |  |  |     <para> | 
					
						
							|  |  |  |      Thirdly, it is because everything target-mentioning only exists to | 
					
						
							|  |  |  |      accommodate compilers with lousy build systems that insist on the compiler | 
					
						
							|  |  |  |      itself and standard library being built together. Of course that is bad | 
					
						
							| 
									
										
										
										
											2019-03-25 17:50:13 -04:00
										 |  |  |      because bigger derivations means longer rebuilds. It is also problematic because | 
					
						
							| 
									
										
										
										
											2019-03-20 18:21:00 -04:00
										 |  |  |      it tends to make the standard libraries less like other libraries than | 
					
						
							|  |  |  |      they could be, complicating code and build systems alike. Because of the | 
					
						
							|  |  |  |      other problems, and because of these innate disadvantages, compilers ought | 
					
						
							|  |  |  |      to be packaged another way where possible. | 
					
						
							|  |  |  |     </para> | 
					
						
							|  |  |  |    </note> | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  |    <note> | 
					
						
							|  |  |  |     <para> | 
					
						
							|  |  |  |      If one explores Nixpkgs, they will see derivations with names like | 
					
						
							|  |  |  |      <literal>gccCross</literal>. Such <literal>*Cross</literal> derivations is | 
					
						
							|  |  |  |      a holdover from before we properly distinguished between the host and | 
					
						
							|  |  |  |      target platforms—the derivation with "Cross" in the name covered the | 
					
						
							|  |  |  |      <literal>build = host != target</literal> case, while the other covered | 
					
						
							|  |  |  |      the <literal>host = target</literal>, with build platform the same or not | 
					
						
							|  |  |  |      based on whether one was using its <literal>.nativeDrv</literal> or | 
					
						
							|  |  |  |      <literal>.crossDrv</literal>. This ugliness will disappear soon. | 
					
						
							|  |  |  |     </para> | 
					
						
							|  |  |  |    </note> | 
					
						
							|  |  |  |   </section> | 
					
						
							| 
									
										
										
										
											2018-05-01 19:54:21 -04:00
										 |  |  |  </section> | 
					
						
							| 
									
										
										
										
											2017-01-22 15:52:35 -05:00
										 |  |  | </chapter> |