Close#15803. This avoids the error:
while setting up the build environment: executing
‘/nix/store/7sb42axk5lrxqz45nldrb2pchlys14s1-bash-4.3-p42/bin/bash’:
Argument list too long
Note: I wanted to make it optional based on buildCommand length,
but that seems pointless as I'm sure it's less performant.
Amended by vcunat:
https://github.com/NixOS/nixpkgs/pull/15803#issuecomment-224841225
This takes another approach at binding FHS directory structure. We
now bind-mount all the root filesystem to directory "/host" in the target tree.
From that we symlink all the directories into the tree if they do not already
exist in FHS structure.
This probably makes `CHROOTENV_EXTRA_BINDS` unnecessary -- its main usecase was
to add bound directories from the host to the sandbox, and we not just symlink
all of them. I plan to get some feedback on its usage and maybe deprecate it.
This also drops old `buildFHSChrootEnv` infrastructure. The main problem with it
is it's very difficult to unmount a recursive-bound directory when mount is not
sandboxed. This problem is a bug even without these changes -- if
you have for example `/home/alice` mounted to somewhere, you wouldn't see
it in `buildFHSChrootEnv` now. With the new directory structure, it's
impossible to use regular bind at all. After some tackling with this I realized
that the fix would be brittle and dangerous (if you don't unmount everything
clearly and proceed to removing the temporary directory, bye-bye fs!). It also
probably doesn't worth it because I haven't heard that someone actually uses it
for a long time, and `buildFHSUserEnv` should cover most cases while being much
more maintainable and safe for the end-user.
1. When multiple versions of the same package are required
$revs is an array.
2. When cargo fetch is run it usually doesn't need a network
connection. But when it does SSL_CERT_FILE isn't set.
find fails when called with an inexistent search path.
That situation may arise when the output is created after by a postFixup hook.
vcunat amended the PR by clarifying one more `return` to `return 0`.
Checkinstall had two problems:
1. when it was called without a version (e.g. with a derivation created
by fetchFromGitHub) it would use `src` as debian version, which caused
dpkg to fail
2. when dpkg failed, it would invoke the pager with the log, which hangs
the build
So now
1. the default version is the dummy `0.0.0`
2. the used pager is `cat`
Source of this change goes back to 2009 and original version of
fetchgit at 205fb0c87e.
The nondeterminism is really caused by changing .git so leave other
files alone as they might be interesting.
Note: this causes a hash mismatch with Hydra's version of Git Plugin
which we should fix to comply.
Move all the dependencies to their own derivation, so that we don't publish all
of them if the wrapper is installed in a profile.
The previous solution just moved them to a custom directory to avoid conflicts,
this refactors that and completely hides them, while preserving the desired
improvement of adding only one directory to each of the emacs search paths
The chroot caps restriction disallows chroot'ed processes from running
any command that requires `CAP_SYS_ADMIN`, breaking `nixos-rebuild`. See
e.g., https://github.com/NixOS/nixpkgs/issues/15293
This significantly weakens chroot protections, but to break
nixos-rebuild out of the box is too severe.
There are no users of it in main tree and recent merge
of multiple outputs branch makes it obsolete for private trees
too.
At the time hook was created, recently merged multiple output
branch was relying on passing flags to autotools to split
outputs, which obviously wasn't working for other build systems
Scatter output was taking different approach where files were
moved out from a build tree based on known paths, which is more
or less what current multiple-outputs.sh hook is able to do too.
A nix specific set of tools for converting icon files
that are not in a freedesktop ready format.
I plan on using these tools for both `keepass` and
`retroarch` packages. It may benifit many other packages.
For every *.{exe,dll} in $output/bin/ we try to find all (potential)
transitive dependencies and symlink those DLLs into $output/bin
so they are found on invocation.
(DLLs are first searched in the directory of the running exe file.)
The links are relative, so relocating whole /nix/store won't break them.
The hook is activated on cygwin and when cross-compiling to mingw.
- the default --docdir is typically DATAROOTDIR/doc/pkgName
- I saw no other way than to employ some magic to guess this `pkgName`
- user can override it by setting $shareDocName
I supplied meta.outputsToInstall automatically in all
mkDerivation products, but some packages still don't use it.
The reported case: jekyll -> bundlerEnv -> buildEnv -> runCommand.
The /nix path in 4d200538 of the layer tar didn't exist for some
packages, such as cacert. This is because cacert just creates an /etc
directory and doesn't depend on any other /nix paths. If we tried
putting this directory in the tar and using overlayfs with it, we'd get
"Invalid argument" when trying to remove the directory.
We now check whether the closure is non-empty before telling tar to
store the /nix directory.
Fixes#14710.
There were two sources of non-determinisim coming into the images. The
first was tar mtimes, the second was pigz/gzip times.
An example image now passes with the --check flag.
I think the intention of this functionality was to provide a simple
alternative to the "runAsRoot" and "contents" attributes.
The implementation caused very slow builds of Docker images. Almost all
of the build time was spent in IO for tar, due to tarballs being
created, immediately extracted, then recreated. I had 30 minute builds
on some of my images which are now down to less than 2 minutes. A couple
of other users on #nix IRC have observed similar improvements.
The implementation also mutated the produced Docker layers without
changing their hashes. Using non-empty tarballs would produce images
which got cached incorrectly in Docker.
I have a commit which just fixes the performance problem but I opted to
completely remove the tarball feature after I found out that it didn't
correctly implement the Docker Image Specification due to the broken
hashing.
This is to avoid unwanted side effects when installing a wrapped emacs in the environment:
* All executables in the dependencies become available in the user environment
* All site-lisp binaries in the dependencies become accessible to unwrapped emacs
Also, both bin and site-lisp would generate conflicts so installing a wrapped emacs becomes really cumbersome
1. Update bower2nix version and add new/updated dependencies into
node-packages-generated.nix. This was done manually, with npm2nix
generating the initial set of derivations. In future, it would be
nice to have an automatic process (see #10358, #9332).
2. Add an override to nodePackages.bower2nix wrapping the commands so
that git is on the PATH.
3. Update fetchbower to support new command-line options of bower2nix,
and to allow github URL tag versions.
Previously, nix-prefetch-git would report the same JSON whether submodules were being fetched or not; with this change, the --fetch-submodules option will cause the JSON output to include "fetchSubmodules": true, so that fetchgit (builtins.fromJSON (builtins.readFile ./path/to/output.json)) will work.
The following parameters are now available:
* hardeningDisable
To disable specific hardening flags
* hardeningEnable
To enable specific hardening flags
Only the cc-wrapper supports this right now, but these may be reused by
other wrappers, builders or setup hooks.
cc-wrapper supports the following flags:
* fortify
* stackprotector
* pie (disabled by default)
* pic
* strictoverflow
* format
* relro
* bindnow
Some recent perl version introduced "keys" to return the keys
in random order. As some of the packages are solved by "provides" and
based on the order, this randomness affects what packages get into the
closure.
This problem may be in other nix perl scripts.
The importance of glibc makes it worthwhile to provide debug
symbols. However, this revealed an issue with separateDebugInfo: it
was indiscriminately adding --build-id to all ld invocations, while in
fact it should only do that for final links. Glibc also uses non-final
("relocatable") links, leading to subsequent failure to apply a build
ID ("Cannot create .note.gnu.build-id section, --build-id
ignored"). So now ld-wrapper.sh only passes --build-id for final
links.
Otherwise, when building glibc and other packages, the "strip" from
bootstrapTools is used, which doesn't recognise some tags produced by
the newer "ld" from binutils.
The two lines I removed technically assert the exact same thing, since `!a -> b`
is equivalent to `a || b`. So, I replaced the two lines with the more symmetric
form to make it clearer.
This commit fixes#6651.
Before this change the `nix-prefetch-git` script would use a different store
name than nix's `fetchgit` function. Because of that it was not possible to
use `nix-prefetch-git` as a way to pre-populate the store (for example when
the user it using private git dependencies that needs access to the ssh agent)
In some cases the $sourceRoot is missing. Skip the hook instead
of showing the following cryptic error:
find: cannot search `': No such file or directory
/nix/store/0p1afvl8jcpi6dvsq2n58i90w9c59vz1-set-source-date-epoch-to-latest.sh: line 12: [: : integer expression expected
vcunat removed the warning; the hook will just skip silently in these cases.
Perhaps someone can improve on it some time.
Regression introduced by 4529ed1259.
I've missed this in #5096, not because of a messed up rebase as I have
guessed from a comment on #12635 but missed this in the first place.
The testing I did while working on the pull request weren't exhaustive
enough to cover this, because I haven't tested with packages that use
the propagatedUserEnvPkgs attribute.
In order to make the test a bit more exhaustive this time, let's test it
using:
nix-build -E 'with import ./. {}; buildEnv {
name = "testenv";
paths = [
pkgs.hello pkgs.binutils pkgs.libsoup pkgs.gnome3.yelp
pkgs.gnome3.totem
];
}'
And with this commit the errors no longer show up and the environment is
built correctly.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Fixes: #12635
- Now `pkg.outputUnspecified = true` but this attribute is missing in
every output, so we can recognize whether the user chose or not.
If (s)he didn't choose, we put `pkg.bin or pkg.out or pkg` into
`systemPackages`.
- `outputsToLink` is replaced by `extraOutputsToLink`.
We add extra outputs *regardless* of whether the user chose anything.
It's mainly meant for outputs with docs and debug symbols.
- Note that as a result, some libraries will disappear from system path.
* authorization token is optional
* registry url is taken from X-Docker-Endpoints header
* pull.sh correctly resumes partial layer downloads
* detjson.py does not fail on missing keys
The comment related to the `deepClone` and `no-deepClone` options was
misleading as these options have no relation with submodules, but on the
the depth in `git clone --depth n`.
Login mode can cause hidden problems, e.g. #12406. Generally we don't want
to read user's .bash_profile when we don't start an interactive shell inside
a chroot.
These environment variables allow using fetchgit with git:// URLs using
the SOCKS proxy technique described in 'Using Git with a SOCKS proxy':
http://www.patthoyts.tk/blog/using-git-with-socks-proxy.html
Briefly, GIT_PROXY_COMMAND is set to a script which invokes connect[1],
which reads SOCKS_PROXY, which might be pointing to a local instance of
'ssh -D'.
[1] pkgs/tools/networking/connect
The ld-wrapper.sh script calls `readlink` in some circumstances. We need
to ensure that this is the `readlink` from the `coreutils` package so
that flag support is as expected.
This is accomplished by explicitly setting PATH at the top of each shell
script.
Without doing this, the following happens with a trivial `main.c`:
```
nix-env -f "<nixpkgs>" -iA pkgs.clang
$ clang main.c -L /nix/../nix/store/2ankvagznq062x1gifpxwkk7fp3xwy63-xnu-2422.115.4/Library -o a.out
readlink: illegal option -- f
usage: readlink [-n] [file ...]
```
The key element is the `..` in the path supplied to the linker via a
`-L` flag. With this patch, the above invocation works correctly on
darwin, whose native `/usr/bin/readlink` does not support the `-f` flag.
The explicit path also ensures that the `grep` called by `cc-wrapper.sh`
is the one from Nix.
Fixes#6447
Building packages requires package-build.el from Melpa, but installing
packages only requires package.el. Packages from ELPA are already built,
so there is no need to involve package-build.el.
When building a package from a Melpa recipe file, get the Emacs package
name from the recipe. Nix is more restrictive about packages names than
Emacs, so the Nix name for a package is sometimes different.
Checking file contents is redundant in this case, because we will go
ahead anyway, regardless of whether the content is the same.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Originally wanted to include ignoreCollisions in cups-progs, but I think
it's better if we use ignoreCollisions only if there are _real_
collisions between files with different contents.
Of course, we also check whether the file permissions match, so you get
a collision if contents are the same but the permissions are different.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
For instance, a binary like libfoo.so will cause a symlink
lib/debug/libfoo.so.debug -> .build-id/<build-ID>.debug to be
created. This is primarily useful for use with eu-addr2line, if you
know the name of a binary and the relative address, but not the build
ID.
- fix in silencing some moveToOutput messages
- allow removing (developer) documentation even without defining outputs
(note: some paths are auto-removed by default, e.g. gtk-doc and man3)
Unfortunately, yesterday Nix got reverted to a version with broken
passAsFile implementation on some Hydra machines, so we have corrupted
files again. (E.g. http://hydra.nixos.org/build/29777678.) Forcing
another gratuitous rebuild to get rid of them.
(cherry picked from commit 75974d9220b8397c736ada76fb24eb934fa62f6c)
Also fix the hash in goPackages.inflect, the only user of the fetcher ATM.
Closes#12002 (different `inflect` fix), fixes#12012.
Using fetchzip-derived functions is likely more efficient than fetchhg,
and it's lighter on dependencies (hash is the same as with fetchhg in this case).
fetchbzr always uses the derivation name `bzr-export`. nix-prefetch-bzr
should use the same name for its output. This avoids duplicate downloads
and problems with forbidden characters in bazaar repository names.
Fixes#10819. emacsWithPackages will know its own package set. This
requires it to be in a package set, rather than at the top level, so it
lives in emacsPackagesNg.
This seems to be the root cause of the random page allocation failures
and @wizeman did a very good job on not only finding the root problem
but also giving a detailed explanation of it in #10828.
Here is an excerpt:
The problem here is that the kernel is trying to allocate a contiguous
section of 2^7=128 pages, which is 512 KB. This is way too much:
kernel pages tend to get fragmented over time and kernel developers
often go to great lengths to try allocating at most only 1 contiguous
page at a time whenever they can.
From the error message, it looks like the culprit is unionfs, but this
is misleading: unionfs is the name of the userspace process that was
running when the system ran out of memory, but it wasn't unionfs who
was allocating the memory: it was the kernel; specifically it was the
v9fs_dir_readdir_dotl() function, which is the code for handling the
readdir() function in the 9p filesystem (the filesystem that is used
to share a directory structure between a qemu host and its VM).
If you look at the code, here's what it's doing at the moment it tries
to allocate memory:
buflen = fid->clnt->msize - P9_IOHDRSZ;
rdir = v9fs_alloc_rdir_buf(file, buflen);
If you look into v9fs_alloc_rdir_buf(), you will see that it will try
to allocate a contiguous buffer of memory (using kzalloc(), which is a
wrapper around kmalloc()) of size buflen + 8 bytes or so.
So in reality, this code actually allocates a buffer of size
proportional to fid->clnt->msize. What is this msize? If you follow
the definition of the structures, you will see that it's the
negotiated buffer transfer size between 9p client and 9p server. On
the client side, it can be controlled with the msize mount option.
What this all means is that, the reason for running out of memory is
that the code (which we can't easily change) tries to allocate a
contiguous buffer of size more or less equal to "negotiated 9p
protocol buffer size", which seems to be way too big (in our NixOS
tests, at least).
After that initial finding, @lethalman tested the gnome3 gdm test
without setting the msize parameter at all and it seems to have resolved
the problem.
The reason why I'm committing this without testing against all of the
NixOS VM test is basically that I think we can only go better but not
worse than the current state.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Previously is was assumed that bash was in the path when calling the
environment setup script. This changes all of the references of bash to
be absolute paths so that the user doesn't have to worry about the
environment they call it with.
Otherwise, if the upstream mirror changes (rather than deletes) a
file, then tarballs.nixos.org won't be used even if it has a copy of
the original file, and so we'll get a hash mismatch.
Emacs packages are commonly distributed as single .el files. This
unpackCmd handles them correctly and sets up sourceRoot. Other sources
are treated in the default manner.
If "fetcher" is a string, then Nix will execute it with bash already, so
the additional bash argument in that string was redundant and apparently
causes trouble on non-Linux platforms.
Hopefully fixes https://github.com/NixOS/nixpkgs/issues/11496.
The list we had before contained a lot of junk, i.e. sites that were no
longer online or no longer in sync. The new list of sites comes from
https://gnupg.org/download/index.html.
The most complex problems were from dealing with switches reverted in
the meantime (gcc5, gmp6, ncurses6).
It's likely that darwin is (still) broken nontrivially.
It turns out that cargo implicitly depends on rustc at runtime: even
`cargo help` will fail if rustc is not in the PATH.
This means that we need to wrap the cargo binary to add rustc to PATH.
However, I have opted into doing something slightly unusual: instead of
tying down a specific cargo to use a specific rustc (i.e., wrap cargo so
that "${rustc}/bin" is prefixed into PATH), instead I'm adding the rustc
used to build cargo as a fallback rust compiler (i.e., wrap cargo so
that "${rustc}/bin" is suffixed into PATH). This means that cargo will
prefer to use a rust compiler that is in the default path, but fallback
into the one used to build cargo only if there wasn't any rust compiler
in the default path.
The reason I'm doing this is that otherwise it could cause unexpected
effects. For example, if you had a build environment with the
rustcMaster and cargo derivations, you would expect cargo to use
rustcMaster to compile your project (since rustcMaster would be the only
compiler available in $PATH), but this wouldn't happen if we tied down
cargo to use the rustc that was used to compile it (because the default
cargo derivation gets compiled with the stable rust compiler).
That said, I have slightly modified makeRustPlatform so that a rust
platform will always use the rust compiler that was used to build cargo,
because this prevents mistakenly depending on two different versions of
the rust compiler (stable and unstable) in the same rust platform,
something which is usually undesirable.
Fixes#11053
The fetch-cargo-deps script is written in bash syntax, but it
erroneously ran under the /bin/sh interpreter.
This wasn't noticed because /bin/sh is actually bash in NixOS, but on
some other systems this is not true.
... because cc-wrapper is meant to propagate man pages into user envs,
and info pages are rather large.
Also replace the duplicate g++ and gcc man1 pages by a symlink.
Note: -B argument seems more like for gcc's main output,
though it's used in a bit strange way here.
(Upstream default is /usr/lib/gcc/ which we don't move.)
Now any developer docs are removed by default, unless "docdev"
is in $outputs or $outputDocdev is defined.
Currently devdoc consists of just man3 and gtk-doc.
While debugging an issue with running NixOps tests, I found out that the
output from debClosureGenerator is not deterministic.
The reason behind this is the way how Provides and Replaces fields are
handled. I haven't yet found out what's the exact issue, but so far
packages "Provides" are more or less picked at random.
So, running the NixOps Hetzner tests we get either mawk, original-awk or
gawk altering on every invocation.
While for the test it isn't poisionous whether wi have mawk or gawk,
having original-awk certainly is, because live-build only works with
mawk or gawk.
The best solution would obviously be to make debClosureGenerator
deterministic, but in the case of "Provides: awk", we can safely pick
mawk by default, because the latter has a "Priority: required" in its
package description.
This also has the advantage that we can safely cherry-pick this to
release-15.09 because it's very unlikely that we'll break the
debClosureGenerator by adding a dependency to commonDebPackages.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
This should avoid accidential expansion of variables, i.e. in
"export PATH=/some/path:$PATH"
$PATH would have been expanded in the environment builder!
'[[ ! -v "$propagatedOutputs" ]]' is incorrect and always evaluates to
true. The correct form using double brackets would be
'[[ ! -v propagatedOutputs ]]', but I strongly dislike '[[ ]]' due to
the totally different quoting rules compared to everything else in bash.
For practical purposes, here are the changes in behavior:
- When fetching from a subdirectory of a repo, do not rebuild because of
changes elsewhere in the repo
- Fetch (not-ignored) untracked files too
It does this by letting git hash and export the directory in question,
which I believes makes for a cleaner implementation than the ad-hoc copying
and hashing that was there before.
Close#9790.
This fixes checkouting for a nasty combination:
1. To be checkouted is a revision which corresponds to tag in a form "<tag>^{}".
2. This revision is not fetched by default.
You can now pass
separateDebugInfo = true;
to mkDerivation. This causes debug info to be separated from ELF
binaries and stored in the "debug" output. The advantage is that it
enables installing lean binaries, while still having the ability to
make sense of core dumps, etc.
Fixes#9044, close#9667. Thanks to @taku0 for suggesting this solution.
Now we have no modes starting with `/` or `+`.
Rewrite the `-perm` parameters of find:
- completely safe: rewrite `/0100` and `+100` to `-0100`,
- slightly semantics-changing: rewrite `+111` to `-0100`.
I cross-verified the `find` manual pages for Linux, Darwin, FreeBSD.
Upstream likes to move "old" releases to an archive mirror as soon as a
new one is released. This is now handled for free by mirrors.nix.
(No idea why cs.utah.edu was used to begin with; it's now added to
mirrors.nix. Note that it doesn't support SSL, but that applies to
several others so I don't see the harm.)
By default `makeWrapper` will not set argv[0] (this is a reversion to
the old default behavior). Based on the breakage we have seen from
changing the default, this is what most people want. The `wrapProgram`
function will send `--argv0 '"$0"'` to `makeWrapper`, i.e. it will
continue to pass-through the argv[0] that the wrapper is called with.
Also, in some cases, the result of fetchBower is different depending on the
value of $out. For now, it seems that it works best if using a local output
directory before copying to $out.
(cherry picked from commit aa4c6b027163abe0891f9ad438899f9679298a64)
Comes in handy if we want to make additional modificiations to the
output file. While I wasn't sure whether to invoke the passed postFetch
directly before the patch or afterwards, I thought it would be better
afterwards because "postFetch of fetchpatch" at least to my intuition
would sound that after whatever "fetchpatch" does - it comes afterwards.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
- Update the instructions for re-generating each of the package set files.
- Provide test-evaluation.nix expression to verify that the package sets evaluates.
- Update list of known broken packages.
- fetchNuGet can fetch binaries from nuget servers
- buildDotnetPackage can build .NET packages using mono/xbuild
- Places nuget & paket as they would clash with nix
- Patch project files because F# targets are expected to be found in
the mono directory (and we know that's not going to happen on nix)
- Find DLLs that were copied from buildInputs and replace by symlink
for sharing
- Export produced DLL via the pkg-config mechanism
- Create wrappers for produced EXEs
- Repackaged this new infrastructure: keepass, monodevelop
- Newly packaged: ExtCore, UnionArgParser, FSharp.Data, Paket, and a
bunch more..
This is a combination of 73 commits.
The point of this is to be able to do `meta.homepage = src.meta.homepage;`
instead of the usual copy-paste for the packages that are hosted
on these hosting services.
Instead it is provided to the user who can choose whether or not
to include it in the final derivati. Example of including would
be:
```nix
callPackage ... (self: { inherit (self.extras) extraThing; })
```
These extras are also available downstream without being built by
default. This is achieved with `passthru`.
- Only they are added to the optional build path (share/agda)
- Only they are are passed as an include dir (share/agda)
- Only they are propigatedBuildInputs
Instead, discover it automatically when building the package.
This makes `buildRustPackage` more future-proof with respect to changes
in how `cargo` generates the hash.
Also, it fixes broken builds in i686 because apparently, cargo generates
a different registry index hash in this architecture (compared to
x86-64).
This is unused, future users can just use override `buildFlags`
and extend/replace as needed. `includeDirs` is provided for this
purpose.
We should add `dirOf self.everythingFile` rather than `.`, but
`dirOf` breaks on relative paths so that is not an option.
This reverts d927da8dae. Having a copy
of gcc-wrapper/setup-hook.sh is bad for maintainability - it had
already started to diverge. Also, gccStdInc gave a nix-env conflict
with the standard gcc. And it wasn't actually used in Nixpkgs.
Instead, if you really need to change "-isystem" to "-I", you can now
set ccIncludeFlag to "-I".
This makes buildRustPackage portable to non-Linux platforms.
Additionally, now we also save the `Cargo.lock` file into the fetch output, so
that we don't have to run $cargoUpdateHook again just before building.
... in a more generic way.
With this commit, if you need to patch a registry package to make it
work with Nix, you just need to add a script to patch-registry-deps
in the same style as the `pkg-config` script.
Instead, move that code into buildRustPackage.
The setup hook was only doing part of the work anyway, and having it in
a separate place was obscuring what was really going on.
Emacs will call package-initialize itself, if required, or the user will
call it in their initialization file. There is no reason to call it in
the wrapper and doing so only increases start-up time.
It turns out that `cargo`, with respect to registry dependencies, was
ignoring the package versions locked in `Cargo.lock` because we changed
the registry index URL.
Therefore, every time `rustRegistry` would be updated, we'd always try
to use the latest version available for every dependency and as a result
the deps' SHA256 hashes would almost always have to be changed.
To fix this, now we do a string substitution in `Cargo.lock` of the
`crates.io` registry URL with our URL. This should be safe because our
registry is just a copy of the `crates.io` registry at a certain point
in time.
Since now we don't always use the latest version of every dependency,
the build of `cargo` actually started to fail because two of the
dependencies specified in its `Cargo.lock` file have build failures.
To fix the latter problem, I've added a `cargoUpdateHook` variable that
gets ran both when fetching dependencies and just before building the
program. The purpose of `cargoUpdateHook` is to do any ad-hoc updating
of dependencies necessary to get the package to build. The use of the
'--precise' flag is needed so that cargo doesn't try to fetch an even
newer version whenever `rustRegistry` is updated (and therefore have to
change depsSha256 as a consequence).
This is useful when `leaveDotGit = true` and some other derivation
expects some branch name to exist.
Previously, `nix-prefetch-git` always created a branch with a
hard-coded name (`fetchgit`).
Now development stuff is propagated from the first output,
and userEnvPkgs from the one with binaries.
Also don't move *.la files (yet). It causes problems, and they're small.
- there were many easy merge conflicts
- cc-wrapper needed nontrivial changes
Many other problems might've been created by interaction of the branches,
but stdenv and a few other packages build fine now.
This patch resolves https://github.com/NixOS/nixpkgs/issues/6395. Deep
cloning is useful in combination with 'leaveDotGit' for builds that want
to run "git describe" to obtain a proper version string, etc., like the
'haskellngPackages.cabal2nix' package does.
This simplifies melpa builder by merging with it my old emacs modes builder,
adds better instructions and support for overrides in emacs-packages.nix,
and renames some emacs-related stuff in all-packages.nix for sanity reasons.
I declare this backwards compatible since direct uses of emacsPackages in
configuration.nix are very unlikely.
- Add a conditional flag for the c++ std lib
- Build binaries that get linked by our own dyld (someday)
- Automatically add framework directories in the setup hook
- nix-template-rpm can now split the generated templates into
a static part that goes into the nixpkgs tree
a dynamic part that can be updated easily to track the rpm spec files
- add lookup mechanism for package names and package paths
- add mechanism to update existing nix-expression with new download files
These packages, and maybe some more include unix.h for some reason.
Creating that file makes them build, and in the case of xdebug also
appear to work.
We now have an alternate setup hook for gcc-wrapper that uses -I to add
include paths rather than -isystem. The latter flag can change the
search order specified by the build system. For KDE 5 packages, we don't
want that!
The default setup-hook for gcc-wrapper adds include directories with
-isystem, which upsets the order -I flags are processed. This adds an
alternative setup-hook that only uses -I flags. The build system's
ordering of -I flags is then respected. This is important when different
packages provide includes with the same name, such as building packages
that depend on Qt4 and Qt5.
This resolves a regression introduced in fc01353703, where providing a
name without a proper extension breaks existing uses of fetchzip (they
now fail to unpack). Of particular note, that commit broke all uses of
fetchFromGitHub because it uses a name like so: "${repo}-${rev}-src"
Fixes#5954
This fixes:
* Passing stripRoot.
* Archives containing a single file.
* Archives where the root folder has the same name as one of it's children.
Fixes#5851
This option tells the kernel to ignore plug-in events of USB devices. Useful to
protect against attacks with malicious hardware. Currently disabled by default,
though.
The name detection didn't work for e.g. http://git.suckless.org/sinit/.
I tested the tarball builds now.
@shlevy claimed nixpkgs requires nix-1.8 features anyway,
so the additional check with message were superfluous.
Problems are worked around by using older gcc wrapper for gnat bootstrap.
(from ca441636f1^)
I tried nicer solutions first, but I was unable to fix all problems for hours.
Especially new users could be confused by this, so we're now marking
services.virtualbox.enable as obsolete and defaulting to
services.virtualboxGuest.enable instead. I believe this now makes it
clear, that this option is for guest additions only.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
This is because libxml/libxslt are not the only implementations that
respect $XML_CATALOG_FILES.
Also, look in share/xml for catalogs (in addition to the
now-deprecated xml/dtd and xml/xsl).
Multi-threaded builds exacerbate the non-determinism in ghc package ids, which
is a serious problem for libraries. Packages that define only executables,
however, should be safe to build with parallelism enabled.
Packages that don't have a Setup.hs file get to use a default version that
lives in the Nix store. By default ghc tries to put the Setup.o and Setup.hi
files in the same directory as the source file, which isn't writable. This
leads to build errors [1]. Thus, we re-direct those paths to a build-local
writable location: $TMPDIR.
Arguably, we could also use "." or copy the /nix/store/deadbeef-Setup.hs file
into the local source directory before compiling, which would work fine, too.
[1] https://github.com/NixOS/nixpkgs/issues/4851
Without this, the generated pack files are non-deterministic.
I didn't notice this issue in my earlier testing, because my test repo
had too few commits for the thread scheduling to take effect. (Test repo
had about 10 commits.)
Add more files to the delete list:
* .git/FETCH_HEAD
* .git/ORIG_HEAD
* .git/refs/remotes/origin/HEAD
* .git/config
Further, remove all remote branches, remove tags not reachable from the
given 'rev', do a full repack and then garbage collect unreferenced
objects.
According to my testing, the result is fully deterministic. As in "any
change done to the upstream repo, ahead of 'rev', will not affect the
hash of the resulting 'clone'". Even changing the clone URL will not
change the output hash, because .git/config is removed.
A new version of git can of course change store format, but that's
unavoidable.
For big repositories, the repack operation may be a bit heavy. But as
far as I can see there is no cheaper way to determinism.
Hydra generates a GHC closure for Darwin that for no apparent reason
contains an ancient, broken Haddock binary -- probably because of an
impurity in the build system. That bug makes those GHC binaries
unusable: <https://github.com/NixOS/nixpkgs/issues/2689>.
This likely exacerbates the non-determinism in ghc package ids, so until
that is fixed let's live with the slow builds.
This reverts commit 817c0e4144.
This should fix the OpenJDK build, which was failing because paxctl is
in sbin and therefore not automatically added to $PATH.
http://hydra.nixos.org/build/15658346
This patch makes two changes.
(1) It memoizes the computation of dependsOnOld.
(2) It replaces rewrittenDerivations with a similar memoized table rewriteMemo.
This prevents the entire tree of run-time dependencies from being traversed and instead only traverses the graph of run-time dependencies.
In the case of deep dependency changes (such as changing one's bash version for an entire NixOS system) this can lead to an exponential speedup in processing time
because shared dependencies are no longer traversed multiple times.
This patch isn't quite derivation-per-derivation equivalent to the original computation.
There are two immaterial differences.
(1) The previous version would always call upon sed to replace oldDependency with newDependency even when the store object being updated doesn't directly depend on
oldDependency.
The new version only replaceds oldDependency with newDependency when the store object being updated actually directly depends on oldDependency (which means there is
actually a hash to replace).
(2) The previous version would list the old store object as a source input of the new store object, *except* for the root derivation being updated. Because the
root derivation being updated has its actual derivation avaiable the previous verions would make the updated root derivation depend on the old derivation as a
derivation input instead of a source input.
The new version always lists the old store object as a source input, including the root derivation.
We still need this because some clang-based packages depend on
it. (The sysroot filtering was originally done by clang-wrapper's
ld-wrapper, but we merged the ld-wrappers in
a4f9b9c8b5ec9ef106671ffdf93e0059835d0ec1.)
http://hydra.nixos.org/build/13906922
Fixes a regression on OS X introduced by f83af95.
Don't use --tmpdir for mktemp, because that flag doesn't exist on OS X.
However, using -t is deprecated in GNU coreutils, so as suggested by
@ip1981 we're now using parameter expansion on ${TMPDIR:-/tmp} to
provide /tmp as a fallback if TMPDIR is not set and use it instead.
Also use this approach for nix-prefetch-cvs now in order to stay
consistent.
Reported-by: Vladimir Kirillov <proger@wilab.org.ua>
Tested-by: Igor Pashev <pashev.igor@gmail.com>
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Instead of relying on $$ to not collide with an existing path.
Quoting the Bash manual about $$:
> Expands to the process ID of the shell. In a () subshell, it expands
> to the process ID of the current shell, not the subshell.
So, this is different from $BASHPID:
> Expands to the process ID of the current bash process. This differs
> from $$ under certain circumstances, such as subshells that do not
> require bash to be re-initialized.
But even $BASHPID is prone to race conditions if the process IDs wrap
around, so to be on the safe side, we're using mktemp here.
Closes#3784.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Now it should contain *all* information from stdenv/setup.sh of
the original mutiple-output branch.
However, the configurability of the output paths is much greater.
For now, we don't NATIVE_SYSTEM_HEADER_DIR because it breaks the
build. However, it points to Glibc in the Nix store (not /usr/include)
so it's kind of okay.
Now gcc is just another build input, making it possible in the future
to have a stdenv that doesn't depend on a C compiler. This is very
useful on NixOS, since it would allow trivial builders like
writeTextFile to work without pulling in the C compiler.
If $src refers to a directory, then always copy it. Previously, we
checked the extension first, so if the directory had an extension like
.tar, unpackPhase would fail.
There was a few files containing timestamp, so we now remove them.
It shouldn't be a problem for logs. However, index might be. Anyway,
that's better than nothing.
Having a separate clang-wrapper is really unfortunate because it
means that we'll forever forget to apply changes to both (e.g.
commit 289895fe2c). This commit
gets rid of the redundant copies of ld-wrapper.sh and utils.sh.
Somewhere the no-sys-dirs.patch got disabled, so gcc was looking in
/usr/local/include and /usr/lib. Since I can't fix the patch easily,
I've borrowed the --sysroot trick from clang-wrapper. This causes
builtin paths to be prefixed with /var/empty
(e.g. /var/empty/usr/lib), which don't exist.
This updates the new stable kernel to 3.14, and the new testing kernel
to 3.15.
This also removes the vserver kernel, since it's probably not nearly as
used.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
If the user explictly gives a ref such as "refs/heads/master", `git
rev-parse` failed because we only checked out the `fetchgit`
branch. Now, we also try `git rev-parse fetchgit` if the first call
fails, which fixes the issue.
HipChat (or rather its copy of Qt) expects to find keyboard data in
/usr/share/X11/xkb. So use a LD_PRELOAD library to intercept and
rewrite the Glibc calls that access those paths. We've been doing the
same thing with packages like Spotify, but now this functionality has
been abstracted into a reusable library, libredirect.so. It uses an
environment variable $NIX_REDIRECTS containing a colon-separated list
of path prefixes to be rewritten, e.g. "/foo=bar:/xyzzy=/fnord".
This now provides a handful of different grsecurity kernels for slightly
different 'flavors' of packages. This doesn't change the grsecurity
module to use them just yet, however.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
fetchpatch is fetchurl that determinizes the patch.
Some parts of generated patches change from time to time, e.g. see #1983 and
http://comments.gmane.org/gmane.linux.distributions.nixos/12815
Using fetchpatch should prevent the hash from changing.
Conflicts (auto-solved):
pkgs/development/libraries/haskell/gitit/default.nix
1) Packages formerly called haskell-haskell-platform-ghcXYZ-VVVV.X.Y.Z are
now called haskell-platform-VVVV.X.Y.Z. The latest version can be
installed by running "nix-env -i haskell-platform".
2) The attributes haskellPackages_ghcXYZ.haskellPlatform no longer exist.
Instead, we have attributes like haskellPlatformPackages."2012_4_0_0".
(The last numeric bit must be quoted when used in a Nix file, but not on
the command line to nix-env, nix-build, etc.) The latest Platform has a
top-level alias called simply haskellPlatform.
3) The haskellPackages_ghcXYZ package sets offer the latest version of every
library that GHC x.y.z can compile. For example, if 2.7 is the latest
version of QuickCheck and if GHC 7.0.4 can compile that version, then
haskellPackages_ghc704.QuickCheck refers to version 2.7.
4) All intermediate GHC releases were dropped from all-packages.nix to
simplify our configuration. What remains is a haskellPackages_ghcXYZ set
for the latest version of every major release branch, i.e. GHC 6.10.4,
6.12.3, 7.0.4, 7.2.2, 7.4.2, 7.6.3, 7.8.2, and 7.9.x (HEAD snapshot).
5) The ghcXYZPrefs functions in haskell-defaults.nix now inherit overrides
from newer to older compilers, i.e. an override configured for GHC 7.0.4
will automatically apply to GHC 6.12.3 and 6.10.4, too. This change has
reduced the redundancy in those configuration functions. The downside is
that overriding an attribute for only one particular GHC version has become
more difficult. In practice, this case doesn't occur much, though.
6) The 'cabal' builder has a brand-new argument called 'extension'. That
function is "self : super : {}" by default and users can override it to
mess with the attribute set passed to cabal.mkDerivation. An example use
would be the definition of darcs in all-packages.nix:
| darcs = haskellPackages.darcs.override {
| cabal = haskellPackages.cabal.override {
| extension = self : super : {
| isLibrary = false;
| configureFlags = "-f-library " + super.configureFlags or "";
| };
| };
| };
In this case, extension disables building the library part of the package
to give us an executable-only version that has no dependencies on GHC or
any other Haskell packages.
The 'self' argument refers to the final version of the attribute set and
'super' refers to the original attribute set.
Note that ...
- Haskell Platform packages always provide the Haddock binary that came with
the compiler.
- Haskell Platform 2009.2.0.2 is broken because of build failures in cgi and
cabal-install.
- Haskell Platform 2010.1.0.0 is broken becasue of build failures in cgi.
This function downloads and unpacks a file in one fixed-output
derivation. This is primarily useful for dynamically generated zip
files, such as GitHub's /archive URLs, where the unpacked content of
the zip file doesn't change, but the zip file itself may (e.g. due to
minor changes in the compression algorithm, or changes in timestamps).
Fetchzip is implemented by extending fetchurl with a "postFetch" hook
that is executed after the file has been downloaded. This hook can
thus perform arbitrary checks or transformations on the downloaded
file.
This allows fonts to be installed from anywhere in an unzipped file
rather than having to cd deep into the directory and come back out in
order for e.g. `forceCopy` to work correctly.
This ensures that the intermediate machine is shut down only after the
migration has finished writing the memory dump to disk, to ensure we
don't end up with empty state files depending on how fast the migration
finished before we actually shut down the VM.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
This ensures that the builder isn't waiting forever if the Windows VM
drops dead while we're waiting for the controller VM to signal that a
particular command has been executed on the Windows VM. It won't ever
happen in such cases so it doesn't make sense to wait for the timeout.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
This hook allows to scatter files in $out to multiple outputs.
For "bin" and "doc" outputs there are prefefined default masks, but
they can be overriden by setting files_<outname>, for example:
files_bin = [ "/bin/*" "/lib/libexec/" ];
To make an effect hook must be specified in buildInputs.
[Note from Austin: I think @edolstra forgot to merge this to master.]
(cherry picked from commit 02b056c5b180b4b8ba22ddc3061d78258e2ef98f on
release-14.04)
These two expressions greatly simplify using the clang-analyzer or
Coverity static analyzer on your C/C++ projects. In fact, they are
identical to nixBuild in every way out of the box, and should 'Just
Work' providing your code can be compiled with Clang already.
The trick is that when running 'make', we actually just alias it to the
appropriate scan build tool, and add a post-build hook that will bundle
up the results appropriately and unalias it.
For Clang, we put the results in $out/analysis and add an 'analysis'
report to $out/nix-support/hydra-build-products pointing to the result
HTML - this means that if the analyzer finds any bugs, the HTML results
will automatically show up Hydra for easy viewing.
For Coverity, it's slightly different. Instead we run the build tool and
after we're done, we tar up the results in a format that Coverity Scan's
service understands. We put the tarball in $out/tarballs under the name
'foo-cov-int.xz' and add an entry for the file to hydra-build-products
as well for easy viewing.
Of course for Coverity you must then upload the build. A Hydra plugin to
do this is on the way, and it will automatically pick up the
cov-int.tar.xz for uploading.
Note that coverityAnalysis requires allowUnfree = true;, as well as the
cov-build tools, which you can download from https://scan.coverity.com -
they're not linked to your account or anything, it's just an annoying
registration wall.
Note this is a first draft. In particular, scan-build fixes the C/C++
compiler to be Clang, and it's perfectly reasonable to want to use Clang
for the analyzer but have scan-build invoke GCC instead.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
This reverts commit a2a398fbda. The
issue *does* still exist in GHC 7.8.2. Compiled binaries have no -rpath
into their own install directory ("$out") and thus cannot find their own
shared libraries. To work around this issue, we pass an explicit -rpath
argument at configure time. We do that only on Linux, though, because
-rpath is known to cause trouble on Darwin, which was the reason I
originally reverted that patch.
This includes a lot of fixes for cross-building to Windows and Mac OS X
and could possibly fix things even for non-cross-builds, like for
example OpenSSL on Windows.
The main reason for merging this in 14.04 already is that we already
have runInWindowsVM in master and it doesn't work until we actually
cross-build Cygwin's setup binary as the upstream version is a fast
moving target which gets _overwritten_ on every new release.
Conflicts:
pkgs/top-level/all-packages.nix
See the comments at f67015cae4
for more information.
Please note: this makes initrd unrepeatable again, but most people will prefer that above an unbootable system.
The gcc-wrapper doesn't wrap 'cpp'. This breaks some software (such as
Buildroot) because the 'cpp' they get come from the non-wrapped gcc
package which doesn't know about any standard include paths.
gcc-cross-wrapper is untested.
zimbatm
Nikolay Amiantov
Tuomas Tynkkynen
Robin Gloster
Domen Kožar
Vladimír Čunát
Kamil Chmielewski
David Craven
Eric Litak
Moritz Ulrich
Guillaume Maudoux
Profpatsch
Franz Pletz
Eelco Dolstra
Thomas Tuegel
Samuel Rivas
Carles Pagès
Joachim Fasting
Peter Simons
Tuomas Tynkkynen
Maxim Ivanov
Frederik Rietdijk
jraygauthier
Guido Zgraggen
Marius Bakke
Brian McKenna
Brian McKenna
Luca Bruno
Luca Bruno
Christoph Hrdinka
Tomasz Kontusz
Lluís Batlle i Rossell
Rodney Lorrimar
Nicolas B. Pierron
Ryan Trinkle
Sander van der Burg
Tristan Helmich
tg(x)
Tim Cuthbertson
Tony White
Jude Taylor
Graham Christensen
Ricardo M. Correia
Scott Olson
Cole Mickens
Asko Soukka
Benjamin Staffin
Eric Seidel
aszlig
Arthur Noel
Arnaud Spiwack
Dan Peebles
Rok Garbas
Anthony Cowley
Tyson Whitehead
Matthew O'Gorman
Michael Raskin
Kevin Cox
Wei-Ming Yang
Shea Levy
Russell O'Connor
Danny Wilson
Danny Wilson
William A. Kennington III
Andrew Rynhard
obadz
goibhniu
Wout Mertens
Demin Dmitriy
Bjørn Forsman
Eric Seidel
ts468
Juho Östman
John Ericson
Mathnerd314
宋文武
Tobias Geerinckx-Rice
Arseniy Seroka
Jan Malakhovski
John Chee
Rob Vermaas
Jeffrey David Johnson
Pascal Wittmann
Mateusz Kowalczyk
Benjamin Staffin
vbgl
Eric Merritt
Edward Tjörnhammar
Robert Scott
Georges Dubus
Ryan Mulligan
Charles Strahan
Daniel Peebles
Alexander V. Nikolaev
Anders Papitto
Joel Taylor
Florian Friesdorf
Dan Peebles
Thomas Strobel
Emil Rangden
Longrin Wischnewski
John Wiegley
Igor Pashev
vi
Bas van Dijk
Vincent Laporte
Raffael Mancini
Chris Double
Alexander Kjeldaas
Aristid Breitkreuz
Russell O'Connor
Mathijs Kwik
Gergely Risko
tv@shackspace.de
Benno Fünfstück
Patrick Mahoney
Georges Dubus
Justin Bedo
Austin Seipp
Paul Colomiets
Jonathan Glines
Simon Hengel
Linquize
Ricky Elrod
ambrop7@gmail.com
Jaka Hudoklin