Since the script running is a failure condition, we should fail the
build properly, not leaving it up to the missing output to determine
that the build went wrong. This should partly address #38952 — nix
build will print out the build log on non-zero exits.
Also fix numberous bugs, such as:
- Not getting confused on more flags taking file arguments.
- Ensuring children reexport their children, but the original
binary/library doesn't.
- Not spawning children when it turns out we just dynamically link
under the threshold but our total number of inputs exceeeds it.
- Children were always named `libunnamed-*`, when that name was
supposed to be the last resort only.
ld-wrapper's own RPATH check hardcodes `.so`, but darwin uses `.dylib`
*and* (in practice due to lousy build systems) `.so`. We don't care
however because we never inject `--rpath` like that in practice on
Darwin. Hopefully someday we won't on linux either.
Pull request #38470 added support for running/building kernels without
modules. This got merged in 38e04bbf29 but
unfortunately while this works perfectly on kernels without modules it
also makes sure that *every* kernel gets no modules.
So all of our VM tests fail since that merge with something like this:
machine# loading module loop...
machine# modprobe: FATAL: Module loop not found in directory /lib/modules/4.14.33
machine# loading module vfat...
machine# modprobe: FATAL: Module vfat not found in directory /lib/modules/4.14.33
machine# loading module nls_cp437...
machine# modprobe: FATAL: Module nls_cp437 not found in directory /lib/modules/4.14.33
machine# loading module nls_iso8859-1...
machine# modprobe: FATAL: Module nls_iso8859-1 not found in directory /lib/modules/4.14.33
machine# loading module fuse...
machine# modprobe: FATAL: Module fuse not found in directory /lib/modules/4.14.33
machine# loading module dm_mod...
machine# modprobe: FATAL: Module dm_mod not found in directory /lib/modules/4.14.33
I shortly tested this against the "misc" VM test and the test is working
again.
In the long term (and I currently don't have time for this) it would be
better to also have a VM test which tests a kernel without modules.
Signed-off-by: aszlig <aszlig@nix.build>
Cc: @roberth, @7c6f434c
This is necessary due to a e2fsprogs update
(e6114781b0fad5345a2430fac3587d618273bda2) that causes mke2fs to
enable a feature (metadata_csum) that depends on crc32c.
https://hydra.nixos.org/build/72636785
Setting the hash to null is a convenient way to bypass the hash check
while developing. It looks like the ability to do this was inadvertently
removed while adding vendor directory support.
This still checks that the user is explicitly setting the value but
allows null as a valid option.
This is to go to a reproducible image build.
Note without this options image are identical from the Docker point of
view but generated docker archives could have different hashes.
This is to improve image creation reproducibility. Since the nar
format doesn't support hard link, the tar stream of a layer can be
different if a dependency of a layer has been built locally or if it
has been fetched from a binary cache.
If the dependency has been build locally, it can contain hard links
which are encoded in the tar stream. If the dependency has been
fetched from a binary cache, the tar stream doesn't contain any hard
link. So even if the content is the same, tar streams are different.
Resolved the following conflicts (by carefully applying patches from the both
branches since the fork point):
pkgs/development/libraries/epoxy/default.nix
pkgs/development/libraries/gtk+/3.x.nix
pkgs/development/python-modules/asgiref/default.nix
pkgs/development/python-modules/daphne/default.nix
pkgs/os-specific/linux/systemd/default.nix
Per @Ericson2314's suggestion [1], make it more clear that the active
hardenings are decided via whitelist; the blacklist is merely for the
debug messages.
1: 36d5ce41d4 (r133279731)
Existing "mips64el" should be "mipsel".
This is just the barest minimum so that nixpkgs can recognize them as
systems - although required for building individual derivations onto
MIPS boards, it is not sufficient if you want to actually build nixos on
those targets
Previously, cargoUpdateHook was meaningful as it was used
in
[`cargo-fetch-deps`](19d3cf81d3/pkgs/build-support/rust/fetch-cargo-deps (L71)).
However, this entire file was removed in
5f8cf0048e. As far as I can
tell, nothing in the code is using it, but it is still
being passed around:
https://github.com/NixOS/nixpkgs/search?q=cargoUpdateHook&type=Code&utf8=%E2%9C%93
There are, however, legitimate use cases for it. For example,
in some software, some dependencies are not locked in Cargo.toml
and this causes Cargo to try fetching another version of them.
This doesn't work well with vendoring crates.
This hook allows to inject patching or whatever necessary workarounds
in the crate vendoring process. I suppose that's what it was for
in there in the first place.
This patch restores this hook and makes it usable again.
Before the code would fail silently for zero values and with some output for
empties. We now currently handle both via defaulting value to zero and making
`let` return success error code when there's no syntax error.
A separate function for building Bazel-bazed packages. Internally it splits the
build into two phases, fetching and building.
Users are expected to provide `fetchArgs.sha256` -- checksum of fetched
dependencies. Local dependencies should be removed in `fetchArgs.preInstall`.
Overall `fetchArgs` and `buildArgs` can be used to add specific steps to fetch
and build.
`find -executable` finds everything with the executable bit set,
including directories. Thats not harmful in this scenario as `cp` won't
copy those directories, but it does result in a few warning messages.
I originally wrote this for packaging proprietary games in Vuizvui[1]
but I thought it would be generally useful as we have a fair amount of
proprietary software lurking around in nixpkgs, which are a bit tedious
to maintain, especially when the library dependencies change after an
update.
So this setup hook searches for all ELF executables and libraries in the
resulting output paths after install phase and uses patchelf to set the
RPATH and interpreter according to what dependencies are available
inside the builder.
For example consider something like this:
stdenv.mkDerivation {
...
nativeBuildInputs = [ autoPatchelfHook ];
buildInputs = [ mesa zlib ];
...
}
Whenever for example an executable requires mesa or zlib, the RPATH will
automatically be set to the lib dir of the corresponding dependency.
If the library dependency is required at runtime, an attribute called
runtimeDependencies can be used to list dependencies that are added to
all executables that are discovered unconditionally.
Beside this, it also makes initial packaging of proprietary software
easier, because one no longer has to manually figure out the
dependencies in the first place.
[1]: https://github.com/openlab-aux/vuizvui
Signed-off-by: aszlig <aszlig@nix.build>
Closes: #34506
Among other things, this will allow *2nix tools to output plain data
while still being composable with the traditional
callPackage/.override interfaces.
Summary:
According to git-submodule manpage,
"git submodule status" prefixes the hash with a '-' if it is not
initialized, and other chars in other circumstances.
(this is consistent on the various git versions tested)
nix-prefetch-git runs "git submodule init" which does you'd think,
but apparently despite this earlier versions of git before 2.16
would still give the hash the '-' suffix.
In particular this is the behavior when using 2.15 and 2.14.1
from the nixos-17.09 and nixos-17.03 channels respectively.
The script then used awk to drop the first char of the first field
which does the wrong thing when there is no prefix emitted:
while there is a space character before the hash, this is not
part of the field and so we ended up eating the first character
of the hash.
To fix this in a way that also works with the previous behavior,
this commit instead uses awk to grab the hash field
and uses tr to delete any '-' chars should they be present.
This seems to work in my testing, and for example can now
successfully fetch the source for "nginxModules.brotli"
where previously it would generate an error:
fatal: '22564a95d9ab58865a096b8d9f7324ea5f2e03e' is not a commit and a branch 'fetchgit' cannot be created from it
(we dropped a '2' from the beginning of the hash)
We still ensure the old and new ones start, respectfully, with `a/` and
`b/`. Use with `stripLen` to ensure tha the old `a/` and `/b` are gone
if a new prefix is added.
cargo-vendor generates almost the right cargo config. Store it with the
vendored files and patch it on use.
This allows to re-use the generated config when using git dependencies.
- All deps go on the PATH
- CC and Bintools wrappers with their host != depender's host still get their
setup hooks run.
- Environment hooks get applied to all packages
This isn't so elegent, but eases the transition on a very significant
PR.
We now have the information to properly determine the role the
cc-wrapper dependency has, by taking advantage of `offset`. No longer
use the soon-to-be-deprecated crossConfig environment variable, the
temp hack used before this change.
Changes:
* doesn't handle root user separately
* doesn't chdir("/") which makes using it seamless
* only bind mounts, doesn't symlink (i.e. files)
Incidentally, fixes#33106.
It's about two times shorter than the previous version, and much
easier to read/follow through. It uses GLib quite heavily, along with
RAII (available in GCC/Clang).
It would be nice to be able to track Nix requests. It's not trustworthy,
but can be helpful for stats and routing in HTTP logs.
Since `fetchurl` is used so widely, we should "magically" get a UA on
`fetchzip`, `fetchFromGitHub`, and other related fetchers.
Since `fetchurl` is only used for fixed-output derivations, this should
cause no mass rebuild.
User-Agent example: curl/7.57.0 Nixpkgs/18.03
* Wrap LEN macro in parantheses
* Drop env_filter in favor of stateful environ_blacklist_filter,
use execvp instead of execvpe, don't explicitly use environ
* Add argument error logging wherever it makes sense
* Drop strjoin in favor of asprintf
* char* -> const char* where appropriate
* Handle stat errors
* Print user messages with fputs, not errorf
* Abstract away is_str_in (previously bind_blacklisted)
* Cleanup temporary directory on error
* Some minor syntactic and naming changes
Thanks to Jörg Thalheim and Tuomas Tynkkynen for the code review!
Factor a bintools (i.e. binutils / cctools) wrapper out of cc-wrapper. While
only LD is wrapped, the setup hook defines environment variables on behalf of
other utilites.
They aren't meant to be critical (uncatchable) errors.
Tested with nix-env + checkMeta:
[ "x86_64-linux" "i686-linux" "x86_64-darwin" "aarch64-linux" ]
This commit adds the CentOS 7.4 base image from the CentOS mirror, for use with
building RPMs or evaluating Nix expressions in a CentOS image.
When CentOS 7.5 comes out, I will swap this URL to the permanently vaulted image.
This change adds granular, non-docker daemon docker image fetchers and
a docker image layer compositor to be used in conjunction with the
`docker2nix` utility provided by the `haskellPackages.hocker` package.
This change includes a hackage package version bump and updated sha256
for recent fixes released to `hocker` resulting from formulating this
patch.
On non-GNU (gcc) compilers, there is no "/lib/gcc/..."
so when this is eventually expanded this is empty
resulting in an incomplete "-idirafter " that
eats the next argument:
-idirafter -B/nix/store/wamjwwdvkmhbf4f2902nhw8jxxzv0hy3-clang-wrapper-4.0.1/bin/
Certain tools, e.g. compilers, are customarily prefixed with the name of
their target platform so that multiple builds can be used at once
without clobbering each other on the PATH. I was using identifiers named
`prefix` for this purpose, but that conflicts with the standard use of
`prefix` to mean the directory where something is installed. To avoid
conflict and confusion, I renamed those to `targetPrefix`.
This continues #23374, which always kept around both attributes, by
always including both propagated files: `propgated-native-build-inputs`
and `propagated-build-inputs`. `nativePkgs` and `crossPkgs` are still
defined as before, however, so this change should only barely
observable.
This is an incremental step to fully keeping the dependencies separate
in all cases.
After #31497 starter quoting all values, there arouse the need to left some
values evaluated.
`--set-default var value` expands to `export var=${var-value}`, where value is
not evaluated and literally assigned to var unless it is already set.
`--set-eval var value` expands to `export var=$(eval echo value)`, where value
is evaluated by `eval`.
Currently we wrap ssh so it can find the config file passed in by
<ssh-config-file>. If one however uses ProxyCommand ssh, then ssh that
is on PATH is taken (which is also unavailable when using nix-shell
--pure), which is the plain ${openssh}/bin/ssh.
This commit makes sure our wrapped ssh is available on PATH.
For a while now, the only thing the 'uboot' attribute does is to tell
whether to add ubootTools to kernel/initrd builds. That can be
determined with platform.kernelTarget == "uImage" just as well.
If a dynamic linker for target is not found the generated script fails
due to unbound variable error (due to "set -u"). Correct by specifying
default value with dynamicLinker:- and not generating ldflagsBefore if
no linker is found.
This problem was found when cross compiling to mingw32 targets
The previous one was very bad and worsened the situation.
But even running with some nix-1.12 I'm unable to reproduce
the original failure. Let's unblock channels for now.
Unlike pathsFromGraph, on Nix 1.12, this function produces a
registration file containing correct NAR hash/size information.
https://hydra.nixos.org/build/62832723
The biggest benefit is that we no longer have to update the registry
package. This means that just about any cargo package can be built by
nix. No longer does `cargo update` need to be feared because it will
update to packages newer then what is available in nixpkgs.
Instead of fetching the cargo registry this bundles all the source code
into a "vendor/" folder.
This also uses the new --frozen and --locked flags which is nice.
Currently cargo-vendor only provides binaries for Linux and
macOS 64-bit. This can be solved by building it for the other
architectures and uploading it somewhere (like the NixOS cache).
This also has the downside that it requires a change to everyone's deps
hash. And if the old one is used because it was cached it will fail to
build as it will attempt to use the old version. For this reason the
attribute has been renamed to `cargoSha256`.
Authors:
* Kevin Cox <kevincox@kevincox.ca>
* Jörg Thalheim <Mic92@users.noreply.github.com>
* zimbatm <zimbatm@zimbatm.com>
This requires some small changes in the stdenv, then working around the
weird choice LLVM made to hardcode @rpath in its install name, and then
lets us remove a ton of annoying workaround hacks in many of our Go
packages. With any luck this will mean less hackery going forward.
Before this fix, it seemed to be trying to merge our postFetch with the
patch normalization logic, but accidentally clobbering the whole thing
with the passed-in value.
cc-wrapper may wrap a cc-compiler, but it doesn't need one to build
itself. (c.f. expand-response-params is a separate derivation.) This
helps avoid cycles on the cross stuff, in addition to removing a
useless dependency edge.
I could have been super careful with overrides in the stdenv to avoid
the mass rebuild, but I don't think it's worth it.
We were using 'Combined Image JSON + Filesystem Changeset Format' [1] to
unpack and pack image and this patch switches to the format used by the registry.
We used the 'repository' file which is not generated by Skopeo when it
pulls an image. Moreover, all information of this file are also in the
manifest.json file.
We then use the manifest.json file instead of 'repository' file. Note
also the manifest.json file is required to push an image with Skopeo.
Fix#29636
[1] 749d90e10f/image/spec/v1.1.md (combined-image-json--filesystem-changeset-format)
The database dump doesn't contain sha and size. This leads to invalid
path in the container. We have to fix the database by using
nix-store.
Note a better way to do this is available in Nix 1.12 (since the
database dump contains all required information).
We also add content output paths in the gcroots since they ca be used
by the container.
Currently, the contents closure is copied to the layer but there is no
nix database initialization. If pkgs.nix is added in the contents,
nix-store doesn't work because there is no nix database.
From the contents of the layer, this commit generates and loads the
database in the nix store of the container. This only works if there
is no parent layer that already have a nix store (to support several
nix layers, we would have to merge nix databases of parent layers).
We also add an example to play with the nix store inside the
container. Note it seems `more` is a missing dependency of the nix
package!
1. `crossDrv` is now the default so we don't need to worry about that in
build != host builds.
2. shell is the build time shell, so `wrapCCCross` doesn't need to
worry, as build == host.
3. `shell.shellPath` will always be appended where useful.
4. Complicated `shell == ""` logic served no purpose.
Fetch into $out and remove all version control files to make it
deterministic (.repo and all .git subdirectories - e.g. the .git/index
files change every time).
Additionally I've changed the default of "useArchive" to false because
fetching with "--archive" will fail for some projects (e.g.
"platform/external/iosched" from the AOSP).
Now, this function should hopefully work for every tag of the AOSP.
Before this patch, a VM was used to spawn docker that pulled the
VM. Now, the tool Skopeo does this job well so we can simplify our
dockerTools since we doesn't need Docker anymore:)
This also fixe the regression described in
https://github.com/NixOS/nixpkgs/issues/29271 : cntlm proxy doesn't
work in 17.09 while it worked in 17.03.
Note Skopeo doesn't produce the same output than docker pull so, we
have to update sha.
This was a problem when run inside a sandbox, e.g. via
"fetchRepoProject". The error message from repo seems unrelated:
fatal: Cannot get https://gerrit.googlesource.com/git-repo/clone.bundle
fatal: error no host given
But the exception is actually thrown due to missing certificates
(/etc/ssl/certs). It should be possible to provide another location via
environment variables (e.g. SSL_CERT_FILE, REQUESTS_CA_BUNDLE or
CURL_CA_BUNDLE) but apparently that doesn't actually work for some
reason (would have to study our Python packaging).
Now "fetchRepoProject" works without the "--no-clone-bundle" option.
The verification was failing with the following error:
gpg: keyblock resource '/tmp/nix-build-XYZ.drv-0/.repo/repo/./.repoconfig/gnupg/pubring.kbx': No such file or directory
Using an absolute path for $HOME fixes this.
And since 175ecbab91 the dependencies on
"git" and "gnupg" aren't required anymore as "gitRepo" already covers
them.
This reverts commit 0a944b345e, reversing
changes made to 61733ed6cc.
I dislike these massive stdenv changes with unclear motivation,
especially when they involve gratuitous mass renames like NIX_CC ->
NIX_BINUTILS. The previous such rename (NIX_GCC -> NIX_CC) caused
months of pain, so let's not do that again.
cctool's as needs to be told use to use gnu as, or else we'd need a
dependency cycle between cctools and clang for this case.
In general, this is not a problem because clang uses its own integrated
assembler where possible, and gnu as otherwise.
To wait for the docker deamon, curl requests are sent. However, if a
http proxy is set, it will respond instead of the docker daemon.
To avoid this, we send docker ps command instead of curl command.
This becomes necessary if more wrappers besides cc-wrapper start
supporting hardening flags. Also good to make the warning into an
error.
Also ensure interface is being used right: Not as a string, not just in
bash.
ccPath is only defined below, so this condition would never be true.
Worse, that's not quite true: what if somebody happend to have `/clang`
and no sandboxing. Boy, wouldn't that be annoying to debug!
set-source-date-epoch-to-latest.sh to ignore files newer than "$NIX_BUILD_TOP/.." (unlike "$NIX_BUILD_TOP" it is root-owned and cannot be touched by nixbld1).
Having multiple compilers in the build environment would result in an
invalid LD_DYLD_PATH like /usr/lib/dyld/usr/lib/dyld.
Since the path is hardcoded in XNU it can't be anything but
/usr/lib/dyld anyway.
This fixes a bug introduced in #27831: `for path in "$dir"/lib*.so` assumed that
all libs match `lib*.so`, but 07674788d6 started
adding libs that match `*.so` and `*.so.*`.
`makeWrapper` and `wrapProgram` are being invoked on all kinds of
wacky things (usually with the help of bash globs or other machine
assistance).
So far, I have come across `wrapProgram` being invoked on a directory,
as well as on the empty string.
As far as I can tell, it's only valid to invoke these utilities on a
normal (non-directory, non-device) executable file. This commit
enforces that precondition.
Previously, makeWrapper would accept arguments it didn't recognize,
potentially allowing argument misspellings or broken callers.
Now, makeWrapper dies with a backtrace if it is called incorrectly.
Also changes `wrapProgram` so that it doesn't pass through the first
argument twice --- this was tripping up the argument checking.
Now is an opportune time to do this, as the infixSalt conversion in
`add-flags.sh` ensures that all the relevant `NIX_*` vars will be
defined even if empty.
This is basically a sed job, in preparation of the next commit. The
rules are more or less:
- s"NIX_(.._WRAPPER_)?([a-zA-Z0-9@]*)"NIX_\1@infixSalt@_\2"g
- except for non-cc-wrapper-specific vars like `NIX_DEBUG`
This is an ugly temp hack for cross compilation, but now we have something better on the way.
Bind `infixSalt` as an environment variable as it will be used in it.
Some programs store the executable in a different place and link it
from the `bin` directory. For example, Polari links `$out/bin/polari`
to `$out/share/polari/org.gnome.Polari`. `wrapGAppsHook` did not follow
symlinks so it was not able to wrap Polari, making it unable to access
GObject introspection definitions required for running the program.
I made the wrapping script follow symlinks to fix this corner case.
In 8d76eff, @Ericson2314 changed the representation of the value that
`findInputs` generated from a whitespace-separated bunch strings to an
actual array of strings.
Expressions that *consume* that value, however, also needed to be
changed to iterate over all the contents of the array, else they would
only select the first value, which turns out to be somewhat limiting.
Fixes#27873
The image json is not exactly the same as the layer json, therefore I
changed the implementation to use the `baseJson` which doesn’t include
layer specific details like `id`, `size` or the checksum of the layer.
Also the `history` entry was missing in the image json. I’m not totally
sure if this field is required, but a I got an error from a docker
registry when I’ve tried to receive the distribution manifest of an
image without those `history` entry:
GET: `http://<registry-host>/v2/<imageName>/manifests/<imageTag>`
```json
{
"errors": [
{
"code": "MANIFEST_INVALID",
"message": "manifest invalid",
"detail": {}
}
]
}
```
I’ve also used a while loop to iterate over all layers which should make
sure that the order of the layers is correct. Previously `find` was
used and I’m not sure if the order was always correct.
callPackage already calls makeOverridable, but that just
makes the function that evaluates to buildEnv overridable,
not buildEnv itself.
If no overridable version of buildEnv is used during construction,
users can't override e.g. `paths` at all
Unified processing of command line arguments in ld-wrapper broke support for
`NIX_DONT_SET_RPATH` and revealed that ld-wrapper adds the directory of its
`-plugin` argument to runpath. This pull request fixes that. It treats
`dir/libname.so` as `-L dir -l name`, because this is how `ld.so` interprets
resulting binary: with `dir` in `RUNPATH` and the bare `libname.so` (without
`dir`) in `NEEDED`, it looks for `libname.so` in each `RUNPATH` and chooses the
first, even when the linker was invoked with an absolute path to `.so`.
As described in https://github.com/NixOS/nixpkgs/issues/18461, MacOS no
longer accepts dylibs which only reexport other dylibs, because their
symbol tables are empty. To get around this, we define an object file
with a single "private extern" symbol, which hopefully won't clobber
anything.
If the base image has been built with nixpkgs.dockerTools, the image
configuration and manifest are readonly so we first need to change
their permissions before removing them.
Fix#27632.
The time to expand rpath was proportional to the number of -L flags times the
number of -l flags. Now it is proportional to their sum (assuming constant
number of files in each directory in an -L flag).
Issue reported by @nh2 at https://github.com/NixOS/nixpkgs/issues/27609#issuecomment-317916623
This setup hook is propagated by gdb. Thus, a typical use is:
$ nix-shell -p gdb nix nix.debug sqlite.debug ...
and gdb will be able find the debug symbols of nix etc. automatically.
The docker loading (docker 1.12.6) of an image with uppercase in the
name fails with the following message:
invalid reference format: repository name must be lowercase
As @oxij points out in [1], this breakage is especially serious because
it changes the contents of built environments without a corresonding
change in their hashes. Also, the revert is easier than I thought.
This reverts commit 3cb745d5a6.
[1]: https://github.com/NixOS/nixpkgs/pull/27427#issuecomment-317293040
Fixes#27406.
Commit 5d4efb2c81 added an assertion to `stopNest'
which requires it be correctly paired with `startNest'. `fetchurl' calls
`stopNest', but never calls `startNest'; the former calls are removed.
The `DISPLAY` environment variable is propagated into chroots built with
`buildFHSUserEnv`, but currently the `XAUTHORITY` variable is not. When
the latter is set, its value is usually necessary in order to connect to
the X server identified by the former.
This matters for users running gdm3, for example, who have `XAUTHORITY`
set to something like `/run/user/1000/gdm/Xauthority` instead of the X
default of `~/.Xauthority`, which doesn't exist in that setup.
Fixes#21532.
Since 3cb745d5a6, the format of
propagated-user-env-packages has changed and propagated packages have not been
included by buildenv, including in the system environment.
The buildenv builder is modified to read propagated-user-env-packages
line-by-line, instead of expecting all packages on one line.
Besides deduplicating overlapping logic, clear warning messages were
added for:
- No glob/path for dynamic linker provided (use default glob)
- Glob did not expand to anything (don't append flag)
- glob expanded to multiple things (take first, like before)
This makes those files a bit easier to read. Also, for what it's worth,
it brings us one baby step closer to handling spaces in store paths.
Also, I optimized handling of many transitive deps with read. Probably,
not very beneficial, but nice to enforce the pkg-per-line structure.
Doing so let me find much dubious code and fix it.
Two misc notes:
- `propagated-user-env-packages` also needed to be adjusted as
sometimes it is copied to/from the propagated input files.
- `local fd` should ensure that file descriptors aren't clobbered
during recursion.
They're additional commits from #26877.
Changing names of the fetched stuff was changing very many hashes,
and I think it's better to avoid that for the moment to reduce work
needed by nixpkgs users. The fetchers are expected to be commonly
used even outside nixpkgs, and the current naming wasn't that bad
usually.
(commit analogical to d10c3cc5eedf58e80e2; I haven't noticed the part of
the PR has already got to master)
The freedesktop specification says that the autostart folder is located at $XDG_CONFIG_DIRS/autostart.
Nixos defines XDG_CONFIG_DIRS as /etc/xdg. So we need to copy the desktop files into the right folder
with make-startupitem.
This will override the existing winsymlinks setting. nativestrict
will cause ln to fail if it's unable to create a native symlink.
Native symlinks are required for the windows dll loader to find the
libraries.
This script is also used for cross-mingw, but setting CYGWIN
shouldn't cause a problem.
This allows users to specify a custom registry src,
because currently every packager would need to create
an outdated Cargo.lock just to be compatible with the
probably outdated rustRegistry in nixpkgs.
Currently there is no easy way to convince cargo to
do that, so this makes that workaround unnecessary.
I think it's ok to export things which aren't wrapped. The cc-wrapper
can be thought of as responsible for all of binutils and the c
compiler, only wrapping those binaries which are necessary to
interposition---as opposed to all binaries it thinks are relevaant.
Conversely, adding the setup hook to the unwrapped compilers would be
unforunate as hooks are ugly hacks and the compilers themselves take
a long time to rebuild. Better to wholely separate "pure packages" from
hacks.
Eventually we should avoid this "pre-wrapping" and just update those
files in nixpkgs. This mass-rebuild change is best done along with
those needed to reduce the disparity between native and cross (i.e.
making native the "identity cross").
We now (on cross) require per-target flag interposition by putting the
triple in the names of the relevant environment variables, e.g:
export NIX_arm_unknown_linux_gnu_CFLAGS_COMPILE=...
The wrapper also has a `infixSalt` attribute (and "_" prefixed and
suffixed variants) to assist downstream packages.
Note how that the dashes are replaced to keep the identifier valid.
Using names like this allows us to keep the settings for different
compilers seperate.
I think it might be even better to use names like `NIX_{BUILD,HOST}...`
using the platform's role rather than the platform itself, but this
would be more work as the previous stages' tools would have to be re-
wrapped to take on their new role. I therefore didn't do this for now,
but that route should be thoroughly explored in the future.
This fixes the Stack Clash issue rediscovered by Qualys. See
https://www.qualys.com/2017/06/19/stack-clash/stack-clash.txt
for more information on the topic, specifically section III.
We don't have the kernel mitigation available because it is a Grsecurity
feature which we don't support anymore. Other distributions like Gentoo
Hardened and Arch already have `-fstack-check` enabled by default.
See the Gentoo page on Stack Clash for more information on this solution:
https://wiki.gentoo.org/wiki/Hardened/Gentoo_Hardened_and_Stack_Clash
This unfortunately doesn't apply to clang because `-fstack-check` is a
noop there. Note that the GCC implementation also has problems that could
be exploited to circumvent these checks but it is still better than
keeping it disabled.
* ultrastardx-beta: init at 1.3.5
* libbass, libbass_fx: init at 24
* ultrastar-creator: init at 2017-04-12
* buildSupport/plugins.nix: add diffPlugins
Helper function to compare expected plugin lists to the found plugins.
* ultrastar-manager: init at 2017-05-24
The plugins are built in their own derivations, speeding up (re-)compilation.
The `diffPlugins` function from `beets` is reused to test for changes in the
plugin list on updates.
* beets: switch to diffPlugins
The function is basically just extracted for better reusability.
This value is require to get c++ std include path for libclang based tools (vim plugins in my case).
I currently extract it this with this rather command:
```
eval echo $(nix-instantiate --eval --expr 'with (import <nixpkgs>) {}; clang.default_cxx_stdlib_compile')
```
it did not trigger any recompilation on my system.
`SSL_CERT_FILE` has been replaced with `NIX_SSL_CERT_FILE`.
Before this change using `fetchdarcs` resulted in an error message like:
```
Identifying repository http://hub.darcs.net/scravy/easyplot inventory
darcs failed: Not a repository: http://hub.darcs.net/scravy/easyplot (Peer certificate cannot be authenticated with given CA certificates)
HINT: Do you have the right URI for the repository?
builder for ‘/nix/store/imyvcs6lvb5yva66krc5wk39931sam8v-fetchdarcs.drv’ failed with exit code 2
```
I was getting the following error building tide from Melpa:
nix-build -E '(import <nixpkgs> {}).emacs25WithPackages (p: [p.melpaPackages.tide])'
File tide-20170509.1134.tar is large (10.2M), really open? (y or n) Error reading from stdin
builder for ‘/nix/store/gs9ik7yf8iilsikkfing74i70m0diax3-emacs-tide-20170509.1134.drv’ failed with exit code 255
cannot build derivation ‘/nix/store/m3p080aani4rw82llp8nqk93cw2nvirk-emacs-with-packages-25.2.drv’: 1 dependencies couldn't be built
Solution was to disable the large file warning threshold when
installing packages.
This still causes some uncached rebuilds, but master(!) and staging
move too fast forward rebuild-wise, so Hydra might never catch up.
(There are also other occasional problems.)
Therefore I merge at this point where the rebuild isn't that bad.
We need to make sure that `$revs` ends with a space, since files must always
end with newlines. The previous code ignored the last entry in `$revs`, because
read already returns non-zero exit code for the last entry, as it does not end
with a space.
This change fixes several defects in the way `wrapGAppsHook` selected
the executable to wrap.
Previously, it would wrap any top-level files in the target `/bin` and
`/libexec` directories, including directories and non-executable
files. In addition, it failed to wrap files in subdirectories.
Now, it uses `find` to iterate over these directory hierarchies,
selecting only executable files for wrapping.
This change fixes several defects in the way `wrapGAppsHook` selected
the executable to wrap.
Previously, it would wrap any top-level files in the target `/bin` and
`/libexec` directories, including directories and non-executable
files. In addition, it failed to wrap files in subdirectories.
Now, it uses `find` to iterate over these directory hierarchies,
selecting only executable files for wrapping.
When not using sandboxing, /usr/share/git-core/templates may leak into the
nix build through the libgit2 hardcoded default template search path. We now
explictly set the templatedir to avoid this problem.
See https://github.com/bennofs/nix-index/issues/2#issuecomment-296268983 for
an example case of nondeterminism.
This typo was likely introduced by copy-pasting the error message from elsewhere and forgetting to change the text, during the MD5 deprecation process (#4491).
This patch fixes file modification times to $SOURCE_DATE_EPOCH, and
ensures that files originating from the store are owned by root:root.
Both changes improve reproducibility, and the latter allows proper
building on a host where the store is owned by a non-root user.
Every Rust derivation used to emit a warning like the following:
```
setting SOURCE_DATE_EPOCH to timestamp 1490877042 of file cargo-6e0c18c/Cargo.lock
warning: file cargo-6e0c18c/Cargo.lock may be generated; SOURCE_DATE_EPOCH may be non-deterministic
```
The reason is that the dependencies are copied without preserving
timestamps. Changing the build script to timestamp-preserving copy
removes the warning.
This is a flag that disables subversion keyword substitution.
Keyword substitution inserts metadata into the files being checked
out, and is therefore somewhat at odds with build reproducibility.
In particular, it can become a problem if you're trying to switch
between svn and a git export of the same thing (keyword substitutions
are normally not exported into git).
This allows for a less blanket approach than nuke-refs, targetting specific
references that we know we don't want rather than all references that we don't
know we want.
Because of bash 4.4 the semantics GLOBIGNORE changed.
This resulted in already compressed manpages to be compressed twice.
Also be careful about symlinks to fix#21777, e.g. the ledger example.
See
8214bb953d
for the cargo commit which deprecated the registry.index key, and
implements this as a replacement. This gets rid of the error message
warning: custom registry support via the `registry.index` configuration is being removed, this functionality will not work in the future
This script is not needed anymore since "nix-prefetch-url --unpack
<url>" and "nix-prefetch-url -A foo.src" (where "foo.src" is a
fetchzip / fetchFromGitHub call) work fine.
This reverts commit 3d9017602b.
This didn't quite work as I had expected ... While it seemed okay at
first, it fails to propagate all the attributes it used to (notably
features). I'll revisit this later but reverting for now.
This adds a `dhallToNix` utility which compiles expression from the Dhall
configuration language to Nix using Nix's support for "import from derivation".
The main motivation of this compiler is to allow users to carve out small typed
subsets of Nix projects. Everything in the Dhall language (except `Double`s)
can be translated to Nix in this way, including functions.
This is required for Aarch64 since a lot of source tarballs ship with
outdated configure scripts that don't recognize aarch64. Simply
replacing the config.guess and config.sub with new versions from
upstream makes them build again.
This same approach is used by at least Buildroot and Fedora. In
principle this could be enabled for all architectures but
conditionalizing this on aarch64 avoids a mass rebuild on x86.
To achieve reproducible results, `cpio` archive members are added in
sorted order and inodes renumbered.
The `cpio-clean.pl` script is made obsolete by setting mtimes via
`touch` & using `cpio --reproducible`. Suggested by @dezgeg in
https://github.com/NixOS/nixpkgs/pull/21273#issuecomment-268116605.
Note that using `--reproducible` means that initial ramdisk creation now
requires at least `cpio` version 2.12 (released in 2015).
Deprecation warnings should not be used in Nixpkgs because they spam
innocent "nix-env -qa" users with (in this case) dozens of messages
that they can't do anything about.
This also reverts commit 2ca8833383.
I believe this reduces surprises and is actually simpler semantically.
This is important e.g. for relative symlinks when moving both source
and target - now the order of moving won't matter.
Fixes#20723 (a particular instance of the surprise).
This makes the response file handling more consistent with GCC.
For example, a reponse file may contain:
"-Wl,$ORIGIN"
GCC will treat this as a double quoted string and not expand the
variable reference. Previously, cc-wrapper would expand the variable
in the same was as if the string was provided on the command line.
This commit extends fetchFromGitHub with ability to fetch GitHub
repositories with submodules, so we can use the function consistently
with all GitHub repositories.
Note it doesn't change the previous behavior.
Include files are typically in the libc package's `dev` output but previously `-isystem` was looking in `out`, resulting in my cross-compilation environment not finding its include files.
The motivation for this change is the following: As gnu-netcat,
e. g. does not support ipv6, it is not suitable as default netcat.
This commit also fixes all obvious build issues caused by this change.
This is a temporary work-around to fix using grsecurity on NixOS with
the new kernelPackages/kernelPatches machinery.
For whatever reason, when `security.grsecurity.enable = true`, the grsec
patch ends up being applied twice, causing the kernel build to fail.
Until the root cause of this is identified, we hack around it by simply
pruning duplicate patches in the grsec kernel builder.
Closes#19698
`stripHash` documentation states that it prints out the stripped name to
the stdout, but the function stored the value in `strippedName`
instead.
Basically all usages did something like
`$(stripHash $foo | echo $strippedName)` which is just braindamaged.
Fixed the implementation and all invocations.
extraFlagsArray should not be exposed outside of `makeWrapper`, it
should only be possible to set it inside a script supplied via the
`--run` argument.
This ensures that most "trivial" derivations used to build NixOS
configurations no longer depend on GCC. For commands that do invoke
gcc, there is runCommandCC.
This is a standard environment that doesn't contain a C/C++
compiler. This is mostly to prevent trivial builders like runCommand
and substituteAll from pulling in gcc for simple configuration changes
on NixOS.
When building an image with multiple layers, files
already included in an underlying layer are supposed to
be excluded from the current layer. However, some subtleties
in the way filepaths are compared seem to be blocking this.
Specifically:
* tar generates relative filepaths with directories ending in '/'
* find generates absolute filepaths with no trailing slashes on directories
That is, paths extracted from the underlying tarball look like:
nix/store/.../foobar/
whereas the layer being generated uses paths like:
/nix/store/.../foobar
This patch modifies the output of "tar -t" to match the latter format.
They now go to devman, devdoc, or $outputMan, in that order. This is
to prevent cases such as the man-pages package quietly losing its
section 3 pages.
This builds elisp to setup an emacs buffer with the packages given
available. See shlevy/nix-buffer for more information.
Currently only modifies $PATH.
This one was already merged into release-16.09, so let's not have the
stable branch is ahead of master and confuse things. In addition to
that, currently we have an odd situation that master has less things
actually finished building than in staging.
Conflicts:
pkgs/data/documentation/man-pages/default.nix
This was one of the ways to build packages, we are trying
hard to minimize different ways so it's easier for newcomers
to learn only one way.
This also:
- removes texLive (old), fixes#14807
- removed upstream-updater, if that code is still used it should be in
separate repo
- changes a few packages like gitit/mit-scheme to use new texlive
Previously, features.grsecurity wasn't actually set due to a bug in the
grsec builder. We now rely on the generic kernel builder to set features
from kernelPatches.
This reverts commit da68127737, reversing
changes made to 2ba494b728.
These changes are reverted because they cause an incompatibility which
they should not.
"emacsWrapper" replaces emacsWithPackages. In addition to "packagesFun",
emacsWrapper has an optional variable called "execStart". execStart can
be used to append elisp to the default site-start.el script. This is
useful for providing a way to load a user's .emacs.d/init.el
file. "emacsWithPackages" is implemented with emacsWrapper for
convenience and compatability.
Sometimes patches start without a leading prefix. We default to strip
one prefix or path component from patches (-p1) in the patchPhase in
stdenv.
As all patches should therefore be in this format, fetchpatch should
have an option to normalize patch paths. This commit introduces a new
argument to fetchpatch called addPrefixes that adds one patch prefix to
the old and new paths in a patch before putting it into the store.
fetchMavenArtifact downloads a Maven artifact given a group id, an artifact id,
and a version.
Example usage:
org_apache_httpcomponents_httpclient_4_5_2 = fetchMavenArtifact {
groupId = "org.apache.httpcomponents";
artifactId = "httpclient";
version = "4.5.2";
sha256 = "0ms00zc28pwqk83nwwbafhq6p8zci9mrjzbqalpn6v0d80hwdzqd";
# Optionally: repos = [ ... urls to some Maven repos to use ... ];
# Optionally: url, urls - pointing directly to a specific jar url.
};
Now `org_apache_httpcomponents_httpclient_4_5_2.jar` points to the downloaded
JAR file, while `org_apache_httpcomponents_httpclient_4_5_2` refers to a
derivation that when used used in `buildInputs` will be automatically added to
the Java classpath.
downloads.sourceforge.net is the official way to download tarballs from
SourceForge. However, it is reported as unreliable due to SF's weird
load balancing system.
This commit gives the official mirror utmost priority, and will use
other configured mirrors (which may be temporary) as a fallback only
when the official one can't be reached/download fails/hangs.
References: NixOs/nixpkgs#16900
This patch fixes#16614 and #16741.
The first issue was caused by the fact that both `/share` and
`/share/fish/vendor_completions.d` end in the `pathsToLink`. The
`pkgs/build-support/buildenv/builder.pl` creates `/share`, then links
`/share/fish` under `/share` and then tries to create the directory
`/share/fish/vendor_completions.d` and fails because it already exists.
The simplest way to reproduce the issue is to build the next Nix
expression:
```nix
let pkgs = import <nixpkgs> { };
in pkgs.buildEnv {
name = "buildenv-issue";
paths = [
pkgs.fish
pkgs.vim
];
pathsToLink = [
"/share"
"/share/fish/vendor_completions.d"
];
}
```
The second issue is more critical and was caused by the fact findFiles
doesn't recurse deep enough. It stops at first unique directory for the
package (e.g., "/share" or even "/") and later the scripts decides it
shouldn't link it as it doesn't match pathsToLink (e.g., "/share/fish"),
so the result is empty.
The test:
```nix
let pkgs = import <nixpkgs> { };
in pkgs.buildEnv {
name = "buildenv-issue";
paths = [
pkgs.fish
pkgs.vim
];
pathsToLink = [
"/share/fish/functions"
];
}
```
or
```nix
let pkgs = import <nixpkgs> { };
in pkgs.buildEnv {
name = "buildenv-issue";
paths = [
pkgs.vim
];
pathsToLink = [
"/share"
];
}
```
Prior to this commit, trailing whitespace would be introduced when
modifying '#!' lines with no arguments. For example (whitespace added):
/nix/store/.../foo: interpreter directive changed
from "/bin/bash"
to "/nix/store/...-bash-4.3-p42/bin/bash "
/nix/store/.../bar: interpreter directive changed
from "/bin/baz wef"
to "/nix/store/...-baz wef "
We add a sed command to strip trailing whitespace, so the above commands
would drop the two spaces after "bash", or the one space after "baz wef".
abbradar: fixed commit title
Closes#16785.
This fixes a regression caused by commit f56ab9e
("nix-prefetch-git: Include the date in the machine-readable [...]")
where a couple of directory paths printed by pushd/popd appeared before
the JSON output on stdout (thus breaking it). Fix it by redirecting the
extraneous output to /dev/null.
Reported by Michael Alan Dorman <mdorman@ironicdesign.com>.
stdout, in strict ISO 8601 format.
This will be helpful for automatically updating fetchgit expressions
and the dates in version numbers associated with them.
stripHash uses a global variable to communicate it's computation
results, but it's not necessary. You can just pipe to stdout in a
subshell. A function mostly behaves like just another command.
baseHash() also introduces a suffix-stripping capability since it's
something the users of the function tend to use.
The tests need to expand passed variable and very carefully.
I could see no other easy way than to change single-quoting in
makeWrapper to double-quoting.
The tests now fail with the same problem as on master...
Henry Till
Tuomas Tynkkynen
Matthew Justin Bauer
Tuomas Tynkkynen
pe@pijul.org
Jean-Philippe Braun
Linus Heckemann
Shea Levy
Piotr Bogdan
John Ericson
John Ericson
aszlig
Michael Raskin
Jan Tojnar
Charles Strahan
Eelco Dolstra
Ryan Trinkle
Kevin Cox
Robert Hensing
Jörg Thalheim
Jörg Thalheim
Will Fancher
Shell Turner
Sarah Brofeldt
lewo
Franz Pletz
Justin Bedo
Nikolay Amiantov
Jan Malakhovski
Vladimír Čunát
Will Dietz
Frederik Rietdijk
Daniel Barlow
Mario Rodas
Eelco Dolstra
Yurii Rashkovskii
Pierre-Etienne Meunier
Timo Kaufmann
Parnell Springmeyer
Stewart Mackenzie
Peter Hoeg
gnidorah
Yegor Timoshenko
Andrew Childs
Robin Gloster
zimbatm
Daiderd Jordan
dywedir
Wei-Ming Yang
Dmitry Kalinkin
Joachim F
Orivej Desh
Dan Peebles
Fahad Sadah
Dylan Simon
Graham Christensen
Andreas Rammhold
Benjamin Hipple
Thomas Tuegel
Thomas Tuegel
Frederik Rietdijk
Bojan Nikolic
knupfer
Falco Peijnenburg
Rodney Lorrimar
Vaibhav Sagar
Dan Haraj
Daniel Peebles
Nikita Uvarov
Michael Weiss
Domen Kožar
volth
Domen Kožar
Peter Simons
Taahir Ahmed
davidak
Michael Alan Dorman
mimadrid
Mathias Schreck
tilpner
Sergey Mironov
adisbladis
Jamey Sharp
luigy
Bastian Köcher
Harmen
hsloan
Michael Fellinger
David McFarland
Bjørn Forsman
Ross MacLeod
Profpatsch
Laverne Schrock
Robert Scott
Joe Hermaszewski
Jaka Hudoklin
Matej Cotman
Mateusz Naściszewski
michael bishop
Benno Fünfstück
Utku Demir
rht
Joachim Fasting
Renzo Carbonara
Sven Slootweg
Tim Stewart
Eric Litak
Christian Kauhaus
Sophie Taylor
Sophie Taylor
HeeL
Arseniy Alekseyev
Anders Papitto
Tim Steinbach
Gabriel Gonzalez
Andraz Bajt
Connor Clark
Jörg Thalheim
Tomas Hlavaty
Aristid Breitkreuz
Lazar Bodor
Alexey Shmalko
Tim Steinbach
Ben Gamari
Test
sternenseemann
Alexander Ried
Tom Boettcher
David McFarland
Vincent Laporte
Damien Cassou
Kirill Boltaev
Markus Hauck
Allen Nelson
Nicolas Dudebout
Eric Sagnes
Luca Bruno
Parnell Springmeyer
obadz
Corbin
cmfwyp
Thomas Tuegel
Maarten Hoogendoorn
Renzo Carbonara
J Phani Mahesh
Vladimír Čunát
Cireo
David Grayson
Nathan Zadoks
Moritz Ulrich
David Craven