‘This release has a nonzero chance to break existing scripts that
use some extension features – I had to quote some tildes in
dot.mkshrc and a variable expansion in ${x/y"$z"} in MirWebseite
(the $z) – twice!. As usual, test!’
-- https://www.mirbsd.org/permalinks/news_e20151212-tg.htm
This fixes#10828 by removing the msize option and also disabling the
writeback cache for the root file system of the test machines.
I've tested this across 5 evaluations on my Hydra, where I run tests for
specific machine configurations:
https://headcounter.org/hydra/eval/304437?filter=testshttps://headcounter.org/hydra/eval/304438?filter=testshttps://headcounter.org/hydra/eval/304439?filter=testshttps://headcounter.org/hydra/eval/304440?filter=testshttps://headcounter.org/hydra/eval/304441?filter=tests
So to summarize on the test failures:
Eval Test failures
304437 quake3, virtualbox
304438
304439 virtualbox
304440 virtualbox
304441
Both the quake3 and virtualbox test failures are unrelated to this merge
and I didn't have to cancel or restart any other jobs. The only jobs I
really had to cancel were the virtualbox jobs, so we no longer should
have "hanging" jobs in the queue due to page allocation failures of the
9p shares.
This is in controst to every evaluation before 304437, where I needed to
constantly restart VM tests.
Thanks a lot to @wizeman for finding the original issue and to
@domenkozar and @lethalman for testing and additional findings (such as
the issue with the cache option).
This seems to be the root cause of the random page allocation failures
and @wizeman did a very good job on not only finding the root problem
but also giving a detailed explanation of it in #10828.
Here is an excerpt:
The problem here is that the kernel is trying to allocate a contiguous
section of 2^7=128 pages, which is 512 KB. This is way too much:
kernel pages tend to get fragmented over time and kernel developers
often go to great lengths to try allocating at most only 1 contiguous
page at a time whenever they can.
From the error message, it looks like the culprit is unionfs, but this
is misleading: unionfs is the name of the userspace process that was
running when the system ran out of memory, but it wasn't unionfs who
was allocating the memory: it was the kernel; specifically it was the
v9fs_dir_readdir_dotl() function, which is the code for handling the
readdir() function in the 9p filesystem (the filesystem that is used
to share a directory structure between a qemu host and its VM).
If you look at the code, here's what it's doing at the moment it tries
to allocate memory:
buflen = fid->clnt->msize - P9_IOHDRSZ;
rdir = v9fs_alloc_rdir_buf(file, buflen);
If you look into v9fs_alloc_rdir_buf(), you will see that it will try
to allocate a contiguous buffer of memory (using kzalloc(), which is a
wrapper around kmalloc()) of size buflen + 8 bytes or so.
So in reality, this code actually allocates a buffer of size
proportional to fid->clnt->msize. What is this msize? If you follow
the definition of the structures, you will see that it's the
negotiated buffer transfer size between 9p client and 9p server. On
the client side, it can be controlled with the msize mount option.
What this all means is that, the reason for running out of memory is
that the code (which we can't easily change) tries to allocate a
contiguous buffer of size more or less equal to "negotiated 9p
protocol buffer size", which seems to be way too big (in our NixOS
tests, at least).
After that initial finding, @lethalman tested the gnome3 gdm test
without setting the msize parameter at all and it seems to have resolved
the problem.
The reason why I'm committing this without testing against all of the
NixOS VM test is basically that I think we can only go better but not
worse than the current state.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Previously is was assumed that bash was in the path when calling the
environment setup script. This changes all of the references of bash to
be absolute paths so that the user doesn't have to worry about the
environment they call it with.
adv_cmds archive actually contains BSDmakefile, not BSDMakefile. While
that probably doesn't matter in default installations, it does matter
for case-sensitive filesystems.
From the changelog:
```
Version 0.7.80, 2015-11-30
+ Matroska: support of MKVMerge statistics tags (duration frame count,
stream size, bit rate) per track, thanks to ndjamena
+ FLAC: Channel positions, thanks to ndjamena
+ FLAC: difference between detected bit depth and stored bit depth
+ MPEG-TS: if DTVCC transport stream is present and no DTVCC service
descriptor, scan also in the middle of the file in order to detect
more caption services
+ Subtitle frame rate computing if frame count and duration are
available (hidden by default)
+ Subtitles in Matroska: count of elements
+ Matroska, MXF and MP4/MOV: detection of truncated files
+ DTS: difference between ES Matrix and ES Discrete
+ DTS: display ES Matrix or ES Discrete even if HRA or MA is present
+ DTS: difference between DTS-HRA with 96k option and pure DTS-96/24
+ DTS: detection of DTS:X
+ Samples per frame info
+ AC-3: detection of Atmos inside TrueHD
+ Video frame rate: showing precision of 1/1.001 frame rates (e.g.
"23.976 (24000/1001) fps" and "23.976 (23976/1000) fps")
+ MPEG-4/MOV: showing the complete list of compatible brands in the
CodecID field
+ MPEG-4/MOV: Alternate groups
+ MPEG-4/MOV: "Disabled" tag
+ MPEG-4/MOV: "Forced" tag
+ MPEG-4/MOV: showing links between tracks (chapters for, subtitles for,
fallback for)
+ MXF: handling of more acquisition metadata items
+ MXF: Package name
+ AVC: Store method of interlaced content (Interleaved Fields or
Separated Fields)
+ EBUCore: acquisition metadata (Proof of concept, for feedback only)
x Matroska: frame rate detection algorithm revisited, less wrong numbers
are expected
x SDP/Teletext: some pages were sometimes (when present in 2 different
SDP lines) displayed several times
x MPEG-4/MOV: some hint tracks were not displayed
+ Hongkongese language added
+ Option "Full parsing"
```
This renames the current openssl to openssl_1_0_1 and defaults
openssl back to openssl_1_0_1 essentially changing nothing currently,
but allowing people to explicitly select openssl_1_0_1 for some software
which needs ABI compatibility to a specific libssl implementation like
binaries (spotify amongst others) and at the same time overriding
openssl to another implementation like libressl.
Shea Levy
Michael Raitza
Rickard Nilsson
viric
Domen Kožar
Frederik Rietdijk
Eric Seidel
Arseniy Seroka
Eric Sagnes
Gleb Peregud
Tobias Geerinckx-Rice
Eric Seidel
Jim Garrison
aszlig
Mate Kovacs
Robert Helgesson
Simon Vandel Sillesen
Kevin Cox
Daniel Peebles
Jakob Gillich
Eelco Dolstra
Damien Cassou
Jascha Geerds
Luis Pedro Coelho
Oliver Charles
Pascal Wittmann
Matthew O'Gorman
Badi' Abdul-Wahid
Bart Brouns
Yann Hodique
Matej Cotman
Aristid Breitkreuz
Alexander Shabalin
Pascal Wittmann
goibhniu
devhell
Franz Pletz
Ambroz Bizjak
JC Brand
Nikolay Amiantov
lethalman
Robin Gloster