public/nixpkgs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
nixpkgs/nixos/tests/chromium.nix

202 lines
5.9 KiB
Nix
Raw Normal View History

nixos/tests/chromium: Allow overriding channel map This has been the case before e45c211, but it turns out that it's very useful to override the channel packages so we can run tests with different Chromium build options. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-20 22:20:39 +01:00
{ system ? builtins.currentSystem
tests: refactor to carry the package set as an argument This way, the package set will be possible to pass without re-importing all the time
2018-11-11 17:41:11 +09:00
, config ? {}
, pkgs ? import ../.. { inherit system config; }
nixos/tests/chromium: Allow overriding channel map This has been the case before e45c211, but it turns out that it's very useful to override the channel packages so we can run tests with different Chromium build options. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-20 22:20:39 +01:00
, channelMap ? {
stable = pkgs.chromium;
beta = pkgs.chromiumBeta;
dev = pkgs.chromiumDev;
}
}:
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 00:20:27 +01:00
tests: refactor to carry the package set as an argument This way, the package set will be possible to pass without re-importing all the time
2018-11-11 17:41:11 +09:00
with import ../lib/testing.nix { inherit system pkgs; };
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 00:20:27 +01:00
with pkgs.lib;
mapAttrs (channel: chromiumPkg: makeTest rec {
name = "chromium-${channel}";
chromium tests: inherit timeout from the package /cc #49442. It should decrease the waste of resources due to abortions.
2018-11-01 20:13:38 +01:00
meta = {
maintainers = with maintainers; [ aszlig ];
# https://github.com/NixOS/hydra/issues/591#issuecomment-435125621
inherit (chromiumPkg.meta) timeout;
all tests: added meta.maintainers section
2015-07-12 12:09:40 +02:00
};
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
nixos/tests/chromium: Detect popup using OCR. This will make the test a lot more reliable, because we no longer need to press ESC multiple times hoping that it will close the popup. Unfortunately in order to run this test I needed to locally revert the gyp update from a305e6855dd8723683c77635f45ae28411c8f36c. With the old gyp version however the test runs fine and it's able to properly detect the popup. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-05-22 11:23:09 +02:00
enableOCR = true;
nixos/tests/chromium: Run tests as normal user The tests have failed because Chromium has started up displaying the following error message in a dialog window: Chromium can not be run as root. Please start Chromium as a normal user. If you need to run as root for development, rerun with the --no-sandbox flag. So let's run as user "alice" and pass all commands using the small helper function "ru" (to keep it short, it's for "Run as User"). Tested it by running the "stable" test on x86_64-linux. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Reported-by: @globin
2017-02-07 07:36:56 +01:00
machine.imports = [ ./common/user-account.nix ./common/x11.nix ];
Grmbl http://hydra.nixos.org/build/24983009
2015-08-25 11:26:32 +02:00
machine.virtualisation.memorySize = 2047;
nixos/tests/chromium: Run tests as normal user The tests have failed because Chromium has started up displaying the following error message in a dialog window: Chromium can not be run as root. Please start Chromium as a normal user. If you need to run as root for development, rerun with the --no-sandbox flag. So let's run as user "alice" and pass all commands using the small helper function "ru" (to keep it short, it's for "Run as User"). Tested it by running the "stable" test on x86_64-linux. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Reported-by: @globin
2017-02-07 07:36:56 +01:00
machine.services.xserver.displayManager.auto.user = "alice";
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 00:20:27 +01:00
machine.environment.systemPackages = [ chromiumPkg ];
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
startupHTML = pkgs.writeText "chromium-startup.html" ''
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Chromium startup notifier</title>
</head>
<body onload="javascript:document.title='startup done'">
<img src="file://${pkgs.fetchurl {
tests-chromium: fix link to svg file closes #11208
2015-11-29 22:43:28 +01:00
url = "http://nixos.org/logo/nixos-hex.svg";
sha256 = "0wxpp65npdw2cg8m0cxc9qff1sb3b478cxpg1741d8951g948rg8";
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
}}" />
</body>
</html>
'';
testScript = let
xdo = name: text: let
xdoScript = pkgs.writeText "${name}.xdo" text;
in "${pkgs.xdotool}/bin/xdotool '${xdoScript}'";
in ''
nixos/tests/chromium: Run tests as normal user The tests have failed because Chromium has started up displaying the following error message in a dialog window: Chromium can not be run as root. Please start Chromium as a normal user. If you need to run as root for development, rerun with the --no-sandbox flag. So let's run as user "alice" and pass all commands using the small helper function "ru" (to keep it short, it's for "Run as User"). Tested it by running the "stable" test on x86_64-linux. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Reported-by: @globin
2017-02-07 07:36:56 +01:00
# Run as user alice
sub ru ($) {
my $esc = $_[0] =~ s/'/'\\${"'"}'/gr;
return "su - alice -c '$esc'";
}
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
sub createNewWin {
$machine->nest("creating a new Chromium window", sub {
nixos/tests/chromium: Run tests as normal user The tests have failed because Chromium has started up displaying the following error message in a dialog window: Chromium can not be run as root. Please start Chromium as a normal user. If you need to run as root for development, rerun with the --no-sandbox flag. So let's run as user "alice" and pass all commands using the small helper function "ru" (to keep it short, it's for "Run as User"). Tested it by running the "stable" test on x86_64-linux. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Reported-by: @globin
2017-02-07 07:36:56 +01:00
$machine->execute(ru "${xdo "new-window" ''
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
search --onlyvisible --name "startup done"
windowfocus --sync
windowactivate --sync
xdotool: 2.20110530.1 -> 3.20150503.1
2015-09-12 12:12:57 +02:00
''}");
nixos/tests/chromium: Run tests as normal user The tests have failed because Chromium has started up displaying the following error message in a dialog window: Chromium can not be run as root. Please start Chromium as a normal user. If you need to run as root for development, rerun with the --no-sandbox flag. So let's run as user "alice" and pass all commands using the small helper function "ru" (to keep it short, it's for "Run as User"). Tested it by running the "stable" test on x86_64-linux. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Reported-by: @globin
2017-02-07 07:36:56 +01:00
$machine->execute(ru "${xdo "new-window" ''
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
key Ctrl+n
''}");
});
}
sub closeWin {
Machine::retry sub {
nixos/tests/chromium: Run tests as normal user The tests have failed because Chromium has started up displaying the following error message in a dialog window: Chromium can not be run as root. Please start Chromium as a normal user. If you need to run as root for development, rerun with the --no-sandbox flag. So let's run as user "alice" and pass all commands using the small helper function "ru" (to keep it short, it's for "Run as User"). Tested it by running the "stable" test on x86_64-linux. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Reported-by: @globin
2017-02-07 07:36:56 +01:00
$machine->execute(ru "${xdo "close-window" ''
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
search --onlyvisible --name "new tab"
windowfocus --sync
windowactivate --sync
xdotool: 2.20110530.1 -> 3.20150503.1
2015-09-12 12:12:57 +02:00
''}");
nixos/tests/chromium: Run tests as normal user The tests have failed because Chromium has started up displaying the following error message in a dialog window: Chromium can not be run as root. Please start Chromium as a normal user. If you need to run as root for development, rerun with the --no-sandbox flag. So let's run as user "alice" and pass all commands using the small helper function "ru" (to keep it short, it's for "Run as User"). Tested it by running the "stable" test on x86_64-linux. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Reported-by: @globin
2017-02-07 07:36:56 +01:00
$machine->execute(ru "${xdo "close-window" ''
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
key Ctrl+w
''}");
for (1..20) {
nixos/tests/chromium: Run tests as normal user The tests have failed because Chromium has started up displaying the following error message in a dialog window: Chromium can not be run as root. Please start Chromium as a normal user. If you need to run as root for development, rerun with the --no-sandbox flag. So let's run as user "alice" and pass all commands using the small helper function "ru" (to keep it short, it's for "Run as User"). Tested it by running the "stable" test on x86_64-linux. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Reported-by: @globin
2017-02-07 07:36:56 +01:00
my ($status, $out) = $machine->execute(ru "${xdo "wait-for-close" ''
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
search --onlyvisible --name "new tab"
''}");
return 1 if $status != 0;
$machine->sleep(1);
}
}
}
sub waitForNewWin {
my $ret = 0;
$machine->nest("waiting for new Chromium window to appear", sub {
for (1..20) {
nixos/tests/chromium: Run tests as normal user The tests have failed because Chromium has started up displaying the following error message in a dialog window: Chromium can not be run as root. Please start Chromium as a normal user. If you need to run as root for development, rerun with the --no-sandbox flag. So let's run as user "alice" and pass all commands using the small helper function "ru" (to keep it short, it's for "Run as User"). Tested it by running the "stable" test on x86_64-linux. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Reported-by: @globin
2017-02-07 07:36:56 +01:00
my ($status, $out) = $machine->execute(ru "${xdo "wait-for-window" ''
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
search --onlyvisible --name "new tab"
windowfocus --sync
windowactivate --sync
''}");
if ($status == 0) {
$ret = 1;
nixos/tests/chromium: Wait 10s after new window This is a very very very ugly workaround and it's because Chromium seems to eat keystroke for a few seconds after a new window is created. I haven't found a better solution yet, so let's at least unbreak the test until we come up with a better way. Thanks to @vcunat for bringing this to my attention and also doing the initial bisect. The change that brought up this problem was 2b29e401531306d044f797a5dfa, which updated Chromium from version 65.0.3325.181 to version 66.0.3359.117. Unfortunately the upstream changelog[1] is way too large to actually guess what the breaking change is. [1]: https://chromium.googlesource.com/chromium/src/+log/65.0.3325.181..66.0.3359.117?pretty=fuller&n=10000 Signed-off-by: aszlig <aszlig@nix.build> Cc: @bendlas, @vcunat
2018-04-28 17:24:31 +02:00
# XXX: Somehow Chromium is not accepting keystrokes for a few
# seconds after a new window has appeared, so let's wait a while.
$machine->sleep(10);
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
last;
}
$machine->sleep(1);
}
});
return $ret;
}
sub createAndWaitForNewWin {
for (1..3) {
createNewWin;
return 1 if waitForNewWin;
}
die "new window didn't appear within 60 seconds";
}
sub testNewWin {
my ($desc, $code) = @_;
createAndWaitForNewWin;
subtest($desc, $code);
closeWin;
}
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 00:20:27 +01:00
$machine->waitForX;
my $url = "file://${startupHTML}";
nixos/tests/chromium: Run tests as normal user The tests have failed because Chromium has started up displaying the following error message in a dialog window: Chromium can not be run as root. Please start Chromium as a normal user. If you need to run as root for development, rerun with the --no-sandbox flag. So let's run as user "alice" and pass all commands using the small helper function "ru" (to keep it short, it's for "Run as User"). Tested it by running the "stable" test on x86_64-linux. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Reported-by: @globin
2017-02-07 07:36:56 +01:00
$machine->execute(ru "ulimit -c unlimited; chromium \"$url\" & disown");
nixos/tests/chromium: Fix popup detection Sometimes it happens that the "Type to search or enter a URL to navigate" popup doesn't show, but all we need to know at this time is whether Chromium has finished starting up. So checking for the "startup done" page is a better option here. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-11-09 09:41:10 +01:00
$machine->waitForText(qr/startup done/);
nixos/tests/chromium: Run tests as normal user The tests have failed because Chromium has started up displaying the following error message in a dialog window: Chromium can not be run as root. Please start Chromium as a normal user. If you need to run as root for development, rerun with the --no-sandbox flag. So let's run as user "alice" and pass all commands using the small helper function "ru" (to keep it short, it's for "Run as User"). Tested it by running the "stable" test on x86_64-linux. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Reported-by: @globin
2017-02-07 07:36:56 +01:00
$machine->waitUntilSucceeds(ru "${xdo "check-startup" ''
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 00:20:27 +01:00
search --sync --onlyvisible --name "startup done"
# close first start help popup
key -delay 1000 Escape
windowfocus --sync
windowactivate --sync
''}");
createAndWaitForNewWin;
$machine->screenshot("empty_windows");
closeWin;
$machine->screenshot("startup_done");
testNewWin "check sandbox", sub {
nixos/tests/chromium: Run tests as normal user The tests have failed because Chromium has started up displaying the following error message in a dialog window: Chromium can not be run as root. Please start Chromium as a normal user. If you need to run as root for development, rerun with the --no-sandbox flag. So let's run as user "alice" and pass all commands using the small helper function "ru" (to keep it short, it's for "Run as User"). Tested it by running the "stable" test on x86_64-linux. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Reported-by: @globin
2017-02-07 07:36:56 +01:00
$machine->succeed(ru "${xdo "type-url" ''
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 00:20:27 +01:00
search --sync --onlyvisible --name "new tab"
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
windowfocus --sync
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 00:20:27 +01:00
type --delay 1000 "chrome://sandbox"
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
''}");
nixos/tests/chromium: Run tests as normal user The tests have failed because Chromium has started up displaying the following error message in a dialog window: Chromium can not be run as root. Please start Chromium as a normal user. If you need to run as root for development, rerun with the --no-sandbox flag. So let's run as user "alice" and pass all commands using the small helper function "ru" (to keep it short, it's for "Run as User"). Tested it by running the "stable" test on x86_64-linux. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Reported-by: @globin
2017-02-07 07:36:56 +01:00
$machine->succeed(ru "${xdo "submit-url" ''
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 00:20:27 +01:00
search --sync --onlyvisible --name "new tab"
windowfocus --sync
key --delay 1000 Return
''}");
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 00:20:27 +01:00
$machine->screenshot("sandbox_info");
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
nixos/tests.chromium: actually notice the tab crash Argh, debugging NixOS tests takes forever… (cherry picked from commit 1afb6e790a6a28a4fc5f310a2b0e5e012eb416fd)
2018-04-04 12:17:19 +02:00
$machine->succeed(ru "${xdo "find-window" ''
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 00:20:27 +01:00
search --sync --onlyvisible --name "sandbox status"
windowfocus --sync
''}");
nixos/tests.chromium: actually notice the tab crash Argh, debugging NixOS tests takes forever… (cherry picked from commit 1afb6e790a6a28a4fc5f310a2b0e5e012eb416fd)
2018-04-04 12:17:19 +02:00
$machine->succeed(ru "${xdo "copy-sandbox-info" ''
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 00:20:27 +01:00
key --delay 1000 Ctrl+a Ctrl+c
''}");
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
nixos/tests/chromium: Run tests as normal user The tests have failed because Chromium has started up displaying the following error message in a dialog window: Chromium can not be run as root. Please start Chromium as a normal user. If you need to run as root for development, rerun with the --no-sandbox flag. So let's run as user "alice" and pass all commands using the small helper function "ru" (to keep it short, it's for "Run as User"). Tested it by running the "stable" test on x86_64-linux. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Reported-by: @globin
2017-02-07 07:36:56 +01:00
my $clipboard = $machine->succeed(ru "${pkgs.xclip}/bin/xclip -o");
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 00:20:27 +01:00
die "sandbox not working properly: $clipboard"
nixos/tests/chromium: Fix sandbox info matching As reported by @andir, the regular expressions that match the sandbox output are no longer matching in the recent Chromium bump as of bb03fbc2c8b97ef5a4f4205eee51029155164b8a. Instead of a boolean field that determines whether namespace sandboxes are on, the namespace sandbox is now an enum within "Layer 1 Sandbox". I've modified the regular expressions accordingly and also ran the test for the stable branch, which now succeeds. Signed-off-by: aszlig <aszlig@nix.build> Issue: https://github.com/NixOS/nixpkgs/issues/49442 Cc: @bendlas, @andir
2018-11-02 10:18:51 +01:00
unless $clipboard =~ /layer 1 sandbox.*namespace/mi
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 00:20:27 +01:00
&& $clipboard =~ /pid namespaces.*yes/mi
&& $clipboard =~ /network namespaces.*yes/mi
&& $clipboard =~ /seccomp.*sandbox.*yes/mi
&& $clipboard =~ /you are adequately sandboxed/mi;
nixos/tests.chromium: actually notice the tab crash Argh, debugging NixOS tests takes forever… (cherry picked from commit 1afb6e790a6a28a4fc5f310a2b0e5e012eb416fd)
2018-04-04 12:17:19 +02:00
$machine->sleep(1);
$machine->succeed(ru "${xdo "find-window-after-copy" ''
search --onlyvisible --name "sandbox status"
''}");
my $clipboard = $machine->succeed(ru "echo void | ${pkgs.xclip}/bin/xclip -i");
$machine->succeed(ru "${xdo "copy-sandbox-info" ''
key --delay 1000 Ctrl+a Ctrl+c
''}");
my $clipboard = $machine->succeed(ru "${pkgs.xclip}/bin/xclip -o");
die "copying twice in a row does not work properly: $clipboard"
nixos/tests/chromium: Fix sandbox info matching As reported by @andir, the regular expressions that match the sandbox output are no longer matching in the recent Chromium bump as of bb03fbc2c8b97ef5a4f4205eee51029155164b8a. Instead of a boolean field that determines whether namespace sandboxes are on, the namespace sandbox is now an enum within "Layer 1 Sandbox". I've modified the regular expressions accordingly and also ran the test for the stable branch, which now succeeds. Signed-off-by: aszlig <aszlig@nix.build> Issue: https://github.com/NixOS/nixpkgs/issues/49442 Cc: @bendlas, @andir
2018-11-02 10:18:51 +01:00
unless $clipboard =~ /layer 1 sandbox.*namespace/mi
nixos/tests.chromium: actually notice the tab crash Argh, debugging NixOS tests takes forever… (cherry picked from commit 1afb6e790a6a28a4fc5f310a2b0e5e012eb416fd)
2018-04-04 12:17:19 +02:00
&& $clipboard =~ /pid namespaces.*yes/mi
&& $clipboard =~ /network namespaces.*yes/mi
&& $clipboard =~ /seccomp.*sandbox.*yes/mi
&& $clipboard =~ /you are adequately sandboxed/mi;
$machine->screenshot("afer_copy_from_chromium");
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 00:20:27 +01:00
};
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 00:20:27 +01:00
$machine->shutdown;
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
'';
nixos/tests/chromium: Allow overriding channel map This has been the case before e45c211, but it turns out that it's very useful to override the channel packages so we can run tests with different Chromium build options. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-20 22:20:39 +01:00
}) channelMap
Reference in New Issue Copy Permalink