public/nixpkgs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
nixpkgs/nixos/tests/chromium.nix

258 lines
7.5 KiB
Nix
Raw Normal View History

nixos/tests/chromium: Allow overriding channel map This has been the case before e45c211, but it turns out that it's very useful to override the channel packages so we can run tests with different Chromium build options. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-20 22:20:39 +01:00
{ system ? builtins.currentSystem
tests: refactor to carry the package set as an argument This way, the package set will be possible to pass without re-importing all the time
2018-11-11 17:41:11 +09:00
, config ? {}
, pkgs ? import ../.. { inherit system config; }
nixos/tests/chromium: Add ungoogled-chromium and Google Chrome By default only `chromium` will be tested but other "channels" can be selected using e.g.: nix-build nixos/tests/chromium.nix -A ungoogled This also adds me as secondary maintainer (I'd like to get notified on PRs/issues and can review them).
2021-01-18 15:16:27 +01:00
, channelMap ? { # Maps "channels" to packages
stable = pkgs.chromium;
beta = pkgs.chromiumBeta;
dev = pkgs.chromiumDev;
ungoogled = pkgs.ungoogled-chromium;
chrome-stable = pkgs.google-chrome;
chrome-beta = pkgs.google-chrome-beta;
chrome-dev = pkgs.google-chrome-dev;
nixos/tests/chromium: Allow overriding channel map This has been the case before e45c211, but it turns out that it's very useful to override the channel packages so we can run tests with different Chromium build options. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-20 22:20:39 +01:00
}
}:
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 00:20:27 +01:00
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
with import ../lib/testing-python.nix { inherit system pkgs; };
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 00:20:27 +01:00
with pkgs.lib;
mapAttrs (channel: chromiumPkg: makeTest rec {
name = "chromium-${channel}";
chromium tests: inherit timeout from the package /cc #49442. It should decrease the waste of resources due to abortions.
2018-11-01 20:13:38 +01:00
meta = {
nixos/tests/chromium: Add ungoogled-chromium and Google Chrome By default only `chromium` will be tested but other "channels" can be selected using e.g.: nix-build nixos/tests/chromium.nix -A ungoogled This also adds me as secondary maintainer (I'd like to get notified on PRs/issues and can review them).
2021-01-18 15:16:27 +01:00
maintainers = with maintainers; [ aszlig primeos ];
chromium tests: inherit timeout from the package /cc #49442. It should decrease the waste of resources due to abortions.
2018-11-01 20:13:38 +01:00
# https://github.com/NixOS/hydra/issues/591#issuecomment-435125621
inherit (chromiumPkg.meta) timeout;
all tests: added meta.maintainers section
2015-07-12 12:09:40 +02:00
};
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
nixos/tests/chromium: Detect popup using OCR. This will make the test a lot more reliable, because we no longer need to press ESC multiple times hoping that it will close the popup. Unfortunately in order to run this test I needed to locally revert the gyp update from a305e6855dd8723683c77635f45ae28411c8f36c. With the old gyp version however the test runs fine and it's able to properly detect the popup. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-05-22 11:23:09 +02:00
enableOCR = true;
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
user = "alice";
nixos/tests/chromium: Run tests as normal user The tests have failed because Chromium has started up displaying the following error message in a dialog window: Chromium can not be run as root. Please start Chromium as a normal user. If you need to run as root for development, rerun with the --no-sandbox flag. So let's run as user "alice" and pass all commands using the small helper function "ru" (to keep it short, it's for "Run as User"). Tested it by running the "stable" test on x86_64-linux. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Reported-by: @globin
2017-02-07 07:36:56 +01:00
machine.imports = [ ./common/user-account.nix ./common/x11.nix ];
Grmbl http://hydra.nixos.org/build/24983009
2015-08-25 11:26:32 +02:00
machine.virtualisation.memorySize = 2047;
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
machine.test-support.displayManager.auto.user = user;
nixos/tests/chromium: Split up into subtests This makes it easier to test just a specific channel rather than to force testing all builds down the users/testers throat. Especially this makes it easier to test NixOS channel upgrades only against the Chromium stable channel instead of just removing the beta/dev channels from the tests entirely (as done in 69ec09f38aa1f1d37baec73ebdf9cf5f21050f94). Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 00:20:27 +01:00
machine.environment.systemPackages = [ chromiumPkg ];
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
startupHTML = pkgs.writeText "chromium-startup.html" ''
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Chromium startup notifier</title>
</head>
<body onload="javascript:document.title='startup done'">
<img src="file://${pkgs.fetchurl {
treewide: use https for nixos.org and hydra.nixos.org tarballs.nixos.org is omitted from the change because urls from there are always hashed and checked
2020-04-18 22:51:19 +02:00
url = "https://nixos.org/logo/nixos-hex.svg";
nixos/tests/chromium: Fix nixos-hex.svg hash
2020-01-07 19:42:52 +01:00
sha256 = "07ymq6nw8kc22m7kzxjxldhiq8gzmc7f45kq2bvhbdm0w5s112s4";
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
}}" />
</body>
</html>
'';
testScript = let
xdo = name: text: let
xdoScript = pkgs.writeText "${name}.xdo" text;
nixos/tests/chromium: Simplify the logic (#110715) - Improve the documentation (Python docstrings for functions and more meaningful xdotool script names). - Make more use of the existing methods (e.g. wait_until_succeeds(), and send_key()) - Note: This increases timeouts from 60 seconds to (currently) 15 minutes but the defaults from test-driver.py should be fine. This should make it simpler to read the code, understand the test output, and drop the custom xdotool scripts in the future.
2021-01-24 20:35:29 +01:00
in "${pkgs.xdotool}/bin/xdotool ${xdoScript}";
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
in ''
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
import shlex
from contextlib import contextmanager, _GeneratorContextManager
nixos/tests/chromium: Run tests as normal user The tests have failed because Chromium has started up displaying the following error message in a dialog window: Chromium can not be run as root. Please start Chromium as a normal user. If you need to run as root for development, rerun with the --no-sandbox flag. So let's run as user "alice" and pass all commands using the small helper function "ru" (to keep it short, it's for "Run as User"). Tested it by running the "stable" test on x86_64-linux. Signed-off-by: aszlig <aszlig@redmoonstudios.org> Reported-by: @globin
2017-02-07 07:36:56 +01:00
# Run as user alice
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
def ru(cmd):
return "su - ${user} -c " + shlex.quote(cmd)
nixos/tests/chromium: Add ungoogled-chromium and Google Chrome By default only `chromium` will be tested but other "channels" can be selected using e.g.: nix-build nixos/tests/chromium.nix -A ungoogled This also adds me as secondary maintainer (I'd like to get notified on PRs/issues and can review them).
2021-01-18 15:16:27 +01:00
def get_browser_binary():
"""Returns the name of the browser binary."""
pname = "${getName chromiumPkg.name}"
if pname.find("chromium") != -1:
return "chromium" # Same name for all channels and ungoogled-chromium
if pname == "google-chrome":
return "google-chrome-stable"
if pname == "google-chrome-dev":
return "google-chrome-unstable"
# For google-chrome-beta and as fallback:
return pname
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
def create_new_win():
nixos/tests/chromium: Simplify the logic (#110715) - Improve the documentation (Python docstrings for functions and more meaningful xdotool script names). - Make more use of the existing methods (e.g. wait_until_succeeds(), and send_key()) - Note: This increases timeouts from 60 seconds to (currently) 15 minutes but the defaults from test-driver.py should be fine. This should make it simpler to read the code, understand the test output, and drop the custom xdotool scripts in the future.
2021-01-24 20:35:29 +01:00
"""Creates a new Chromium window."""
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
with machine.nested("Creating a new Chromium window"):
nixos/tests/chromium: Simplify the logic (#110715) - Improve the documentation (Python docstrings for functions and more meaningful xdotool script names). - Make more use of the existing methods (e.g. wait_until_succeeds(), and send_key()) - Note: This increases timeouts from 60 seconds to (currently) 15 minutes but the defaults from test-driver.py should be fine. This should make it simpler to read the code, understand the test output, and drop the custom xdotool scripts in the future.
2021-01-24 20:35:29 +01:00
machine.wait_until_succeeds(
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
ru(
nixos/tests/chromium: Simplify the logic (#110715) - Improve the documentation (Python docstrings for functions and more meaningful xdotool script names). - Make more use of the existing methods (e.g. wait_until_succeeds(), and send_key()) - Note: This increases timeouts from 60 seconds to (currently) 15 minutes but the defaults from test-driver.py should be fine. This should make it simpler to read the code, understand the test output, and drop the custom xdotool scripts in the future.
2021-01-24 20:35:29 +01:00
"${xdo "create_new_win-select_main_window" ''
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
search --onlyvisible --name "startup done"
windowfocus --sync
windowactivate --sync
''}"
)
)
nixos/tests/chromium: Simplify the logic (#110715) - Improve the documentation (Python docstrings for functions and more meaningful xdotool script names). - Make more use of the existing methods (e.g. wait_until_succeeds(), and send_key()) - Note: This increases timeouts from 60 seconds to (currently) 15 minutes but the defaults from test-driver.py should be fine. This should make it simpler to read the code, understand the test output, and drop the custom xdotool scripts in the future.
2021-01-24 20:35:29 +01:00
machine.send_key("ctrl-n")
# Wait until the new window appears:
machine.wait_until_succeeds(
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
ru(
nixos/tests/chromium: Simplify the logic (#110715) - Improve the documentation (Python docstrings for functions and more meaningful xdotool script names). - Make more use of the existing methods (e.g. wait_until_succeeds(), and send_key()) - Note: This increases timeouts from 60 seconds to (currently) 15 minutes but the defaults from test-driver.py should be fine. This should make it simpler to read the code, understand the test output, and drop the custom xdotool scripts in the future.
2021-01-24 20:35:29 +01:00
"${xdo "create_new_win-wait_for_window" ''
search --onlyvisible --name "New Tab"
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
windowfocus --sync
windowactivate --sync
''}"
)
)
nixos/tests/chromium: Simplify the logic (#110715) - Improve the documentation (Python docstrings for functions and more meaningful xdotool script names). - Make more use of the existing methods (e.g. wait_until_succeeds(), and send_key()) - Note: This increases timeouts from 60 seconds to (currently) 15 minutes but the defaults from test-driver.py should be fine. This should make it simpler to read the code, understand the test output, and drop the custom xdotool scripts in the future.
2021-01-24 20:35:29 +01:00
def close_new_tab_win():
"""Closes the Chromium window with the title "New Tab"."""
machine.wait_until_succeeds(
ru(
"${xdo "close_new_tab_win-select_main_window" ''
search --onlyvisible --name "New Tab"
windowfocus --sync
windowactivate --sync
''}"
)
)
machine.send_key("ctrl-w")
# Wait until the closed window disappears:
machine.wait_until_fails(
ru(
"${xdo "close_new_tab_win-wait_for_close" ''
search --onlyvisible --name "New Tab"
''}"
)
)
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
@contextmanager
def test_new_win(description):
nixos/tests/chromium: Simplify the logic (#110715) - Improve the documentation (Python docstrings for functions and more meaningful xdotool script names). - Make more use of the existing methods (e.g. wait_until_succeeds(), and send_key()) - Note: This increases timeouts from 60 seconds to (currently) 15 minutes but the defaults from test-driver.py should be fine. This should make it simpler to read the code, understand the test output, and drop the custom xdotool scripts in the future.
2021-01-24 20:35:29 +01:00
create_new_win()
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
with machine.nested(description):
yield
nixos/tests/chromium: Simplify the logic (#110715) - Improve the documentation (Python docstrings for functions and more meaningful xdotool script names). - Make more use of the existing methods (e.g. wait_until_succeeds(), and send_key()) - Note: This increases timeouts from 60 seconds to (currently) 15 minutes but the defaults from test-driver.py should be fine. This should make it simpler to read the code, understand the test output, and drop the custom xdotool scripts in the future.
2021-01-24 20:35:29 +01:00
# Close the newly created window:
machine.send_key("ctrl-w")
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
machine.wait_for_x()
url = "file://${startupHTML}"
nixos/tests/chromium: Add ungoogled-chromium and Google Chrome By default only `chromium` will be tested but other "channels" can be selected using e.g.: nix-build nixos/tests/chromium.nix -A ungoogled This also adds me as secondary maintainer (I'd like to get notified on PRs/issues and can review them).
2021-01-18 15:16:27 +01:00
machine.succeed(ru(f'ulimit -c unlimited; "{get_browser_binary()}" "{url}" & disown'))
if get_browser_binary().startswith("google-chrome"):
# Need to click away the first window:
machine.wait_for_text("Make Google Chrome the default browser")
machine.screenshot("google_chrome_default_browser_prompt")
machine.send_key("ret")
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
machine.wait_for_text("startup done")
machine.wait_until_succeeds(
ru(
"${xdo "check-startup" ''
search --sync --onlyvisible --name "startup done"
# close first start help popup
key -delay 1000 Escape
windowfocus --sync
windowactivate --sync
''}"
)
)
nixos/tests/chromium: Simplify the logic (#110715) - Improve the documentation (Python docstrings for functions and more meaningful xdotool script names). - Make more use of the existing methods (e.g. wait_until_succeeds(), and send_key()) - Note: This increases timeouts from 60 seconds to (currently) 15 minutes but the defaults from test-driver.py should be fine. This should make it simpler to read the code, understand the test output, and drop the custom xdotool scripts in the future.
2021-01-24 20:35:29 +01:00
create_new_win()
# Optional: Wait for the new tab page to fully load before taking the screenshot:
machine.wait_for_text("Web Store")
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
machine.screenshot("empty_windows")
nixos/tests/chromium: Simplify the logic (#110715) - Improve the documentation (Python docstrings for functions and more meaningful xdotool script names). - Make more use of the existing methods (e.g. wait_until_succeeds(), and send_key()) - Note: This increases timeouts from 60 seconds to (currently) 15 minutes but the defaults from test-driver.py should be fine. This should make it simpler to read the code, understand the test output, and drop the custom xdotool scripts in the future.
2021-01-24 20:35:29 +01:00
close_new_tab_win()
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
machine.screenshot("startup_done")
with test_new_win("check sandbox"):
machine.succeed(
ru(
"${xdo "type-url" ''
nixos/tests/chromium: Simplify the logic (#110715) - Improve the documentation (Python docstrings for functions and more meaningful xdotool script names). - Make more use of the existing methods (e.g. wait_until_succeeds(), and send_key()) - Note: This increases timeouts from 60 seconds to (currently) 15 minutes but the defaults from test-driver.py should be fine. This should make it simpler to read the code, understand the test output, and drop the custom xdotool scripts in the future.
2021-01-24 20:35:29 +01:00
search --sync --onlyvisible --name "New Tab"
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
windowfocus --sync
type --delay 1000 "chrome://sandbox"
''}"
)
)
machine.succeed(
ru(
"${xdo "submit-url" ''
nixos/tests/chromium: Simplify the logic (#110715) - Improve the documentation (Python docstrings for functions and more meaningful xdotool script names). - Make more use of the existing methods (e.g. wait_until_succeeds(), and send_key()) - Note: This increases timeouts from 60 seconds to (currently) 15 minutes but the defaults from test-driver.py should be fine. This should make it simpler to read the code, understand the test output, and drop the custom xdotool scripts in the future.
2021-01-24 20:35:29 +01:00
search --sync --onlyvisible --name "New Tab"
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
windowfocus --sync
key --delay 1000 Return
''}"
)
)
machine.screenshot("sandbox_info")
machine.succeed(
ru(
"${xdo "find-window" ''
nixos/tests/chromium: Simplify the logic (#110715) - Improve the documentation (Python docstrings for functions and more meaningful xdotool script names). - Make more use of the existing methods (e.g. wait_until_succeeds(), and send_key()) - Note: This increases timeouts from 60 seconds to (currently) 15 minutes but the defaults from test-driver.py should be fine. This should make it simpler to read the code, understand the test output, and drop the custom xdotool scripts in the future.
2021-01-24 20:35:29 +01:00
search --sync --onlyvisible --name "Sandbox Status"
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
windowfocus --sync
''}"
)
)
machine.succeed(
ru(
"${xdo "copy-sandbox-info" ''
key --delay 1000 Ctrl+a Ctrl+c
''}"
)
)
clipboard = machine.succeed(
ru("${pkgs.xclip}/bin/xclip -o")
)
filters = [
"layer 1 sandbox.*namespace",
"pid namespaces.*yes",
"network namespaces.*yes",
"seccomp.*sandbox.*yes",
"you are adequately sandboxed",
]
if not all(
re.search(filter, clipboard, flags=re.DOTALL | re.IGNORECASE)
for filter in filters
):
assert False, f"sandbox not working properly: {clipboard}"
machine.sleep(1)
machine.succeed(
ru(
"${xdo "find-window-after-copy" ''
nixos/tests/chromium: Simplify the logic (#110715) - Improve the documentation (Python docstrings for functions and more meaningful xdotool script names). - Make more use of the existing methods (e.g. wait_until_succeeds(), and send_key()) - Note: This increases timeouts from 60 seconds to (currently) 15 minutes but the defaults from test-driver.py should be fine. This should make it simpler to read the code, understand the test output, and drop the custom xdotool scripts in the future.
2021-01-24 20:35:29 +01:00
search --onlyvisible --name "Sandbox Status"
nixosTests.chromium: Port to Python
2020-02-06 14:50:47 +01:00
''}"
)
)
clipboard = machine.succeed(
ru(
"echo void | ${pkgs.xclip}/bin/xclip -i"
)
)
machine.succeed(
ru(
"${xdo "copy-sandbox-info" ''
key --delay 1000 Ctrl+a Ctrl+c
''}"
)
)
clipboard = machine.succeed(
ru("${pkgs.xclip}/bin/xclip -o")
)
if not all(
re.search(filter, clipboard, flags=re.DOTALL | re.IGNORECASE)
for filter in filters
):
assert False, f"copying twice in a row does not work properly: {clipboard}"
machine.screenshot("after_copy_from_chromium")
machine.shutdown()
nixos: Add rudimentary VM tests for Chromium. Currently, the test is only for testing the user namespace sandbox and even that isn't very representative, because we're running the tests as root. But apart from that, we should have functionality for opening/closing windows and the main goal here is to get them as deterministic as possible, because Chromium usually isn't very nice to chained xdotool keystrokes. And of course, the most important "test" we have here: We know at least whether Chromium works _at_all_. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-18 18:41:56 +01:00
'';
nixos/tests/chromium: Allow overriding channel map This has been the case before e45c211, but it turns out that it's very useful to override the channel packages so we can run tests with different Chromium build options. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-20 22:20:39 +01:00
}) channelMap
Reference in New Issue Copy Permalink