In 5500dc8 we introduced the --keep-vm-state flag and defaulted to that flag not being set. This lead to the `runInMachine` tests not longer working and that going unnoticed for quite some time now.