If empty directory isn't deleted, referer depenedencies will
fail with:
cp: missing destination file operand after '/tmp/nix-build-cabal-helper-0.8.0.2.drv-0/setup-package.conf.d/'
This is currently only the case for cabal-install, as cabal2nix
doesn't handle well buildable=False flags due to long-standing bugs
in Cabal itself.
6fd5287c36