Flake input `git+file:./${submodule}` no longer works #9708

bryango · 2024-01-08T15:21:33Z

Describe the bug

Flake input of a git submodule in a relative subdirectory git+file:./${submodule}

works on 2.18.1 (nixpkgs.nixVersions.nix_2_18)
but no longer works on 2.19.2 (nixpkgs.nixVersions.nix_2_19)

Update: it seems that the regression happens since #9061. An example of this flake ref:

https://github.com/bryango/cheznix/blob/c445c8c82697de2bde53c65e9acacb9790fcd54e/flake.nix#L20-L23

The breakage would be acceptable to me if there is any alternative syntax that works, but currently I could not find anything that can replace git+file:./${submodule}.

Steps To Reproduce

$ nix --version
nix (Nix) 2.19.2

## suppose we have `inputs.nixpkgs.url = git+file:./nixpkgs;`
$ nix flake update nixpkgs
fatal: '/nixpkgs' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
warning: could not read HEAD ref from repo at 'file://file:./nixpkgs', using 'master'
fatal: '/nixpkgs' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
error:
       … while updating the lock file of flake 'git+file:///home/$USER/.config/home-manager?ref=refs/heads/master&rev=c445c8c82697de2bde53c65e9acacb9790fcd54e'

       … while updating the flake input 'nixpkgs'

       … while fetching the input 'git+file://file:./nixpkgs'

       error: program 'git' failed with exit code 128

Expected behavior

I believe submodule sub-flake is quite common and git+file:./${submodule} should still work.

nix-env --version output

evaluating file '/derivation-internal.nix'
nix-env (Nix) 2.19.2
System type: x86_64-linux
Additional system types: i686-linux, x86_64-v1-linux, x86_64-v2-linux, x86_64-v3-linux
Features: gc, signed-caches
System configuration file: /etc/nix/nix.conf
User configuration files: /home/$USER/.config/nix/nix.conf:/etc/xdg/nix/nix.conf
Store directory: /nix/store
State directory: /nix/var/nix
Data directory: /nix/store/0k52aibl2flync375nfw9a0kdjzvrzrc-nix-2.19.2/share

Additional context

N/A

Priorities

Add 👍 to issues you find important.

The text was updated successfully, but these errors were encountered:

roberth · 2024-01-08T23:32:29Z

In the upcoming release 2.20, the git fetcher is overhauled, with improved submodule support, but the submodule flag for the fetcher is not enabled by default.
@edolstra I vaguely recall that we'd enable submodules by default in fetchTree. Did we decide that?
Otherwise, we could solve this flake problem by having a flake meta-attribute and "re-fetch" themselves with changed flags such as submodules = true;.

bryango · 2024-01-09T02:09:02Z

I tried the latest static binary from hydra which is built from 8e865f3, and it has a similar regression:

$ latestStatic=/nix/store/x1vkrxsxakk9wj7x0sydwifqa22jvqpx-nix-static-x86_64-unknown-linux-musl-2.20.0pre20240106_8e865f3
$ nix copy --from https://cache.nixos.org "$latestStatic"
$ nix shell "$latestStatic" ## below is from within the nix shell

$ nix --version
nix (Nix) 2.20.0pre20240106_8e865f3

$ nix flake update nixpkgs
fatal: '/nixpkgs' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
warning: could not read HEAD ref from repo at 'file://file:./nixpkgs', using 'master'
error:
       … while updating the lock file of flake 'git+file:///home/$USER/.config/home-manager?ref=refs/heads/master&rev=c445c8c82697de2bde53c65e9acacb9790fcd54e'

       … while updating the flake input 'nixpkgs'

       … while fetching the input 'git+file://file:./nixpkgs'

       error: fetching 'refs/heads/master:refs/heads/master' from 'file://file:./nixpkgs': failed to resolve path 'file://file:./nixpkgs': No such file or directory

bryango · 2024-01-10T05:00:02Z

I tried a bisect using hydra static builds, and it seems that the breaking change is made:

strictly after: https://hydra.nixos.org/build/237666981
and before: https://hydra.nixos.org/build/238054861

... which narrows down the commits to this range: 301623f...d070d8b. Looks like it is indeed related to fetchTree.

Update: I think it is precisely:

fetchTree cleanup #9061

Update: I found that reverting these lines fixes the issue:

856fe13#diff-4de86ddf9ae3dd43a49cc14cc56e2f6b29f51fde7a0603e3ab7c5cd34599b5d7L296-L308

But I need to understand why. Then I can make a PR to try to fix it.

simonzkl · 2024-02-05T17:57:58Z

Before I open a new issue, I also noticed that this broke:

nix build '.?ref=my-branch#my-package'

fatal: ambiguous argument 'my-branch#m': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
error:
       … while fetching the input 'git+file:///home/foo/project?ref=my-branch%23m'

       error: program 'git' failed with exit code 128

Is this the same issue by any chance?

It's possible to work around this issue with:

nix build '.?ref=my-branch&#my-package'

bryango · 2024-02-06T06:46:54Z

nix build '.?ref=my-branch#my-package'

fatal: ambiguous argument 'my-branch#m': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
error:
       … while fetching the input 'git+file:///home/foo/project?ref=my-branch%23m'

       error: program 'git' failed with exit code 128

Is this the same issue by any chance?

I think this is a different issue, though it is also related to flake URI parsing.

bryango · 2024-02-06T06:52:23Z

@roberth @fricklerhandwerk would you be interested in taking a look at #9897? 🥺 It is a super simple PR which fixes the issue for the 2.19 series (but not 2.20 yet, due to new complications from submodules & switching to libgit2).

DamienCassou · 2024-11-11T17:11:05Z

I'm using 2.24.10 and I have a working flake.nix file looking like:

{
  inputs = {
    emacs-overlay.url = "git+file:./emacs-overlay?shallow=1";
    ...
  };
}

This seems to work fine. Should this issue be closed?

bryango · 2024-12-23T07:51:41Z

Ha, this broke again between 2.24.10...2.24.11

$ /nix/store/nmpx2d5vlypzbc62rzcbplyaygxq6kk6-nix-2.24.10/bin/nix flake update submodule
• Updated input 'submodule':
    'git+file:./submodule?ref=refs/heads/dev&rev=62315f886eed0edf1b711afd2ae1a89afdbe2073' (2024-12-21)
  → 'git+file:./submodule?ref=refs/heads/dev&rev=4d0e38dce0405c5990df95b560ca58b1d81e1a2d' (2024-12-22)

$ nix run nixpkgs\#nixVersions.nix_2_24 -- --version
nix (Nix) 2.24.11

$ nix run nixpkgs\#nixVersions.nix_2_24 -- flake update submodule
nix: src/libutil/posix-source-accessor.cc:13: nix::PosixSourceAccessor::PosixSourceAccessor(std::filesystem::__cxx11::path&&): Assertion `root.empty() || root.is_absolute()' failed.
[1]    2741534 IOT instruction (core dumped)  nix run nixpkgs\#nixVersions.nix_2_24 -- flake update submodule

Update: I don't understand how this happened through the diff. Could it be c556c20 ?

DamienCassou · 2024-12-23T10:39:56Z

I confirm the issue

bryango · 2024-12-23T12:06:19Z

Indeed it's something between these two hydra builds:

... which translates to: 6e095dd...ef21dfa. Basically, the culprit is indeed c556c20, which is backported from #11837.

However, it appears that #11837 is a bugfix commit. So perhaps we have been depending on this bug for git+file:./${submodule} to work 😆

nixos-discourse · 2024-12-23T16:08:20Z

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-12-23-nix-team-meeting-minutes-205/57783/1

NixOS/nix#9708 (comment)

ipimiutil broken in unstable as of 03539677da5a6caba6a6b3d2143ec63b116902d3 And on top of that `nix flake update` is crashing with ``` nix: src/libutil/posix-source-accessor.cc:13: nix::PosixSourceAccessor::PosixSourceAccessor(std::filesystem::__cxx11::path&&): Assertion `root.empty() || root.is_absolute()' failed. Aborted (core dumped) ``` I might need to try an older or newer version... Update / note to self: It worked with nix from nixpkgs unstable at c77dab3ae5c71a5eaef539968ebfb13dce79b0ea I eventually found NixOS/nix#9708 which was for a much older issue is being used to track the new regression. Nixpkgs downgraded back to Nix 2.24.10 in DamienCassou/nixpkgs@0bb2579 so I'll probably just use that.

bryango · 2024-12-27T13:31:31Z

I just submitted a fix! Please see #12107.

bryango added the bug label Jan 8, 2024

bryango changed the title ~~Flake input git+file:./${submodule} works on 2.18.1 but no longer works on 2.19.2~~ Flake input git+file:./${submodule} works on 2.18.1 but no longer works on 2.19.2 [regression] Jan 8, 2024

roberth added regression Something doesn't work anymore fetching Networking with the outside (non-Nix) world, input locking labels Jan 8, 2024

roberth added the flakes label Jan 8, 2024

bryango mentioned this issue Jan 10, 2024

fetchTree cleanup #9061

Merged

bryango changed the title ~~Flake input git+file:./${submodule} works on 2.18.1 but no longer works on 2.19.2 [regression]~~ Flake input git+file:./${submodule} no longer works since #9061 [regression] Jan 11, 2024

fricklerhandwerk changed the title ~~Flake input git+file:./${submodule} no longer works since #9061 [regression]~~ Flake input git+file:./${submodule} no longer works Jan 15, 2024

bryango mentioned this issue Feb 1, 2024

libutil/url: fix git+file:./ parse error #9897

Merged

thufschmitt assigned tomberek Feb 16, 2024

thufschmitt added this to the Flakes milestone Feb 28, 2024

tomberek closed this as completed Nov 12, 2024

tomberek reopened this Dec 23, 2024

tomberek added this to Nix team Dec 23, 2024

github-project-automation bot moved this to To triage in Nix team Dec 23, 2024

bryango linked a pull request Dec 26, 2024 that will close this issue

fetchers/git: make relative path absolute for local repo #12107

Open

DamienCassou added a commit to DamienCassou/nixpkgs that referenced this issue Dec 26, 2024

Downgrade nix

0bb2579

NixOS/nix#9708 (comment)

Mic92 moved this from To triage to 🏁 Review in Nix team Jan 8, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Flake input `git+file:./${submodule}` no longer works #9708

Flake input `git+file:./${submodule}` no longer works #9708

bryango commented Jan 8, 2024 •

edited

Loading

roberth commented Jan 8, 2024

bryango commented Jan 9, 2024 •

edited

Loading

bryango commented Jan 10, 2024 •

edited

Loading

simonzkl commented Feb 5, 2024 •

edited

Loading

bryango commented Feb 6, 2024

bryango commented Feb 6, 2024 •

edited

Loading

DamienCassou commented Nov 11, 2024

bryango commented Dec 23, 2024 •

edited

Loading

DamienCassou commented Dec 23, 2024

bryango commented Dec 23, 2024

nixos-discourse commented Dec 23, 2024

bryango commented Dec 27, 2024

Flake input git+file:./${submodule} no longer works #9708

Flake input git+file:./${submodule} no longer works #9708

Comments

bryango commented Jan 8, 2024 • edited Loading

roberth commented Jan 8, 2024

bryango commented Jan 9, 2024 • edited Loading

bryango commented Jan 10, 2024 • edited Loading

simonzkl commented Feb 5, 2024 • edited Loading

bryango commented Feb 6, 2024

bryango commented Feb 6, 2024 • edited Loading

DamienCassou commented Nov 11, 2024

bryango commented Dec 23, 2024 • edited Loading

DamienCassou commented Dec 23, 2024

bryango commented Dec 23, 2024

nixos-discourse commented Dec 23, 2024

bryango commented Dec 27, 2024

Flake input `git+file:./${submodule}` no longer works #9708

Flake input `git+file:./${submodule}` no longer works #9708

bryango commented Jan 8, 2024 •

edited

Loading

bryango commented Jan 9, 2024 •

edited

Loading

bryango commented Jan 10, 2024 •

edited

Loading

simonzkl commented Feb 5, 2024 •

edited

Loading

bryango commented Feb 6, 2024 •

edited

Loading

bryango commented Dec 23, 2024 •

edited

Loading