When creating a stack, support relative paths for volumes when using a git repository

[huib-portainer]

Problem statement

I'm unable to use relative paths for volumes when deploying from Git.

As an example, I have static HTML content in a git repo, as well as a yaml to deploy nginx that wants to use that static HTML.

I have the following compose yaml in my git repo:

version: '3.1'
services:
  webapp:
    image: nginx:alpine
    restart: always
    ports:
      - "3002:80"
    volumes:
      - ./static:/usr/share/nginx/html

And I have the static HTML in a dir in git called static.

Deploy it:

Results in:
invalid mount config for type "bind": bind source path does not exist: /data/compose/56/stack/static

Which is referencing the Portainer volume.
And even if I'd mount the Portainer volume to my nginx container, I wouldn't know the compose id.

I’m using swarm, so the services are getting rejected

Click on the name of a task

At the moment I have to create a volume, put the static content in it in some other way and reference it as external:

version: '3.1'
services:
  webapp:
    image: nginx:alpine
    restart: always
    ports:
      - "3002:80"
    volumes:
      - html:/usr/share/nginx/html
volumes:
  html:
    ext

Workaround on Docker and NOT using an Agent

To make this work on Docker, use the local Portainer environment.
Use the --data flag and use a bind mount for Portainer as well, so that the volume gets shared:

docker run --rm -p 8000:8000 -p 9000:9000 -p 9443:9443 --name portainer \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /tmp/portainer-test-dir:/tmp/portainer-test-dir \
  portainer/portainer-ce:latest --data=/tmp/portainer-test-dir

So nginx has a bind mount with a relative path (which is on the docker host).
And it means that every container deployed via Portainer can access Portainer’s data

Deploying using an Agent

When using the Agent, we can't share the volume between the Portainer instance and where the stack is getting deployed.
(the bind mount for Portainer would be on a different host than the bind mount for the Agent)
And named volumes can't be shared between hosts either.
So that's currently not working.

Swarm

The stack in the first example above for nginx uses the relative bind mount of ./static.
However, that's not allowed for Swarm:

deploying to a non-local platform MUST reject Compose files which use relative host paths

https://github.com/compose-spec/compose-spec/blob/master/spec.md#volumes

Considered solutions

Use named volumes and clone directly into that:
Automatically clone the repo in each remote instance directly.
e.g. allow the user to specify the name of the volume in the UI when deploying a stack/app from git. Assuming the volume exist, we then clone to that volume; the yaml for the container can then reference that volume reliably.
This could have severe security implications; the Agent would need to obtain the Git credentials.
However, when not using an Agent, we can’t mount a new volume to a running Portainer instance, so that’s why the agent has the bind mount /var/lib/docker/volumes:/var/lib/docker/volumes.

Use named volumes and copy the files into them:
Automatically copy the cloned git files into a persistent volume.
Portainer clones Git, creates a persistent vol and copies the files into it. Then deploy the compose file with the persistence defined in the compose file repointed to the new populated volume.
This would require parsing the compose file to check any bind-mount paths in it for “relative-ness”, and then fix them.
(when using an Agent: after cloning the Git repo but before deploying, push the contents of the repo to the remote agent so that they are available there)
However, when not using an Agent, we can’t mount a new volume to a running Portainer instance, so that’s why the agent has the bind mount /var/lib/docker/volumes:/var/lib/docker/volumes.

Use a network share:
Make a network shared volume that all nodes can see, and then copy the files into it.

[huib-portainer]

Duplicates: #2368, #4571, #1647, #6243

[BobSilent]

I created a dirty workaround:
On the host system I created a link to the docker volume used by portainer

ln -s /var/lib/docker/volumes/portainer_data/_data /data

so that the path /data/compose/56/stack/static exists.

[m-rgba]

Not a perfect solution, but I'm currently working around this using a temporary service container to checkout our Git files and then mount to the target container.

Roughly something like:

version: '2'
services:

  # Service container
  # Purge any leftover files, clone repo, copy necessary files to the temporary volume
  # Note: you could also use a sparse checkout for this (size of clone, performance), but I didn't since my use case was pretty small
  git-checkout:
    image: bitnami/git:latest
    working_dir: /temp
    command: >
      bash -c "
        find /temp -mindepth 1 -delete && 
        find /files -mindepth 1 -delete &&
        git clone https://${GITHUB_TOKEN}@github.com/<username>/<repo>.git &&
        cp -R ./<repo>/<directory>/. /<volume_location>
      "
    volumes:
      - service_files:/<volume_location>

  # Target container -> mounts files
  target-container:
    image: <image>
    volumes:
      - service_files:<target_directory_for_files>
   
volumes:
  service_files:

[jerrac]

I've been testing Portainer to see how well it might work for me and just ran into this issue.

Here's my thought on what a solution might look like:

Inspect the portainer container, find the /data mount, find what directory the current stack is checked out into, and then replace any relative paths on the stack's docker-compose with the full path from the host?

Or if it's too much hassle to edit the docker-compose file before running it, set a variable that can easily be replaced, then ask users to put that in in place of the relative path.

Anyway, just a thought I hope can start some brainstorming. :)

[ConstantinFF]

I hit the same issue.

Here is my workaround. Pretty much what @jerrac suggests using env.
I must note that I am running TrueNAS with k8s and docker-compose app.

The portainer config is pretty straight forward:

services:
    portainer:
        image: portainer/portainer-ce:alpine
        volumes:
            - /etc/localtime:/etc/localtime:ro
            - /var/run/docker.sock:/var/run/docker.sock:ro
            - ./data:/data
        ports:
            - 9000:9000

Add new stack from private git repository.
Add dummy env var PROJECT_PATH set to /tmp for example
The project docker-compose.yml is:

services:
  node:
    image: node:alpine
    volumes:
      - ${PROJECT_PATH:-$PWD}:/app
      - /app/node_modules
    working_dir: /app
    entrypoint: [ "/bin/sh", "-c" ]
    command:
      - |
        npm install
        npm start
    ports:
      - 3000:3000

This will fail because /app folder will be empty.

But you can go into the Container -> Container details -> Labels
and check the com.docker.compose.project.working_dir

Get this value and set the PROJECT_PATH env var to it.
Also you might need to prepend it with the full path to portainer

Ex. com.docker.compose.project.working_dir = /data/compose/5
Portainer absolute path is /mnt/ssd-pool/apps/portainer
so the PROJECT_PATH is /mnt/ssd-pool/apps/portainer/data/compose/5
Then you can Pull and redeploy

Also docker-compose has default value $PWD that is for local development.

[maciek16180]

We are also interested in this issue getting a proper solution.

[huib-portainer]

Hi everyone, you can give this a try by using the image portainerci/portainer-ee:pr1810.
Note that this is a development build and should not be used in a production environment.

We've added a toggle to enable relative paths:

This is currently scheduled to become part of the 2.17 Business Edition release.

[loloolllool]

Hi everyone, you can give this a try by using the image portainerci/portainer-ee:pr1810. Note that this is a development build and should not be used in a production environment.

We've added a toggle to enable relative paths:

This is currently scheduled to become part of the 2.17 Business Edition release.

Tried it out and it works 👍

Related feature request: Make this available as a global default
For the simple usecase of just wanting relative paths from git repo it seems unecessary to set the local filesystem path individually for each stack.

[Buckler89]

I'm trying the portainerci/portainer-ee:pr1810.
To do that i run on the server:
docker run -d -p 8000:8000 -p 9000:9000 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainerci/portainer-ee:pr1810

When i configure an edge_agent portainer show me the command to run on the edge (I run the following in another device):

root@ipc127e:/data/compose# docker run -d   -v /var/run/docker.sock:/var/run/docker.sock   -v /var/lib/docker/volumes:/var/lib/docker/volumes   -v /:/host   -v portainer_agent_data:/data   --restart always   -e EDGE=1   -e EDGE_ID=184908d7-e459-4398-802c-ef46965d247f   -e EDGE_KEY=aHR0cHM6Ly8xOTIuMTY4LjU5LjIyNzo5MDAwfDE5Mi4xNjguNTkuMjI3OjgwMDB8YWE6NDg6N2U6Y2Y6MWU6NzE6MTM6M2Q6YzM6ZTU6NjY6MTk6ZGM6NGY6NWE6NmR8NA   -e EDGE_INSECURE_POLL=1   --name portainer_edge_agent   portainer/agent:2.17.0

Unable to find image 'portainer/agent:2.17.0' locally
docker: Error response from daemon: manifest for portainer/agent:2.17.0 not found: manifest unknown: manifest unknown.

As you can see, unfortunately the agent version 2.17.0 is not avilabe on dockerhub, so i switch to the latest tag. Any incompatibilities here?

However, I've tried setting up a stack from a git repo, but the option enable relative path volumens is not present.
Someone can teel me why?

[xucian]

Any news on this for the ce edition?
Also, the $PWD var doesn't work -- is this a bug, will it be solved, is it 'by-design'?

[patrickmichalina]

When will this be launched as non-dev build?

[huib-portainer]

It will be in the Business Edition 2.17.0 release.

[melchoir55]

It will be in the Business Edition 2.17.0 release.

Any estimate on a date for this release? Or a stable development build? pr1810 is no longer available

[DataSailNZ]

We're expecting to release that one early Feb.
In the mean time, you can try and use portainerci/portainer-ee:develop, which is very much work in progress.

[JMLX42]

It will be in the Business Edition 2.17.0 release.

@huib-portainer does it mean it will be a BE only feature ?

[NikolaSivkov]

Given the following Tree layout
What should my compose file point to, in order to mount the caddy file?
Also, please add more detailed documentation.

[patzm]

in the setup screen of a new stack, you can enable this setting:

I set the following value: /mnt/ssd-1tb/docker/portainer-stacks.
Portainer then creates a sub-folder called portainer-compose-unpacker, with a folder in it called stacks. This folder will contain one sub-folder per stack you configure to use the new relative volume paths. That folder in turn will simply clone your repository.
To summarize, the path I think composes of

• relative-volume-path: as set in the UI, this is a path on the actual host OS
• stack-name: also as set in the UI
• repo-content: the content of your repo, you can descend in there however you like

as

<relative-volume-path>/portainer-compose-unpacker/stacks/<stack-name>/<repo-content>

So in your case, if you set the relative volume path to say /mnt/vol1 and your stack is called vikunja, mount the file like

/mnt/vol1/portainer-compose-unpacker/stacks/vikunja/compose-templates/vikunja/Caddyfile

And to the portainer folks: more documentation would help 🙏. Thanks for adding the feature! Much appreciated ❤️.

[NikolaSivkov]

Couple of things i discovered so far :

you can specify path like this :

      - ./Caddyfile.conf:/etc/caddy/Caddyfile

Where the ./ will be the root of your repo docker-compose.yaml file relative to the mount path specified in the setup + the other variables.
So, in the end the path resolves to

      - /mnt/vol1/portainer-compose-unpacker/stacks/vikunja/compose-templates/vikunja/Caddyfile.conf

And right now, not sure if it's a bug, but i can't mount a file, only a directory.
Even the file gets creates as a folder in the server path /mnt/vol1/portainer-compose-unpacker/stacks/vikunja/compose-templates/vikunja/Caddyfile.conf

If i attempt to mount a non existent path, it will create a directory relative to the docker-compose.yaml file. IMO, that's neat! (and not a bug)

[patzm]

oh awesome, I didn't know that the ./ gets replaced properly.

[jer-0]

Hello this also fixes: #7254
https://portal.portainer.io/knowledge/can-i-build-an-image-while-deploying-a-stack/application-from-git

[darioackermann]

I am having the impression that since I'm using this feature, I am unable to use env_file with the stack.env value set. AWS based calls now fail with "Unable to locate credentials", although the corresponding environment variables have been set.

[atanasenko]

Stacks that are created with the 'Enable relative path volumes' option are marked as Limited control (This stack was created outside of portainer) since they are effectively created from outside by 'portainer/compose-unpacker'.

[Marek00Malik]

Can someone advice with this relative path option?
I'm trying to use this relative path option in my Swarm stack. I've configured an NFS volume that is mounted to the container.

But when I toggle the Enable relative path volumes and select the network filesystem path (for example: /mnt) I'm always getting this error:

An error occurred while running unpacker container with exit code 255

I'm not sure if there is some value that should be 'valid' for this path or its more on the host configuration side.

[Marek00Malik]

the folder:portainer-compose-unpacker/ is being created under /mnt on my host, it's empty thought. I'm not sure if this is correct or it should be created where the nfs volume is mounted. But in both cases the same error occurs.

[AlexChesters]

I've tried to make sense of the above discussion as to what it the future is for people in the community edition (I'm an individual, the business edition does not make sense for my use case).

I can understand why the relative paths feature is paywall'd by the BE, but is there potential for something like #6390 (comment) to be in the CE?

Speaking for myself I think being able to mount local files when deploying from Git (either by relative paths or by having a predictable volume name on the host) is a critical feature; I cannot use Portainer without this.

[AlexChesters]

Just found out that the BE has 3 nodes free, looks like that solves my problem! 😄

[baterja]

I've just started trying Portainer for my homelab and that's the first major obstacle I've encountered almost instantly. That's a no-go for me.

[jamescarppe]

As the relative path volume feature has been rolled out to Portainer Business Edition for a while now, I'm closing this discussion for now. For future visitors to this thread, Portainer Business Edition is available for free for up to 3 nodes via our website, and includes relative path volume support for Git stacks (among other features).

[ersefa]

Thanks in advance for your great work.

What about relative paths and webhook deployment? Is there any incompatibility between them?
Im deploying my project using relative paths (building the image directly from composer "in real time" using my repo src what is awesome), but when I try to use webhooks on this stack (to force rebuild and redeployment), im receiving this response:

curl -k -X POST -H "Content-Type: application/json" https://portainer.domain.internal/api/stacks/webhooks/76a1d3fd-d120-4806-8bf9-fa2ee68edae3

{"message":"Failed to update the stack","details":"failed to deploy a docker compose stack 77: an error occurred while running unpacker container with exit code 255: authentication required"}

[ersefa]

At first I supposed to be a problem with gitlab pull, but configured at store level my credentials on the hosts machine where the /mnt/ relative path exists. When I call the webhook or using fetch interval mode, the content on the relative path (/mnt/) dissapears. This means that not only is not updating (rebuilding) my project, also is breaking it removing the files.

[HunterDG]

Just spent a few hours going down this rabbit hole, for what now seems a waste of time 😖

Stacks that are created with the 'Enable relative path volumes' option are marked as Limited control (This stack was created outside of portainer) since they are effectively created from outside by 'portainer/compose-unpacker'.

Before I bite the BE 3-node bullet, is this ^ still the case? If so, not sure why I'd bother, since the whole reason I'm using Portainer's Git Repo functionality is to gain "Total" control

[asage-me]

Is this feature coming to Portainer CE?