Docker (rootless)

Docker is a powerful solution for setting up Services. This on will give you hints how to setup Docker in a good way.

Currently i am Experimenting on that topic, so maybe this documentation will be ready to use, maybe not.

Docker itself is nice, but it will run as root per default, which is a no-go at all. This will setup Docker in rootless- mode on OpenSuSE (currently Leap 15.5).

While Docker-Rootless is not in the main Repositories, first add the AddOn- Repository: https://download.opensuse.org/repositories/Virtualization:/containers/${releasever}/

Then install the package zypper in docker-rootless-extras docker-compose and its dependencies.

Create a new user and maybe give this user another home-directory, as it will store all files of docker.

Let's say choose home /srv/docker-user for user docker.

Also it may be a good choice, not to have this user added to users group - choose docker as default group.

OpenSuSE Leap 15.5 does not seem to have cGroups v2 enabled.

You may see a warnign when running docker info:

WARNING: Running in rootless-mode without cgroups. To enable cgroups in rootless-mode, you need to boot the system in cgroup v2 mode.

According to this documentation https://rootlesscontaine.rs/getting-started/common/cgroup2/ it needs to be enabled by appending the yast/bootloader command line with:

systemd.unified_cgroup_hierarchy=1

and also the delegation for the user of cpu is needed:

$ sudo mkdir -p /etc/systemd/system/user@.service.d
$ cat <<EOF | sudo tee /etc/systemd/system/user@.service.d/delegate.conf
[Service]
Delegate=cpu cpuset io memory pids
EOF
$ sudo systemctl daemon-reload

after this, reboot and check if /sys/fs/cgroup/cgroup.controllers is present and if docker info says:

Cgroup Driver: systemd
Cgroup Version: 2

Warning: You CANNOT sudo to this user and install docker this way, because pam is not loaded when doing this. You need to:

If you login in the system using either of
- graphical session
- login on terminal (username and password)
- ssh
then the PAM machinery will call pam_systemd, and this will setup all needed hooks to use systemctl;
if you switch user using sudo or su, this will not happen.

I chose to ssh into my machine directly:

#> ssh localhost -l docker
#> pwd
/srv/docker-user

#> dockerd-rootless-setuptool.sh check
[INFO] Requirements are satisfied

#> dockerd-rootless-setuptool.sh install
[INFO] Creating /srv/docker-user/.config/systemd/user/docker.service
[INFO] starting systemd service docker.service
+ systemctl --user start docker.service
+ sleep 3
+ systemctl --user --no-pager --full status docker.service
● docker.service - Docker Application Container Engine (Rootless)
    Loaded: loaded (/srv/docker-user/.config/systemd/user/docker.service; disabled; vendor preset: disabled)
    Active: active (running) since Tue 2023-12-19 17:53:39 CET; 3s ago
      Docs: https://docs.docker.com/go/rootless/
  Main PID: 6774 (rootlesskit)
    CGroup: /user.slice/user-1001.slice/user@1001.service/app.slice/docker.service
            ├─ 6774 rootlesskit --net=slirp4netns --mtu=65520 --slirp4netns-sandbox=auto --slirp4netns-seccom
p=auto --disable-host-loopback --port-driver=builtin --copy-up=/etc --copy-up=/run --propagation=rslave /usr/b
in/dockerd-rootless.sh
            ├─ 6785 /proc/self/exe --net=slirp4netns --mtu=65520 --slirp4netns-sandbox=auto --slirp4netns-sec
comp=auto --disable-host-loopback --port-driver=builtin --copy-up=/etc --copy-up=/run --propagation=rslave /us
r/bin/dockerd-rootless.sh
            ├─ 6804 slirp4netns --mtu 65520 -r 3 --disable-host-loopback --enable-sandbox --enable-seccomp 67
85 tap0
            ├─ 6811 dockerd
            └─ 6833 containerd --config /run/user/1001/docker/containerd/containerd.toml
+ DOCKER_HOST=unix:///run/user/1001/docker.sock
+ /usr/bin/docker version
Client:
Version:           24.0.7-ce
API version:       1.43
Go version:        go1.20.12
Git commit:        311b9ff0aa93
Built:             Fri Oct 27 12:00:00 2023
OS/Arch:           linux/amd64
Context:           default

Server:
Engine:
 Version:          24.0.7-ce
 API version:      1.43 (minimum version 1.12)
 Go version:       go1.20.12
 Git commit:       311b9ff0aa93
 Built:            Fri Oct 27 12:00:00 2023
 OS/Arch:          linux/amd64
 Experimental:     false
containerd:
 Version:          v1.7.8
 GitCommit:        8e4b0bde866788eec76735cc77c4720144248fb7
runc:
 Version:          1.1.10
 GitCommit:        v1.1.10-0-g18a0cb0f32bc
docker-init:
 Version:          0.1.7_catatonit
 GitCommit:
rootlesskit:
 Version:          1.1.1
 ApiVersion:       1.1.1
 NetworkDriver:    slirp4netns
 PortDriver:       builtin
 StateDir:         /tmp/rootlesskit687654985
slirp4netns:
 Version:          1.2.0
 GitCommit:        unknown
+ systemctl --user enable docker.service
Created symlink /srv/docker-user/.config/systemd/user/default.target.wants/docker.service → /srv/docker-user/.
config/systemd/user/docker.service.
[INFO] Installed docker.service successfully.
[INFO] To control docker.service, run: `systemctl --user (start|stop|restart) docker.service`
[INFO] To run docker.service on system startup, run: `sudo loginctl enable-linger docker`

[INFO] Creating CLI context "rootless"
Successfully created context "rootless"
[INFO] Using CLI context "rootless"
Current context is now "rootless"

[INFO] Make sure the following environment variable(s) are set (or add them to ~/.bashrc):
export PATH=/usr/bin:$PATH

[INFO] Some applications may require the following environment variable too:
export DOCKER_HOST=unix:///run/user/1001/docker.sock

So, this looks nice.

Mind, that now your volumes are under /srv/docker-user/.local/share/docker/volumes what may not be wanted. i would suggest, that you specify other volumes-directories in the yml-files for each service.

in rootless-mode, the file to configure docker is here:

~/.config/docker/daemon.json

by default, the path and the file is not existent, create it with the user.

For example use another directory für your data in that file:

{
 "log-level": "warn",
 "log-driver": "json-file",
 "experimental": true,
 "ip6tables": true,
 "data-root": "/srv/docker",
 "storage-driver": "overlay2",
 "log-opts": {
   "max-size": "10m",
   "max-file": "5"
 }
}

Here, ipv6 support has been turned on. See https://docs.docker.com/config/daemon/ipv6/ for details.

Notice: Don't use userns-remap - this won't work and makes no sense.

Mind, that at the time writing overlay2 is the way to go as storage driver in docker, but it only supports xfs as backing filesystems (with d_type=true which means ftype=1 ).

I personally dislike xfs, especially while its not rubust and won't shrink. I use it anyway, because of its srong advise to do so - with kernel 5.19+ it should be possible to get overlay2 working on btrfs.

For checking which filesystem is in use, see docker info, lines under Storage Driver:

Storage Driver: overlay2
 Backing Filesystem: xfs
 Supports d_type: true
 Using metacopy: false
 Native Overlay Diff: false
 userxattr: true

If you create a XFS-Filesystem with Leap 15.5 it will have ftype=1 set, check output of xfs_info <volumename>

For me, i like btrfs a bit more than xfs, so i chose to have three different locations for docker on two filesystems (which are raided lvms. Check out the other docs here):

#> mount | grep /srv
/dev/mapper/vgdata-lvdata on /srv type btrfs (rw,noatime,nodiratime,compress=zstd:3,space_cache,autodefrag,subvolid=5,subvol=/)
/dev/mapper/vgdata-lvdocker on /srv/docker type xfs (rw,**noexec**,noatime,nodiratime,attr2,inode64,logbufs=8,logbsize=32k,sunit=128,swidth=256,noquota)

and in /srv i have

#> ls /srv/ | grep docker
docker #-> for storing docker-data using xfs
docker-compose #-> for my yaml-files
docker-user #-> this is the home of the docker user, containing the executables in user-space

For me i would like the files only be read/writeable for user docker and group docker, which is very safe:

#:~ # cd /srv
#:/srv # chown docker docker* -R
#:/srv # chgrp docker docker* -R
#:/srv # setfacl -m d:u::rwX -R docker*
#:/srv # setfacl -m d:g::rwX -R docker*
#:/srv # setfacl -m d:o::--- -R docker*
#:/srv # setfacl -m u::rwX -R docker*
#:/srv # setfacl -m g::rwX -R docker*
#:/srv # setfacl -m o::--- -R docker*

That way, others do not have any access to the files or folders.

There are Dockers out there, that are not aware of rootless-modes. One example (date up to now) is phpmyadmin. It will create files in docker-start- script and they will have the default permissions - which are safe, but now won't work any more.

So you may change the default for others to be readable:

#:~ # cd /srv
#:/srv # setfacl -m d:o::r-X -R /srv/docker
#:/srv # setfacl -m o::r-X -R /srv/docker
#:/srv # setfacl -m d:o::--- -R /srv/docker/volumes
#:/srv # setfacl -m o::--- -R /srv/docker/volumes

The docker- daemon will create all files that are downloaded and created in a container as the facls are set if not forced to have other permissions by the docker-app itself.

Usually all the data- files are kept in Volumes. Unfortunatelly, docker does not care about linux acls and user permissions in Volumes - or to be more precise: it overwrites them.

So if you would like to have your files really secured from other user, you can create the volume with an umask - e.g.:

docker volume create portainer_data –opt o=umask=0007
That would make all files in the Volume have the right permissions.

In Docker-Compose use this in your yml:

volumes:
  volumename:
    driver_opts:
      device: ""
      type: ""
      o: "umask=0007"

Device and type need to be there, but can be empty to use default values.

Now - finally its time for our first running Container. As the Portainer- App is the most important Management- Software in Docker, let's run it in a safe userspaced way now.

As always, SSH into your docker- user and than create the folders and yml-files for portainer.

obel1x@server:~> ssh localhost -l docker

Password:
docker@server:~> cd /srv/docker-compose/
docker@server:~> mkdir portainer
docker@server:/srv/docker-compose> cd portainer
docker@server:/srv/docker-compose/portainer> getfacl .
# file: .
# owner: docker
# group: docker
user::rwx
group::rwx
other::---
default:user::rwx
default:group::rwx
default:other::---

docker@server:/srv/docker-compose/portainer> touch docker-compose.yml

put the following into that file:

version: "3.8"
services:
 portainer:
   restart: always
   image: portainer/portainer-ce
   ports:
     - 9000:9000
     - 9433:9433
   volumes:
     - portainer_data:/data
     - /run/user/1001/docker.sock:/var/run/docker.sock

volumes:
 portainer_data:
   driver_opts:
     device: ""
     type: ""
     o: "umask=0007"

now start your app:

docker@server:~> docker-compose up -d
[...]
⠿ Container portainer-portainer-1    Started

Check the volume has been created at the right location and do have the right umask set:

docker@server:~> docker volume inspect portainer_portainer_data
[
{
"CreatedAt": "2023-12-21T09:37:16+01:00",
"Driver": "local",
"Labels": {
"com.docker.compose.project": "portainer",
"com.docker.compose.version": "2.14.2",
"com.docker.compose.volume": "portainer_data"
},
"Mountpoint": "**/srv/docker/volumes/portainer_portainer_data/_data**",
"Name": "portainer_portainer_data",
"Options": {
"device": "_ckgedit>,
**"o": "umask=0007"**,
"type": "_ckgedit>
},
"Scope": "local"
}
]

and finally if the files have been created only readable by the user:

ls /srv/docker/volumes/portainer_portainer_data/* -l -R
-rw——- 1 docker docker 77 21. Dez 09:37 /srv/docker/volumes/portainer_portainer_data/opts.json

/srv/docker/volumes/portainer_portainer_data/_data:
insgesamt 40
drwx——+ 2 docker docker 6 21. Dez 09:37 bin
drwx——+ 2 docker docker 37 21. Dez 09:37 certs
drwx——+ 2 docker docker 29 21. Dez 09:37 chisel
drwx——+ 2 docker docker 6 21. Dez 09:37 compose
drwx——+ 2 docker docker 25 21. Dez 09:37 docker_config
-rw——- 1 docker docker 65536 21. Dez 09:37 portainer.db
-rw——- 1 docker docker 227 21. Dez 09:37 portainer.key
-rw——- 1 docker docker 190 21. Dez 09:37 portainer.pub
drwx——+ 2 docker docker 6 21. Dez 09:37 tls

/srv/docker/volumes/portainer_portainer_data/_data/bin:
insgesamt 0

/srv/docker/volumes/portainer_portainer_data/_data/certs:
insgesamt 8
-rw——- 1 docker docker 518 21. Dez 09:37 cert.pem
-rw——- 1 docker docker 227 21. Dez 09:37 key.pem

/srv/docker/volumes/portainer_portainer_data/_data/chisel:
insgesamt 4
-rw——- 1 docker docker 227 21. Dez 09:37 private-key.pem

/srv/docker/volumes/portainer_portainer_data/_data/compose:
insgesamt 0

/srv/docker/volumes/portainer_portainer_data/_data/docker_config:
insgesamt 4
-rw-r—– 1 docker docker 389 21. Dez 09:37 config.json

/srv/docker/volumes/portainer_portainer_data/_data/tls:
insgesamt 0

this is amazing, right ;) ?

Now you can go to http://localhost:9000 and finish the setup of Portainer using the local environement and enjoy the docker-party:

Thats all here, cheers!