Unterschiede

Hier werden die Unterschiede zwischen zwei Versionen angezeigt.

--- content:serverbasics:docker [2024/04/20 13:29] – [Install rootless Docker] Daniel
+++ content:serverbasics:docker [2025/02/08 11:30] (aktuell) – [Update] Daniel
@@ Zeile 1: / Zeile 1: @@
-====== Docker (rootless) ======
+====== Docker (rootless) + Portainer ======
-Docker is a powerful solution for setting up Services. This on will give you hints how to setup Docker in a good way.
+Docker is a powerful solution for setting up Services. This short Introducion will give you hints how to setup Docker in a good way in userspace, so no root-access is needed for Docker.
-Currently i am Experimenting on that topic, so maybe this documentation will be ready to use, maybe not.
+Currently i am experimenting on that topic, so maybe this documentation will be ready to use, maybe not.
 Docker itself is nice, but it will run as root per default, which is a no-go at all. This will setup Docker in rootless- mode on OpenSuSE (currently Leap 15.5).
-===== Package =====
+__**Warning: This is a very strong advise NOT to use docker default in rootmode at all! **__  The reason is, that any service is able to talk to the Docker Daemon if there is a connection to the Docker socket in the Volumes (which some services require) - or simply if thers a bug somewhere. By that way, the Docker Container will be able to set up ANY service and bind ANY location on the Host, that the docker user may be able to see. So if the Service gets taken over and the service ist runnig as root… you know where you are.
-I had really a lot of troubles with the package in the AddOn- Repository: ''[[https://download.opensuse.org/repositories/Virtualization:/containers/${releasever}/|https://download.opensuse.org/repositories/Virtualization:/containers/${releasever}/]]'' while they are not installing docker the same way, that docker would do. For example they will not be installed in User-Subspace only, but will use systems Docker executables installed in global paths. This is a problem when using btrfs - as btrfs is not fully compatible with docker. So i won't use this any more.
+So just: Don't set up Docker rootfull at all if possible - its even not neeeded nowadays.
+===== Filesystem Layout =====
+Mind, that at the time writing, ''overlay2'' is the way to go as storage driver in docker, but it only supports xfs as backing filesystems (with ''d_type=true'' which means ''ftype=1'' ) for full support.
+I personally dislike xfs, especially while its not rubust and won't shrink. I use it anyway, because of its srong advise to do so - with kernel 5.19+ it should be possible to get overlay2 working on btrfs, but still there are things that may not work even with that kernel - in the worst case, docker is unable to unlink files, so there will be huge Containers and Volumes and maybe Services will break.
+So make sure, that the Home-Directory of you docker user is on XFS. The ftype is already ok on SuSE 15.5, check output of ''xfs_info <volumename>''.
+Warning: you may have umask set your way - i prefer 007 as written before. But if you change umask and permissions be very cautious, as docker uses userid- mapping and may change the permissions and ownersets of files in its directory to the subuserid.
+That may change the ownership in a way, that even the docker user on the host cannot access the Files, which is OK !
+__**STRONG WARNING: Don't change permissions or ownership of docker- directories on the Host directly as this will change them in the container, making them unavaiable and break your Services !!!**__
+__**The only way to manage Volume- File- Permissions is to bash inside the running container itself and to change them there (to the right values of course)!**__
+A short hint: Docker rootless uses Sub(g)uids, which is a feature of Linux. That means each user has a range of userids (quite a huge range) and groupid which the user may use. Those will be exclusive reserved for that user. But it does not mean, that the User can acces the Files created by those Subuids! Also the UIDs are onyl a number - not a real user in Linux having a username- They cannot be used to logon or to work with. Docker manages internally which Host-Subuserid is assigned to which caontainer and to which userid inside the running container/service. Inside the Container, you may become that user having a real username and a (different) uid.
+===== Packages NOT to install =====
+I had really a lot of troubles with the package Docker-Rootless in the AddOn- Repository: ''[[https://download.opensuse.org/repositories/Virtualization:/containers/${releasever}/|https://download.opensuse.org/repositories/Virtualization:/containers/${releasever}/]]'' while they are not installing docker the same way, that docker would do. For example they will not be installed in User-Subspace only, but will use systems Docker executables installed in global paths. This is a problem when using btrfs - as btrfs is not fully compatible with docker. So i won't use this any more.
+So i disabled the following packages and locked them to never install:
+  * docker
+  * docker-compose
+  * containerd
 Check out beneath for install the docker way.
-===== User =====
+===== Docker- User =====
-Create a new **group** called **docker** and a new **user** called **docker**. Make the user have the **default group docker**.
+Create a new **group**  called **docker**  and a new **user**  called **docker**. Make the user is in the **default group docker**.
 Attention: The Home Directory should be on a volume having XFS as btrfs or others are not fully supported right now (20.04.2024 - patches in new Kernel 5.19 are incoming, but this Kernel is not released until now and still there are some problems open in developement there).
@@ Zeile 21: / Zeile 49: @@
 ===== cGroups v2 =====
-OpenSuSE Leap 15.5 does not seem to have cGroups v2 enabled.
+OpenSuSE Leap 15.5 does not have cGroups v2 enabled, which are needed by docker.
 You may see a warning (later) when running ''docker info'':
@@ Zeile 43: / Zeile 71: @@
 </file>
-after this, reboot and check if ''/sys/fs/cgroup/cgroup.controllers'' is present
+after this, reboot and check if ''/sys/fs/cgroup/cgroup.controllers''  is present
-After installing docker (see beneath), check if ''docker info'' says:
+After installing docker (see beneath), check if ''docker info''  says:
 <file>
@@ Zeile 165: / Zeile 193: @@
 </code>
-So, this looks very nice. **Important:** Do what the Installation says with the file ''~/.bashrc''
+So, this looks very nice. **Important:**  Do what the Installation says with the file ''~/.bashrc''
+===== Check Docker install =====
-==== Optional Docker- Directory ====
+Log out of docker user if you are still in from install. Then, log back in to apply the bashrc- settings.
+Ceck the Environement to have the settings:
+<code>
+docker@pcserver2023:~> Abgemeldet
+Connection to localhost closed.
+obel1x@pcserver2023:~> ssh localhost -l docker
+Password:
+Last login: Sat Apr 20 15:18:56 2024 from ::1
+Have a lot of fun...
+docker@pcserver2023:~> echo $DOCKER_HOST
+unix:///run/user/1001/docker.sock
+</code>
+Now check ''docker info'':
+<code>
+docker@pcserver2023:~> docker info
+Client:
+ Version:    25.0.2
+ Context:    default
+ Debug Mode: false
+Server:
+ Containers: 0
+  Running: 0
+  Paused: 0
+  Stopped: 0
+ Images: 0
+ Server Version: 25.0.2
+ Storage Driver: overlay2
+  Backing Filesystem: xfs
+  Supports d_type: true
+  Using metacopy: false
+  Native Overlay Diff: false
+  userxattr: true
+ Logging Driver: json-file
+ Cgroup Driver: systemd
+ Cgroup Version: 2
+ Plugins:
+  Volume: local
+  Network: bridge host ipvlan macvlan null overlay
+  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
+ Swarm: inactive
+ Runtimes: io.containerd.runc.v2 runc
+ Default Runtime: runc
+ Init Binary: docker-init
+ containerd version: 7c3aca7a610df76212171d200ca3811ff6096eb8
+ runc version: v1.1.12-0-g51d5e94
+ init version: de40ad0
+ Security Options:
+  seccomp
+   Profile: builtin
+  rootless
+  cgroupns
+ Kernel Version: 5.14.21-150500.55.52-default
+ Operating System: openSUSE Leap 15.5
+ OSType: linux
+ Architecture: x86_64
+ CPUs: 8
+ Total Memory: 30.79GiB
+ Name: pcserver2023
+ ID: 45699224-ea9c-4865-8dea-a53bb20b788c
+ Docker Root Dir: /home/docker/.local/share/docker
+ Debug Mode: false
+ Experimental: false
+ Insecure Registries:
+.0.0.0/8
+ Live Restore Enabled: false
+ Product License: Community Engine
+</code>
+==== Additional knowledge ====
+  * Storage driver and FS-Type : overlay2 should always be used, btrfs is outdated! XFS and d_type are important!
+  * CGroup Version needs to be 2 or better
+  * If you see Docker complaining about Module aufs at start: do not care about - that module is obsolete
+===== IP-Filter =====
+When starting Docker, an the log says:
+<code>
+level=warning msg="Running modprobe bridge br_netfilter failed with message: modprobe: ERROR: could not insert 'br_netfilter': Operation not permitted\ninsmod /lib/modul>
+level=info msg="skipping firewalld management for rootless mode"
+</code>
+You first need to load the module with modprobe.
+For system startup, use ''/etc/modules-load.d''  and creat e a file ''docker-rootless.conf''  in it, containing that module.
+===== IPTables =====
+If you see ''docker info''  saying:
+<code>
+WARNING: bridge-nf-call-iptables is disabled
+WARNING: bridge-nf-call-ip6tables is disabled
+</code>
+This should be fixed by:
+<code>
+# sudo echo "net.bridge.bridge-nf-call-iptables = 1">> /etc/sysctl.conf
+# sudo echo "net.bridge.bridge-nf-call-ip6tables = 1">> /etc/sysctl.conf
+# sudo modprobe br_netfilter
+# sudo sysctl --system
+</code>
+===== Configuring Docker Daemon =====
 in rootless-mode, the file to configure docker is here:
@@ Zeile 174: / Zeile 318: @@
 ''~/.config/docker/daemon.json''
-by default, the path and the file is not existent, create it with the user.
+**by default, the path and the file is not existent, __create it new__  within the docker user**.
-For example use another directory für your data in that file:
+For example, enable IPv6. See [[https://docs.docker.com/config/daemon/ipv6/|https://docs.docker.com/config/daemon/ipv6/]] for details.
 <file>
 {
- "log-level": "warn",
- "log-driver": "json-file",
- "experimental": true,
  "ip6tables": true,
- "data-root": "/srv/docker",
+ "ipv6": true,
- "storage-driver": "overlay2",
+ "fixed-cidr-v6": "fd12:3456:789a:1::/64",
  "log-opts": {
    "max-size": "10m",
@@ Zeile 194: / Zeile 335: @@
 </file>
-Here, ipv6 support has been turned on. See [[https://docs.docker.com/config/daemon/ipv6/|https://docs.docker.com/config/daemon/ipv6/]] for details.
+Notice: Don't use ''userns-remap''  - this won't work and makes no sense in rootless!
-Notice: Don't use ''userns-remap'' - this won't work and makes no sense.
+Edit: "experimental": true has been removed for ipv6 with docker v27.
-===== Filesystem Layout =====
-Mind, that at the time writing ''overlay2'' is the way to go as storage driver in docker, but it only supports xfs as backing filesystems (with ''d_type=true'' which means ''ftype=1'' ).
+You need to adjust cidr to some unique ULA. ULAs are non internet routable adresses (like 192.X.X.X in ipv4). Select an unique adress only for that internal Docker network - you can choose anything that is not assigned anywhere else on your network to not cause trouble.
-I personally dislike xfs, especially while its not rubust and won't shrink. I use it anyway, because of its srong advise to do so - with kernel 5.19+ it should be possible to get overlay2 working on btrfs.
+Maybe use this tool to generate: [[https://www.unique-local-ipv6.com|https://www.unique-local-ipv6.com]]
-For checking which filesystem is in use, see ''docker info'', lines under ''Storage Driver'':
+===== Networking in Docker rootless =====
-<file>
-Storage Driver: overlay2
+If you read docs in the net about networking with Docker you may see docker0 as bridge network. While this network is also there in docker rootless, you will not find that network as interface on your host like you would on a rootful docker.
- Backing Filesystem: xfs
- Supports d_type: true
- Using metacopy: false
- Native Overlay Diff: false
- userxattr: true
-</file>
+Instead the network is encapsulated in the environement of rootlesskit and not visible to the host. From the Hosts view Docker is just another Application running on your Host talking to the internet like some app would do.
-If you create a XFS-Filesystem with Leap 15.5 it will have ''ftype=1'' set, check output of ''xfs_info <volumename>''
+===== Install docker compose =====
-For me, i like btrfs a bit more than xfs, so i chose to have three different locations for docker on two filesystems (which are raided lvms. Check out the other docs here):
+**This Chapter may be obsolete as since docker v27 the compose plugin is part of installation script - check your output of ''docker info''  for the installed Plugins and if ''docker compose version''  already has a version. If so, skip this.**
-<file>
+The command ''docker-compose''  has been obsoleted and been replaced by a plugin ''compose''  for docker (see [[https://docs.docker.com/compose/install/|https://docs.docker.com/compose/install/]]).
-#> mount | grep /srv
-/dev/mapper/vgdata-lvdata on /srv type btrfs (rw,noatime,nodiratime,compress=zstd:3,space_cache,autodefrag,subvolid=5,subvol=/)
-/dev/mapper/vgdata-lvdocker on /srv/docker type xfs (rw,**noexec**,noatime,nodiratime,attr2,inode64,logbufs=8,logbsize=32k,sunit=128,swidth=256,noquota)
-</file>
+Installing it the manual way:
-and in /srv i have
-<file>
-#> ls /srv/ | grep docker
-docker #-> for storing docker-data using xfs
-docker-compose #-> for my yaml-files
-docker-user #-> this is the home of the docker user, containing the executables in user-space
-</file>
-==== ... and the permissions of the files ====
-For me i would like the files only be read/writeable for user docker and group docker, which is very safe:
+Edit the File ''~/.bashrc''  and add:
 <code>
-#:~ # cd /srv
+export DOCKER_CONFIG=${DOCKER_CONFIG:-$HOME/.docker}
-#:/srv # chown docker docker* -R
-#:/srv # chgrp docker docker* -R
-#:/srv # setfacl -m d:u::rwX -R docker*
-#:/srv # setfacl -m d:g::rwX -R docker*
-#:/srv # setfacl -m d:o::--- -R docker*
-#:/srv # setfacl -m u::rwX -R docker*
-#:/srv # setfacl -m g::rwX -R docker*
-#:/srv # setfacl -m o::--- -R docker*
 </code>
-That way, others do not have any access to the files or folders.
+Then relog to the docker user and do as the doc says to install and check you install:
-=== Compatibility Mode ===
-There are Dockers out there, that are not aware of rootless-modes. One example (date up to now) is phpmyadmin. It will create files in docker-start- script and they will have the default permissions - which are safe, but now won't work any more.
-So you may change the default for others to be readable:
 <code>
-#:~ # cd /srv
+docker@pcserver2023:~> mkdir -p $DOCKER_CONFIG/cli-plugins
-#:/srv # setfacl -m d:o::r-X -R /srv/docker
+docker@pcserver2023:~> curl -SL https://github.com/docker/compose/releases/download/v2.26.1/docker-compose-linux-x86_64 -o $DOCKER_CONFIG/cli-plugins/docker-compose
-#:/srv # setfacl -m o::r-X -R /srv/docker
+  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
-#:/srv # setfacl -m d:o::--- -R /srv/docker/volumes
+                                 Dload  Upload   Total   Spent    Left  Speed
-#:/srv # setfacl -m o::--- -R /srv/docker/volumes
+     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
+59.8M  100 59.8M    0     0  9951k      0  0:00:06  0:00:06 --:--:-- 11.4M
+docker@pcserver2023:~> chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose
+docker@pcserver2023:~> docker compose version
+Docker Compose version v2.26.1
+docker@pcserver2023:~>
 </code>
-The docker- daemon will create all files that are downloaded and created in a container as the facls are set if not forced to have other permissions by the docker-app itself.
+Your done with the compose plugin
-=== Volume- Permissions ===
+===== Update =====
-Usually all the data- files are kept in Volumes. Unfortunatelly, docker does not care about linux acls and user permissions in Volumes - or to be more precise: it overwrites them.
+If you want to update your docker- installation, there is not update- process but to use the same script again:
-So if you would like to have your files really secured from other user, you can create the volume with an umask - e.g.:
+<file>
+#!/bin/bash
+#Upgrade docker rootless and plugin
+./docker_stop_all.sh
+sleep 5
+systemctl --user stop docker
+sleep 5
+#uninstall
+rm -f ~/bin/dockerd
+rm ~/.config/systemd/user/docker.service.bak
+mv ~/.config/systemd/user/docker.service ~/.config/systemd/user/docker.service.bak
+#reinstall docker compose
+COMPOSE_VER='2.32.4'
+rm $DOCKER_CONFIG/cli-plugins/docker-compose
+echo "Download Docker Compose Release ${COMPOSE_VER} - please check at https://github.com/docker/compose/releases for the newes Version and change this File"
+curl -SL https://github.com/docker/compose/releases/download/v${COMPOSE_VER}/docker-compose-linux-x86_64 -o $DOCKER_CONFIG/cli-plugins/docker-compose
+chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose
+#install docker
+curl -fsSL https://get.docker.com/rootless | sh
+#need to give the new binary permissions to acces privileged network ports (beneath 1024)
+sudo setcap 'cap_net_bind_service=+ep' ~/bin/rootlesskit
+#this should be everything
+docker info
-''docker volume create portainer_data –opt o=umask=0007'' \\
+</file>
-That would make all files in the Volume have the right permissions.
-In Docker-Compose use this in your yml:
-<code>
+===== Create a place for Yamls =====
-volumes:
-  volumename:
-    driver_opts:
-      device: ""
-      type: ""
-      o: "umask=0007"
-</code>
+Now, that you have compose, you can use it to setup your services with YAML- Files. Each service should have a directory for its own.
-Device and type need to be there, but can be empty to use default values.
+Make a directory with ''mkdir ~/docker_compose''  and change to it.
 ===== First Docker App: Portainer =====
-Now - finally its time for our first running Container. As the Portainer- App is the most important Management- Software in Docker, let's run it in a safe userspaced way now.
+Now - finally its time for our first running Container. As the Portainer- App is an important Management- Software in Docker for inexperienced users, let's run it in a safe userspaced way now.
-As always, SSH into your docker- user and than create the folders and yml-files for portainer.
+As always, SSH into your docker- user and than create the folders and yml-files for docker compose and portainer.
+<code>
-<code>
 obel1x@server:~> ssh localhost -l docker
 Password:
-docker@server:~> cd /srv/docker-compose/
+docker@pcserver2023:~> cd ~/docker_compose
-docker@server:~> mkdir portainer
+docker@pcserver2023:~/docker_compose> mkdir portainer
-docker@server:/srv/docker-compose> cd portainer
+docker@pcserver2023:~/docker_compose> cd portainer
-docker@server:/srv/docker-compose/portainer> getfacl .
+docker@pcserver2023:~/docker_compose> touch docker-compose.yml
-# file: .
+docker@pcserver2023:~/docker_compose>
-# owner: docker
-# group: docker
-user::rwx
-group::rwx
-other::---
-default:user::rwx
-default:group::rwx
-default:other::---
-docker@server:/srv/docker-compose/portainer> touch docker-compose.yml
 </code>
-put the following into that file:
+Put the following into that file:
 <file>
-version: "3.8"
 services:
  portainer:
@@ Zeile 340: / Zeile 452: @@
 volumes:
  portainer_data:
-   driver_opts:
-     device: ""
-     type: ""
-     o: "umask=0007"
 </file>
-now start your app:
+Check, that the Socket- Path is the correct one.
-<code>
+Now start your app and look the magic:
-docker@server:~> docker-compose up -d
-[...]
-⠿ Container portainer-portainer-1    Started
-</code>
-Check the volume has been created at the right location and do have the right umask set:
 <code>
-docker@server:~> docker volume inspect portainer_portainer_data
+docker@pcserver2023:~/docker_compose/portainer> docker compose up -d
-[
+[+] Running 12/12
-{
+ ✔ portainer Pulled                                                                                                            17.6s
-"CreatedAt": "2023-12-21T09:37:16+01:00",
+   ✔ 379538b6d68e Pull complete                                                                                                 0.5s
-"Driver": "local",
+   ✔ 4ea3e2c3a39b Pull complete                                                                                                 0.5s
-"Labels": {
+   ✔ 5171176db7f2 Pull complete                                                                                                 3.8s
-"com.docker.compose.project": "portainer",
+   ✔ 52e9438966a5 Pull complete                                                                                                 6.5s
-"com.docker.compose.version": "2.14.2",
+   ✔ 43d4775415ac Pull complete                                                                                                 6.7s
-"com.docker.compose.volume": "portainer_data"
+   ✔ c1cad9f5200f Pull complete                                                                                                 9.6s
-},
+   ✔ 22eab514564f Pull complete                                                                                                 7.1s
-"Mountpoint": "**/srv/docker/volumes/portainer_portainer_data/_data**",
+   ✔ 962b9fa821a2 Pull complete                                                                                                10.0s
-"Name": "portainer_portainer_data",
+   ✔ c153fefda5ce Pull complete                                                                                                10.9s
-"Options": {
+   ✔ bed990c4615b Pull complete                                                                                                10.2s
-"device": "_ckgedit>,
+   ✔ 4f4fb700ef54 Pull complete                                                                                                10.5s
-**"o": "umask=0007"**,
+[+] Running 3/3
-"type": "_ckgedit>
+ ✔ Network portainer_default          Created                                                                                   0.2s
-},
+ ✔ Volume "portainer_portainer_data"  Created                                                                                   0.1s
-"Scope": "local"
+ ✔ Container portainer-portainer-1    Started                                                                                   0.3s
-}
+docker@pcserver2023:~/docker_compose/portainer>
-]
 </code>
-and finally if the files have been created only readable by the user:
+Now you can go to [[http://localhost:9000|http://localhost:9000]] and pick a password to finish the setup of Portainer using the local Environment and enjoy the docker-party:
-<code>
+{{  .:screenshot_20231221_101150.png?968x260  }}
-ls /srv/docker/volumes/portainer_portainer_data/* -l -R
--rw——- 1 docker docker 77 21. Dez 09:37 /srv/docker/volumes/portainer_portainer_data/opts.json
-/srv/docker/volumes/portainer_portainer_data/_data:
+Thats all: Docker is running and serving your services, cheers!
-insgesamt 40
-drwx——+ 2 docker docker 6 21. Dez 09:37 bin
-drwx——+ 2 docker docker 37 21. Dez 09:37 certs
-drwx——+ 2 docker docker 29 21. Dez 09:37 chisel
-drwx——+ 2 docker docker 6 21. Dez 09:37 compose
-drwx——+ 2 docker docker 25 21. Dez 09:37 docker_config
--rw——- 1 docker docker 65536 21. Dez 09:37 portainer.db
--rw——- 1 docker docker 227 21. Dez 09:37 portainer.key
--rw——- 1 docker docker 190 21. Dez 09:37 portainer.pub
-drwx——+ 2 docker docker 6 21. Dez 09:37 tls
-/srv/docker/volumes/portainer_portainer_data/_data/bin:
+===== Fast Stop of all Containers =====
-insgesamt 0
-/srv/docker/volumes/portainer_portainer_data/_data/certs:
+This makes life easy ''docker_stop_all.sh'':
-insgesamt 8
+<file>
--rw——- 1 docker docker 518 21. Dez 09:37 cert.pem
--rw——- 1 docker docker 227 21. Dez 09:37 key.pem
-/srv/docker/volumes/portainer_portainer_data/_data/chisel:
+#!/bin/bash
-insgesamt 4
+docker stop $(docker ps -a -q)
--rw——- 1 docker docker 227 21. Dez 09:37 private-key.pem
+docker rm $(docker ps -a -q)
-/srv/docker/volumes/portainer_portainer_data/_data/compose:
+</file>
-insgesamt 0
-/srv/docker/volumes/portainer_portainer_data/_data/docker_config:
-insgesamt 4
--rw-r—– 1 docker docker 389 21. Dez 09:37 config.json
-/srv/docker/volumes/portainer_portainer_data/_data/tls:
-insgesamt 0
-</code>
-this is amazing, right ;) ?
-Now you can go to [[http://localhost:9000|http://localhost:9000]] and finish the setup of Portainer using the local environement and enjoy the docker-party:
-{{  .:screenshot_20231221_101150.png?968x260  }}
-Thats all here, cheers!