# kubernetes persistence volume issues **URL:** \ **Category:** General Discussions\ **Tags:** kubernetes\ **Created:** [January 15, 2023, 5:22am UTC](https://community.solace.com/t/kubernetes-persistence-volume-issues/1547 "2023-01-15T05:22:51Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![sulfred](https://yyz1.discourse-cdn.com/solace/user_avatar/community.solace.com/sulfred/32/48_2.png) [@sulfred](https://community.solace.com/u/sulfred)\ **Post date:** [January 15, 2023, 5:22am UTC](https://community.solace.com/t/kubernetes-persistence-volume-issues/1547/1 "2023-01-15T05:22:51Z")
Hi all, > I am using k8s to deploy solace pubsub +. While setting the persistence volumes, I hit a permission issue and here are some error messages: > Starting PubSub+ Software Event Broker Container: Sun Jan 15 05:04:27 UTC 2023 > Setting umask to 022 > Sun Jan 15 05:04:27 UTC 2023 INFO: startup-broker.sh-Node ordinal: 0 > Sun Jan 15 05:04:27 UTC 2023 INFO: startup-broker.sh-Waiting for management API to become available > SolOS Version: soltr\_10.2.1.32 > management host is not responding > Sun Jan 15 05:04:27 UTC 2023 INFO: startup-broker.sh-Waited 0 seconds, Management API not yet accessible > ERROR: Required directory /var/lib/solace is not writable by current user > ERROR: Unable to create required directory /var/lib/solace/diagnostics: [Errno 13] Permission denied: ‘/var/lib/solace/diagnostics’ > ERROR: Required directory /var/lib/solace/diagnostics does not exist > ERROR: Unable to create required directory /var/lib/solace/spool: [Errno 13] Permission denied: ‘/var/lib/solace/spool’ > ERROR: Required directory /var/lib/solace/spool does not exist > ERROR: Unable to create required directory /var/lib/solace/spool-cache: [Errno 13] Permission denied: ‘/var/lib/solace/spool-cache’ > ERROR: Required directory /var/lib/solace/spool-cache does not exist > ERROR: Unable to create required directory /var/lib/solace/spool-cache-backup: [Errno 13] Permission denied: ‘/var/lib/solace/spool-cache-backup’ > ERROR: Required directory /var/lib/solace/spool-cache-backup does not exist > ERROR: Unable to create required directory /var/lib/solace/spool-cache-backup/image: [Errno 13] Permission denied: ‘/var/lib/solace/spool-cache-backup’ > ERROR: Required directory /var/lib/solace/spool-cache-backup/image does not exist I am using: ubuntu 20.04 kubectl version \> $ kubectl version > WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version. > Client Version: version.Info{Major:“1”, Minor:“26”, GitVersion:“v1.26.0”, GitCommit:“b46a3f887ca979b1a5d14fd39cb1af43e7e5d12d”, GitTreeState:“clean”, BuildDate:“2022-12-08T19:58:30Z”, GoVersion:“go1.19.4”, Compiler:“gc”, Platform:“linux/amd64”} > Kustomize Version: v4.5.7 > Server Version: version.Info{Major:“1”, Minor:“25”, GitVersion:“v1.25.3”, GitCommit:“434bfd82814af038ad94d62ebe59b133fcb50506”, GitTreeState:“clean”, BuildDate:“2022-10-12T10:49:09Z”, GoVersion:“go1.19.2”, Compiler:“gc”, Platform:“linux/amd64”} > minikube 1.28.0 helm 3.10.3 \> I install solace with commands: > $ helm repo add solace [https://solaceproducts.github.io/pubsubplus-kubernetes-quickstart/helm-charts/](https://solaceproducts.github.io/pubsubplus-kubernetes-quickstart/helm-charts/) > $ helm pull solace/pubsubplus --version 3.2.0 --untar ture > > # prepare persistent storage > > $ kubectl apply -f ./pubsubplus/custom\_config/dev/storage.yaml > > # start solace pub sub > > $ helm install solace-mb ./pubsubplus/ --values ./pubsubplus/custom\_config/dev/values.customer.yaml > values.customer.yaml > solace: >   usernameAdminPassword: adminpw > storage: >   persistent: true >   customVolumeMount: | >     persistentVolumeClaim: >       claimName: solace-volume >   useStorageClass: standard >   useStorageGroup: true > storage.yaml # storage request > ## apiVersion: v1 kind: PersistentVolumeClaim metadata:   name: solace-volume spec:   storageClassName: solace-storage   accessModes:     - ReadWriteOnce   resources:     requests:       storage: 30Gi > > # storage implement > > apiVersion: v1 > kind: PersistentVolume > metadata: >   name: local-storage > spec: >   storageClassName: solace-storage >   capacity: >     storage: 30Gi >   accessModes: >     - ReadWriteOnce >   hostPath: >     path: “/mnt/volumes/solaceVolume/” >     type: DirectoryOrCreate > From minikube vm, I see that the volume is created successfully and data are put to there by solace. > $ cd /mnt/volumes/solaceVolume/ > $ ls -hal > total 0 > drwxr-xr-x 8 root root 160 Jan 15 04:18 . > drwxr-xr-x 3 root root  60 Jan 15 04:18 .. > drwxr-xr-x 2 root root  40 Jan 15 04:18 adb > drwxr-xr-x 2 root root  40 Jan 15 04:18 diags > drwxr-xr-x 3 root root  60 Jan 15 04:18 internalSpool > drwxr-xr-x 2 root root  40 Jan 15 04:18 jail > drwxr-xr-x 2 root root  40 Jan 15 04:18 softAdb > drwxr-xr-x 2 root root  40 Jan 15 04:18 var It seems that there is some configuration missing. I would like to ask if anyone has idea with this? Thanks ---
**Author:** ![Aaron](https://yyz1.discourse-cdn.com/solace/user_avatar/community.solace.com/aaron/32/843_2.png) [@Aaron](https://community.solace.com/u/Aaron)\ **Post date:** [January 18, 2023, 6:15pm UTC](https://community.solace.com/t/kubernetes-persistence-volume-issues/1547/2 "2023-01-18T18:15:59Z")
Hi @sulfred , how you doing? I am asking around for someone with Kubernetes experience to take a look at this ticket. But just read through it more carefully and have some questions: In the very first code block, you seem to be getting permission errors for writing to /var/lib/solace/ but your yaml config file is referencing /mnt/volumes/solaceVolume/ … is this intentional, or a copy-paste error from a different configuration? I’m assuming you’re not running this as root, due to permission errors. Have you tried (if you can) running as root? And if running as non-root, there’s some new documentation here you might want to take a look at? [Rootless Containers](https://docs.solace.com/Software-Broker/Container-Tasks/rootless-containers.htm#Rootless_Containers) What version of PubSub+ broker are you installing? I assume it’s the most recent? Anyhow, I’ll ask around internally if someone can take a look at this. ---
**Author:** ![pkondrat](https://yyz1.discourse-cdn.com/solace/user_avatar/community.solace.com/pkondrat/32/2451_2.png) [@pkondrat](https://community.solace.com/u/pkondrat)\ **Post date:** [January 18, 2023, 7:56pm UTC](https://community.solace.com/t/kubernetes-persistence-volume-issues/1547/3 "2023-01-18T19:56:55Z")
Hi @sulfred , The issue is with the hostPath in your persistent volume. There are some limitations when using the hostPath. From the Kubernetes docs ( Volumes | Kubernetes ): “The files or directories created on the underlying hosts are only writable by root. You either need to run your process as root in a  privileged Container  or modify the file permissions on the host to be able to write to a  hostPath  volume.” You can see that the directories that the broker created are owned by root and only writeable by root. The processes in the container by default run as user 1000001. You can open up the permissions to allow group write or change the ownership to 1000001. The other option (and probably easiest) is to use Minikube’s default storage class. Best Regards, Paul ---
**Author:** ![sulfred](https://yyz1.discourse-cdn.com/solace/user_avatar/community.solace.com/sulfred/32/48_2.png) [@sulfred](https://community.solace.com/u/sulfred)\ **Post date:** [January 19, 2023, 12:21am UTC](https://community.solace.com/t/kubernetes-persistence-volume-issues/1547/4 "2023-01-19T00:21:15Z")
Thanks @Aaron , @pkondrat for the information. I will have a look to use the default storage class instead of `hostPath` . This minikube setting is just for testing so there is no urgency for this issue. I will try your suggestions and read the document first. Thanks ---
**Author:** ![prashantk2000](https://avatars.discourse-cdn.com/v4/letter/p/71e660/32.png) [@prashantk2000](https://community.solace.com/u/prashantk2000)\ **Post date:** [February 10, 2023, 7:23am UTC](https://community.solace.com/t/kubernetes-persistence-volume-issues/1547/5 "2023-02-10T07:23:39Z")
> [@pkondrat](#): > > Hi @sulfred , > The issue is with the hostPath in your persistent volume. There are some limitations when using the hostPath. From the Kubernetes docs ( Volumes | Kubernetes ): > “The files or directories created on the underlying hosts are only writable by root. You either need to run your process as root in a  privileged Container  or modify the file permissions on the host to be able to write to a  hostPath  volume.” You can see that the directories that the broker created are owned by root and only writeable by root. The processes in the container by default run as user 1000001. You can open up the permissions to allow group write or change the ownership to 1000001. The other option (and probably easiest) is to use Minikube’s default storage class. > Best Regards, > Paul Hi @pkondrat , How does this works when kuberenets is running in local windows? as securityContext.runAsGroup  - not possible on Windows as there is no GID support and securityContext.privileged  - Windows doesn’t support privileged containers. I am facing the same problem in windows. Below kuberenets docs I am referring [Windows containers in Kubernetes | Kubernetes](https://kubernetes.io/docs/concepts/windows/intro/)[Create a Windows HostProcess Pod | Kubernetes](https://kubernetes.io/docs/tasks/configure-pod-container/create-hostprocess-pod/)[Configure RunAsUserName for Windows pods and containers | Kubernetes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-runasusername/) This is how I have created the pv and pvc ![image.png](https://canada1.discourse-cdn.com/solace/original/2X/b/b63d445302dee6588c5b47e7a59733af4a4b65f6.png) ---
**Author:** ![pkondrat](https://yyz1.discourse-cdn.com/solace/user_avatar/community.solace.com/pkondrat/32/2451_2.png) [@pkondrat](https://community.solace.com/u/pkondrat)\ **Post date:** [February 10, 2023, 1:52pm UTC](https://community.solace.com/t/kubernetes-persistence-volume-issues/1547/6 "2023-02-10T13:52:26Z")
Hi @prashantk2000 , What Kunbernetes distribution are you using on your Windows machine? I wrote a codelab a couple of years ago where I used Docker for Windows on Windows 10 PC to deploy an instance of PubSub+. Here is the link: Windows 10 Helm / Kubernetes Development Environment Setup (solace.dev) Maybe it will help. Best Regards, Paul