Enable Container run with readonly root filesystem

[alwinmark]

Therefore moved home directory into data volume and fix right of Erlang Cookie in case of reclaiming a volume on kubernetes.

fixes: #243

[hsudbrock]

I have the same issue that @alwinmark wants to resolve here: The root filesystem on my pod in Kubernetes is read-only, and only the persistent volume mounted into the pod under vernemq/data is read-write.

The solution proposed in this pull request by @alwinmark solved the issue for me.

Is there anything where I could help to get this pull request merged?

[ioolkos]

@hsudbrock hm, I see the issue. I just wasn't sure about any possible further consequences in making the defined data mount the home directory.

Does this actually mean that most users run this with a write-enabled Kubernetes root file system; or is your situation the default?
(I ask because the issue hasn't come up much)

[hsudbrock]

I just wasn't sure about any possible further consequences in making the defined data mount the home directory.

I also did not look more deeply into any further consequences. What I tried was running verneMQ with the change suggested by @alwinmark, and I did not see any problems yet (but I only published and consumed a few simple messages in my test, nothing fancy). We could also think about alternatives, where we use a second mount for the home directory (so we don't mix it with the data mount); maybe one could even use a mount that does not require a PVC (e.g., mounting a pre-filled erlang cookie file from a configMap, or using an emptyDir mount). With the emptyDir I am not sure if this makes problems, because with that approach the erlang cookie would not survive restarts of the verneMQ pods, and I cannot say with my current Erlang knowledge if we need the erlang cookie to survive pod restarts.

Does this actually mean that most users run this with a write-enabled Kubernetes root file system; or is your situation the default?

To be honest, I don't know what the more common situation is. In my case, the cluster has strict security policies in place, including a read-only root file system. From "googling", this seems to be a security best practice, to lower the attack surface of the cluster (see, e.g., https://docs.bridgecrew.io/docs/bc_k8s_21 as an example for such a policy with a short description). I would guess that such security policies are more common in enterprise settings.

[ioolkos]

@hsudbrock Thanks for the elaborations. I'm researching alternatives and see what other Erlang projects do. I think if the cookie is the only reason, we can do it from a ConfigMap.
https://blog.differentpla.net/blog/2022/01/09/erlang-cookies-and-kubernetes/

[hsudbrock]

Short update:

• I attempted to go the way with the ConfigMap; this was not successful, because I did not manage to mount the erlang cookie while giving it access rights for the owner only (which is required by erlang). When mounting from a ConfigMap, the file always had access rights for more than the owner.
• For my purposes, I could now work around requiring a root-only root filesystem (so I do not need a solution for my own purposes anymore) - but am of course still open to helping finding a solution for this pull request here.

[ioolkos]

@hsudbrock Thanks for your valuable testing & update! I also think, we'll need to merge the general solution using a mount.

Need to check vernemq.sh script for correctness of all directory paths with this change

[subhasniveus]

much needed fix.