Container-in-container also pushes a number of responsibilities from the the App Service down into the head container, like ingress, service management, health, logging etc. You might as well run compose/swarm on your own VM(s) than reinvent those wheels.
AKS + kompose might be a not too bad option. AKS can deploy on the private vnet and can do private endpoints. The cluster will probably fall over once a year for but it will mostly manage itself. If you leave k8s to auto upgrade, run a microk8s instance as a test env somewhere and it will hit the upgrade issues before AKS releases a k8s version.
- https://github.com/kubernetes/kompose
- https://learn.microsoft.com/en-us/azure/aks/private-clusters...
Since a new server is launched for every preview, it would be cool if the packaging used cloud-init vs docker-compose.
If containers are a focus perhaps you could support k8s deployment/svc manifests and compose via something like https://github.com/kubernetes/kompose
For k8s you could quickly launch a single node cluster with kind
I'd strongly recommend using a hosted k8s - either GKE, EKS, or I believe digital ocean have just released one.
If you want to use an existing VPS just to test it out, see the docs here https://kubernetes.io/docs/setup/independent/create-cluster-...
Once you have the cluster running, kompose[1] might be a nice tool if you're used to using docker-compose, however I'd say just use it as a guideline - you'll probably want to rewrite most of what it generates at one point or another
or
or keep using docker's dev client, it can target a locally running k8s now.
I still keep hoping for kubernetes kompose (https://github.com/kubernetes/kompose) to bring the simplicity of Docker Swarmkit to k8s.
Or will Docker Infrakit bring creeping sophistication first and eat kuberentes lunch ? (https://github.com/docker/infrakit/pull/601)