docker | k8s | WSL
Table of Content

We talk about WSL1 here on win10 early version.

The main limitation between WSL1 and WSL2 is that WSL1 is not full Linux, everything looks good just without Linux Kernel so cannot run docker and k8s server version, only client version to work with remote vm; however WSL2 has full Linux kernel which is fact VM running on top of Hyper-V.

Here are lists we needed for basic devops toolchaining:

  • python3/pip3
  • ansible
  • vagrant
  • docker client
  • kubectl client

hostname

change to own hostname

 
sudo hostname dreamcloud
re-launch wsl 
oldhorse@dreamcloud:~$

python3

by default, python3 already installed in wsl, 

oldhorse@dreamcloud:~$ python3 --version
Python 3.8.10

install pip3,

oldhorse@dreamcloud:~$ sudo apt install python3-pip
oldhorse@dreamcloud:~$ pip --version
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)
oldhorse@dreamcloud:~$ pip3 --version
pip 20.0.2 from sr/lib/python3/dist-packages/pip (python 3.8)

ansible


oldhorse@dreamcloud:~$ pip install ansible
...
Successfully built ansible ansible-core
Installing collected packages: pyparsing, packaging, resolvelib, ansible-core, ansible
Successfully installed ansible-4.6.0 ansible-core-2.11.5 packaging-21.0 pyparsing-2.4.7 resolvelib-0.5.4

oldhorse@dreamcloud:~$ ll .local/bin
total 92
drwxr-xr-x 1 oldhorse oldhorse  4096 Oct  9 15:52 ./
drwx------ 1 oldhorse oldhorse  4096 Oct  9 15:52 ../
-rwxr-xr-x 1 oldhorse oldhorse  6437 Oct  9 15:52 ansible*
-rwxr-xr-x 1 oldhorse oldhorse  6437 Oct  9 15:52 ansible-config*
-rwxr-xr-x 1 oldhorse oldhorse 12919 Oct  9 15:52 ansible-connection*
-rwxr-xr-x 1 oldhorse oldhorse  6437 Oct  9 15:52 ansible-console*
-rwxr-xr-x 1 oldhorse oldhorse  6437 Oct  9 15:52 ansible-doc*
-rwxr-xr-x 1 oldhorse oldhorse  6437 Oct  9 15:52 ansible-galaxy*
-rwxr-xr-x 1 oldhorse oldhorse  6437 Oct  9 15:52 ansible-inventory*
-rwxr-xr-x 1 oldhorse oldhorse  6437 Oct  9 15:52 ansible-playbook*
-rwxr-xr-x 1 oldhorse oldhorse  6437 Oct  9 15:52 ansible-pull*
-rwxr-xr-x 1 oldhorse oldhorse   793 Oct  9 15:52 ansible-test*
-rwxr-xr-x 1 oldhorse oldhorse  6437 Oct  9 15:52 ansible-vault*

It is located at ~/.local/bin, so need to add it to $PATH,

export PATH=~/.local/bin:$PATH

oldhorse@dreamcloud:~$ which -a ansible
/home/oldhorse/.local/bin/ansible

oldhorse@dreamcloud:~$ ansible --version
ansible [core 2.11.5]
  config file = None
  configured module search path = ['/home/oldhorse/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/oldhorse/.local/lib/python3.8/site-packages/ansible
  ansible collection location = /home/oldhorse/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/oldhorse/.local/bin/ansible
  python version = 3.8.10 (default, Sep 28 2021, 16:10:42) [GCC 9.3.0]
  jinja version = 2.10.1
  libyaml = True

vagrant

You have two options – directly use vagrant windows or install vagrant for Linux, for my test vagrant for windows is working well, 

oldhorse@dreamcloud:~$ which -a vagrant.exe
/mnt/c/dclab/portableapps/vagrant/bin/vagrant.exe

oldhorse@dreamcloud:~$ vagrant.exe version
Installed Version: 2.2.18
Latest Version: 2.2.18

You're running an up-to-date version of Vagrant!

also make sure Virtualbox in $PATH,

oldhorse@dreamcloud:~$ which -a VBoxManage.exe
/mnt/c/Program_Files/Oracle/VirtualBox/VBoxManage.exe

docker client use case 1: Docker Machine

you can create own docker machine which is boot2docker vm with dedicate IP, then you need to directly access to dedicate ip and port 2376.

oldhorse@dreamcloud:~$ docker-machine.exe create test
Running pre-create checks...
Creating machine...
(test) Copying C:\Users\oldhorse\.docker\machine\cache\boot2docker.iso to C:\Users\oldhorse\.docker\machine\machines\test\
boot2docker.iso...
...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine.exe env test
oldhorse@dreamcloud:~$ docker-machine.exe ls
NAME   ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER      ERRORS
test   -        virtualbox   Running   tcp://192.168.99.122:2376           v19.03.12

you need to config env for docker to access docker server in remote docker machine,

oldhorse@dreamcloud:~$ docker-machine.exe env test --shell=bash
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.99.122:2376"
export DOCKER_CERT_PATH="C:\Users\oldhorse\.docker\machine\machines\test"
export DOCKER_MACHINE_NAME="test"
export COMPOSE_CONVERT_WINDOWS_PATHS="true"
# Run this command to configure your shell:
# eval $(docker-machine.exe env test --shell=bash)

The window path won’t work when run on wsl, here is fix, 

export DOCKER_CERT_PATH="/mnt/c/Users/oldhorse/.docker/machine/machines/test"

then you can run docker locally, 

oldhorse@dreamcloud:~$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.6.3-docker)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 19.03.12
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.19.130-boot2docker
 Operating System: Boot2Docker 19.03.12 (TCL 10.1)
 OSType: linux
 Architecture: x86_64
 CPUs: 1
 Total Memory: 985.4MiB
 Name: test
 ID: ONBA:CHPR:SM45:BPF6:2U5A:HI6L:SAHQ:CCZB:GUF2:4JUI:IKAD:NDZU
 Docker Root Dir: /mnt/sda1/var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
  provider=virtualbox
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

docker env script for docker-machine vm

it is handy function as part of https://github.com/robertluwang/portabledevops

# function to setup env for docker-machine vm host
    denv(){
    if [ $(uname|cut -d'_' -f1) = 'Linux' ] && [ $(uname -a|awk '{print $4}'|cut -d'-' -f2) = Microsoft ]; then
            eval $(dm env "$@" --shell bash |sed -e 's|\\|/|g' -e 's|C:/|/mnt/c/|g')
    else
            eval $(dm env "$@" --shell bash)
    fi
    }

    export -f denv

after docker-machine created new machine test, first thing is to setup docker env, here is sample how to use denv,

 
oldhorse@dreamcloud:~$ denv test
oldhorse@dreamcloud:~$ docker context ls
NAME        DESCRIPTION                               DOCKER ENDPOINT             KUBERNETES ENDPOINT   ORCHESTRATOR
default *   Current DOCKER_HOST based configuration   tcp://192.168.99.122:2376                         swarm

docker client use case 2: remote Linux docker vm

more details please see previous post,

http://dreamcloud.artark.ca/wsl-for-remote-docker-and-k8s-cluster/

here is summary of steps:

1) ubuntu 

changed docker server to expose to outside via tcp IP and port, use un-encrypted port 2375,
vagrant@k8s-master$ sudo mkdir -p /etc/systemd/system/docker.service.d
vagrant@k8s-master$ sudo nano /etc/systemd/system/docker.service.d/options.conf
vagrant@k8s-master$ cat /etc/systemd/system/docker.service.d/options.conf
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H unix:// -H tcp://0.0.0.0:2375

reload config and verify exposed port, 
vagrant@k8s-master$ sudo systemctl daemon-reload
vagrant@k8s-master$ sudo systemctl restart docker
vagrant@k8s-master$ ps -aux |grep dockerd

2) virtualbox GUI 
next need to map laptop or wsl host port to virtualbox vm which is Docker server port 2375 
127.0.0.1:2375 -> :2375

3) WSL 
oldhorse@dreamcloud:~$ unset DOCKER_TLS_VERIFY
oldhorse@dreamcloud:~$ export DOCKER_HOST="tcp://127.0.0.1:2375"

docker env script for remote Linux docker vm

it is handy function as part of https://github.com/robertluwang/portabledevops

# function to setup env for docker vm on Virtualbox, assume dockerd listen on tcp:0.0.0.0:2375
    denv2vm(){
    unset DOCKER_TLS_VERIFY
    export DOCKER_HOST="tcp://127.0.0.1:2375"
    }
    
    export -f denv2vm

usage sample,

 
oldhorse@dreamcloud:~$ denv2vm
oldhorse@dreamcloud:~$ docker context ls
NAME        DESCRIPTION                               DOCKER ENDPOINT        KUBERNETES ENDPOINT   ORCHESTRATOR
default *   Current DOCKER_HOST based configuration   tcp://127.0.0.1:2375                         swarm

kubectl client

Please see more details in previous post,

http://dreamcloud.artark.ca/wsl-for-remote-docker-and-k8s-cluster/

Here are summary of steps:

1) install kubectk client 
oldhorse@dreamcloud:~$ curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"
oldhorse@dreamcloud:~$ chmod +x ./kubectl
oldhorse@dreamcloud:~$ sudo mv ./kubectl /usr/local/bin/kubectl
oldhorse@dreamcloud:~$ kubectl version --client
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.2", GitCommit:"8b5a19147530eaac9476b0ab82980b4088bbc1b2", GitTreeState:"clean", BuildDate:"2021-09-15T21:38:50Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}

2) transfer .kube from ubuntu to wsl, 
oldhorse@dreamcloud:~$ cd
oldhorse@dreamcloud:~$ scp -r vagrant@192.168.22.23:/home/vagrant/.kube .

3) verify kubectl client is working
oldhorse@dreamcloud:~$ kubectl get no
NAME         STATUS     ROLES                  AGE   VERSION
k8s-master   Ready      control-plane,master   10d   v1.21.4
k8s-node1    NotReady                    10d   v1.21.4
k8s-node2    NotReady                    10d   v1.21.4

short and completion,

add below to ~/.bashrc

# short for kubectl
source <(kubectl completion bash)
alias k=kubectl
complete -F __start_kubectl k
export do='--dry-run=client -o yaml'