Table of Content
kubectl installation
oldhorse@ubdev:~$ curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 64.2M 100 64.2M 0 0 355k 0 0:03:05 0:03:05 --:--:-- 83005
oldhorse@ubdev:~$ chmod +x ./kubectl
oldhorse@ubdev:~$ sudo mv ./kubectl /usr/local/bin/kubectl
verify by
oldhorse@ubdev:~$ which kubectl
/usr/local/bin/kubectl
oldhorse@ubdev:~$ kubectl version
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.0", GitCommit:"925c127ec6b946659ad0fd596fa959be43f0cc05", GitTreeState:"clean", BuildDate:"2017-12-15T21:07:38Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}
enable shell autocompletion for kubectl by:
oldhorse@ubdev:~$ echo "source > ~/.bashrc
minikube installation
oldhorse@ubdev:~$ curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.24.1/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 39.4M 100 39.4M 0 0 343k 0 0:01:57 0:01:57 --:--:-- 302k
here you go,
oldhorse@ubdev:~$ which minikube /usr/local/bin/minikube oldhorse@ubdev:~$ minikube version minikube version: v0.24.1
install docker on ubuntu
There is not docker-ce on ubuntu 17.10, but can install from 17.04 repository:
sudo apt-get install apt-transport-https ca-certificates curl software-properties-common curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu zesty stable" sudo apt-get update
verify here,
oldhorse@ubdev:~$ which -a docker /usr/bin/docker oldhorse@ubdev:~$ docker version Client: Version: 17.09.1-ce API version: 1.32 Go version: go1.8.3 Git commit: 19e2cf6 Built: Thu Dec 7 22:24:28 2017 OS/Arch: linux/amd64
start minikube on ubuntu
By default minikube will look for virtualbox when start minikube,
oldhorse@ubdev:~$ minikube start
Starting local Kubernetes v1.8.0 cluster...
Starting VM...
Downloading Minikube ISO
140.01 MB / 140.01 MB [============================================] 100.00% 0s
E1221 10:01:17.902321 6719 start.go:150] Error starting host: Error creating host: Error executing step: Running precreate checks.
: VBoxManage not found. Make sure VirtualBox is installed and VBoxManage is in the path.
Retrying.
E1221 10:01:17.903734 6719 start.go:156] Error starting host: Error creating host: Error executing step: Running precreate checks.
: VBoxManage not found. Make sure VirtualBox is installed and VBoxManage is in the path
================================================================================
An error has occurred. Would you like to opt in to sending anonymized crash
information to minikube to help prevent future errors?
To opt out of these messages, run the command:
minikube config set WantReportErrorPrompt false
================================================================================
Please enter your response [Y/n]:
Prefer using KVM for this minikube test.
pre-install before kvm driver
sudo apt install libvirt-bin qemu-kvm sudo usermod -a -G libvirt $(whoami) newgrp libvirt
install kvm2 driver
We need to install new KVM2 driver following here.
oldhorse@ubdev:~$ curl -LO https://storage.googleapis.com/minikube/releases/latest/docker-machine-driver-kvm2 && chmod +x docker-machine-driver-kvm2 && sudo mv docker-machine-driver-kvm2 /usr/bin/
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 34.2M 100 34.2M 0 0 310k 0 0:01:52 0:01:52 --:--:-- 76117
oldhorse@ubdev:~$ which docker-machine-driver-kvm2
/usr/bin/docker-machine-driver-kvm2
However got new error,
oldhorse@ubdev:~$ minikube start --vm-driver kvm2 Starting local Kubernetes v1.8.0 cluster... Starting VM... E1210 09:08:29.593703 16013 start.go:150] Error starting host: Error creating host: Error creating machine: Error in driver during machine creation: creating domain: Error defining domain xml: virError(Code=8, Domain=44, Message='invalid argument: could not find capabilities for domaintype=kvm ')
it seems related to nested virtualization since I ran ubuntu vm on top of vmware on win7.
Turn on virtualization in vm setting,
delete minikube and run minikube start again,
oldhorse@ubdev:~$ minikube start --vm-driver kvm2 Starting local Kubernetes v1.8.0 cluster... Starting VM... E1210 09:28:14.513072 1879 start.go:150] Error starting host: Error creating host: Error creating machine: Error in driver during machine creation: Machine didn't return an IP after 120 seconds.
run with debug on,
(minikube) Creating domain... (minikube) Waiting to get IP... (minikube) DBG | Waiting for machine to come up 0/40 (minikube) DBG | Waiting for machine to come up 1/40
it is waiting for vm up but timeout.
opened the issue for minikube, it is more like the bug.
test kvm plugin
oldhorse@ubdev:~$ curl -Lo docker-machine-driver-kvm https://github.com/dhiltgen/docker-machine-kvm/releases/download/v0.10.0/docker-machine-driver-kvm-ubuntu16.04 && chmod +x docker-machine-driver-kvm && sudo mv docker-machine-driver-kvm /usr/local/bin
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 627 0 627 0 0 990 0 --:--:-- --:--:-- --:--:-- 988
100 11.3M 100 11.3M 0 0 260k 0 0:00:44 0:00:44 --:--:-- 232k
oldhorse@ubdev:~$ minikube start --vm-driver kvm --v=7
(minikube) DBG | Starting VM minikube
(minikube) Failed to start: virError(Code=1, Domain=10, Message='internal error: process exited while connecting to monitor: qemu-system-x86_64: /build/qemu-SJhlrB/qemu-2.8+dfsg/target-i386/kvm.c:1805: kvm_put_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.')
E1221 11:24:50.308431 4768 start.go:150] Error starting host: Error creating host: Error creating machine: Error in driver during machine creation: virError(Code=1, Domain=10, Message='internal error: process exited while connecting to monitor: qemu-system-x86_64: /build/qemu-SJhlrB/qemu-2.8+dfsg/target-i386/kvm.c:1805: kvm_put_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.').
virt-manager
This GUI is very helpful to create/manage vm including kvm
sudo apt-get install qemu-kvm libvirt-bin bridge-utils virt-manager
CPU model
From GUI can see the CPU model is host-passthrough, when I start vm in GUI
will get same error of "Assertion `ret == cpu->kvm_msr_buf->nmsrs’ failed" ; however if I changed to Hypervisor Default model, I can start it in GUI.
the minikube vm running with below setting:
hypervisor: KVM CPU: 1 CPU model: Hypervisor Default memory: 2014
so just run
minikube start --v=7
It looks good now,
(minikube) DBG | IP address: 192.168.42.33 (minikube) DBG | Unable to locate IP address for MAC 52:54:00:85:56:eb (minikube) Calling .GetSSHPort (minikube) Calling .GetSSHKeyPath (minikube) Calling .GetSSHUsername Moving files into cluster... Setting up certs... Connecting to cluster... (minikube) Calling .GetURL (minikube) DBG | GetURL called (minikube) DBG | GetIP called for minikube (minikube) DBG | Failed to retrieve dnsmasq leases from /var/lib/libvirt/dnsmasq/docker-machines.leases (minikube) DBG | IP address: 192.168.42.33 (minikube) DBG | Unable to locate IP address for MAC 52:54:00:85:56:eb Setting up kubeconfig... Starting cluster components... Kubectl is now configured to use the cluster. Loading cached images from config file. Making call to close driver server (minikube) Calling .Close (minikube) DBG | Closing plugin on server side Successfully made call to close driver server Making call to close connection to plugin binary Making call to close driver server (minikube) Calling .Close (minikube) DBG | Closing plugin on server side Successfully made call to close driver server Making call to close connection to plugin binary
minikube test
oldhorse@ubdev:~$ minikube status
minikube: Running
cluster: Running
kubectl: Correctly Configured: pointing to minikube-vm at 192.168.42.33
oldhorse@ubdev:~$ minikube ssh
_ _
_ _ ( ) ( )
___ ___ (_) ___ (_)| |/') _ _ | |_ __
/' _ _ \| |/' _ \| || , < ( ) ( )| '_\ /'__`\
| ( ) ( ) || || ( ) || || |\`\ | (_) || |_) )( ___/
(_) (_) (_)(_)(_) (_)(_)(_) (_)\___/'(_,__/'\____)
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3f1bb5ac2583 fed89e8b4248 "/sidecar --v=2 --..." 11 seconds ago Up 10 seconds k8s_sidecar_kube-dns-86f6f55dd5-sgn55_kube-system_10a1ca52-e6a1-11e7-a7e5-5254006d0510_0
64e46895866a 459944ce8cc4 "/dnsmasq-nanny -v..." 11 seconds ago Up 11 seconds k8s_dnsmasq_kube-dns-86f6f55dd5-sgn55_kube-system_10a1ca52-e6a1-11e7-a7e5-5254006d0510_0
957dbec5dd7e 512cd7425a73 "/kube-dns --domai..." 12 seconds ago Up 11 seconds k8s_kubedns_kube-dns-86f6f55dd5-sgn55_kube-system_10a1ca52-e6a1-11e7-a7e5-5254006d0510_0
c8106498f4a0 gcr.io/google_containers/pause-amd64:3.0 "/pause" 12 seconds ago Up 12 seconds k8s_POD_kube-dns-86f6f55dd5-sgn55_kube-system_10a1ca52-e6a1-11e7-a7e5-5254006d0510_0
$ exit
logout
oldhorse@ubdev:~$ minikube docker-env
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.42.33:2376"
export DOCKER_CERT_PATH="/home/oldhorse/.minikube/certs"
export DOCKER_API_VERSION="1.23"
# Run this command to configure your shell:
# eval $(minikube docker-env)
oldhorse@ubdev:~$ eval $(minikube docker-env)
oldhorse@ubdev:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
983e211282d9 fed89e8b4248 "/sidecar --v=2 --..." 17 seconds ago Up 16 seconds k8s_sidecar_kube-dns-86f6f55dd5-wdg64_kube-system_2a098bb7-e6a1-11e7-a7e5-5254006d0510_0
b05806114b3f 459944ce8cc4 "/dnsmasq-nanny -v..." 18 seconds ago Up 17 seconds k8s_dnsmasq_kube-dns-86f6f55dd5-wdg64_kube-system_2a098bb7-e6a1-11e7-a7e5-5254006d0510_0
c20e6848714f 512cd7425a73 "/kube-dns --domai..." 18 seconds ago Up 17 seconds k8s_kubedns_kube-dns-86f6f55dd5-wdg64_kube-system_2a098bb7-e6a1-11e7-a7e5-5254006d0510_0
6c8c76ae957d gcr.io/google_containers/pause-amd64:3.0 "/pause" 18 seconds ago Up 17 seconds k8s_POD_kube-dns-86f6f55dd5-wdg64_kube-system_2a098bb7-e6a1-11e7-a7e5-5254006d0510_0