create centos7 k8s vagrant box from dreamcloud-centos7

Table of Content

launch dev vm

it is based on my dev ready vagrant box dreancloud/centos7.

create project folder

$ mdkir ~/vagrant/centos7k8sbox
$ cd ~/vagrant/centos7k8sbox

prepare the Vagrantfile,

Vagrant.configure("2") do |config|
    config.vm.box="dreamcloud/centos7"
    
    config.vm.define "centos7k8s" do |ck8s|
        ck8s.vm.hostname = "centos7k8s"
        ck8s.vm.provider :virtualbox do |vb|
            vb.name="centos7k8s"
            vb.memory=1024
        end
    end
end

then boot up the vm,

$ vagrant up
Bringing machine 'centos7k8s' up with 'virtualbox' provider...
==> centos7k8s: Importing base box 'dreamcloud/centos7'...
==> centos7k8s: Matching MAC address for NAT networking...
==> centos7k8s: Checking if box 'dreamcloud/centos7' is up to date...
==> centos7k8s: Setting the name of the VM: centos7k8s
==> centos7k8s: Clearing any previously set network interfaces...
==> centos7k8s: Preparing network interfaces based on configuration...
    centos7k8s: Adapter 1: nat
==> centos7k8s: Forwarding ports...
    centos7k8s: 22 (guest) => 2222 (host) (adapter 1)
==> centos7k8s: Running 'pre-boot' VM customizations...
==> centos7k8s: Booting VM...
==> centos7k8s: Waiting for machine to boot. This may take a few minutes...
    centos7k8s: SSH address: 127.0.0.1:2222
    centos7k8s: SSH username: vagrant
    centos7k8s: SSH auth method: private key
    centos7k8s:
    centos7k8s: Vagrant insecure key detected. Vagrant will automatically replace
    centos7k8s: this with a newly generated keypair for better security.
    centos7k8s:
    centos7k8s: Inserting generated public key within guest...
    centos7k8s: Removing insecure key from the guest if it's present...
    centos7k8s: Key inserted! Disconnecting and reconnecting using new SSH key...
==> centos7k8s: Machine booted and ready!
[centos7k8s] GuestAdditions 5.1.30 running --- OK.
==> centos7k8s: Checking for guest additions in VM...
==> centos7k8s: Setting hostname...
==> centos7k8s: Mounting shared folders...
    centos7k8s: /vagrant => C:/oldhorse/portableapps/msys64/home/oldhorse/vagrant/centos7k8sbox

$ vagrant ssh

reset ssh keypair

The reason I re-init ssh keypair for new vm centos7k8s here, because the keypair has been changed when I launch this vm from vagrant box dreamcloud/centos7. It will be failed if we don’t reset keypair, vagrant has no idea for this new keypair. It is common practice if we build new vagrant box based on existing box.

[vagrant@centos7k8s ~]$ curl -Lo /home/vagrant/.ssh/authorized_keys \
> https://raw.githubusercontent.com/hashicorp/vagrant/master/keys/vagrant.pub
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   409  100   409    0     0    559      0 --:--:-- --:--:-- --:--:--   560
[vagrant@centos7k8s ~]$ chmod 0600 /home/vagrant/.ssh/authorized_keys

docker CE

$ curl -fsSL get.docker.com -o get-docker.sh
$ sudo sh get-docker.sh
$ sudo systemctl enable docker
$ sudo systemctl start docker
$ sudo systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
   Active: active (running) since Mon 2018-01-01 11:27:22 EST; 1s ago

add vagrant to docker group

vagrant@centos7k8s:~$ sudo usermod -aG docker vagrant

relogin, then can run docker as normal user,

vagrant@centos7k8s:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

compatability for cgroup driver,

[vagrant@centos7k8s ~]$ cat << EOF > /tmp/daemon.json
{
  "exec-opts": ["native.cgroupdriver=cgroupfs"]
}
EOF

[vagrant@centos7k8s ~]$ sudo mv /tmp/daemon.json /etc/docker/

restart docker,

$ sudo systemctl start docker

k8s installation

kubernetes.repo

[vagrant@centos7k8s ~]$ cat  <<EOF  > /tmp/repo
[kubernetes] 
name=Kubernetes 
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64 
enabled=1 
gpgcheck=1 
repo_gpgcheck=1 
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg 
EOF

[vagrant@centos7k8s ~]$ sudo mv /tmp/repo /etc/yum.repos.d/kubernetes.repo

install k8s,

[vagrant@centos7k8s ~]$ sudo yum install -y kubelet kubeadm kubectl

start k8s,

[vagrant@centos7k8s ~]$ sudo systemctl enable kubelet && sudo systemctl start kubelet
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /etc/systemd/system/kubelet.service.

iptables

[vagrant@centos7k8s ~]$ cat <<EOF  > /tmp/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

[vagrant@centos7k8s ~]$ sudo mv /tmp/k8s.conf /etc/sysctl.d/
[vagrant@centos7k8s ~]$ sudo sysctl --system
* Applying /usr/lib/sysctl.d/00-system.conf ...

verify by,

[vagrant@centos7k8s ~]$ which kubeadm
/usr/bin/kubeadm

[vagrant@centos7k8s ~]$ which kubectl
/usr/bin/kubectl

[vagrant@centos7k8s ~]$ which kubelet
/usr/bin/kubelet

make sure kubelet cgroup driver matchs with docker:

$ docker info | grep -i cgroup
Cgroup Driver: cgroupfs

$ sudo sed -i "s/cgroup-driver=systemd/cgroup-driver=cgroupfs/g"
/etc/systemd/system/kubelet.service.d/10-kubeadm.conf

defragmentation the vm

sudo dd if=/dev/zero of=/EMPTY bs=1M
sudo rm -f /EMPTY

clean up vm

sudo dd if=/dev/zero of=/EMPTY bs=1M
sudo rm -f /EMPTY
sudo yum clean all
sudo rm -rf /var/cache/yum
cat /dev/null > ~/.bash_history && history -c

package vagrant box

from local shell, when give vm name ub17k8s we just created, vagrant will compress it as box format,

$ cd ~/vagrant/centos7k8sbox
$ vagrant package --output dreamcloud-centos7k8s-vagrant.box --base centos7k8s
==> centos7k8s: Attempting graceful shutdown of VM...
==> centos7k8s: Clearing any previously set forwarded ports...
==> centos7k8s: Exporting VM...
==> centos7k8s: Compressing package to: C:/oldhorse/portableapps/msys64/home/oldhorse/vagrant/centos7k8sbox/dreamcloud-centos7k8s-vagrant.box

dreamcloud-centos7k8s-vagrant.box 726M.

test vagrant box

$ cd ~/vagrant/ctk8stest
$ vagrant box add dreamcloud/centos7k8s dreamcloud-centos7k8s-vagrant.box
$ vagrant box list
$ vagrant init dreamcloud/centos7k8s

prepare test Vagrantfile,

Vagrant.configure("2") do |config|
    config.vm.box="dreamcloud/centos7k8s"

    config.vm.define "centos7k8s-master" do |master|
        master.vm.hostname = "centos7k8s-master"
        master.vm.network :private_network, ip: "10.120.0.15"
        master.vm.network "forwarded_port", guest: 8443, host: 8443, protocol: "tcp"
        master.vm.network "forwarded_port", guest: 30000, host: 30000, protocol: "tcp"
        master.vm.provider :virtualbox do |vb|
            vb.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
            vb.name="centos7k8s-master"
            vb.memory=1024
        end
    end

    config.vm.define "centos7k8s-node" do |node|
        node.vm.hostname = "centos7k8s-node"
        node.vm.network :private_network, ip: "10.120.0.16"
        node.vm.provider :virtualbox do |vb|
            vb.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
            vb.name="centos7k8s-node"
            vb.memory=1024
        end
    end
end

Let’s boot k8s cluster up,

$ vagrant up
$ vagrant status
Current machine states:
centos7k8s-master running (virtualbox)
centos7k8s-node running (virtualbox)