what is minishift
minishift is one node openshift 3.x, it ported from minikube, so they look very similar.
- one single file written in Go
- easy installation by download it and place to one folder, then add path to $PATH
- support all platform
- best for developer also good to admin training
- run on docker inside vm
install minishift on win10
We want to run minishift on wsl1 on win10, choice window version of minishift because it naturely integrated with windows Virtualbox.
Download minishift for win from https://github.com/minishift/minishift/releases.
extract single binary minishift.exe from package, and place to local folder, add this folder to %PATH%, then you can run it anywhere, including wsl1.
run minishift on wsl1
From you wsl1 you only can run minishift.exe instead of minishift, you can create alias for it.
alias minishift=minishift.exe
Let’s start minishift on wsl1,
oldhorse@myhost:~$ minishift start -- Starting profile 'minishift' -- Check if deprecated options are used ... OK -- Checking if https://github.com is reachable ... OK -- Checking if requested OpenShift version 'v3.11.0' is valid ... OK -- Checking if requested OpenShift version 'v3.11.0' is supported ... OK -- Checking if requested hypervisor 'virtualbox' is supported on this platform ... OK -- Checking if VirtualBox is installed ... OK -- Checking the ISO URL ... OK -- Checking if provided oc flags are supported ... OK -- Starting the OpenShift cluster using 'virtualbox' hypervisor ... -- Minishift VM will be configured with ... Memory: 4 GB vCPUs : 2 Disk size: 20 GB -- Starting Minishift VM ............................... OK -- Checking for IP address ... OK -- Checking for nameservers ... OK -- Checking if external host is reachable from the Minishift VM ... Pinging 8.8.8.8 ... OK -- Checking HTTP connectivity from the VM ... Retrieving http://minishift.io/index.html ... OK -- Checking if persistent storage volume is mounted ... OK -- Checking available disk space ... 1% used OK -- Writing current configuration for static assignment of IP address ... OK Importing 'openshift/origin-control-plane:v3.11.0' . CACHE MISS Importing 'openshift/origin-docker-registry:v3.11.0' . CACHE MISS Importing 'openshift/origin-haproxy-router:v3.11.0' . CACHE MISS -- OpenShift cluster will be configured with ... Version: v3.11.0 -- Pulling the OpenShift Container Image ........................... OK -- Copying oc binary from the OpenShift container image to VM ... OK -- Starting OpenShift cluster .......................................................................... Getting a Docker client ... Checking if image openshift/origin-control-plane:v3.11.0 is available ... E0203 04:40:47.133168 2836 helper.go:173] Reading docker config from /home/docker/.docker/config.json failed: open /home/dock er/.docker/config.json: no such file or directory, will attempt to pull image docker.io/openshift/origin-cli:v3.11.0 anonymously Pulling image openshift/origin-cli:v3.11.0 Image pull complete E0203 04:40:48.175832 2836 helper.go:173] Reading docker config from /home/docker/.docker/config.json failed: open /home/dock er/.docker/config.json: no such file or directory, will attempt to pull image docker.io/openshift/origin-node:v3.11.0 anonymousl y Pulling image openshift/origin-node:v3.11.0 Pulled 5/6 layers, 84% complete Pulled 6/6 layers, 100% complete Extracting Image pull complete Checking type of volume mount ... Determining server IP ... Using public hostname IP 192.168.99.100 as the host IP Checking if OpenShift is already running ... Checking for supported Docker version (=>1.22) ... Checking if insecured registry is configured properly in Docker ... Checking if required ports are available ... Checking if OpenShift client is configured properly ... Checking if image openshift/origin-control-plane:v3.11.0 is available ... I0203 04:41:24.611906 2836 config.go:40] Running "create-master-config" Starting OpenShift using openshift/origin-control-plane:v3.11.0 ... I0203 04:41:26.607561 2836 config.go:46] Running "create-node-config" I0203 04:41:27.951762 2836 flags.go:30] Running "create-kubelet-flags" I0203 04:41:28.385720 2836 run_kubelet.go:49] Running "start-kubelet" I0203 04:41:28.605406 2836 run_self_hosted.go:181] Waiting for the kube-apiserver to be ready ... I0203 04:42:41.623268 2836 interface.go:26] Installing "kube-proxy" ... I0203 04:42:41.623917 2836 interface.go:26] Installing "kube-dns" ... I0203 04:42:41.623925 2836 interface.go:26] Installing "openshift-service-cert-signer-operator" ... I0203 04:42:41.623930 2836 interface.go:26] Installing "openshift-apiserver" ... I0203 04:42:41.623999 2836 apply_template.go:81] Installing "openshift-apiserver" I0203 04:42:41.624682 2836 apply_template.go:81] Installing "kube-dns" I0203 04:42:41.624902 2836 apply_template.go:81] Installing "openshift-service-cert-signer-operator" I0203 04:42:41.625820 2836 apply_template.go:81] Installing "kube-proxy" I0203 04:42:47.546697 2836 interface.go:41] Finished installing "kube-proxy" "kube-dns" "openshift-service-cert-signer-operat or" "openshift-apiserver" I0203 04:45:06.647985 2836 run_self_hosted.go:242] openshift-apiserver available I0203 04:45:06.648720 2836 interface.go:26] Installing "openshift-controller-manager" ... I0203 04:45:06.648738 2836 apply_template.go:81] Installing "openshift-controller-manager" I0203 04:45:10.817807 2836 interface.go:41] Finished installing "openshift-controller-manager" Adding default OAuthClient redirect URIs ... Adding router ... Adding sample-templates ... Adding persistent-volumes ... Adding web-console ... Adding centos-imagestreams ... Adding registry ... I0203 04:45:10.841221 2836 interface.go:26] Installing "openshift-router" ... I0203 04:45:10.841231 2836 interface.go:26] Installing "sample-templates" ... I0203 04:45:10.841238 2836 interface.go:26] Installing "persistent-volumes" ... I0203 04:45:10.841247 2836 interface.go:26] Installing "openshift-web-console-operator" ... I0203 04:45:10.841255 2836 interface.go:26] Installing "centos-imagestreams" ... I0203 04:45:10.841261 2836 interface.go:26] Installing "openshift-image-registry" ... I0203 04:45:10.842121 2836 interface.go:26] Installing "sample-templates/mariadb" ... I0203 04:45:10.842131 2836 interface.go:26] Installing "sample-templates/dancer quickstart" ... I0203 04:45:10.842139 2836 interface.go:26] Installing "sample-templates/jenkins pipeline ephemeral" ... I0203 04:45:10.842145 2836 interface.go:26] Installing "sample-templates/cakephp quickstart" ... I0203 04:45:10.842152 2836 interface.go:26] Installing "sample-templates/django quickstart" ... I0203 04:45:10.842159 2836 interface.go:26] Installing "sample-templates/nodejs quickstart" ... I0203 04:45:10.842165 2836 interface.go:26] Installing "sample-templates/rails quickstart" ... I0203 04:45:10.842172 2836 interface.go:26] Installing "sample-templates/sample pipeline" ... I0203 04:45:10.842179 2836 interface.go:26] Installing "sample-templates/mongodb" ... I0203 04:45:10.842186 2836 interface.go:26] Installing "sample-templates/mysql" ... I0203 04:45:10.842192 2836 interface.go:26] Installing "sample-templates/postgresql" ... I0203 04:45:10.842234 2836 apply_list.go:67] Installing "sample-templates/postgresql" I0203 04:45:10.842989 2836 apply_template.go:81] Installing "openshift-web-console-operator" W0203 04:45:10.843588 2836 create_secret.go:78] Error reading $HOME/.docker/config.json: open /home/docker/.docker/config.jso n: no such file or directory, imagestream import credentials will not be setup I0203 04:45:10.843621 2836 apply_list.go:67] Installing "centos-imagestreams" I0203 04:45:10.843742 2836 apply_list.go:67] Installing "sample-templates/mariadb" I0203 04:45:10.843842 2836 apply_list.go:67] Installing "sample-templates/dancer quickstart" I0203 04:45:10.843982 2836 apply_list.go:67] Installing "sample-templates/jenkins pipeline ephemeral" I0203 04:45:10.844093 2836 apply_list.go:67] Installing "sample-templates/cakephp quickstart" I0203 04:45:10.844184 2836 apply_list.go:67] Installing "sample-templates/django quickstart" I0203 04:45:10.844274 2836 apply_list.go:67] Installing "sample-templates/nodejs quickstart" I0203 04:45:10.844367 2836 apply_list.go:67] Installing "sample-templates/rails quickstart" I0203 04:45:10.844460 2836 apply_list.go:67] Installing "sample-templates/sample pipeline" I0203 04:45:10.844551 2836 apply_list.go:67] Installing "sample-templates/mongodb" I0203 04:45:10.844714 2836 apply_list.go:67] Installing "sample-templates/mysql" I0203 04:45:35.249245 2836 interface.go:41] Finished installing "sample-templates/mariadb" "sample-templates/dancer quickstar t" "sample-templates/jenkins pipeline ephemeral" "sample-templates/cakephp quickstart" "sample-templates/django quickstart" "sam ple-templates/nodejs quickstart" "sample-templates/rails quickstart" "sample-templates/sample pipeline" "sample-templates/mongod b" "sample-templates/mysql" "sample-templates/postgresql" I0203 04:47:05.845076 2836 interface.go:41] Finished installing "openshift-router" "sample-templates" "persistent-volumes" "o penshift-web-console-operator" "centos-imagestreams" "openshift-image-registry" Login to server ... Creating initial project "myproject" ... Server Information ... OpenShift server started. The server is accessible via web console at: https://192.168.99.100:8443/console You are logged in as: User: developer Password: <any value> To login as administrator: oc login -u system:admin -- Exporting of OpenShift images is occuring in background process with pid 20800.
default minishift vm:
- Memory: 4 GB
- vCPUs : 2
- Disk size: 20 GB
You also can set specific capacity to minishift vm.
--cpus int default is 2 --memory string default "4GB" --disk-size string default "20GB"
minishift vm is centos-7 based, you can check out directly on vm node,
minishift ssh [docker@minishift ~]$ uname -a Linux minishift 3.10.0-1127.19.1.el7.x86_64 #1 SMP Tue Aug 25 17:23:54 UTC 2020 x86_64 x86_64 x--memory string86_64 GNU/Linux [docker@minishift ~]$ cat /etc/os-release NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="[https://www.centos.org/](https://www.centos.org/)" BUG_REPORT_URL="[https://bugs.centos.org/](https://bugs.centos.org/)" CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7" VARIANT="minishift" VARIANT_VERSION="1.17.0" BUILD_ID="a18b447-25092020060406-local"
minishift status
oldhorse@myhost:~$ minishift version minishift v1.34.3+4b58f89 oldhorse@myhost:~$ minishift status Minishift: Running Profile: minishift OpenShift: Running (openshift v3.11.0+32a500f-598) DiskUsage: 16% of 19G (Mounted On: /mnt/sda1) CacheUsage: 1.711 GB (used by oc binary, ISO or cached images) oldhorse@myhost:~$ minishift ip 192.168.99.100
minishift console
You can printout console url or directly launched console web gui,
oldhorse@myhost:~$ minishift console --url https://192.168.99.100:8443/console oldhorse@myhost:~$ minishift console --machine-readable HOST=192.168.99.100 PORT=8443 CONSOLE_URL=https://192.168.99.100:8443/console oldhorse@myhost:~$ minishift console Opening the OpenShift Web console in the default browser... '\\wsl$\Ubuntu\home\oldhorse' CMD.EXE was started with the above path as the current directory. UNC paths are not supported. Defaulting to Windows directory. default console login: developer/developer
minishift openshift
minishift can interactive local openshift cluster directly, common use case is to check openshift config details,
oldhorse@myhost:~$ minishift openshift config view admissionConfig: pluginConfig: openshift.io/ImagePolicy: configuration: apiVersion: v1 executionRules: - matchImageAnnotations: - key: images.openshift.io/deny-execution value: "true" name: execution-denied onResources: - resource: pods - resource: builds reject: true skipOnResolutionFailure: true kind: ImagePolicyConfig location: "" aggregatorConfig: proxyClientInfo: certFile: openshift-aggregator.crt keyFile: openshift-aggregator.key apiLevels: - v1 apiVersion: v1 auditConfig: auditFilePath: "" enabled: false logFormat: "" maximumFileRetentionDays: 0 maximumFileSizeMegabytes: 0 maximumRetainedFiles: 0 policyConfiguration: null policyFile: "" webHookKubeConfig: "" webHookMode: "" authConfig: oauthMetadataFile: "" requestHeader: clientCA: frontproxy-ca.crt clientCommonNames: - system:openshift-aggregator extraHeaderPrefixes: - X-Remote-Extra- groupHeaders: - X-Remote-Group usernameHeaders: - X-Remote-User webhookTokenAuthenticators: null controllerConfig: controllers: - '*' election: null serviceServingCert: signer: certFile: service-signer.crt keyFile: service-signer.key controllers: "" corsAllowedOrigins: - //127\.0\.0\.1(:|$) - //192\.168\.99\.100:8443$ - //localhost(:|$) dnsConfig: allowRecursiveQueries: true bindAddress: 0.0.0.0:8053 bindNetwork: tcp4 etcdClientInfo: ca: ca.crt certFile: master.etcd-client.crt keyFile: master.etcd-client.key urls: - https://192.168.99.100:4001 etcdConfig: address: 192.168.99.100:4001 peerAddress: 192.168.99.100:7001 peerServingInfo: bindAddress: 0.0.0.0:7001 bindNetwork: tcp4 certFile: etcd.server.crt clientCA: ca.crt keyFile: etcd.server.key namedCertificates: null servingInfo: bindAddress: 0.0.0.0:4001 bindNetwork: tcp4 certFile: etcd.server.crt clientCA: ca.crt keyFile: etcd.server.key namedCertificates: null storageDirectory: /var/lib/etcd etcdStorageConfig: kubernetesStoragePrefix: kubernetes.io kubernetesStorageVersion: v1 openShiftStoragePrefix: openshift.io openShiftStorageVersion: v1 imageConfig: format: openshift/origin-${component}:v3.11.0 latest: false imagePolicyConfig: disableScheduledImport: false maxImagesBulkImportedPerRepository: 50 maxScheduledImageImportsPerMinute: 60 scheduledImageImportMinimumIntervalSeconds: 900 jenkinsPipelineConfig: autoProvisionEnabled: true parameters: null serviceName: jenkins templateName: jenkins-ephemeral templateNamespace: openshift kind: MasterConfig kubeletClientInfo: ca: ca.crt certFile: master.kubelet-client.crt keyFile: master.kubelet-client.key port: 10250 kubernetesMasterConfig: apiLevels: - v1 apiServerArguments: null controllerArguments: null disabledAPIGroupVersions: {} masterEndpointReconcileTTL: 15 masterIP: 192.168.99.100 podEvictionTimeout: 5m proxyClientInfo: certFile: master.proxy-client.crt keyFile: master.proxy-client.key schedulerArguments: null schedulerConfigFile: "" servicesNodePortRange: 30000-32767 servicesSubnet: 172.30.0.0/16 masterClients: openshiftLoopbackClientConnectionOverrides: acceptContentTypes: application/vnd.kubernetes.protobuf,application/json burst: 600 contentType: application/vnd.kubernetes.protobuf qps: 300 openshiftLoopbackKubeConfig: openshift-master.kubeconfig masterPublicURL: https://192.168.99.100:8443 networkConfig: clusterNetworks: - cidr: 10.128.0.0/14 hostSubnetLength: 9 externalIPNetworkCIDRs: null ingressIPNetworkCIDR: 172.29.0.0/16 networkPluginName: "" serviceNetworkCIDR: 172.30.0.0/16 vxlanPort: 4789 oauthConfig: alwaysShowProviderSelection: false assetPublicURL: https://192.168.99.100:8443/console/ grantConfig: method: auto serviceAccountMethod: prompt identityProviders: - challenge: true login: true mappingMethod: claim name: anypassword provider: apiVersion: v1 kind: AllowAllPasswordIdentityProvider masterCA: ca-bundle.crt masterPublicURL: https://192.168.99.100:8443 masterURL: https://192.168.99.100:8443 sessionConfig: sessionMaxAgeSeconds: 300 sessionName: ssn sessionSecretsFile: "" templates: null tokenConfig: accessTokenMaxAgeSeconds: 86400 authorizeTokenMaxAgeSeconds: 300 policyConfig: userAgentMatchingConfig: defaultRejectionMessage: "" deniedClients: null requiredClients: null projectConfig: defaultNodeSelector: "" projectRequestMessage: "" projectRequestTemplate: "" securityAllocator: mcsAllocatorRange: s0:/2 mcsLabelsPerProject: 5 uidAllocatorRange: 1000000000-1999999999/10000 routingConfig: subdomain: 192.168.99.100.nip.io serviceAccountConfig: limitSecretReferences: false managedNames: - default - builder - deployer masterCA: ca-bundle.crt privateKeyFile: serviceaccounts.private.key publicKeyFiles: - serviceaccounts.public.key servingInfo: bindAddress: 0.0.0.0:8445 bindNetwork: tcp4 certFile: /var/serving-cert/tls.crt clientCA: ca.crt keyFile: /var/serving-cert/tls.key maxRequestsInFlight: 1200 namedCertificates: null requestTimeoutSeconds: 3600 volumeConfig: dynamicProvisioningEnabled: true
docker env
minishift is docker based vm, it is possible to access docker daemon using local docker client, minishift provides handy tool to setup connection,
oldhorse@myhost:~$ minishift docker-env --shell bash export DOCKER_TLS_VERIFY="1" export DOCKER_HOST="tcp://192.168.99.100:2376" export DOCKER_CERT_PATH="C:\Users\myuser\.minishift\certs" # Run this command to configure your shell: # eval $(minishift docker-env)
However this format not working for WSL1, it should be
export DOCKER_CERT_PATH="/mnt/c/Users/myuser/.minishift/certs"
here is remedy,
eval $(minishift docker-env "$@" --shell bash |sed -e 's|\\|/|g' -e 's|C:/|/mnt/c/|g')
you can check k8s and openshift containers as below,
docker ps |grep kube docker ps |grep openshift
oc env on WSL1
oc is openshift client, shipping with minishift, you need to setup its path to $PATH on wsl1 to allow run it.
oldhorse@myhost:~$ minishift oc-env --shell bash export PATH="C:\Users\myuser\.minishift\cache\oc\v3.11.0\windows:$PATH" # Run this command to configure your shell: # eval $(minishift oc-env)
this is not expected on wsl1, the correct format should be:
export PATH="/mnt/c/Users/myuser/.minishift/cache/oc/v3.11.0/windows:$PATH"
here is handy script to correct PATH format,
eval $(minishift oc-env "$@" --shell bash |sed -e 's|\\|/|g' -e 's|C:/|/mnt/c/|g')
we still get issue to start oc client,
oldhorse@myhost:~$ oc status error: Missing or incomplete configuration info. Please login or point to an existing, complete config file: 1. Via the command-line flag --config 2. Via the KUBECONFIG environment variable 3. In your home directory as ~/.kube/config To view or setup config directly use the 'config' command.
oc client replaces kubectl to deal with k8s and openshift cluster with similar config setting, the generated config located at /mnt/c/Users/myuser/.minishift/machines/minishift_kubeconfig, we use window version minishift.exe so by default it will look for kube config at /mnt/c/Users/myuser/.kube, you can directly copy it to default kube config to make k8s and minishift connection working smoothly.
cp /mnt/c/Users/myuser/.minishift/machines/minishift_kubeconfig /mnt/c/Users/myuser/.kube/config
since path is windows format with backflash in config, also we know k8s or oc will check ~/.kube/config on WSL1, as remedy, re-format config file and place to ~/.kube/config.
cat /mnt/c/Users/myuser/.kube/config|sed -e 's|\\|/|g' -e 's|C:/|/mnt/c/|g' > /home/oldhorse/.kube/config
test oc connection,
oldhorse@myhost:~$ oc status In project My Project (myproject) on server https://192.168.99.100:8443 You have no services, deployment configs, or build configs. Run 'oc new-app' to create an application.
we put all config tips together in script, you can place into ~/.bashrc or startup script under /etc/profile.d,
# minishift oc-env if [ $(which minishift.exe) ];then alias minishift=minishift.exe source <(minishift completion bash) ocenv(){ set -x if [ $(uname|cut -d'_' -f1) = 'Linux' ] && [ $(uname -a|awk '{print $4}'|cut -d'-' -f2) = Microsoft ]; then eval $(minishift oc-env "$@" --shell bash |sed -e 's|\\|/|g' -e 's|C:/|/mnt/c/|g') eval $(minishift docker-env "$@" --shell bash |sed -e 's|\\|/|g' -e 's|C:/|/mnt/c/|g') winuser=$(minishift oc-env --shell bash|grep PATH|awk -F\\ '{print $3}') if [ ! -d /mnt/c/Users/$winuser/.kube ]; then mkdir /mnt/c/Users/$winuser/.kube fi cp /mnt/c/Users/$winuser/.minishift/machines/minishift_kubeconfig /mnt/c/Users/$winuser/.kube/config if [ ! -d /home/$USERNAME/.kube ];then mkdir /home/$USERNAME/.kube fi touch /home/$USERNAME/.kube/config cat /mnt/c/Users/$winuser/.kube/config|sed -e 's|\\|/|g' -e 's|C:/|/mnt/c/|g' > /home/$USERNAME/.kube/config else eval $(minishift oc-env "$@" --shell bash) eval $(minishift docker-env "$@" --shell bash) fi set +x alias oc=oc.exe source <(oc completion bash) } fi export -f ocenv
Here is how ocenv function works, it fix all oc, docker, kube setup issues on WSL1,
oldhorse@myhost:~$ ocenv oldhorse@myhost:~$ oc status In project My Project (myproject) on server https://192.168.99.100:8443 You have no services, deployment configs, or build configs. Run 'oc new-app' to create an application.
login oc as sysadm
oldhorse@myhost:~$ oc login -u system:admin Logged into "https://192.168.99.100:8443" as "system:admin" using existing credentials. You have access to the following projects and can switch between them with 'oc project <projectname>': default kube-dns kube-proxy kube-public kube-system * myproject openshift openshift-apiserver openshift-controller-manager openshift-core-operators openshift-infra openshift-node openshift-service-cert-signer openshift-web-console Using project "myproject".
access minishift from local k8s client
As we know minishift based on k8s, so it is possible to access k8s, containers directly from kubectl client.
oldhorse@myhost:~$ kubectl get node NAME STATUS ROLES AGE VERSION localhost Ready <none> 9d v1.11.0+d4cacc0 oldhorse@myhost:~$ kubectl config get-contexts CURRENT NAME CLUSTER AUTHINFO NAMESPA CE * myproject/192-168-99-100:8443/system:admin 192-168-99-100:8443 system:admin/192-168-99-100:8443 myproject
Conclusion
- covers all pain points to run window version of minishift on WSL1
- provides handy script to hide all setup details
- supports oc client, docker client and k8s client access to one single node openshift cluster