Tạo Gói Offline

1. Tải xuống và giải nén KubeKey bằng lệnh sau:

   export KKZONE=cn
   curl -sfL https://get-kk.kubesphere.io | VERSION=v3.0.13 sh -
   

2. Tạo tệp `manifest.yaml` với nội dung sau:

   ---
   apiVersion: kubekey.kubesphere.io/v1alpha2
   kind: Manifest
   metadata:
     name: sample
   spec:
     arches:
     - amd64
     operatingSystems:
     - arch: amd64
       type: linux
       id: centos
       version: "7"
       repository:
         iso:
           localPath:
           url: https://github.com/kubesphere/kubekey/releases/download/v3.0.10/centos7-rpms-amd64.iso
     - arch: amd64
       type: linux
       id: ubuntu
       version: "20.04"
       repository:
         iso:
           localPath:
           url: https://github.com/kubesphere/kubekey/releases/download/v3.0.10/ubuntu-20.04-debs-amd64.iso
     kubernetesDistributions:
     - type: kubernetes
       version: v1.23.15
     components:
       helm:
         version: v3.9.0
       cni:
         version: v1.2.0
       etcd:
         version: v3.4.13
       calicoctl:
         version: v3.23.2
       containerRuntimes:
       - type: docker
         version: 20.10.8
       - type: containerd
         version: 1.6.4
       crictl:
         version: v1.24.0
       docker-registry:
         version: "2"
       harbor:
         version: v2.5.3
       docker-compose:
         version: v2.2.2
     images:
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-apiserver:v1.23.15
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controller-manager:v1.23.15
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-proxy:v1.23.15
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-scheduler:v1.23.15
     - registry.cn-beijing.aliyuncs.com/kubesphereio/pause:3.6
     - registry.cn-beijing.aliyuncs.com/kubesphereio/coredns:1.8.6
     - registry.cn-beijing.aliyuncs.com/kubesphereio/cni:v3.23.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controllers:v3.23.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/node:v3.23.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/pod2daemon-flexvol:v3.23.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/typha:v3.23.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/flannel:v0.12.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/provisioner-localpv:3.3.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/linux-utils:3.3.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/haproxy:2.3
     - registry.cn-beijing.aliyuncs.com/kubesphereio/nfs-subdir-external-provisioner:v4.0.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/k8s-dns-node-cache:1.15.12
     - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-installer:v3.4.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-apiserver:v3.4.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-console:v3.4.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-controller-manager:v3.4.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.22.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.21.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.20.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kubefed:v0.8.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/tower:v0.2.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/minio:RELEASE.2019-08-07T01-59-21Z
     - registry.cn-beijing.aliyuncs.com/kubesphereio/mc:RELEASE.2019-08-07T23-14-43Z
     - registry.cn-beijing.aliyuncs.com/kubesphereio/snapshot-controller:v4.0.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/nginx-ingress-controller:v1.1.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/defaultbackend-amd64:1.4
     - registry.cn-beijing.aliyuncs.com/kubesphereio/metrics-server:v0.4.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/redis:5.0.14-alpine
     - registry.cn-beijing.aliyuncs.com/kubesphereio/haproxy:2.0.25-alpine
     - registry.cn-beijing.aliyuncs.com/kubesphereio/alpine:3.14
     - registry.cn-beijing.aliyuncs.com/kubesphereio/openldap:1.3.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/netshoot:v1.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/cloudcore:v1.13.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/iptables-manager:v1.13.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/edgeservice:v0.3.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/gatekeeper:v3.5.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/openpitrix-jobs:v3.3.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/devops-apiserver:ks-v3.4.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/devops-controller:ks-v3.4.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/devops-tools:ks-v3.4.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-jenkins:v3.4.0-2.319.3-1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/inbound-agent:4.10-2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-base:v3.2.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-nodejs:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.1-jdk11
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-python:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.16
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.17
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.18
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-base:v3.2.2-podman
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-nodejs:v3.2.0-podman
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.0-podman
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.1-jdk11-podman
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-python:v3.2.0-podman
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.0-podman
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.16-podman
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.17-podman
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.18-podman
     - registry.cn-beijing.aliyuncs.com/kubesphereio/s2ioperator:v3.2.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/s2irun:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/s2i-binary:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java11-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java11-runtime:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java8-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java8-runtime:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/java-11-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/java-8-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/java-8-runtime:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/java-11-runtime:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/nodejs-8-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/nodejs-6-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/nodejs-4-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/python-36-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/python-35-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/python-34-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/python-27-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/argocd:v2.3.3
     - registry.cn-beijing.aliyuncs.com/kubesphereio/argocd-applicationset:v0.4.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/dex:v2.30.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/redis:6.2.6-alpine
     - registry.cn-beijing.aliyuncs.com/kubesphereio/configmap-reload:v0.7.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/prometheus:v2.39.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/prometheus-config-reloader:v0.55.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/prometheus-operator:v0.55.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-rbac-proxy:v0.11.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-state-metrics:v2.6.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/node-exporter:v1.3.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/alertmanager:v0.23.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/thanos:v0.31.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/grafana:8.3.3
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-rbac-proxy:v0.11.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/notification-manager-operator:v2.3.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/notification-manager:v2.3.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/notification-tenant-sidecar:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/elasticsearch-curator:v5.7.6
     - registry.cn-beijing.aliyuncs.com/kubesphereio/elasticsearch-oss:6.8.22
     - registry.cn-beijing.aliyuncs.com/kubesphereio/opensearch:2.6.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/opensearch-dashboards:2.6.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/opensearch-curator:v0.0.5
     - registry.cn-beijing.aliyuncs.com/kubesphereio/fluentbit-operator:v0.14.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/docker:19.03
     - registry.cn-beijing.aliyuncs.com/kubesphereio/fluent-bit:v1.9.4
     - registry.cn-beijing.aliyuncs.com/kubesphereio/log-sidecar-injector:v1.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/filebeat:6.7.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-events-operator:v0.6.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-events-exporter:v0.6.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-events-ruler:v0.6.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-auditing-operator:v0.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-auditing-webhook:v0.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/pilot:1.14.6
     - registry.cn-beijing.aliyuncs.com/kubesphereio/proxyv2:1.14.6
     - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-operator:1.29
     - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-agent:1.29
     - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-collector:1.29
     - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-query:1.29
     - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-es-index-cleaner:1.29
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kiali-operator:v1.50.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kiali:v1.50
     - registry.cn-beijing.aliyuncs.com/kubesphereio/busybox:1.31.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/nginx:1.14-alpine
     - registry.cn-beijing.aliyuncs.com/kubesphereio/wget:1.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/hello:plain-text
     - registry.cn-beijing.aliyuncs.com/kubesphereio/wordpress:4.8-apache
     - registry.cn-beijing.aliyuncs.com/kubesphereio/hpa-example:latest
     - registry.cn-beijing.aliyuncs.com/kubesphereio/fluentd:v1.4.2-2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/perl:latest
     - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-productpage-v1:1.16.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-reviews-v1:1.16.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-reviews-v2:1.16.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-details-v1:1.16.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-ratings-v1:1.16.3
     - registry.cn-beijing.aliyuncs.com/kubesphereio/scope:1.13.0
   

3. Xuất artifact bằng lệnh sau:

   export KKZONE=cn
   ./kk artifact export -m manifest.yaml -o kubesphere.tar.gz
   

Cài đặt Offline

1. Sao chép KubeKey và artifact đã tải xuống vào nút cài đặt môi trường offline bằng ổ USB hoặc phương tiện khác.

2. Tạo tệp cấu hình cluster offline bằng lệnh sau:

   ./kk create config --with-kubesphere v3.4.1 --with-kubernetes v1.23.15 -f config-sample.yaml
   

3. Chỉnh sửa tệp cấu hình vừa tạo:

   • Cấu hình lại thông tin nút theo môi trường thực tế.
   • Chỉ định nút triển khai kho lưu trữ registry (để KubeKey triển khai kho lưu trữ Harbor tự lưu trữ).
   • Trong cấu hình registry, phải đặt `type` là `harbor`, nếu không mặc định sẽ cài đặt Docker registry.

4. Cài đặt kho lưu trữ image bằng lệnh sau:

   ./kk init registry -f config-sample.yaml -a kubesphere.tar.gz
   

5. Tạo dự án Harbor:

   • Do hạn chế về kiểm soát truy cập (RBAC) của dự án Harbor, chỉ những người dùng có vai trò được chỉ định mới có thể thực hiện một số thao tác. Nếu bạn không tạo dự án, image sẽ không thể được đẩy lên Harbor. Harbor có hai loại dự án:
     • Dự án công khai (Public): Bất kỳ người dùng nào cũng có thể kéo image từ dự án này.
     • Dự án riêng tư (Private): Chỉ những người dùng là thành viên của dự án mới có thể kéo image.
   • Tài khoản quản trị viên Harbor: `admin`, mật khẩu: `Harbor12345`. Tệp cài đặt Harbor nằm trong `/opt/harbor`.
   • Đăng nhập vào kho lưu trữ Harbor để tạo dự án. Đặt dự án thành công khai để mọi người dùng có thể kéo image.

6. Chỉnh sửa lại tệp cấu hình cluster `config-sample.yaml`:

     ...
     registry:
       type: harbor
       auths:
         "dockerhub.kubekey.local":
           username: admin
           password: Harbor12345
       privateRegistry: "dockerhub.kubekey.local"
       namespaceOverride: "kubesphereio"
       registryMirrors: []
       insecureRegistries: []
     addons: []
   

   • Thêm cấu hình `auths` với `dockerhub.kubekey.local`, tên người dùng và mật khẩu.
   • Thêm `privateRegistry` là `dockerhub.kubekey.local`.
   • Thêm `namespaceOverride` là `kubesphereio`.

7. Cài đặt KubeSphere cluster bằng lệnh sau:

   ./kk create cluster -f config-sample.yaml -a kubesphere.tar.gz
   

8. Xem trạng thái cluster bằng lệnh sau:

   kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l 'app in (ks-install, ks-installer)' -o jsonpath='{.items[0].metadata.name}') -f
   

   Sau khi cài đặt hoàn tất, bạn sẽ thấy thông tin đăng nhập.