如何將地端Kubernets叢集加入Azure Arc，以MicroK8s叢集為例

首先當然是要準備好 MicroK8s 叢集。

$ kubectl get nodes -o wide
NAME   STATUS   ROLES    AGE   VERSION   INTERNAL-IP     EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
wk8s   Ready    <none>   14h   v1.28.3   192.168.8.184   <none>        Ubuntu 22.04.3 LTS   5.15.0-89-generic   containerd://1.6.15
mk8s   Ready    <none>   14h   v1.28.3   192.168.8.182   <none>        Ubuntu 22.04.3 LTS   5.15.0-89-generic   containerd://1.6.15

在主要的 Control Plane 需要先安裝 Azure CLI。

$ curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash

Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease
Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB]
Hit:3 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.28/deb  InRelease
Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease
Get:5 http://archive.ubuntu.com/ubuntu jammy-security InRelease [110 kB]
Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,224 kB]
Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [255 kB]
Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,018 kB]
Get:9 http://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,015 kB]
Get:10 http://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [195 kB]
Get:11 http://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [814 kB]
Fetched 4,750 kB in 4s (1,207 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
lsb-release is already the newest version (11.1.0ubuntu4).
lsb-release set to manually installed.
curl is already the newest version (7.81.0-1ubuntu1.15).
gnupg is already the newest version (2.2.27-3ubuntu2.1).
gnupg set to manually installed.
apt-transport-https is already the newest version (2.4.11).
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.
Get:1 https://packages.microsoft.com/repos/azure-cli jammy InRelease [3,595 B]
Get:2 https://packages.microsoft.com/repos/azure-cli jammy/main all Packages [1,099 B]
Get:4 https://packages.microsoft.com/repos/azure-cli jammy/main amd64 Packages [1,242 B]
Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease
Hit:3 https://prod-cdn.packages.k8s.io/repositories/isv:/kubernetes:/core:/stable:/v1.28/deb  InRelease
Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease
Hit:7 http://archive.ubuntu.com/ubuntu jammy-backports InRelease
Hit:8 http://archive.ubuntu.com/ubuntu jammy-security InRelease
Fetched 5,936 B in 1s (4,853 B/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
  azure-cli
0 upgraded, 1 newly installed, 0 to remove and 3 not upgraded.
Need to get 65.6 MB of archives.
After this operation, 802 MB of additional disk space will be used.
Get:1 https://packages.microsoft.com/repos/azure-cli jammy/main amd64 azure-cli amd64 2.55.0-1~jammy [65.6 MB]
Fetched 65.6 MB in 19s (3,421 kB/s)
Selecting previously unselected package azure-cli.
(Reading database ... 74423 files and directories currently installed.)
Preparing to unpack .../azure-cli_2.55.0-1~jammy_amd64.deb ...
Unpacking azure-cli (2.55.0-1~jammy) ...
Setting up azure-cli (2.55.0-1~jammy) ...
NEEDRESTART-VER: 3.5
NEEDRESTART-KCUR: 5.15.0-89-generic
NEEDRESTART-KEXP: 5.15.0-89-generic
NEEDRESTART-KSTA: 1

接著登入你的 Azure 訂閱：

$ az login
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code {YourCode} to authenticate.

安裝 connetedk8s 擴充指令集。

$ az extension add --name connectedk8s

註冊 Azure Arc 的 Kubernetes 提供者：

az provider register --namespace Microsoft.Kubernetes
az provider register --namespace Microsoft.KubernetesConfiguration
az provider register --namespace Microsoft.ExtendedLocation

註冊最多可能需要 10 多分鐘的時間。可以去休息一下。

az provider show -n Microsoft.Kubernetes -o table
az provider show -n Microsoft.KubernetesConfiguration -o table
az provider show -n Microsoft.ExtendedLocation -o table

Namespace             RegistrationPolicy    RegistrationState
--------------------  --------------------  -------------------
Microsoft.Kubernetes  RegistrationRequired  Registered
Namespace                          RegistrationPolicy    RegistrationState
---------------------------------  --------------------  -------------------
Microsoft.KubernetesConfiguration  RegistrationRequired  Registered
Namespace                   RegistrationPolicy    RegistrationState
--------------------------  --------------------  -------------------
Microsoft.ExtendedLocation  RegistrationRequired  Registered

註冊之後，您應該會看到 RegistrationState 這些命名空間的狀態變更為 Registered。

建立一個測試用的資源群組：

az group create --name AzureArcTest --location EastUS --output table

確認一下必要條件都已完成。正常而言，你能下 kubectl get nodes 而且 az login 取得的身份權限夠的話，那麼續繼往下進行應該沒有問題的。

接下來就進行 Azure Arc Agent 的安裝與連線：

az connectedk8s connect --name AzureArcTest1 --resource-group AzureArcTest

上面從官網複製下來指令是不能用的。看了好久才看懂這句話「在此範例中，叢集的名稱是 AzureArcTest1。」每個叢集都有內定的名稱，當你擁有多個叢集時，可以重新命名以方便管理。要取得叢集的名稱也不難：

$ kubectl config get-clusters
NAME
microk8s-cluster

以正確的叢集名稱再執行一次：

$ az connectedk8s connect --name microk8s-cluster --resource-group AzureArcTest
This operation might take a while...

Downloading kubectl client for first time. This can take few minutes...
To check existing issues, please visit: https://github.com/Azure/azure-cli/issues


Downloading helm client for first time. This can take few minutes...
The required pre-checks for onboarding have succeeded.
Azure resource provisioning has begun.
Azure resource provisioning has finished.
Starting to install Azure arc agents on the Kubernetes cluster.
{
  "agentPublicKeyCertificate": "Key",
  "agentVersion": null,
  "connectivityStatus": "Connecting",
  "distribution": "generic",
  "id": "id",
  "identity": {
    "principalId": "id",
    "tenantId": "id",
    "type": "SystemAssigned"
  },
  "infrastructure": "generic",
  "kubernetesVersion": null,
  "lastConnectivityTime": null,
  "location": "eastus",
  "managedIdentityCertificateExpirationTime": null,
  "name": "microk8s-cluster",
  "offering": null,
  "provisioningState": "Succeeded",
  "resourceGroup": "AzureArcTest",
  "systemData": {
    "createdAt": "2023-12-07T03:19:29.627180+00:00",
    "createdBy": "email",
    "createdByType": "User",
    "lastModifiedAt": "2023-12-07T03:19:29.627180+00:00",
    "lastModifiedBy": "email",
    "lastModifiedByType": "xxxx"
  },
  "tags": {},
  "totalCoreCount": null,
  "totalNodeCount": null,
  "type": "microsoft.kubernetes/connectedclusters"
}

完成之後，到 AzureArcTest 看一下。

這樣就很順利把 Hyper-V 裡的 MicroK8s 叢集加入到 Azure Arc 清單之中了。

查詢一下 Pods，也可以發現，Azure Arc Agent 會部屬不少的 Pods 來協助 Azure Arc 作業。

$ kubectl get pods -o wide -n azure-arc
NAME                                          READY   STATUS    RESTARTS      AGE    IP             NODE   NOMINATED NODE   READINESS GATES
flux-logs-agent-6f5cd97458-8kgtq              1/1     Running   0             159m   10.1.28.201    wk8s   <none>           <none>
logcollector-d49c9479b-cfjxt                  1/1     Running   0             159m   10.1.28.200    wk8s   <none>           <none>
cluster-metadata-operator-5d986d58c6-fn56l    2/2     Running   0             159m   10.1.28.197    wk8s   <none>           <none>
clusteridentityoperator-66db66859c-gpntk      2/2     Running   0             159m   10.1.28.202    wk8s   <none>           <none>
metrics-agent-56bc97476f-h7s5c                2/2     Running   0             159m   10.1.28.194    wk8s   <none>           <none>
resource-sync-agent-79684d7494-92hnd          2/2     Running   0             159m   10.1.28.199    wk8s   <none>           <none>
extension-events-collector-56869469b9-gzszl   2/2     Running   0             159m   10.1.28.203    wk8s   <none>           <none>
clusterconnect-agent-fc94b6574-5qp7t          3/3     Running   0             159m   10.1.28.198    wk8s   <none>           <none>
config-agent-7858f8777b-wwq7s                 2/2     Running   0             159m   10.1.28.195    wk8s   <none>           <none>
controller-manager-5bc7d9876d-mjzls           2/2     Running   2 (72s ago)   159m   10.1.215.202   mk8s   <none>           <none>
extension-manager-ddbd59d4b-v99l2             3/3     Running   1 (81s ago)   159m   10.1.28.196    wk8s   <none>           <none>
kube-aad-proxy-588dd5cdd9-gblm2               1/2     Running   3 (18s ago)   159m   10.1.215.201   mk8s

因為我們內網會利用 Acr Artifact Cache 來取得映像檔，順便整理一下它們使用的映像檔資訊：

mcr.microsoft.com/azurearck8s/flux-logs-agent
mcr.microsoft.com/azurearck8s/fluent-bit-collector
mcr.microsoft.com/azurearck8s/cluster-metadata-operator
mcr.microsoft.com/azurearck8s/cluster-identity
mcr.microsoft.com/azurearck8s/metrics-agent
mcr.microsoft.com/azurearck8s/resource-sync
mcr.microsoft.com/azurearck8s/extension-events-collector
mcr.microsoft.com/azurearck8s/clusterconnectservice-operator
mcr.microsoft.com/azurearck8s/clusterconnect-agent
mcr.microsoft.com/azurearck8s/config-agent
mcr.microsoft.com/azurearck8s/configoperator
mcr.microsoft.com/azurearck8s/healthstatecontroller
mcr.microsoft.com/azurearck8s/extensionoperator
mcr.microsoft.com/azurearck8s/kube-aad-proxy

KingKong Bruce記事

網頁

如何將地端Kubernets叢集加入Azure Arc，以MicroK8s叢集為例

如何將地端Kubernets叢集加入Azure Arc，以MicroK8s叢集為例

沒有留言:

張貼留言