Kubernetes Clusters ADVANCED ENTERPRISE¶

Requirements¶

• Agent installation is required for Master and Worker Nodes. Refer to Morpheus Agent section for additional information.

• Access to Cloud Front, Image copy access and permissions for System and Uploaded Images used in Cluster Layouts

  Image(s) used in Cluster Layouts must either exist in destination cloud/resource or be able to be copied to destination by Morpheus, typically applicable for non-public clouds. For the initial provision, Morpheus System Images are streamed from Cloud Front through Morpheus to target destination. Subsequent provisions clone the local Image.

• System Kubernetes Layouts require Master and Worker nodes to access to the following over 443 during K8s install and configuration:

  • Morpheus Appliance url (443)

  • https://packages.cloud.google.com

  • https://storage.googleapis.com

  • https://docs.projectcalico.org

  • https://openebs.github.io

  • https://cloud.weave.works

• Morpheus Role permission Infrastructure: Clusters > Full required for Viewing, Creating, Editing and Deleting Clusters.

• Morpheus Role permission Infrastructure: Clusters > Read required for Viewing Cluster list and detail pages.

For HKS networking and planned node service, see HKS kube-vip and HKS Node Maintenance.

CNCF Conformance Claim¶

This documentation does not bundle an official CNCF conformance record that identifies an HKS product name, Kubernetes version, certification status, and validity period. It therefore makes no claim that HKS generally, or every HKS-provisioned Kubernetes version, is CNCF Certified Kubernetes.

Certification is version-specific. Confirm a claim only against the CNCF Certified Kubernetes product list and its linked conformance record. If the exact HKS offering and version are not present, contact HPE before representing the deployment as certified. Kubernetes API compatibility or a successful conformance test run is not, by itself, a published CNCF certification.

Creating Kubernetes Clusters¶

Provisions a new Kubernetes Cluster in selected target Cloud using selected Layout.

HPE Morpheus Software maintains a number of default Kubernetes Cluster Layouts which are updated frequently to offer support for current versions. AKS & GKE Kubernetes versions will dynamically update to the providers supported versions. HPE Morpheus Software also supports creation of custom Kubernetes Cluster Layouts, a process which is described in detail in a later section.

To create a new Kubernetes Cluster:

1. Navigate to Infrastructure > Clusters

2. Select + ADD CLUSTER

3. Select Kubernetes Cluster

4. Select a Group for the Cluster

5. Select NEXT

6. Populate the following:

   CLOUD

   Select target Cloud

   CLUSTER NAME

   Name for the Kubernetes Cluster

   RESOURCE NAME

   Name for Kubernetes Cluster resources

   DESCRIPTION

   Description of the Cluster

   VISIBILITY

   Public

   Available to all Tenants

   Private

   Available to Master Tenant

   LABELS

   Internal label(s)

7. Select NEXT

8. Populate the following:

   Note

   VMware sample fields provided. Actual options depend on Target Cloud

   LAYOUT

   Select from available layouts. System provided layouts include Single Master and Cluster Layouts.

   PLAN

   Select plan for Kubernetes Master

   VOLUMES

   Configure volumes for Kubernetes Master

   NETWORKS

   Select the network for Kubernetes Master & Worker VM’s

   CUSTOM CONFIG

   Add custom Kubernetes annotations and config hash

   CLUSTER HOSTNAME

   Cluster address Hostname (cluster layouts only)

   POD CIDR

   POD network range in CIDR format ie 192.168.0.0/24 (cluster layouts only)

   WORKER PLAN

   Plan for Worker Nodes (cluster layouts only)

   NUMBER OF WORKERS

   Specify the number of workers to provision

   LOAD BALANCER

   Select an available Load Balancer (cluster layouts only) }

   User Config

   CREATE YOUR USER

   Select to create your user on provisioned hosts (requires Linux user config in HPE Morpheus Software User Profile)

   USER GROUP

   Select User group to create users for all User Group members on provisioned hosts (requires Linux user config in HPE Morpheus Software User Profile for all members of User Group)

   Advanced Options

   DOMAIN

   Specify Domain override for DNS records

   HOSTNAME

   Set hostname override (defaults to Instance name unless an Active Hostname Policy applies)

9. Select NEXT

10. Select optional Workflow to execute

11. Select NEXT

12. Review and select COMPLETE

    • The Master Node(s) will provision first.

    • Upon successful completion of VM provision, Kubernetes scripts will be executed to install and configure Kubernetes on the Masters.

      Note

      Access to the sites listed in the Requirements section is required from Master and Worker nodes over 443

    • After Master or Masters are successfully provisioned and Kubernetes is successfully installed and configured, the Worker Nodes will provision in parallel.

    • Provision status can be viewed:

      • From the Status next to the Cluster in Infrastructure > Clusters

      • Status bar with eta and current step available on Cluster detail page, accessible by selecting the Cluster name from Infrastructure > Clusters

    • All process status and history is available - From the Cluster detail page History tab, accessible by selecting the Cluster name from Infrastructure > Clusters and the History tab - From Operations - Activity - History - Individual process output available by clicking i on target process

13. Once all Master and Worker Nodes are successfully provisioned and Kubernetes is installed and configured, the Cluster status will turn green.

    Important

    Cluster provisioning requires successful creation of VMs, Agent Installation, and execution of Kubernetes workflows. Consult process output from ``Infrastructure > Clusters - Details and morpheus-ui current logs at Administration - Health - Morpheus Logs for information on failed Clusters.

Intra-Kubernetes Cluster Port Requirements¶

The table below includes port requirements for the machines within the cluster (not for the HPE Morpheus Software appliance itself). Check that the following ports are open on Control-plane and Worker nodes:

Adding Worker Nodes¶

1. Navigate to Infrastructure - Clusters

2. Select v MORE for the target cluster

3. Select ADD (type) Kubernetes Worker

   NAME

   Name of the Worker Node. Auto=populated with ${cluster.resourceName}-worker-${seq}

   DESCRIPTION

   Description of the Worker Node, displayed in Worker tab on Cluster Detail pages, and on Worker Host Detail page

   CLOUD

   Target Cloud for the Worker Node.

4. Select NEXT

5. Populate the following:

   Note

   VMware sample fields provided. Actual options depend on Target Cloud

   SERVICE PLAN

   Service Plan for the new Worker Node

   NETWORK

   Configure network options for the Worker node.

   HOST

   If Host selection is enabled, optionally specify target host for new Worker node

   FOLDER

   Optionally specify target folder for new Worker node

   Advanced Options

   DOMAIN

   Specify Domain override for DNS records

   HOSTNAME

   Set hostname override (defaults to Instance name unless an Active Hostname Policy applies)

6. Select NEXT

7. Select optional Workflow to execute

8. Select NEXT

9. Review and select COMPLETE

Kubernetes Cluster Detail Pages¶

The Kubernetes Cluster Detail page provides a high degree of monitoring and control over Kubernetes Clusters. This includes monitoring of all nodes in the Cluster, kubectl command line, account and role control, workload management, and more. The upper section of the page (which is persistent regardless of the currently-selected tab) provides high level costing and monitoring information, including a current aggregate metric for the CPU, memory and storage use.

The upper section also includes the ACTIONS menu which includes the following functions:

• REFRESH: Forces a routine sync of the cluster status

• PERMISSIONS: View and edit the Group, Service Plan, and Tenant access permissions for the cluster

• VIEW API TOKEN: Displays the API token for the cluster

• VIEW KUBE CONFIG: Displays the cluster configuration

• RUN WORKLOAD: Run deployments, stateful sets, daemon sets, or jobs and target them to a specific namespace

• UPGRADE CLUSTER: Upgrade the cluster to a higher version of Kubernetes

  Important

  For HKS 8.1.0, 8.1.1, and 8.1.2 clusters upgrading to Kubernetes v1.35.x or later, remove the deprecated --pod-infra-container-image flag from /var/lib/kubelet/kubeadm-flags.env on every control-plane and worker node. Leaving the flag in place causes the upgrade to fail. Current upgrade scripts remove this flag as part of the node upgrade; verify it is absent if an upgrade is interrupted or uses older scripts. This note records the requirement for those historical release lines on the current canonical HKS page because separate versioned source pages are not present in this documentation set.

• ADD KUBERNETES WORKER: Launches a wizard which allows users to configure a new worker for the cluster

Additional monitoring and control panes are located within tabs, some of which contain subtabs.

SummaryControlAccessNodesWorkloadsNetworkStorageContainersMonitoringHistoryWiki

The summary tab contains high-level details on health and makeup of the cluster.

Adding External Kubernetes Clusters¶

HPE Morpheus Software supports the management and consumption of Kubernetes clusters provisioned outside of HPE Morpheus Software. These are referred to as External Kubernetes Clusters in HPE Morpheus Software UI. This could be used, for example, to onboard and manage OpenShift clusters. In order to fully integrate the Kubernetes cluster with the HPE Morpheus Software feature set, you may need to create a service account for HPE Morpheus Software. Without first taking that step, some features may not work fully, such as listing all namespaces. The process for creating a service account and integrating the Cluster with HPE Morpheus Software is described here.

First, create the Service Account within the Kubernetes cluster:

kubectl create serviceaccount morpheus

Next, create the Role Binding:

kubectl create clusterrolebinding morpheus-admin \
--clusterrole=cluster-admin --serviceaccount=default:morpheus \
--namespace=default

With those items created, we can gather the API URL and the API token which will be used to add the existing cluster to HPE Morpheus Software in the next step:

kubectl config view --minify | grep server | cut -f 2- -d ":" | tr -d " "

SECRET_NAME=$(kubectl get secrets | grep ^morpheus | cut -f1 -d ' ')
kubectl describe secret $SECRET_NAME | grep -E '^token' | cut -f2 -d':' | tr -d " "

After finishing those steps, we can now create the external cluster in HPE Morpheus Software. Navigate to Infrastructure ‣ Clusters. Click + ADD CLUSTER and then select “External Kubernetes Cluster”. Set the following fields, you will have to advance through the pages of the wizard to see all fields indicated:

• GROUP: A previously created HPE Morpheus Software Group

• CLOUD: A previously-integrated Cloud

• CLUSTER NAME: A friendly name for the onboarded cluster in HPE Morpheus Software UI

• RESOURCE NAME: The resource name will be pre-pended to Kubernetes hosts associated with this cluster when shown in HPE Morpheus Software UI

• LAYOUT: Set an associated Layout

• API URL: Enter the API URL gathered in the previous step

• API TOKEN: Enter the API Token gathered in the previous step

• KUBE CONFIG: Enter Kubeconfig YAML to authenticate the cluster

The above are the required fields, others may be optionally configured depending on the situation. Complete the wizard and HPE Morpheus Software will begin the process of onboarding the existing cluster into management within HPE Morpheus Software UI. Once things are finalized and statuses are green, the cluster can be monitored and consumed as any other cluster provisioned from HPE Morpheus Software.

Additional Kubernetes Topics¶