Posted January 4, 2022 by Alexander Ullah2 Comments Kubernetes

Tanzu Packages Explained

In this blog post, I will describe what Tanzu Packages are and how they can be used. You might have read about Tanzu Kubernetes Grid (TKG) Extensions in my previous blog post or the official VMware documentation. However, Tanzu Packages are the evolution of TKG Extensions. They easily install add-ons such as Cert-Manager, Contour, Prometheus, Harbor, and others onto your TKG clusters via the Tanzu CLI. Tanzu Packages have been introduced with the Tanzu Community Edition and as part of TKG 1.4. If you have played with both solutions, you will have realized that they are using different package repositories with slightly different content. The reason being is that the TKG 1.4 repository includes all compatible, tested, and VMware-supported packages. Tanzu Community Edition is open-source, and therefore packages will be maintained and supported by the community. A comparison between the different Tanzu Editions and their content can be found here.

List of TKG 1.4 supported packages:

A list of Tanzu Community Edition packages can be found here:

If you are using Tanzu Mission Control (TMC), it also has a new catalog feature that deploys Tanzu Packages on your TMC managed TKG clusters.

The TKG 1.4 Tanzu Packages can also be used on vSphere with Tanzu, also known as Tanzu Kubernetes Grid Service (TKGS). Nevertheless, as of writing this blog post, there is the limitation that the observability packages have not been validated on vSphere with Tanzu yet.

Background

So what are Tanzu Packages really, and what are they built on. First of all, Tanzu Packages are based on the open-source project CARVEL. Carvel is a set of composable and single-purpose tools with the goal of easing the application management on Kubernetes. And with management, I mean the entire life-cycle starting with building, templating, packaging, deploying, and day2. Here is an excellent session (only 26 minutes) from VMware’s Helen George and João Pereira at KubeCon Europe 2021 about Carvel:

Let’s have a closer look at Tanzu CLI and how it uses Carvel to deploy and manage Tanzu Packages.

Tanzu CLI

The Tanzu CLI provides an easy way to manage our TKG clusters and Tanzu Packages. It hides a lot of the complexity and creates all of the required Kubernetes resources for you. We will look at the workflow from a Tanzu CLI consumer perspective.

But before we start, please download and install Tanzu CLI, kubectl, and the Carvel tools described here. You can also configure Tanzu CLI auto-completion for your shell (bash, zsh):

➜  ~ tanzu completion --help

Output shell completion code for the specified shell [bash zsh].

The shell completion code must be evaluated to provide completion. See Examples
for how to perform this for your given shell.

Note for bash users: make sure the bash-completions package has been installed.
Note for zsh users: zsh >= 5.2 is required for command completion.

Usage:
  tanzu completion [command]

Examples:

# Bash instructions:

  ## Load only for current session:
  source <(tanzu completion bash)

  ## Load for all new sessions:
  tanzu completion bash >  $HOME/.tanzu/completion.bash.inc
  printf "\n# Tanzu shell completion\nsource '$HOME/.tanzu/completion.bash.inc'\n" >> $HOME/.bash_profile

# Zsh instructions:

  ## Load only for current session:
  source <(tanzu completion zsh)

  ## Load for all new sessions:
  echo "autoload -U compinit; compinit" >> ~/.zshrc
  tanzu completion zsh > "${fpath[1]}/_tanzu"

Tanzu Packages

For this scenario, I will use a Tanzu Kubernetes Cluster (TKC), which I’ve easily and fast deployed in my Workload Management enabled vSphere with Tanzu environment. The process for Tanzu Kubernetes Grid Multi-Cloud (TKGM) is almost the same. The only difference is that the kapp-controller is already installed on a TKGM 1.4 based cluster. The first thing we have to do is install the kapp-controller, but what is it?

Kapp-controller

The kapp-controller is a package manager for Kubernetes, and it is a crucial component of Carvel and Tanzu Packages. It provides declarative APIs for the continuous delivery of applications and packages. Additionally, it gives Developers the flexibility to choose from various supported sources (e.g., Git, Helm, imgpkg) and templating tools (e.g., ytt, Helm). Applications are defined as part of an App CustomResourceDefinition (CRD) and deployed via kapp. Kapp-controller comes with multiple CRDs, and we will look at the most important ones during the process.

kapp-controller = declarative APIs, package management, and reconciliation for your applications

The App CRD defines a set of Kubernetes resources and breaks down the application specification into three steps.

1. Fetch – From which source do you want to fetch your image and configuration (Git, Helm, imgpkg …).
2. Template: Which tool do you want to use for templating (ytt, Helm, …).
3. Deploy: Kapp is currently the only tool that is supported for deployment. However, there is the possibility that other tools will be added in the future.

apiVersion: kappctrl.k14s.io/v1alpha1
kind: App
metadata:
  name: simple-app
  namespace: default
spec:
  serviceAccountName: default-ns-sa
  fetch:
  - git:
      url: https://github.com/vmware-tanzu/carvel-simple-app-on-kubernetes
      ref: origin/develop
      subPath: config-step-2-template
  template:
  - ytt: {}
  deploy:
  - kapp: {}

After applying an application via the App CR (Custom Resource), the kapp-controller will continuously reconcile it with your Kubernetes cluster.

To install the kapp-controller onto our TKC, we can simply use the manifest from here and apply it to our cluster.

➜  ~ k apply -f kapp-controller.yaml
namespace/tkg-system created
namespace/tanzu-package-repo-global created
apiservice.apiregistration.k8s.io/v1alpha1.data.packaging.carvel.dev created
service/packaging-api created
customresourcedefinition.apiextensions.k8s.io/internalpackagemetadatas.internal.packaging.carvel.dev created
customresourcedefinition.apiextensions.k8s.io/internalpackages.internal.packaging.carvel.dev created
customresourcedefinition.apiextensions.k8s.io/apps.kappctrl.k14s.io created
customresourcedefinition.apiextensions.k8s.io/packageinstalls.packaging.carvel.dev created
customresourcedefinition.apiextensions.k8s.io/packagerepositories.packaging.carvel.dev created
configmap/kapp-controller-config created
deployment.apps/kapp-controller created
serviceaccount/kapp-controller-sa created
clusterrole.rbac.authorization.k8s.io/kapp-controller-cluster-role created
clusterrolebinding.rbac.authorization.k8s.io/kapp-controller-cluster-role-binding created
clusterrolebinding.rbac.authorization.k8s.io/pkg-apiserver:system:auth-delegator created
rolebinding.rbac.authorization.k8s.io/pkgserver-auth-reader created

You should see the kapp-controller pod coming up under the tkg-system namespace. If not, check if your Pod Security Policies are configured ok.

➜  ~ k get pods -n tkg-system
NAME                               READY   STATUS    RESTARTS   AGE
kapp-controller-5d8f7d9477-nk8fc   1/1     Running   0          66s

Now that we have the kapp-controller running, we can add the Tanzu Package Repository to our TKC. But wait, what is a Package Repository?

Package Repository

A Package Repository is a collection of Packages that can be provided and maintained by the Software Producer. In our case, this is where we find all the Tanzu Packages (Contour, Prometheus, Fluent Bit, etc…) that we might want to deploy onto our TKC. The PackageRepository CRD is created by the Software Consumer to tell the kapp-controller where the Package Repository can be found and make the Packages available on a Kubernetes cluster. It references an Image Bundle “imgpkgBundle”, which I cover in the next section.

Package Repository = collection of Packages a user can install from

apiVersion: packaging.carvel.dev/v1alpha1
kind: PackageRepository
metadata:
  name: simple-package-repository
  namespace: default
spec:
  fetch:
    imgpkgBundle:
      image: k8slt/corp-com-pkg-repo:1.0.0

Imgpkg

With the Carvel tool imgpkg, we can efficiently distribute and relocate multiple Packages via a single immutable OCI (Open Container Initiative) artifact. It combines configuration files and dependent OCI images of your applications into a Bundle. A Bundle is essentially an OCI image that can be used with every OCI compliant image registry. If you want to learn how to create a Bundle yourself, look at this basic workflow.

Bundle = single OCI artifact that combines configuration files and dependent OCI images of multiple applications

You probably guessed it already, the Tanzu Package Repositories are Bundles as well, and we can use the imgpkg copy command to copy them for, e.g., air-gapped scenarios.

We can now add the Tanzu Package Repository to our TKG cluster via Tanzu CLI. We will use the “tanzu-package-repo-global” namespace, which was created via the kapp-controller manifest. Kapp-controller is configured to share all packages within the “tanzu-package-repo-global” namespace cluster-wide; see Namespacing section of the kapp-controller documentation for more details. Please note: If you choose a different namespace, the Packages will only be visible and deployable in that particular namespace.

➜  ~ tanzu package repository add repo --url projects.registry.vmware.com/tkg/packages/standard/repo:v1.4.0 -n tanzu-package-repo-global
- Adding package repository 'repo'...
 Added package repository 'repo'

➜  ~ tanzu package repository list -n tanzu-package-repo-global
- Retrieving repositories...
  NAME  REPOSITORY                                                      STATUS       DETAILS
  repo  projects.registry.vmware.com/tkg/packages/standard/repo:v1.4.0  Reconciling

➜  ~ k get packagerepositories -n tanzu-package-repo-global
NAME   AGE     DESCRIPTION
repo   5m46s   Reconcile succeeded

We have successfully added the Tanzu Packages Repository to our TKG cluster. Now we can check what Packages are available for installation. But first, let’s cover what a Package is?

Package

A Package is a combination of an OCI Image and configuration metadata. The Package CRD wraps the App CRD and adds attributes such as “version” and “refName” to it. It provides an easy way to define your application and share it as a Package that can be reused in multiple environments.

Package = OCI Image plus configuration metadata of an application

apiVersion: data.packaging.carvel.dev/v1alpha1
kind: Package
metadata:
  name: simple-app.corp.com.1.0.0
spec:
  refName: simple-app.corp.com
  version: 1.0.0
  releaseNotes: |
        Initial release of the simple app package
  template:
    spec:
      fetch:
      - imgpkgBundle:
          image: index.docker.io/k8slt/kctrl-example-pkg@sha256:8ffa7f9352149dba1d539d0006b38eda357917edcdd39b82497a61dab2c27b75
      template:
      - ytt:
          paths:
          - config/
      - kbld:
          paths:
          - '-'
          - .imgpkg/images.yml
      deploy:
      - kapp: {}
...

Now let’s check if we have available Packages via the newly added repository. We will again use the Tanzu CLI to list all available Packages.

➜  ~ tanzu package available list
\ Retrieving available packages...
  NAME                           DISPLAY-NAME  SHORT-DESCRIPTION
  cert-manager.tanzu.vmware.com  cert-manager  Certificate management
  contour.tanzu.vmware.com       Contour       An ingress controller
  external-dns.tanzu.vmware.com  external-dns  This package provides DNS synchronization functionality.
  fluent-bit.tanzu.vmware.com    fluent-bit    Fluent Bit is a fast Log Processor and Forwarder
  grafana.tanzu.vmware.com       grafana       Visualization and analytics software
  harbor.tanzu.vmware.com        Harbor        OCI Registry
  multus-cni.tanzu.vmware.com    multus-cni    This package provides the ability for enabling attaching multiple network interfaces to pods in Kubernetes
  prometheus.tanzu.vmware.com    prometheus    A time series database for your metrics

As you can see, we don’t have to specify a namespace as we used the “tanzu-package-repo-global” namespace for our repository, and all Packages are available cluster-wide. Let’s identify a package that we want to install first. In this case, I am going to look at the cert-manager package. We can get the available package version and additional details with the following command.

➜  ~ tanzu package available list cert-manager.tanzu.vmware.com
\ Retrieving package versions for cert-manager.tanzu.vmware.com...
  NAME                           VERSION               RELEASED-AT
  cert-manager.tanzu.vmware.com  1.1.0+vmware.1-tkg.2  2020-11-24T18:00:00Z

➜  ~ tanzu package available get cert-manager.tanzu.vmware.com
| Retrieving package details for cert-manager.tanzu.vmware.com...
NAME:                 cert-manager.tanzu.vmware.com
DISPLAY-NAME:         cert-manager
SHORT-DESCRIPTION:    Certificate management
PACKAGE-PROVIDER:     VMware
LONG-DESCRIPTION:     Provides certificate management provisioning within the cluster
MAINTAINERS:          [{Nicholas Seemiller}]
SUPPORT:              Support provided by VMware for deployment on TKG 1.4+ clusters. Best-effort support for deployment on any conformant Kubernetes cluster. Contact support by opening a support request via VMware Cloud Services or my.vmware.com.
CATEGORY:             [certificate management

Finally, we can install the Package via Tanzu CLI. I will create a “packages” namespace for all the Custom Resources created during the process as we don’t want to have anything in the “default” namespace.

➜  ~ k create ns packages
namespace/packages created

➜  ~ tanzu package install cert-manager --package-name cert-manager.tanzu.vmware.com --version 1.1.0+vmware.1-tkg.2 -n packages
- Installing package 'cert-manager.tanzu.vmware.com'
\ Getting namespace 'packages'
- Getting package metadata for 'cert-manager.tanzu.vmware.com'
\ Creating service account 'cert-manager-packages-sa'
- Creating cluster admin role 'cert-manager-packages-cluster-role'
\ Creating cluster role binding 'cert-manager-packages-cluster-rolebinding'
| Creating package resource
/ Package install status: Reconciling


 Added installed package 'cert-manager' in namespace 'packages'

➜  ~ tanzu package installed list -n packages
| Retrieving installed packages...
  NAME          PACKAGE-NAME                   PACKAGE-VERSION       STATUS
  cert-manager  cert-manager.tanzu.vmware.com  1.1.0+vmware.1-tkg.2  Reconcile succeeded

➜  ~ k get packageinstalls,app -n packages
NAME                                               PACKAGE NAME                    PACKAGE VERSION        DESCRIPTION           AGE
packageinstall.packaging.carvel.dev/cert-manager   cert-manager.tanzu.vmware.com   1.1.0+vmware.1-tkg.2   Reconcile succeeded   67s

NAME                                DESCRIPTION           SINCE-DEPLOY   AGE
app.kappctrl.k14s.io/cert-manager   Reconcile succeeded   63s            67s

➜  ~ k get ns
NAME                           STATUS   AGE
cert-manager                   Active   57s
default                        Active   13d
kube-node-lease                Active   13d
kube-public                    Active   13d
kube-system                    Active   13d
packages                       Active   94s
tanzu-package-repo-global      Active   12d
tkg-system                     Active   12d
vmware-system-auth             Active   13d
vmware-system-cloud-provider   Active   13d
vmware-system-csi              Active   13d

➜  ~ k get pods -n cert-manager
NAME                                       READY   STATUS    RESTARTS   AGE
cert-manager-84856fcf54-g8p8h              1/1     Running   0          60s
cert-manager-cainjector-5c8f5b649b-qzq82   1/1     Running   0          60s
cert-manager-webhook-6986456c8-9vd7n       1/1     Running   0          60s

We successfully installed the cert-manager package onto our cluster. We can see it created the App and PackageInstall custom resources in our “packages” namespace and the actual Pods into a different “cert-manager” namespace. But what is the PackageInstall custom resource?

PackageInstall

The PackageInstall CRD is the actual implementation of a Package onto a Kubernetes cluster. The idea is that the Package & Package CR is created by the Software Producer, and the PackageInstall CR is implemented by the Software Consumer.

PackageInstall = represents the deployment of a Package on a Kubernetes cluster

apiVersion: packaging.carvel.dev/v1alpha1
kind: PackageInstall
metadata:
  name: pkg-demo
  namespace: default
spec:
  serviceAccountName: default-ns-sa
  packageRef:
    refName: simple-app.corp.com
    versionSelection:
      constraints: 1.0.0

Conclusion

You might think, wait, isn’t there a solution that does the same thing called Helm? Helm has a similar scope, but Carvel is complementary and adds additional functionality. Helm can still be used as a source and for templating. However, when it comes to actual application deployment, Carvel is using kapp. The significant benefit is that Carvel and, more precisely, kapp-controller continuously reconcile and come with declarative APIs. The Software Producer can leverage all the functionality provided by the Carvel tools and the described way via Packages, Package Repositories, and Bundle OCI Images to provide and distribute Software Packages.

Tanzu Packages are much easier to deploy and manage in comparison to the Tanzu Extensions. A lot of the yaml file handling disappeared as this is done automatically via the Tanzu CLI. For the Software Consumer or Cluster Admin, Tanzu CLI is the easy way to deploy and manage all of the supported Tanzu Packages on their TKG clusters.

Sources

Alexander Ullah View All

2 thoughts on “Tanzu Packages Explained” Leave a comment ›

Pingback: Quick guide to install cert-manager, contour, prometheus and grafana into TKG using Tanzu Packages (Kapp) – VMwire
Pingback: GitOps with FluxCD and Tanzu Packages – Beyond Elastic