OpenShift CI

The OpenShift CI platform provides two ways through which a Konflux pipeline can interact with it.

Run a ProwJob

The most straightforward way is to run a test that has already been defined in OpenShift CI. To learn how to add a custom integration test that would run a CI job in Konflux, see the run-prowjob task.

Provision an Ephemeral Cluster

The provisioning task creates an OpenShift cluster for running Konflux tests on. Familiarity with the OpenShift CI platform is required (see docs.ci.openshift.org) since many components from that domain are involved: ProwJobs, ci-operator, release payloads and more.
This guide provides links to the relevant OpenShift CI documentation where additional context may be needed.

Although it’s not strictly mandatory, having past experience in how to setup and troubleshoot OpenShift CI jobs might be helpful to better understand these paragraphs.

Provisioning a cluster (an ephemeral cluster) is done by executing the provision-ephemeral-cluster task.

apiVersion: tekton.dev/v1
kind: PipelineRun
spec:
  pipelineSpec:
    tasks:
    - name: provision-ephemeral-cluster
      taskRef:
        resolver: git
        params:
        - name: url
          value: https://github.com/openshift/konflux-tasks
        - name: revision
          value: main
        - name: pathInRepo
          value: tasks/provision-ephemeral-cluster/0.1/provision-ephemeral-cluster.yaml

Task `provision-ephemeral-cluster`

The ultimate goal of this task is to create a valid ci-operator configuration (see docs.ci.openshift.org/docs/architecture/ci-operator) that contains all the information needed to spin up an ephemeral cluster, run it in a ProwJob and, finally, retrieve a kubeconfig. The task takes several parameters that maps directly to their ci-operator configuration counterparts.
What follows is a list of the most important ones (for a complete list see github.com/openshift/konflux-tasks/tasks/provision-ephemeral-cluster/0.1/README.md):

Task Parameter ci-operator stanza Description OpenShift CI Docs

Task Parameter	ci-operator stanza	Description	OpenShift CI Docs
`params[*].name.releases`	`releases`	Release payload(s) that are going to be used to install OpenShift on the cluster	Testing with OCP releases
`params[*].name.workflow`	`tests[0].steps[0].workflow`	The OpenShift CI workflow that prepares the cloud infrastructure and creates the ephemeral cluster	What is a workflow and Workflows Catalog
`params[*].name.clusterProfile`	`tests[0].steps[0].cluster_profile`	The cluster profile that holds what’s needed (ex.: credentials) to access the cloud infrastructure in which we provision the ephemeral cluster	Multi-Stage Tests and the Test Step Registry and How to add a new Cluster Profile
`params[*].name.env`	`tests[0].steps[0].env`	Environment variables to pass to the workflow	Multi-Stage Tests and the Test Step Registry
`params[*].name.clusterClaim`	`tests[0].cluster_claim`	Claim a cluster from the Hosted Management cluster rather than provision a new one from scratch	Testing With a Cluster From a Cluster Pool

params[*].name.releases

releases

Release payload(s) that are going to be used to install OpenShift on the cluster

Testing with OCP releases

params[*].name.workflow

tests[0].steps[0].workflow

The OpenShift CI workflow that prepares the cloud infrastructure and creates the ephemeral cluster

What is a workflow and Workflows Catalog

params[*].name.clusterProfile

tests[0].steps[0].cluster_profile

The cluster profile that holds what’s needed (ex.: credentials) to access the cloud infrastructure in which we provision the ephemeral cluster

Multi-Stage Tests and the Test Step Registry and How to add a new Cluster Profile

params[*].name.env

tests[0].steps[0].env

Environment variables to pass to the workflow

Multi-Stage Tests and the Test Step Registry

params[*].name.clusterClaim

tests[0].cluster_claim

Claim a cluster from the Hosted Management cluster rather than provision a new one from scratch

Testing With a Cluster From a Cluster Pool

The task also accepts an optional env-files workspace for passing environment variable values as files. Each file in the workspace root is injected as an environment variable where the filename becomes the variable name and the file content becomes the value. Values from the workspace override same-named keys in the env parameter. This is useful when a value is too large to pass inline through the env JSON parameter.

The files can be populated from a ConfigMap or written by a prior task in the pipeline.

For example, the hypershift-hostedcluster-workflow supports an IMAGE_CONTENT_SOURCES environment variable for configuring alternate image registries. Create a ConfigMap with the mirror mapping:

apiVersion: v1
kind: ConfigMap
metadata:
  name: my-env-files
data:
  IMAGE_CONTENT_SOURCES: |
    - source: registry.example.com/my-product/my-operator
      mirrors:
        - quay.io/my-org/my-operator

Then bind it to the env-files workspace on the PipelineRun:

workspaces:
- name: env-files
  configMap:
    name: my-env-files

Provision an Ephemeral Cluster from a release payload

We want to start an ephemeral cluster built from the bleeding edge CI integration stream:

Choose an appropriate workflow from the OpenShift CI step registry
Create a new cluster profile if you don’t have one: docs.ci.openshift.org/docs/how-tos/adding-a-cluster-profile
Use the provision-ephemeral-cluster task in your pipeline:

tasks:
- name: provision-ephemeral-cluster
  params:
  - name: ownerName
    value: $(context.pipelineRun.name) # Internal usage only, use always this value here
  - name: ownerUid
    value: $(context.pipelineRun.uid) # Internal usage only, use always this value here
  - name: workflow
    value: ipi-aws
  - name: clusterProfile
    value: my-cluster-profile
  - name: releases
    value: |
      {
        "initial": {
          "integration": {
            "name": "4.21",
            "namespace": "ocp"
          }
        },
        "latest": {
          "integration": {
            "include_built_images": true,
            "name": "4.21",
            "namespace": "ocp"
          }
        }
      }
  taskRef:
    resolver: git
    params:
    - name: url
      value: https://github.com/openshift/konflux-tasks
    - name: revision
      value: main
    - name: pathInRepo
      value: tasks/provision-ephemeral-cluster/0.1/provision-ephemeral-cluster.yaml

The resulting ci-operator configuration is:

build_root:
  image_stream_tag:
    name: release
    namespace: openshift
    tag: rhel-9-release-golang-1.24-openshift-4.20
releases:
  initial:
    integration:
      name: "4.21"
      namespace: ocp
  latest:
    integration:
      name: "4.21"
      namespace: ocp
resources:
  '*':
    limits:
      memory: 400Mi
    requests:
      cpu: 200m
tests:
- as: cluster-provisioning
  steps:
    cluster_profile: my-cluster-profile
    test:
    - as: wait-test-complete
      commands: |
        # Omitted, internals that are unnecessary for this discussion
      from: cli
      resources:
        limits:
          memory: 100Mi
        requests:
          cpu: 10m
    workflow: ipi-aws

You don’t have full control over the tests[] stanza. Regardless of what parameters we pass, there is always one test as: cluster-provisioning generated.

As stated before, each parameter passed to the task maps to a ci-operator stanza, whereas others (see resources and build_root) are assigned to default values when not set.

Claim an Ephemeral Cluster from Hypershift

We want to claim a cluster from Hypershift (see Testing with a Cluster from Hypershift):

Choose hypershift-hostedcluster-workflow workflow from the OpenShift CI step registry
Identify a cluster profile that has the required credentials for the cloud provider.
Define and pass the latest release payload on the releases parameter.
Use the provision-ephemeral-cluster task in your pipeline:

tasks:
- name: provision-ephemeral-cluster
  params:
  - name: ownerName
    value: $(context.pipelineRun.name) # Internal usage only, use always this value here
  - name: ownerUid
    value: $(context.pipelineRun.uid) # Internal usage only, use always this value here
  - name: workflow
    value: hypershift-hostedcluster-workflow
  - name: clusterProfile
    value: my-cluster-profile
  - name: env
    value: '{"COMPUTE_NODE_TYPE": "m5.xlarge", "HYPERSHIFT_NODE_COUNT": "3"}'
  - name: releases
    value: |
      {"latest":{"release":{"channel":"stable","version":"4.21","architecture":"multi"}}}
  taskRef:
    resolver: git
    params:
    - name: url
      value: https://github.com/openshift/konflux-tasks
    - name: revision
      value: main
    - name: pathInRepo
      value: tasks/provision-ephemeral-cluster/0.1/provision-ephemeral-cluster.yaml

The resulting ci-operator configuration is:

build_root:
  image_stream_tag:
    name: release
    namespace: openshift
    tag: rhel-9-release-golang-1.24-openshift-4.20
base_images:
  # This image is used internally and gets injected by default
  cli:
    name: "4.22"
    namespace: ocp
    tag: cli
releases:
  latest:
    release:
      channel: stable
      version: "4.21"
      architecture: multi
resources:
  '*':
    limits:
      memory: 400Mi
    requests:
      cpu: 200m
tests:
- as: cluster-provisioning
  steps:
    cluster_profile: my-cluster-profile
    test:
    - as: wait-test-complete
      commands: |
        # Omitted, internals that are unnecessary for this discussion
      from: cli
      resources:
        limits:
          memory: 100Mi
        requests:
          cpu: 10m
    workflow: hypershift-hostedcluster-workflow

Claim an Ephemeral Cluster from Hive

Claim a pre-provisioned cluster from a Hive cluster pool (see Testing with a Cluster from a Cluster Pool):

Choose generic-claim workflow from the OpenShift CI step registry
Create a new cluster pool if you don’t have one: docs.ci.openshift.org/docs/how-tos/cluster-claim/
Use the provision-ephemeral-cluster task in your pipeline:

tasks:
- name: provision-ephemeral-cluster
  params:
  - name: ownerName
    value: $(context.pipelineRun.name) # Internal usage only, use always this value here
  - name: ownerUid
    value: $(context.pipelineRun.uid) # Internal usage only, use always this value here
  - name: workflow
    value: generic-claim
  - name: clusterClaim
    value: |
      {
        "architecture": "amd64",
        "as": "unused",
        "cloud": "aws",
        "labels": {
          "region": "us-east-1"
        },
        "owner": "openshift-ci",
        "product": "ocp",
        "timeout": "1h0m0s",
        "version": "4.19"
      }
  taskRef:
    resolver: git
    params:
    - name: url
      value: https://github.com/openshift/konflux-tasks
    - name: revision
      value: main
    - name: pathInRepo
      value: tasks/provision-ephemeral-cluster/0.1/provision-ephemeral-cluster.yaml

The resulting ci-operator configuration is:

build_root:
  image_stream_tag:
    name: release
    namespace: openshift
    tag: rhel-9-release-golang-1.24-openshift-4.20
base_images:
  # This image is used internally and gets injected by default
  cli:
    name: "4.22"
    namespace: ocp
    tag: cli
resources:
  '*':
    limits:
      memory: 400Mi
    requests:
      cpu: 200m
tests:
- as: cluster-provisioning
  steps:
    test:
    - as: wait-test-complete
      cluster_claim:
        architecture: amd64
        as: unused
        cloud: aws
        labels:
          region: us-east-1
        owner: openshift-ci
        product: ocp
        timeout: 1h0m0s
        version: "4.22"
      commands: |
        # Omitted, internals that are unnecessary for this discussion
      from: cli
      resources:
        limits:
          memory: 100Mi
        requests:
          cpu: 10m
    workflow: generic-claim

Consuming `tasks.provision-ephemeral-cluster.results`

Upon completion, the provisioning task provides a Secret containing a kubeconfig with admin privileges over the ephemeral cluster. Depending on the workflow, the secret may also contain a kubeAdminPassword key.
Mount the credentials secret as a volume and set KUBECONFIG to the mount path:

tasks:
- name: run-test
  runAfter:
  - provision-ephemeral-cluster
  params:
  - name: clusterCredentialsSecretRef
    value: $(tasks.provision-ephemeral-cluster.results.secretRef)
  taskSpec:
    params:
    - name: clusterCredentialsSecretRef
      type: string
      description: "The secret that holds the cluster credentials"
    steps:
    - name: run-test
      image: quay.io/konflux-ci/konflux-test:v1.4.52@sha256:aa2c97da9bb73a4e8d1c6b41950f8d902b74461be0e042debe89277fdc4ebe49
      env:
      - name: KUBECONFIG
        value: /var/run/secrets/cluster/kubeconfig
      volumeMounts:
      - name: cluster-credentials
        mountPath: /var/run/secrets/cluster
        readOnly: true
      script: |
        #!/usr/bin/env bash
        set -eo pipefail

        oc whoami
    volumes:
    - name: cluster-credentials
      secret:
        secretName: $(params.clusterCredentialsSecretRef)

How to keep track of the provisioning status

While it is running, the task outputs, as soon as it becomes available, the ProwJob’s status link through which a user can monitor the ongoing provisioning procedures.
This is how a ProwJob status link looks like:

https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-openshift-service-mesh-proxy-release-1.27-copy-artifacts-gcs/1972650688026513408

FAQ

Q: The workflow foobar needs a specific image defined as a base_images on the ci-operator configuration. How can you add that?

A: The provision-ephemeral-cluster task accepts the baseImages parameter as well.
You could pass that parameter as follow:

tasks:
- name: provision-ephemeral-cluster
  params:
  - name: baseImages
    value: |
      {"base":{"name":"4.10","namespace":"ocp","tag":"base"}}
  taskRef:
    resolver: git
    params:
    - name: url
      value: https://github.com/openshift/konflux-tasks
    - name: revision
      value: main
    - name: pathInRepo
      value: tasks/provision-ephemeral-cluster/0.1/provision-ephemeral-cluster.yaml

Examples

Here are some examples of how to provision or claim an ephemeral cluster.

Task `copy-secrets-to-ephemeral-cluster`

Applications with private image repositories will likely require pull secrets be placed on ephemeral clusters for testing purposes. The copy-secrets-to-ephemeral-cluster task bulk-copies Secrets from the Konflux namespace to a namespace on the ephemeral cluster.

tasks:
- name: copy-secrets-to-ephemeral-cluster
  runAfter:
  - provision-ephemeral-cluster
  taskRef:
    resolver: git
    params:
    - name: url
      value: https://github.com/openshift/konflux-tasks
    - name: revision
      value: main
    - name: pathInRepo
      value: tasks/copy-secrets-to-ephemeral-cluster/0.1/copy-secrets-to-ephemeral-cluster.yaml
  params:
  - name: clusterCredentialsSecretRef
    value: $(tasks.provision-ephemeral-cluster.results.secretRef)
  - name: namespace
    value: default
  - name: labelSelector
    value: my-app=pull-secrets

Task `deprovision-ephemeral-cluster`

Cluster resources are automatically cleaned up when the owning PipelineRun or TaskRun is deleted. Optionally, the deprovision-ephemeral-cluster task can be used to wait for cluster teardown to complete before the pipeline finishes.

In most cases, relying on the automatic cleanup is sufficient and avoids adding a blocking deprovisioning step to the pipeline. Omitting this task can significantly reduce pipeline execution time since the pipeline does not need to wait for the cluster to be fully torn down.

When used, define it within a finally block to ensure it runs regardless of pipeline outcome. The parameters it accepts are directly produced by the provisioning task’s results (for a complete list see github.com/openshift/konflux-tasks/tasks/deprovision-ephemeral-cluster/0.1/README.md):

finally:
- name: deprovision-ephemeral-cluster
  taskRef:
    resolver: git
    params:
    - name: url
      value: https://github.com/openshift/konflux-tasks
    - name: revision
      value: main
    - name: pathInRepo
      value: tasks/deprovision-ephemeral-cluster/0.1/deprovision-ephemeral-cluster.yaml
  params:
  - name: testPlatformClusterClaimName
    value: $(tasks.provision-ephemeral-cluster.results.testPlatformClusterClaimName)
  - name: testPlatformClusterClaimNamespace
    value: $(tasks.provision-ephemeral-cluster.results.testPlatformClusterClaimNamespace)

How to keep track of the deprovision status

Refer to How to keep track of the provisioning status since the ProwJob that provision and deprovision an ephemeral cluster is always the same.

OpenShift CI

Run a ProwJob

Provision an Ephemeral Cluster

Task provision-ephemeral-cluster

Provision an Ephemeral Cluster from a release payload

Claim an Ephemeral Cluster from Hypershift

Claim an Ephemeral Cluster from Hive

Consuming tasks.provision-ephemeral-cluster.results

How to keep track of the provisioning status

FAQ

Examples

Task copy-secrets-to-ephemeral-cluster

Task deprovision-ephemeral-cluster

How to keep track of the deprovision status

Task `provision-ephemeral-cluster`

Consuming `tasks.provision-ephemeral-cluster.results`

Task `copy-secrets-to-ephemeral-cluster`

Task `deprovision-ephemeral-cluster`