Context

Definition of the environment context

As a user I want to define a context so that test, plan or story metadata can be adjusted to it accordingly.

This specification defines all standard context dimensions which can be used together with the adjust attribute to modify object metadata for particular context. For a detailed description of the concept itself see the fmf context documentation.

The context can be defined either on the command line using the --context or the -c options or under the plan context key which can be used to adjust all tests included in that particular plan. The context definition provided on the command line overrides context defined in the plan.

All context dimensions are optional and all context dimension values are handled in case-insensitive way. It is possible to define your own dimensions to describe the context, just make sure the dimension name does not conflict with reserved names.

Note

Always make sure the needed context dimensions are explicitly provided either on the command line or in the plan.

There is no support for detecting context from other command line options or from the provisioned environment. Similarly, provision options, such as distro image, are not detected from the context values.

Because of the chicken-and-egg problem (plan definitions, including the provision step, can be modified using adjust rules as well) there is no plan to implement context autodetection from the provisioned environment.

Examples:

# Full option name
tmt --context distro=fedora-33 run
tmt --context product=rhscl run
tmt --context trigger=code run

# Short version
tmt -c distro=fedora test show
tmt -c distro=fedora plan show

# Provide multiple context dimensions
tmt --context distro=fedora-33 --context trigger=commit run

# Provide multiple context dimension values
tmt --context distro=fedora-33,fedora-34 run

# Read context from a file
tmt --context @ci.yaml run

# In most use cases you want to define both distro image for
# the provision step and distro context dimension because
# context is not detected from image and vice versa.
tmt --context distro=fedora-44 run --all provision --how virtual --image fedora-44

# When context is not defined, adjust rules checking for the
# specific distro version will not be applied
tmt run --all provision --how virtual --image fedora-44

# Distro images are not detected from the context value,
# default image will be used when not explicitly specified
tmt --context distro=fedora-44 run --all provision --how container

dimension

Supported context dimensions

As a tester I want to clearly specify dimensions, such as ‘distro’ or ‘trigger’, so that I can well describe the context in which a test is running.

The following dimensions are reserved for storing dedicated information as described below:

distro

The operating system distribution on which the application runs (fedora, fedora-33, centos, centos-8, centos-8.4, centos-stream, centos-stream-9, rhel, rhel-8, rhel-8.4).

deployment-mode

Deployment mode of the distribution, package for the standard way using packages or image for the new way of distributing using container images.

variant

The distribution variant (Client, Desktop, Server Workstation, Silverblue, CoreOS).

arch

The guest architecture (aarch64, armhfp, i386, ppc64, ppc64le, s390x, x86_64).

component

Name of the relevant component, should match the source package name (bash, php, httpd).

collection

The Red Hat Software Collection name (python27, rh-python34, rh-mariadb100, httpd24).

module

Module name with an optional stream specification (mariadb, mariadb:10.5, httpd, httpd:2.4, perl, perl:5.32, ruby, ruby:2.7).

initiator

Name of the service, pipeline or tool which initiated the testing or special value human for manual testing. See the initiator section for details.

trigger

The event which triggered testing, see the trigger section for the full list of supported values.

Examples:

context:
    distro: fedora-33
    variant: Workstation
    arch: x86_64

context:
    product: rhscl
    collection: httpd24

Status: implemented

initiator

Definition of the initiator dimension

As a user I want to run a custom set of tests based on the service, pipeline or tool which initiated the testing.

Sometimes it is useful to choose a custom set of tests for particular automation service or when tests are run manually. The following values are defined:

human

Test execution was initiated manually by a human.

packit

For jobs initiated by the Packit service.

fedora-ci

Testing in the Fedora CI pipeline. To be implemented.

centos-ci

Testing in the CentOS CI pipeline. To be implemented.

rhel-ci

Testing in the RHEL CI pipeline. To be implemented.

ewa

Errata Workflow Automation jobs. To be implemented.

konflux

Application builder. To be implemented.

Each service, pipeline or tool is responsible to set the value to its name. For manual testing --context initiator=human is expected on the command line.

Examples:

# Enable the plan only when run manually
summary: Tests requiring special environment
discover:
    how: fmf
    filter: tag:special
execute:
    how: tmt
adjust:
    enabled: false
    when: initiator != human

Status: implemented

trigger

Definition of the trigger dimension

As a user I want to run a custom set of tests for different events which trigger the testing.

Testing can be triggered by various events and a different set of tests is usually executed for each of them. The trigger dimension supports the following values:

commit

Project source code has changed. This can be either a pull/merge request creation or update, new branch created or a new commit pushed to an existing branch.

build

There has been a new official package built in koji or brew. It can be a non-scratch build created directly or a draft build which has been promoted to non-draft.

update

A new bodhi update or errata advisory with one or more builds has been created or updated.

compose

There is a new compose ready for integration testing.

This context dimension will usually be provided from the command line.

Examples:

summary: Full test coverage (takes three days)
discover:
    how: fmf
execute:
    how: tmt
adjust:
    enabled: false
    when: trigger == code

Status: implemented