Skip to content

Drop STI migration section #5078

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
LecrisUT wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Drop STI migration section #5078

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
271 changes: 0 additions & 271 deletions docs/questions.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -306,274 +306,3 @@ Custom workflow can then consume generated ids and perform desired
actions such as fetch the tests and execute them.

__ https://fmf.readthedocs.io/en/latest/concept.html#identifiers


How do I migrate STI tests to tmt?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

`Standard Test Interface`__ tests are enabled using ``tests.yml``
Ansible playbooks together with the `Standard Test Roles`__ which
make it easier to execute tests, check their results and perform
additional actions such as installing required packages. The
configuration, however, can sometimes be confusing and quite hard
to understand.

With ``tmt`` it is possible to achieve the same result with much
more concise and clean syntax. For majority of existing tests the
conversion is relatively straightforward. Let's demonstrate it on
a couple of real-life examples. Below you can see the original STI
ansible playbooks and their ``tmt`` equivalents for inspiration.

As the first step, initialize the metadata tree using the ``tmt
init`` command in the root of the git repository. Then store the
new config files with the ``.fmf`` extension. Naming and location
of the files is up to you. See the :ref:`guide` for more details.

.. note::

With ``tmt``, files used in the ``script`` or ``test``
key are expected to be executable, unlike with STI.

__ https://docs.fedoraproject.org/en-US/ci/standard-test-interface/
__ https://docs.fedoraproject.org/en-US/ci/standard-test-roles/


Simple Script
------------------------------------------------------------------

Running a simple binary using STI:

.. code-block:: yaml

- hosts: localhost
roles:
- role: standard-test-basic
tags:
- classic
tests:
- simple:
dir: .
run: binary --help

The equivalent ``tmt`` plan has only two lines:

.. code-block:: yaml

execute:
script: binary --help

Store them for example as ``/plans/smoke.fmf`` and you're done.


Required Packages
------------------------------------------------------------------

This example prepares testing environment by installing
required packages.

STI example:

.. code-block:: yaml

- hosts: localhost
tags:
- atomic
- classic
- container
roles:
- role: standard-test-beakerlib
tests:
- cmd-line-options
required_packages:
- which
- rpm-build
- libtool
- gettext

tmt example plan (L2 metadata):

.. code-block:: yaml

summary: Check basic command line options
prepare:
how: install
package:
- which
- rpm-build
- libtool
- gettext
execute:
script: cmd-line-options


Remote Repository
------------------------------------------------------------------

Tests in the following example are fetched from a remote
repository and filtered by the provided condition.

STI example:

.. code-block:: yaml

- hosts: localhost
roles:
- role: standard-test-beakerlib
tags:
- classic
repositories:
- repo: "https://src.fedoraproject.org/tests/shell.git"
dest: "shell"
fmf_filter: "tier: 1"

tmt example plan (L2 metadata):

.. code-block:: yaml

summary: Tier 1 shell test plan
discover:
how: fmf
url: https://src.fedoraproject.org/tests/shell.git
filter: "tier: 1"
execute:
how: tmt


Multiple Tests
------------------------------------------------------------------

In this migration of STI a single plan (L2 metadata) is created
and each original test is stored in a separate L1 metadata file
(test). This approach allows the setup of different environment
variables and required packages for each test.

STI example:

.. code-block:: yaml

- hosts: localhost
roles:
- role: standard-test-basic
tags:
- classic
tests:
- smoke27:
dir: tests
run: VERSION=2.7 METHOD=virtualenv ./venv.sh
- smoke37:
dir: tests
run: VERSION=3.7 ./venv.sh
required_packages:
- python27
- python37
- python2-virtualenv
- python3-virtualenv
- python2-devel
- python3-devel


tmt example: plan (L2 metadata) and tests (L1 metadata)

.. code-block:: yaml
:caption: plans/example.fmf

discover:
how: fmf
execute:
how: tmt

.. code-block:: yaml
:caption: tests/smoke27.fmf

test: ./venv.sh
environment:
VERSION: 2.7
METHOD: virtualenv
require:
- python27
- python2-virtualenv
- python2-devel

.. code-block:: yaml
:caption: tests/smoke37.fmf

test: ./venv.sh
environment:
VERSION: 3.7
require:
- python37
- python3-virtualenv
- python3-devel

This arrangement can be especially useful when a large number of
tests is stored in the repository.


Dist Git Source
------------------------------------------------------------------

Use the ``dist-git-source`` feature of the ``discover`` step to
extract tests from the (rpm) sources.

STI example:

.. code-block:: yaml

- hosts: localhost
tags:
- classic
roles:
- role: standard-test-source

tmt example plan (L2 metadata):

.. code-block:: yaml

discover:
how: shell
dist-git-source: true

See the :tmt:story:`/spec/plans/discover/dist-git-source` documentation for
more details.


Migrating provision.fmf
------------------------------------------------------------------

The ``provision.fmf`` file is used to specify storage and network
devices. In this migration, the contents of the ``provision.fmf``
file are moved to the ``provision`` step under ``hardware``
specification.

``provision.fmf`` example:

.. code-block:: yaml

standard-inventory-qcow2:
qemu:
drive:
- size: 10737418240
- size: 10737418240
- size: 10737418240

tmt example plan (L2 metadata):

.. code-block:: yaml

provision:
how: virtual
hardware:
disk:
- size: ">10GiB"
- size: ">10GiB"
- size: ">10GiB"

See the :tmt:story:`/spec/hardware/disk` and :tmt:story:`/spec/hardware/network`
documentation for more details about these hardware specifications
in tmt plans.

If you were using ``provision.fmf`` with Testing Farm, check out
the `Testing Farm docs`__ on this HW requirement for more details
and how Testing Farm works with tmt metadata.

__ https://docs.testing-farm.io/Testing%20Farm/0.1/test-request.html
Loading