diff --git a/tests/.fmf/version b/tests/.fmf/version new file mode 100644 index 0000000..d00491f --- /dev/null +++ b/tests/.fmf/version @@ -0,0 +1 @@ +1 diff --git a/tests/README.md b/tests/README.md new file mode 100644 index 0000000..af2be65 --- /dev/null +++ b/tests/README.md @@ -0,0 +1,56 @@ +[Anaconda](https://github.com/rhinstaller/anaconda) installer gating tests. + +Additionally to the tests (`tests*.yml`) contains also playbooks for runnning the tests from localhost on a remote test runner. The runner can be provisioned by `linchpin`. See [run_tests_remotely.sh](run_tests_remotely.sh) script as an example of the playbooks usage. + +Running the tests remotely +-------------------------- + +### Test runner + +The remote test runner can be provided in any way. To be used by the playbooks: + +* It has to allow ssh access as a remote user which is allowed to become root, using the ssh key configured by `private_key_file` in [ansible config](remote_config/ansible.cfg). By default the `remote_user` for running the tests is `root`. +* Test runner host name / IP should be configured for the playbooks in `gating_test_runner` group of [remote_config/inventory](remote_config/inventory). + +The runner can be provisioned in a cloud by linchpin as in the [script](run_tests_remotely.sh): + +* The cloud credentials need to be configured in the file and profile reffered by `credentials` variable of [topology](linchpin/topologies/gating-test.yml). So the credentials file [`clouds.yml`](linchpin/credentials/clouds.yml) should contain profile `ci-rhos`. The file can be placed to `~/.config/linchpin` directory or the directory containing the file can be set by `linchpin` `--creds-path` option. +* The ssh key is set by `keypair` value of linchpin [topology](linchpin/topologies/gating-test.yml) file. It should correspond to the key defined in [ansible config](remote_config/ansible.cfg). The [topology](linchpin/topologies/gating-test.yml) file also defines image to be used for test runner. +* The script populates the [inventory](remote_config/inventory) for playbooks with [inventory](linchpin/layouts/gating-test.yml) generated by linchpin. +* The script tries to find out which remote user should be used (`root`, `fedora`, `cloud-user`) and updates [ansible config](remote_config/ansible.cfg) with the value. + +### Test runner environment + +Test runner environment is prepared by [`prepare-test-runner.yml`](prepare-test-runner.yml) playbook: + +* It is possible to add repositories to the runner by defining [`test_runner_repos`](roles/prepare-test-runner/defaults/main.yml) variable. It can be useful for example for adding a repository with scratch build to be tested or adding repositories for test dependencies missing on remote runner. +* Empty directory for storing test artifacts is created on test runner based on the [`artifacts`](roles/prepare-test-runner/vars/main.yml) variable. + +### Test playbooks configuration + +#### Running on the remote runner: + +Normally the testing system runs all the `tests*.yml` playbooks. + +**WARNING:** +The test playbooks are run on `localhost` (test runner provided by the testing system). They change the test runner environment (eg install packages) so most probably you don't want to run them as they are - on your local host. +The [script](run_tests_remotely.sh) updates `hosts` value of the test playbooks to use remote host from [`gating_test_runner`](remote_config/inventory/hosts) group as test runner (using a [playbook](set_tests_to_run_on_remote.yml)). +If you want to run the tests playbooks separately make sure the `hosts` variable in the test playbook is set to remote test runner (eg. `gating_test_runner`). + + + + +The test playbooks need [`artifacts`](roles/prepare-test-runner/vars/main.yml) variable supplied as can be seen in the [script](run_tests_remotely.sh). (Normally testing system takes care of this.) + +#### Installation repositories: + +Repositories (base and additional) used for installation test are defined in [repos](roles/installation-repos/defaults/main.yml) configuration. Their URL can be either defined explicitly or looked up in specified repositories of the test runner. + +#### dirinstall test + +There is a text and a vnc variant of dirinstall test. Both will run all the kickstarts found in [roles/dirinstall/templates/kickstarts](roles/dirinstall/templates/kickstarts). + + +### The results + +The results and logs are fetched from remote host into local host directory defined by `artifacts` variable passed to the playbooks. This value can be passed also to the [script](run_tests_remotely.sh) with `-a` option. diff --git a/tests/check_and_set_remote_user.yml b/tests/check_and_set_remote_user.yml new file mode 100644 index 0000000..59df611 --- /dev/null +++ b/tests/check_and_set_remote_user.yml @@ -0,0 +1,25 @@ +--- +# Check if remote_user is reachable by ansible and set ansible.cfg +# if so. + +- hosts: gating_test_runner + become: True + gather_facts: False + remote_user: "{{ remote_user }}" + + tasks: + - name: Try a raw command as a check + raw: echo "CHECK OK" + register: result + + - debug: + msg: "{{ result }}" + + - name: Set ansible.cfg remote user to "{{ remote_user }}" + become: no + local_action: + module: lineinfile + path: ./remote_config/ansible.cfg + regexp: ^remote_user + line: "remote_user = {{ remote_user }}" + when: result.stdout_lines[0] == "CHECK OK" diff --git a/tests/linchpin/PinFile b/tests/linchpin/PinFile new file mode 100644 index 0000000..4438a09 --- /dev/null +++ b/tests/linchpin/PinFile @@ -0,0 +1,4 @@ +--- +gating-test: + topology: gating-test.yml + layout: gating-test.yml diff --git a/tests/linchpin/credentials/clouds.yml b/tests/linchpin/credentials/clouds.yml new file mode 100644 index 0000000..aafd9c4 --- /dev/null +++ b/tests/linchpin/credentials/clouds.yml @@ -0,0 +1,7 @@ +clouds: + ci-rhos: + auth: + auth_url: + project_name: + username: + password: diff --git a/tests/linchpin/inventories/empty b/tests/linchpin/inventories/empty new file mode 100644 index 0000000..10acfd5 --- /dev/null +++ b/tests/linchpin/inventories/empty @@ -0,0 +1 @@ +[gating_test_runner] diff --git a/tests/linchpin/layouts/gating-test.yml b/tests/linchpin/layouts/gating-test.yml new file mode 100644 index 0000000..d511ace --- /dev/null +++ b/tests/linchpin/layouts/gating-test.yml @@ -0,0 +1,11 @@ +--- +inventory_layout: + #inventory_file: "{% raw -%}{{ workspace }}/inventories/gating-test.inventory{%- endraw%}" + vars: + hostname: __IP__ + hosts: + gating_test_runner: + count: 1 + host_groups: + - gating_test_runner + diff --git a/tests/linchpin/linchpin.conf b/tests/linchpin/linchpin.conf new file mode 100644 index 0000000..81364e0 --- /dev/null +++ b/tests/linchpin/linchpin.conf @@ -0,0 +1,295 @@ +# This file is a well-documented, and commented out (mostly) file, which +# covers the configuration options available in LinchPin +# +# Used to override default configuration settings for LinchPin +# Defaults exist in linchpin/linchpin.constants file +# +# Uncommented options enable features found in v1.5.1 or newer and +# can be turned off by commenting them out. +# +# structured in INI style +# use %% to allow code interpolation +# use % to use config interpolation +# + +[DEFAULT] +# name of the python package (Redundant, but easier than programmatically +# obtaining the value. It's very unlikely to change.) +pkg = linchpin + +# Useful for storing the RunDB or other components like global credentials +# travis-ci doesn't like ~/.config/linchpin, use /tmp +#default_config_path = ~/.config/linchpin + +# When creating an provider not already included in LinchPin, this path +# extends where LinchPin will look to run the appropriate playbooks +#external_providers_path = %(default_config_path)s/linchpin-x + +# When adding anything to the lp section, it should be general for +# the entire application. +[lp] + +# load custom ansible modules from here +#module_folder = library + +# rundb tracks provisioning transactions +# If you add a new one, rundb/drivers.py needs to be updated to match +# rundb_conn is the location of the run database. +# A common reason to move it is to use the rundb centrally across +# the entire system, or in a central db on a shared filesystem. +# System-wide RunDB: rundb_conn = ~/.config/linchpin/rundb-::mac::.json +#rundb_conn = {{ workspace }}/.rundb/rundb-::mac::.json +rundb_conn = ~/.config/linchpin/rundb-::mac::.json + +# name the type of Run database. Currently only TinyRunDB exists +#rundb_type = TinyRunDB + +# How to connect to the RunDB, if it's on a separate server, +# it may be tcp or ssh +#rundb_conn_type = file + +# The schema is used because TinyDB is a NoSQL db. Another DB +# may use this as a way to manage fields in a specific table. +#rundb_schema = {"action": "", +# "inputs": [], +# "outputs": [], +# "start": "", +# "end": "", +# "rc": 0, +# "uhash": ""} + +# each entry in the RunDB contains a unique-ish hash (uhash). This +# sets the hashing mechanism used to generate the uhash. +#rundb_hash = sha256 + +# The default dateformat used in LinchPin. Specifically used in the +# RunDB for recording start and end dates, but also used elsewhere. +#dateformat = %%m/%%d/%%Y %%I:%%M:%%S %%p + +# The name of the pinfile. Someone could adjust this and use TopFile +# or somesuch. The ramifications of this would mean that the file in +# the workspace that linchpin reads would change to this value. +#default_pinfile = PinFile + +# By default, whenever linchpin performs an action +# (linchpin up/linchpin destroy), the data is read from the PinFile. +# Enabling 'use_rundb_for_actions' will allow destroy and certain up +# actions (specifically when using --run-id or --tx-id) to pull data +# from the RunDB instead. +#use_rundb_for_actions = False +use_rundb_for_actions = True + +# A user can request specific data distilled from the RunDB. This flag +# enables the Context Distiller. +# NOTE: This flag requires generate_resources = False. +#distill_data = False +distill_data = True + +# If desired, enabling distill_on_error will distill any successfully (and +# possibly failed) provisioned resources. This is predicated on the data +# being written to the RunDB (usually means _async tasks may never record +# data upon failure). +distill_on_error = False + +# User can make linchpin use the actual return codes for final return code +# if enabled True, even if one target provision is successfull linchpin +# returns exit code zero else returns the sum of all the return codes +# use_actual_rcs = False + +# LinchPin sets several extra_vars (evars) that are passed to the playbooks. +# This section controls those items. +[evars] + +# enables the ansible --check option +# _check_mode = False + +# enables the ansible async ability. For some providers, it allows multiple +# provisioning tasks to happen at once, then will collect the data afterward. +# The default is perform the provision actions in serial. +#_async = False + +# How long to wait before failing (in seconds) for an async task. +#async_timeout = 1000 + +# the uhash value will still exist, but will not be added to +# instances or the inventory_path +#enable_uhash = False +enable_uhash = True + +# in older versions of linchpin (/linchpin +# determined in the load_config method of # linchpin.cli.LinchpinCliContext + +# Each of the following items controls the path (usually along with the +# default values below) to the corresponding item. + +# In the workspace (generally), this is the location of the layouts and +# topologies looked up by the PinFile. If either of these change, the +# value in linchpin/templates must also change. +#layouts_folder = layouts +#topologies_folder = topologies + +# The relative location for hooks +#hooks_folder = hooks + +# The relative location for provider roles +#roles_folder = roles + +# The relative location for storing inventories +#inventories_folder = inventories + +# The relative location for resources output (deprecated) +#resources_folder = resources + +# The relative location to find schemas (deprecated) +#schemas_folder = schemas + +# The relative location to find playbooks +#playbooks_folder = provision + +# The default path to schemas for validation (deprecated) +#default_schemas_path = {{ lp_path }}/defaults/%(schemas_folder)s + +# The default path to topologies if they aren't in the workspace +#default_topologies_path = {{ lp_path }}/defaults/%(topologies_folder)s + +# The default path to inventory layouts if they aren't in the workspace +#default_layouts_path = {{ lp_path }}/defaults/%(layouts_folder)s + +# The default path for outputting ansible static inventories +#default_inventories_path = {{ lp_path }}/defaults/%(inventories_folder)s + +# The default path to the ansible roles which control the providers +#default_roles_path = {{ lp_path }}/%(playbooks_folder)s/%(roles_folder)s + +# In older versions (<1.2.x), the schema was held here. These schemas are +# deprecated. +#schema_v3 = %(default_schemas_path)s/schema_v3.json +#schema_v4 = %(default_schemas_path)s/schema_v4.json + +# The location where default credentials data would exist. This path doesn't +# automatically exist +#default_credentials_path = %(default_config_path)s + +# If desired, one could overwrite the location of the generated inventory path +#inventory_path = {{ workspace }}/{{inventories_folder}}/happy.inventory + +# Libvirt images can be stored almost anywhere (not /tmp). +# Unprivileged users need not setup sudo to manage a path to which they have rights. +# The following are specific settings to manage libvirt images and instances + +# the location to store generated ssh keys and the like +#default_ssh_key_path = ~/.ssh + +# Where to store the libvirt images for copying/booting instances +#libvirt_image_path = /var/lib/libvirt/images/ + +# What user to use to access libvirt. +# Using root means sudo without password must be setup +#libvirt_user = root + +# When using root or any privileged user, this must be set to yes. +# sudo without password must also be setup +#libvirt_become = yes + +# This section covers settings for the `linchpin init` command +#[init] + +# source path of files generated by linchpin init +#source = templates/ + +# formal name of the generated PinFile. Can be changed as desired. +#pinfile = PinFile + +# This section covers logging setup +[logger] + +# Turns off and on the logger functionality +#enable = True + +# Full path to the location of the linchpin log file +file = ~/.config/linchpin/linchpin.log + +# Log format used. See https://docs.python.org/2/howto/logging-cookbook.html +#format = %%(levelname)s %%(asctime)s %%(message)s + +# Date format used. See https://docs.python.org/2/howto/logging-cookbook.html +#dateformat = %%m/%%d/%%Y %%I:%%M:%%S %%p + +# Level of logging provided +#level = logging.DEBUG + +# Logging to the console via STDERR +#[console] + +# logging to the console should also be possible +# NOTE: Placeholder only, cannot disable. +#enable = True + +# Log format used. See https://docs.python.org/2/howto/logging-cookbook.html +#format = %%(message)s + +# Level of logging provided +#level = logging.INFO + +# LinchPin hooks have several states depending on the action. Currently, there +# are three hook states relating to tasks being completed. +# * up - when performing the up (provision) action +# * destroy - when performing the destroy (teardown) action +# * inv - when performing the internal inventory generation action +# (currently unimplemented) +#[hookstates] + +# when performing the up action, these hooks states are run +#up = pre,post,inv + +# when performing the inv action, these hooks states are run +#inv = post + +# when performing the destroy action, these hooks states are run +#destroy = pre,post + +# This section covers file extensions for generating or looking +# up specific files +#[extensions] + +# When looking for provider playbooks, use this extension +#playbooks = .yml + +# When generating inventory files, use this extension +#inventory = .inventory + +# This section controls the ansible settings for display or other settings +#[ansible] + +# If set to true, this enables verbose output automatically to the screen. +# This is equivalent of passing `-v` to the linchpin command line shell. +#console = False + +# When linchpin is run, certain states are called at certain points along the +# execution timeline. These STATES are defined below. +#[states] +# in future each state will have comma separated substates + +# The name of the state which occurs before (pre) provisioning (up) +#preup = preup + +# The name of the state which occurs before (pre) teardown (destroy) +#predestroy = predestroy + +# The name of the state which occurs after (post) provisioning (up) +#postup = postup + +# The name of the state which occurs after (pre) teardown (destroy) +#postdestroy = postdestroy + +# The name of the state which occurs after (post) inventory is generated (inv) +#postinv = inventory + diff --git a/tests/linchpin/topologies/gating-test.yml b/tests/linchpin/topologies/gating-test.yml new file mode 100644 index 0000000..e34043a --- /dev/null +++ b/tests/linchpin/topologies/gating-test.yml @@ -0,0 +1,18 @@ +--- +topology_name: gating-test +resource_groups: + - resource_group_name: gating-test + resource_group_type: openstack + resource_definitions: + - name: "gating_test_runner" + role: os_server + flavor: m1.small + image: RHEL-8.0-x86_64-nightly-latest + count: 1 + keypair: kstests + fip_pool: 10.8.240.0 + networks: + - installer-jenkins-priv-network + credentials: + filename: clouds.yml + profile: ci-rhos diff --git a/tests/prepare-test-runner.yml b/tests/prepare-test-runner.yml new file mode 100644 index 0000000..f2f5f59 --- /dev/null +++ b/tests/prepare-test-runner.yml @@ -0,0 +1,7 @@ +--- +# prepare test runner +- hosts: gating_test_runner + become: true + + roles: + - prepare-test-runner diff --git a/tests/provision.fmf b/tests/provision.fmf new file mode 100644 index 0000000..62a6eba --- /dev/null +++ b/tests/provision.fmf @@ -0,0 +1,5 @@ +--- + +standard-inventory-qcow2: + qemu: + m: 2G diff --git a/tests/remote_config/ansible.cfg b/tests/remote_config/ansible.cfg new file mode 100644 index 0000000..6795cfa --- /dev/null +++ b/tests/remote_config/ansible.cfg @@ -0,0 +1,5 @@ +[defaults] +inventory = inventory +remote_user = root +host_key_checking = False +private_key_file = /path/to/private_key diff --git a/tests/remote_config/inventory/hosts b/tests/remote_config/inventory/hosts new file mode 100644 index 0000000..73cac65 --- /dev/null +++ b/tests/remote_config/inventory/hosts @@ -0,0 +1,3 @@ +[gating_test_runner] +[gating_test_runner:vars] +ansible_python_interpreter=/usr/libexec/platform-python diff --git a/tests/roles/dirinstall/tasks/ks-run.yml b/tests/roles/dirinstall/tasks/ks-run.yml new file mode 100644 index 0000000..e33227e --- /dev/null +++ b/tests/roles/dirinstall/tasks/ks-run.yml @@ -0,0 +1,78 @@ +--- + +- set_fact: + kickstart: "{{ kickstart_template | basename }}" + +- set_fact: + test_name_with_ks: "{{ test_name }}.{{ kickstart }}" + +- debug: + msg: "Running '{{ test_name }}' with kickstart '{{ kickstart }}'" + +- name: Copy installation kickstart + template: + src: "templates/kickstarts/{{ kickstart }}" + dest: "{{ kickstart_dest }}" + mode: 0755 + +- name: Clean target directory + file: + path: "{{ install_dir }}" + state: "{{ item }}" + mode: 0755 + with_items: + - absent + - directory + +- name: Clean installation logs + file: + path: "/tmp/{{ item }}" + state: absent + with_items: "{{ installation_logs }}" + +- name: Run dirinstall + shell: timeout -k 10s 3600s anaconda --dirinstall {{ install_dir }} --kickstart {{ kickstart_dest }} {{ method }} --noninteractive 2>&1 + register: result + ignore_errors: True + +- debug: + msg: "{{ result }}" + +- set_fact: + result_str: "PASS" + +- set_fact: + result_str: "FAIL" + global_result: "FAIL" + when: result.rc != 0 + +- name: Update global test.log + lineinfile: + path: "{{ local_artifacts }}/test.log" + line: "{{ result_str }} {{ test_name_with_ks }}" + create: yes + insertafter: EOF + +- name: Create this test log + copy: + content: "{{ result.stdout }}" + dest: "{{ local_artifacts }}/{{ result_str }}_{{ test_name_with_ks }}.log" + +- name: Create installation logs dir in artifacts + file: + path: "{{ local_artifacts }}/{{ test_name_with_ks }}" + state: directory + +- name: Copy input kickstart to artifacts + copy: + remote_src: True + src: "{{ kickstart_dest }}" + dest: "{{ local_artifacts }}/{{ test_name_with_ks }}/{{ kickstart_dest | basename }}" + +- name: Copy installation logs to artifacts + copy: + remote_src: True + src: "/tmp/{{ item }}" + dest: "{{ local_artifacts }}/{{ test_name_with_ks }}/{{ item }}" + with_items: "{{ installation_logs }}" + ignore_errors: True diff --git a/tests/roles/dirinstall/tasks/main.yml b/tests/roles/dirinstall/tasks/main.yml new file mode 100644 index 0000000..66cf52a --- /dev/null +++ b/tests/roles/dirinstall/tasks/main.yml @@ -0,0 +1,13 @@ +--- +- name: Install vnc install dependencies + dnf: + name: + - metacity + state: latest + when: method == "--vnc" + +- include_tasks: ks-run.yml + with_fileglob: + - templates/kickstarts/* + loop_control: + loop_var: kickstart_template diff --git a/tests/roles/dirinstall/templates/kickstarts/basic b/tests/roles/dirinstall/templates/kickstarts/basic new file mode 100644 index 0000000..d0c0db1 --- /dev/null +++ b/tests/roles/dirinstall/templates/kickstarts/basic @@ -0,0 +1,12 @@ +{{ base_repo_command }} +{{ "\n".join(repo_commands) }} +{{ "\n".join(additional_repo_commands) }} +lang en_US.UTF-8 +keyboard --vckeymap=us --xlayouts='us' +rootpw --plaintext redhat +timezone --utc Europe/Prague + +shutdown + +%packages +%end diff --git a/tests/roles/dirinstall/vars/main.yml b/tests/roles/dirinstall/vars/main.yml new file mode 100644 index 0000000..5438297 --- /dev/null +++ b/tests/roles/dirinstall/vars/main.yml @@ -0,0 +1,5 @@ +--- + +install_dir: "/root/installdir" +kickstart_dest: "/root/ks.dirinstall.cfg" + diff --git a/tests/roles/global-result/tasks/main.yml b/tests/roles/global-result/tasks/main.yml new file mode 100644 index 0000000..80ec312 --- /dev/null +++ b/tests/roles/global-result/tasks/main.yml @@ -0,0 +1,6 @@ +--- +# Make the test fail based on individual test failures + +- fail: + msg: "The test has failed." + when: global_result is defined and global_result == "FAIL" diff --git a/tests/roles/installation-repos/defaults/main.yml b/tests/roles/installation-repos/defaults/main.yml new file mode 100644 index 0000000..fad262e --- /dev/null +++ b/tests/roles/installation-repos/defaults/main.yml @@ -0,0 +1,43 @@ +--- + +### Base repository + +# Base repository command for kickstart + +#base_repo_command: "url --url=http://download.englab.brq.redhat.com/pub/fedora/development-rawhide/Everything/x86_64/os/" + +# If base_repo_command is not defined, look for base repo url +# in [base_repo_from_runner.repo] repository of +# /etc/yum.repos.d/base_repo_from_runner.file on test runner + +base_repo_from_runner: + file: rhel.repo + repo: rhel + + +### Additional repositories + +# Additional repo commands for kickstart: +# - undefine to allow detecting of repos from test runner by +# repos_from_runner variable +# - set to [] for no additional repositories + +#repo_commands: [] +#repo_commands: +# - "repo --baseurl=http://download.englab.brq.redhat.com/pub/fedora/development-rawhide/Everything/x86_64/os/" + +# If repo_commands is not defined, look for additinal repositories +# in [repo] repository of /etc/yum.repos.d/file of test runner. +# Multiple repositories can be defined here. + +repos_from_runner: + - file: rhel.repo + repo: rhel + - file: rhel.repo + repo: rhel-AppStream + + +# Additional repo commands to be used in any case, +# ie even in case additional repos are detected by repos_from_runner + +additional_repo_commands: [] diff --git a/tests/roles/installation-repos/tasks/main.yml b/tests/roles/installation-repos/tasks/main.yml new file mode 100644 index 0000000..0686004 --- /dev/null +++ b/tests/roles/installation-repos/tasks/main.yml @@ -0,0 +1,87 @@ +--- + +### Set up local facts from system repositories + +- name: Create facts directory for repository custom facts + file: + state: directory + recurse: yes + path: /etc/ansible/facts.d + +- name: Install base repository facts + copy: + remote_src: yes + src: "/etc/yum.repos.d/{{ base_repo_from_runner.file }}" + dest: "/etc/ansible/facts.d/{{ base_repo_from_runner.file}}.fact" + when: base_repo_command is not defined and base_repo_from_runner is defined + +- name: Install additional repositories facts + copy: + remote_src: yes + src: "/etc/yum.repos.d/{{ item.file }}" + dest: "/etc/ansible/facts.d/{{ item.file}}.fact" + with_items: "{{ repos_from_runner }}" + when: repo_commands is not defined and repos_from_runner is defined + +- name: Setup repository facts + setup: + filter: ansible_local + +### Base repository + +- name: Set base installation repository from system base metalink repository + set_fact: + base_repo_command: "url --metalink={{ ansible_local[base_repo_from_runner.file][base_repo_from_runner.repo]['metalink'] }}" + when: ansible_local[base_repo_from_runner.file][base_repo_from_runner.repo]['metalink'] is defined + +- name: Set base installation repository from system base mirrorlist repository + set_fact: + base_repo_command: "url --mirrorlist={{ ansible_local[base_repo_from_runner.file][base_repo_from_runner.repo]['mirrorlist'] }}" + when: ansible_local[base_repo_from_runner.file][base_repo_from_runner.repo]['mirrorlist'] is defined + +- name: Set base installation repository from system base url repository + set_fact: + base_repo_command: "url --url={{ ansible_local[base_repo_from_runner.file][base_repo_from_runner.repo]['baseurl'] }}" + when: ansible_local[base_repo_from_runner.file][base_repo_from_runner.repo]['baseurl'] is defined + +### Additional repositories + +- name: Look for system metalink repositories + set_fact: + repos_metalink: "{{ repos_metalink | default([]) + [ 'repo --name=' + item.repo + ' --metalink=' + ansible_local[item.file][item.repo]['metalink'] ] }}" + #ignore_errors: true + with_items: "{{ repos_from_runner }}" + when: repo_commands is not defined and ansible_local[item.file][item.repo]['metalink'] is defined + +- name: Look for system mirrorlist repositories + set_fact: + repos_mirrorlist: "{{ repos_mirrorlist | default([]) + [ 'repo --name=' + item.repo + ' --mirrorlist=' + ansible_local[item.file][item.repo]['mirrorlist'] ] }}" + #ignore_errors: true + with_items: "{{ repos_from_runner }}" + when: repo_commands is not defined and ansible_local[item.file][item.repo]['mirrorlist'] is defined + +- name: Look for system baseurl repositories + set_fact: + repos_baseurl: "{{ repos_baseurl | default([]) + [ 'repo --name=' + item.repo + ' --baseurl=' + ansible_local[item.file][item.repo]['baseurl'] ] }}" + #ignore_errors: true + with_items: "{{ repos_from_runner }}" + when: repo_commands is not defined and ansible_local[item.file][item.repo]['baseurl'] is defined + + +- name: Set additional metalink installation repositories from system + set_fact: + repo_commands: "{{ repo_commands | default([]) + [ item ] }}" + with_items: "{{ repos_metalink }}" + when: repos_metalink is defined + +- name: Set additional mirrorlist installation repositories from system + set_fact: + repo_commands: "{{ repo_commands | default([]) + [ item ] }}" + with_items: "{{ repos_mirrorlist }}" + when: repos_mirrorlist is defined + +- name: Set additional baseurl installation repositories from system + set_fact: + repo_commands: "{{ repo_commands | default([]) + [ item ] }}" + with_items: "{{ repos_baseurl }}" + when: repos_baseurl is defined diff --git a/tests/roles/prepare-env/tasks/main.yml b/tests/roles/prepare-env/tasks/main.yml new file mode 100644 index 0000000..856ac28 --- /dev/null +++ b/tests/roles/prepare-env/tasks/main.yml @@ -0,0 +1,6 @@ +--- +- name: Prepare testing environment + dnf: + name: + - anaconda + state: latest diff --git a/tests/roles/prepare-test-runner/defaults/main.yml b/tests/roles/prepare-test-runner/defaults/main.yml new file mode 100644 index 0000000..8332f3e --- /dev/null +++ b/tests/roles/prepare-test-runner/defaults/main.yml @@ -0,0 +1,10 @@ +--- + +# Additional repos added to test runner, eg repo with builds to be tested +test_runner_repos: + rhel: + name: rhel + source: "baseurl=http://download.eng.bos.redhat.com/rhel-9/nightly/RHEL-9-Beta/latest-RHEL-9/compose/BaseOS/x86_64/os/" + rhel-AppStream: + name: rhel-AppStream + source: "baseurl=http://download.eng.bos.redhat.com/rhel-9/nightly/RHEL-9-Beta/latest-RHEL-9/compose/AppStream/x86_64/os/" diff --git a/tests/roles/prepare-test-runner/tasks/main.yml b/tests/roles/prepare-test-runner/tasks/main.yml new file mode 100644 index 0000000..2c01d47 --- /dev/null +++ b/tests/roles/prepare-test-runner/tasks/main.yml @@ -0,0 +1,18 @@ +--- + +- name: Add repositories + template: + src: repo.j2 + dest: "/etc/yum.repos.d/{{ test_runner_repos[item]['name']}}.repo" + with_items: "{{ test_runner_repos }}" + +- name: Create empty artifacts directory on local host + become: no + local_action: + module: file + path: "{{ artifacts }}" + state: "{{ item }}" + mode: 0755 + with_items: + - absent + - directory diff --git a/tests/roles/prepare-test-runner/templates/repo.j2 b/tests/roles/prepare-test-runner/templates/repo.j2 new file mode 100644 index 0000000..557cec8 --- /dev/null +++ b/tests/roles/prepare-test-runner/templates/repo.j2 @@ -0,0 +1,5 @@ +[{{ test_runner_repos[item]['name'] }}] +name={{ test_runner_repos[item]['name'] }} +{{ test_runner_repos[item]['source'] }} +enabled=1 +gpgcheck=0 diff --git a/tests/roles/sync-artifacts/defaults/main.yml b/tests/roles/sync-artifacts/defaults/main.yml new file mode 100644 index 0000000..0c46caa --- /dev/null +++ b/tests/roles/sync-artifacts/defaults/main.yml @@ -0,0 +1,3 @@ +--- + +artifacts: "{{ lookup('env', 'TEST_ARTIFACTS')|default('./artifacts', true) }}" diff --git a/tests/roles/sync-artifacts/tasks/main.yml b/tests/roles/sync-artifacts/tasks/main.yml new file mode 100644 index 0000000..449fcac --- /dev/null +++ b/tests/roles/sync-artifacts/tasks/main.yml @@ -0,0 +1,13 @@ +--- + +- name: Make sure rsync required to fetch artifacts is installed + dnf: + name: + - rsync + +- name: Fetch artifacts + synchronize: + mode: pull + delete: yes + src: "{{ local_artifacts }}/" + dest: "{{ artifacts }}" diff --git a/tests/run_tests_remotely.sh b/tests/run_tests_remotely.sh new file mode 100755 index 0000000..309d36c --- /dev/null +++ b/tests/run_tests_remotely.sh @@ -0,0 +1,161 @@ +#!/bin/sh + +usage () { + cat <] + + Run gating tests on test runners provisioned by linchpin and deployed with ansible, + syncing artifacts to localhost. + + -c Run configuration check. + -a Local host directory for fetching artifacts from test runner. +HELP_USAGE +} + +CHECK_ONLY="no" +ARTIFACTS="/tmp/artifacts" + +while getopts "ca:" opt; do + case $opt in + c) + # Run only configuration check + CHECK_ONLY="yes" + ;; + a) + # Set up directory for fetching artifacts + ARTIFACTS="${OPTARG}" + ;; + *) + echo "Usage:" + usage + exit 1 + ;; + esac +done + +DEFAULT_CRED_FILENAME="clouds.yml" +CRED_DIR="${HOME}/.config/linchpin" +CRED_FILE_PATH=${CRED_DIR}/${DEFAULT_CRED_FILENAME} +TOPOLOGY_FILE_PATH="linchpin/topologies/gating-test.yml" +ANSIBLE_CFG_PATH="remote_config/ansible.cfg" + +CHECK_RESULT=0 + + +############################## Check the configuration + +echo +echo "========= Dependencies are installed" +echo "linchpin and ansible are required to be installed." +echo "For linchpin installation instructions see:" +echo "https://linchpin.readthedocs.io/en/latest/installation.html" +echo + +if ! type ansible &> /dev/null; then + echo "=> FAILED: ansible package is not installed" + CHECK_RESULT=1 +else + echo "=> OK: ansible is installed" +fi + +if ! type linchpin &> /dev/null; then + echo "=> FAILED: linchpin is not installed" + CHECK_RESULT=1 +else + echo "=> OK: linchpin is installed" +fi + + +echo +echo "========= Linchpin cloud credentials configuration" +echo "The credentials file for linchpin provisioner should be in ${CRED_DIR}" +echo "The name of the file and the profile to be used is defined by" +echo " resource_groups.credentials variables in the topology file" +echo " (${TOPOLOGY_FILE_PATH})" +echo + +config_changed=0 +if [[ -f ${TOPOLOGY_FILE_PATH} ]]; then + grep -q 'filename:.*'${DEFAULT_CRED_FILENAME} ${TOPOLOGY_FILE_PATH} + config_changed=$? +fi + +if [[ ${config_changed} -eq 0 ]]; then + if [[ -f ${CRED_FILE_PATH} ]]; then + echo "=> OK: ${CRED_FILE_PATH} exists" + else + echo "=> FAILED: ${CRED_FILE_PATH} does not exist" + CHECK_RESULT=1 + fi +else + echo "=> NOT CHECKING: seems like this has been configured in a different way" +fi + + +echo +echo "========== Deployment ssh key configuration" +echo "The ssh key used for deployment with ansible has to be defined by" +echo "private_key_file variable in ${ANSIBLE_CFG_PATH}" +echo "and match the key used for provisioning of the machines with linchpin" +echo "which is defined by resource_groups.resource_definitions.keypair variable" +echo "in topology file (${TOPOLOGY_FILE_PATH})." +echo + + +deployment_key_defined_line=$(grep 'private_key_file.*=.*[^\S]' ${ANSIBLE_CFG_PATH}) +if [[ -n "${deployment_key_defined_line}" ]]; then + echo "=> OK: ${ANSIBLE_CFG_PATH}: ${deployment_key_defined_line}" +else + echo "=> FAILED: deployment ssh key not defined in ${ANSIBLE_CFG_PATH}" + CHECK_RESULT=1 +fi + +linchpin_keypair=$(grep "keypair:" ${TOPOLOGY_FILE_PATH} | uniq) +echo "=> INFO: should be the same key as ${TOPOLOGY_FILE_PATH}: ${linchpin_keypair}" + + +if [[ ${CHECK_RESULT} -ne 0 ]]; then +echo +echo "=> Configuration check FAILED, see FAILED messages above." +echo +fi + +if [[ ${CHECK_ONLY} == "yes" || ${CHECK_RESULT} -ne 0 ]]; then + exit ${CHECK_RESULT} +fi + + +############################## Run the tests + +set -x + +### Clean the linchpin generated inventory +rm -rf linchpin/inventories/*.inventory + +### Provision test runner +linchpin -v --workspace linchpin -p linchpin/PinFile -c linchpin/linchpin.conf up + +### Pass inventory generated by linchpin to ansible +cp linchpin/inventories/*.inventory remote_config/inventory/linchpin.inventory + +### Use remote hosts in tests playbooks +ansible-playbook set_tests_to_run_on_remote.yml + +### Use the ansible configuration for running tests on remote host +export ANSIBLE_CONFIG=${ANSIBLE_CFG_PATH} + +### Configure remote user for playbooks +# By default root is used but it can be fedora or cloud-user for cloud images +for USER in root fedora cloud-user; do + ansible-playbook --extra-vars="remote_user=$USER" check_and_set_remote_user.yml +done + +### Prepare test runner +ansible-playbook --extra-vars="artifacts=${ARTIFACTS}" prepare-test-runner.yml +### Run test on test runner (supply artifacts variable which is testing system's job) +ansible-playbook --extra-vars="artifacts=${ARTIFACTS}" tests.yml + +### Destroy the test runner +linchpin -v --workspace linchpin -p linchpin/PinFile -c linchpin/linchpin.conf destroy + diff --git a/tests/set_tests_to_run_on_remote.yml b/tests/set_tests_to_run_on_remote.yml new file mode 100644 index 0000000..4a3b6c9 --- /dev/null +++ b/tests/set_tests_to_run_on_remote.yml @@ -0,0 +1,17 @@ +--- +# Replace hosts in test playbooks to run on remote host instead of localhost + +- hosts: localhost + become: False + gather_facts: False + + tasks: + - name: Replace hosts in tests*.yml playbooks + lineinfile: + path: "{{ item }}" + regexp: "- hosts: localhost\\S*" + line: "- hosts: gating_test_runner" + backrefs: yes + with_fileglob: + - tests*.yml + diff --git a/tests/tests.yml b/tests/tests.yml new file mode 100644 index 0000000..e78df87 --- /dev/null +++ b/tests/tests.yml @@ -0,0 +1,22 @@ +--- +# test anaconda +- hosts: localhost + become: true + tags: + - classic + vars_files: + - vars_tests.yml + + roles: + - role: prepare-env + tags: + - prepare-env + - role: installation-repos + - role: dirinstall + vars: + method: "--text" + test_name: dirinstall-text + tags: + - dirinstall-text + - role: sync-artifacts + - role: global-result diff --git a/tests/vars_tests.yml b/tests/vars_tests.yml new file mode 100644 index 0000000..3f23934 --- /dev/null +++ b/tests/vars_tests.yml @@ -0,0 +1,12 @@ +--- +# variables for tests.yml +installation_logs: + - anaconda.log + - dbus.log + - dnf.librepo.log + - hawkey.log + - ifcfg.log + - packaging.log + - program.log + - storage.log +local_artifacts: /tmp/artifacts