MBBox deployment guide

Introduction

This guide will describe the deployment of MBBox operator in OpenShift 4 cluster and it's prerequisites.

MBBox upstream documentation can be found at: https://mbbox-operator.readthedocs.io/en/latest/.

The MBBox operator has kubernetes custom resources to deploy the following components:

• koji-hub (includes koji-web)
• koji-builder
• kojira
• MBS Backend
• MBS Frontend

Dependencies

MBBox depends on two components/services to be already deployed/available:

• PostgreSQL (>= 10.4)
• Fedora Messaging

The operator does not deploy the above components but needs two secrets, one for each component, in the same namespace custom resources (koji, koji-builder, etc) are being created.

PostgreSQL

Secret format (all data values need to be base64 encoded):

apiVersion: v1
kind: Secret
metadata:
  name: postgres
  labels:
    app: postgres
data:
  POSTGRES_HOST: cG9zdGdyZXNxbA== # postgresql
  POSTGRES_DB: bWJveGRi # mboxdb
  POSTGRES_USER: a29qaQ== # koji
  POSTGRES_PASSWORD: bWJveA== # mbox

Creating the secret from the command line:

oc create secret generic postgres \
--from-literal=POSTGRES_HOST=postgresql \
--from-literal=POSTGRES_DB=mboxdb \
--from-literal=POSTGRES_USER=koji \
--from-literal=POSTGRES_PASSWORD=mbox

Fedora Messaging

Secret format (all data values need to be base64 encoded):

apiVersion: v1
kind: Secret
metadata:
  name: koji-hub-msg
data:
  koji.ca: |-
    Y2hhbmdlbWU=
  koji.crt: |-
    Y2hhbmdlbWU=
  koji.key: |-
    Y2hhbmdlbWU=

The secret can also be created from the command line:

oc create secret generic koji-hub-msg \
--from-file=koji.ca=/path/to/ca.pem \
--from-file=koji.crt=/path/to/cert.pem \
--from-file=koji.key=/path/to/key.pem

All certificates must be in PEM format.

Storage

NOTE: this section describes how to deal with Openshift Persistent Volumes if the cluster being used is not using any storage provisioning operator which may require extra PV setup from the cluster admin.

The MBBox needs several Persistent Volumes created in prior to deployment:

• mbox-registry (Recommended: 100 Gi) - Used as image registry
• httpd (Minimum: 1Gi) - Used by koji-hub httpd server
• koji (Minimum: 50Gi) - Used for shared koji space
• postgres (Minimum: 5Gi) - Used by postgreSQL database

Persistent Volumes

ClaimRef

ClaimRef should be set in created Persistent Volumes (PVs) if no storage class is set in the cluster scope:

kind: PersistentVolume
apiVersion: v1
metadata:
  annotations:
    pv.kubernetes.io/bound-by-controller: 'yes'
  selfLink: /api/v1/persistentvolumes/stg-postgres
  resourceVersion: '39237794'
  name: stg-postgres
  uid: d0e3960e-af9e-45af-8f87-ac03def7b8a5
  creationTimestamp: '2020-11-23T01:01:35Z'
...
spec:
  capacity:
    storage: 5Gi
  claimRef:
    apiVersion: v1 
    kind: PersistentVolumeClaim
    namespace: mbox # pvc namespace
    name: postgres # pvc name to bound to
  persistentVolumeReclaimPolicy: Retain
  storageClassName: slow
  volumeMode: Filesystem
  ...

Both uid and resourceVersion will need to be removed from the claimRef property if the PV needs to be rebound to another PVC:

claimRef:
    apiVersion: v1 
    kind: PersistentVolumeClaim
    namespace: mbox-stg # new namesapce
    name: postgres-stg # new pvc name
    uid: 775c0823-47c1-46a2-b07c-612d59592430 # remove this
    resourceVersion: '39235159' #remove this

Both PVC and PV should have a bound status after a few seconds.

A note on volumes: recently created volumes may need a permission change the volume itself, the oficial openshift docs has a page dedicated to this problem and how to fix it: https://access.redhat.com/solutions/3508731

Deployment

There are three flavors of deployment:

• Development
• Production: Root CA Only
• Production: All Certificates

What differs each deployment flavor is the amount of required steps before the actual deployment process.

You can follow the upstream documentation once those preparation steps are done: https://mbbox-operator.readthedocs.io/en/latest/deployment-guide.html

Development

Ideal for running MBBox locally and automated tests.

The only requirement is a running openshift cluster with admin access.

All certificates, incluing the Root CA will be self signed using openssl.

Production: Root CA Only

This deployment path requires an openshift secret in the same namespace other components will be created, such as koji.

This secret should contain a root CA certificate and private key, this way the operator will create and sign all required certificates from this root CA instead of using a self signed one.

Secret format:

apiVersion: v1
kind: Secret
metadata:
  name: koji-hub-ca-cert
  labels:
    app: koji-hub
data:
  cert: -|
    cert_in_base64_format
  key: -|
    key_in_base64_format

You may also create it from the command line:

oc create secret generic koji-hub-ca-cert \
--from-file=cert=/ca.pem \
--from-file=key=/key.pem

You may then proceed in deployment MBBox.

Production: All Certificates

Similar to the previous section but it needs all certificates and keys to be manually created.

[Koji-Hub] Root CA Secret

The root certifcate which all other certificates were generated/signed from.

We only need the certificate in this case.

Secret format:

apiVersion: v1
kind: Secret
metadata:
  name: koji-hub-ca-cert
  labels:
    app: koji-hub
data:
  cert: -|
    cert_in_base64_format

You may also create it from the command line:

oc create secret generic koji-hub-ca-cert \
--from-file=cert=/ca.pem

[Koji-Hub] HTTPD Secret

Apache HTTPD certificate and private key.

NOTES:

• The certificate CN field must use the same value as the public hostname for koji;
• The following subject altnames must be used:
  • DNS:$KOJI_PUBLIC_HOSTNAME
  • DNS:koji-hub
  • DNS:koji-hub.$OPENSHIFT_NAMESPACE.svc
  • DNS:koji-hub.$OPENSHIFT_NAMESPACE.svc.cluster

Secret format:

apiVersion: v1
kind: Secret
metadata:
  name: koji-hub-service-cert
  labels:
    app: koji-hub
data:
  tls.crt: -|
    cert_in_base64_format
  tls.key: -|
    key_in_base64_format

You may also create it from the command line:

oc create secret generic koji-hub-service-cert \
--from-file=cert=/cert.pem \
--from-file=key=/key.pem

[Koji-Hub] Web Client Secret

Koji-Web client secret for koji-hub API authentication.

NOTES:

• The client.pem file must contain BOTH private key and certificate;
• The certificate CN field must use the same value used in web_client_username

Secret format:

apiVersion: v1
kind: Secret
metadata:
  name: koji-hub-web-client-cert
  labels:
    app: koji-hub
data:
  client.pem: -|
    key_and_cert_in_base64_format

You may also create it from the command line:

oc create secret generic koji-hub-web-client-cert \
--from-file=client.pem=/key_and_cert.pem

[Koji-Hub] Admin Client Secret

Koji-Hub admin client secret for koji-hub API authentication.

This certificate is used in several admin level operations by the operator.

NOTES:

• The client.pem file must contain BOTH private key and certificate;
• The certificate CN field must use the same value used in admin_username

Secret format:

apiVersion: v1
kind: Secret
metadata:
  name: koji-hub-admin-cert
  labels:
    app: koji-hub
data:
  client.pem: -|
    key_and_cert_in_base64_format

You may also create it from the command line:

oc create secret generic koji-hub-admin-cert \
--from-file=client.pem=/key_and_cert.pem

[Koji-Builder] Client Secret

Koji-Builder client secret for koji-hub API authentication.

NOTES:

• The client.pem file must contain BOTH private key and certificate;
• The certificate CN field must use the same value used in host_name

Secret format:

apiVersion: v1
kind: Secret
metadata:
  name: koji-builder-client-cert
  labels:
    app: koji-builder
data:
  client.pem: -|
    key_and_cert_in_base64_format

You may also create it from the command line:

oc create secret generic koji-builder-client-cert \
--from-file=client.pem=/key_and_cert.pem

[Kojira] Client Secret

Kojira client secret for koji-hub API authentication.

NOTES:

• The client.pem file must contain BOTH private key and certificate;
• The certificate CN field must use the same value used in hub_username

Secret format:

apiVersion: v1
kind: Secret
metadata:
  name: kojira-client-cert
  labels:
    app: kojira
data:
  client.pem: -|
    key_and_cert_in_base64_format

You may also create it from the command line:

oc create secret generic kojira-client-cert \
--from-file=client.pem=/key_and_cert.pem

[MBS-Backend] Client Secret

MBS Backend client secret for koji-hub API authentication.

NOTES:

• The client.pem file must contain BOTH private key and certificate;
• The certificate CN field must use the same value used in hub_username

Secret format:

apiVersion: v1
kind: Secret
metadata:
  name: mbs-client-cert
  labels:
    app: mbs-backend
data:
  client.pem: -|
    key_and_cert_in_base64_format

You may also create it from the command line:

oc create secret generic mbs-client-cert \
--from-file=client.pem=/key_and_cert.pem

[MBS-Frontend] Client Secret

MBS Frontend client secret for koji-hub API authentication.

NOTES:

• The client.pem file must contain BOTH private key and certificate;
• The certificate CN field must use the same value used in hub_username

Secret format:

apiVersion: v1
kind: Secret
metadata:
  name: mbs-frontend-client-cert
  labels:
    app: mbs-frontend
data:
  client.pem: -|
    key_and_cert_in_base64_format

You may also create it from the command line:

oc create secret generic mbs-frontend-client-cert \
--from-file=client.pem=/key_and_cert.pem

[MBS-Frontend] HTTPD Secret

Apache HTTPD certificate and private key.

NOTES:

• The certificate CN field must use the same value as the public hostname for mbs-frontend;
• The following subject altnames must be used:
  • DNS:$MBS_FRONTEND_PUBLIC_HOSTNAME
  • DNS:mbs-frontend
  • DNS:mbs-frontend.$OPENSHIFT_NAMESPACE.svc
  • DNS:mbs-frontend.$OPENSHIFT_NAMESPACE.svc.cluster

Secret format:

apiVersion: v1
kind: Secret
metadata:
  name: mbs-frontend-service-cert
  labels:
    app: koji-hub
data:
  tls.crt: -|
    cert_in_base64_format
  tls.key: -|
    key_in_base64_format

You may also create it from the command line:

oc create secret generic mbs-frontend-service-cert \
--from-file=cert=/cert.pem \
--from-file=key=/key.pem