| |
@@ -1,63 +1,480 @@
|
| |
# MBBox deployment guide
|
| |
|
| |
- ## Description
|
| |
+ ## Introduction
|
| |
|
| |
This guide will describe the deployment of [MBBox](https://github.com/fedora-infra/mbbox) operator in OpenShift 4 cluster and it's prerequisites.
|
| |
|
| |
- ## Prerequisites
|
| |
+ MBBox upstream documentation can be found at: https://mbbox-operator.readthedocs.io/en/latest/.
|
| |
|
| |
- ### Persistent Volumes
|
| |
+ The MBBox operator has kubernetes custom resources to deploy the following components:
|
| |
+
|
| |
+ * koji-hub (includes koji-web)
|
| |
+ * koji-builder
|
| |
+ * kojira
|
| |
+ * MBS Backend
|
| |
+ * MBS Frontend
|
| |
+
|
| |
+ ## Dependencies
|
| |
+
|
| |
+ MBBox depends on two components/services to be already deployed/available:
|
| |
+
|
| |
+ * PostgreSQL (>= 10.4)
|
| |
+ * Fedora Messaging
|
| |
+
|
| |
+ The operator does not deploy the above components but needs two secrets, one for each component, in the same namespace custom resources (koji, koji-builder, etc) are being created.
|
| |
+
|
| |
+ ### PostgreSQL
|
| |
+
|
| |
+ Secret format (all `data` values need to be base64 encoded):
|
| |
+
|
| |
+ ```yaml
|
| |
+ apiVersion: v1
|
| |
+ kind: Secret
|
| |
+ metadata:
|
| |
+ name: postgres
|
| |
+ labels:
|
| |
+ app: postgres
|
| |
+ data:
|
| |
+ POSTGRES_HOST: cG9zdGdyZXNxbA== # postgresql
|
| |
+ POSTGRES_DB: bWJveGRi # mboxdb
|
| |
+ POSTGRES_USER: a29qaQ== # koji
|
| |
+ POSTGRES_PASSWORD: bWJveA== # mbox
|
| |
+ ```
|
| |
+
|
| |
+ Creating the secret from the command line:
|
| |
+
|
| |
+ ```
|
| |
+ oc create secret generic postgres \
|
| |
+ --from-literal=POSTGRES_HOST=postgresql \
|
| |
+ --from-literal=POSTGRES_DB=mboxdb \
|
| |
+ --from-literal=POSTGRES_USER=koji \
|
| |
+ --from-literal=POSTGRES_PASSWORD=mbox
|
| |
+ ```
|
| |
+
|
| |
+ ### Fedora Messaging
|
| |
+
|
| |
+ Secret format (all `data` values need to be base64 encoded):
|
| |
+
|
| |
+ ```yaml
|
| |
+ apiVersion: v1
|
| |
+ kind: Secret
|
| |
+ metadata:
|
| |
+ name: koji-hub-msg
|
| |
+ data:
|
| |
+ koji.ca: |-
|
| |
+ Y2hhbmdlbWU=
|
| |
+ koji.crt: |-
|
| |
+ Y2hhbmdlbWU=
|
| |
+ koji.key: |-
|
| |
+ Y2hhbmdlbWU=
|
| |
+ ```
|
| |
+
|
| |
+ The secret can also be created from the command line:
|
| |
+
|
| |
+ ```
|
| |
+ oc create secret generic koji-hub-msg \
|
| |
+ --from-file=koji.ca=/path/to/ca.pem \
|
| |
+ --from-file=koji.crt=/path/to/cert.pem \
|
| |
+ --from-file=koji.key=/path/to/key.pem
|
| |
+ ```
|
| |
+
|
| |
+ All certificates must be in PEM format.
|
| |
+
|
| |
+ ### Storage
|
| |
+
|
| |
+ **NOTE:** this section describes how to deal with Openshift Persistent Volumes if the cluster being used is not using any storage provisioning operator which may require extra PV setup from the cluster admin.
|
| |
|
| |
The MBBox needs several Persistent Volumes created in prior to deployment:
|
| |
|
| |
* mbox-registry (Recommended: 100 Gi) - Used as image registry
|
| |
- * httpd (Recommended: 1Gi) - Used by koji-hub httpd server
|
| |
- * koji (Recommended: 50Gi) - Used for shared koji space
|
| |
- * postgres (Recommended: 5Gi) - Used by postgreSQL database
|
| |
-
|
| |
- ### Postgres
|
| |
-
|
| |
- MBBox needs PostgreSQL database for various tasks.
|
| |
-
|
| |
- * Version: 10.4
|
| |
- * Volumes: postgredb:/var/lib/postgresql
|
| |
-
|
| |
- ### Secrets
|
| |
-
|
| |
- MBBox needs plenty of secrets to be created for the components. Following is the list of secrets.
|
| |
-
|
| |
- * ca-cert
|
| |
- Description: Certificate for Certification Authority
|
| |
- Key/value: cert/CA certificate
|
| |
- * koji-builder-client-cert
|
| |
- Description: Koji builder client certificate (must have the CN: koji builder host name)
|
| |
- Key/value: tls.pem/Combined certificate with key signed by CA
|
| |
- * koji-hub-service-cert
|
| |
- Description: Server certificate for koji-hub
|
| |
- Key/value: tls.crt/Koji server certificate signed by CA
|
| |
- Key/value: tls.key/Private key for the tls.crt
|
| |
- * koji-hub-admin-cert
|
| |
- Description: Certificate for koji admin user (must have the CN: koji admin username)
|
| |
- Key/value: client.pem/Combined certificate with key signed by CA
|
| |
- * koji-hub-web-client-cert
|
| |
- Description: Certificate for koji web client user (must have the CN: koji web client username)
|
| |
- Key/value: client.pem/Combined certificate with key signed by CA
|
| |
- * koji-hub-msg
|
| |
- Description: Koji hub messaging certificate for fedora messaging
|
| |
- Key/value: koji.ca/CA certificate for messaging
|
| |
- Key/value: koji.crt/Messaging certificate
|
| |
- Key/value: koji.key/Private key for messaging certificate
|
| |
- * kojira-client-cert
|
| |
- Description: Kojira client certificate to communicate with koji-hub (must have the CN: koji hub username for kojira)
|
| |
- Key/value: client.pem/Combined certificate with key signed by CA
|
| |
- * mbs-client-cert
|
| |
- Description: MBS backend client certificate to communicate with koji-hub (must have the CN: koji hub username for MBS)
|
| |
- Key/value: client.pem/Combined certificate with key signed by CA
|
| |
- * mbs-frontend-client-cert
|
| |
- Description: MBS frontend client certificate to communicate with koji-hub (must have the CN: koji hub hostname)
|
| |
- Key/value: client.pem/Combined certificate with key signed by CA
|
| |
- * mbs-frontend-client-cert
|
| |
- Description: MBS frontend server certificate
|
| |
- Key/value: tls.crt/MBS server certificate signed by CA
|
| |
- Key/value: tls.key/Private key for server certificate
|
| |
-
|
| |
+ * httpd (Minimum: 1Gi) - Used by koji-hub httpd server
|
| |
+ * koji (Minimum: 50Gi) - Used for shared koji space
|
| |
+ * postgres (Minimum: 5Gi) - Used by postgreSQL database
|
| |
+
|
| |
+ #### Persistent Volumes
|
| |
+
|
| |
+ ##### ClaimRef
|
| |
+
|
| |
+ ClaimRef should be set in created Persistent Volumes (PVs) if no storage class is set in the cluster scope:
|
| |
+
|
| |
+ ```yaml
|
| |
+ kind: PersistentVolume
|
| |
+ apiVersion: v1
|
| |
+ metadata:
|
| |
+ annotations:
|
| |
+ pv.kubernetes.io/bound-by-controller: 'yes'
|
| |
+ selfLink: /api/v1/persistentvolumes/stg-postgres
|
| |
+ resourceVersion: '39237794'
|
| |
+ name: stg-postgres
|
| |
+ uid: d0e3960e-af9e-45af-8f87-ac03def7b8a5
|
| |
+ creationTimestamp: '2020-11-23T01:01:35Z'
|
| |
+ ...
|
| |
+ spec:
|
| |
+ capacity:
|
| |
+ storage: 5Gi
|
| |
+ claimRef:
|
| |
+ apiVersion: v1
|
| |
+ kind: PersistentVolumeClaim
|
| |
+ namespace: mbox # pvc namespace
|
| |
+ name: postgres # pvc name to bound to
|
| |
+ persistentVolumeReclaimPolicy: Retain
|
| |
+ storageClassName: slow
|
| |
+ volumeMode: Filesystem
|
| |
+ ...
|
| |
+ ```
|
| |
+
|
| |
+ Both `uid` and `resourceVersion` will need to be removed from the `claimRef` property if the PV needs to be rebound to another PVC:
|
| |
+
|
| |
+ ```yaml
|
| |
+ claimRef:
|
| |
+ apiVersion: v1
|
| |
+ kind: PersistentVolumeClaim
|
| |
+ namespace: mbox-stg # new namesapce
|
| |
+ name: postgres-stg # new pvc name
|
| |
+ uid: 775c0823-47c1-46a2-b07c-612d59592430 # remove this
|
| |
+ resourceVersion: '39235159' #remove this
|
| |
+ ```
|
| |
+
|
| |
+ Both PVC and PV should have a `bound` status after a few seconds.
|
| |
+
|
| |
+ A note on volumes: recently created volumes may need a permission change the volume itself, the oficial openshift docs has a page dedicated to this problem and how to fix it: https://access.redhat.com/solutions/3508731
|
| |
+
|
| |
+
|
| |
+ ## Deployment
|
| |
+
|
| |
+ There are three flavors of deployment:
|
| |
+
|
| |
+ * Development
|
| |
+ * Production: Root CA Only
|
| |
+ * Production: All Certificates
|
| |
+
|
| |
+ What differs each deployment flavor is the amount of required steps before the actual deployment process.
|
| |
+
|
| |
+ You can follow the upstream documentation once those preparation steps are done: https://mbbox-operator.readthedocs.io/en/latest/deployment-guide.html
|
| |
+
|
| |
+ ### Development
|
| |
+
|
| |
+ Ideal for running MBBox locally and automated tests.
|
| |
+
|
| |
+ The only requirement is a running openshift cluster with admin access.
|
| |
+
|
| |
+ All certificates, incluing the Root CA will be self signed using openssl.
|
| |
+
|
| |
+ ### Production: Root CA Only
|
| |
+
|
| |
+ This deployment path requires an openshift secret in the same namespace other components will be created, such as koji.
|
| |
+
|
| |
+ This secret should contain a root CA certificate and private key, this way the operator will create and sign all required certificates from this root CA instead of using a self signed one.
|
| |
+
|
| |
+ Secret format:
|
| |
+
|
| |
+ ```yaml
|
| |
+ apiVersion: v1
|
| |
+ kind: Secret
|
| |
+ metadata:
|
| |
+ name: koji-hub-ca-cert
|
| |
+ labels:
|
| |
+ app: koji-hub
|
| |
+ data:
|
| |
+ cert: -|
|
| |
+ cert_in_base64_format
|
| |
+ key: -|
|
| |
+ key_in_base64_format
|
| |
+ ```
|
| |
+
|
| |
+ You may also create it from the command line:
|
| |
+
|
| |
+ ```sh
|
| |
+ oc create secret generic koji-hub-ca-cert \
|
| |
+ --from-file=cert=/ca.pem \
|
| |
+ --from-file=key=/key.pem
|
| |
+ ```
|
| |
+
|
| |
+ You may then proceed in deployment MBBox.
|
| |
+
|
| |
+ ### Production: All Certificates
|
| |
+
|
| |
+ Similar to the previous section but it needs all certificates and keys to be manually created.
|
| |
+
|
| |
+ #### [Koji-Hub] Root CA Secret
|
| |
+
|
| |
+ The root certifcate which all other certificates were generated/signed from.
|
| |
+
|
| |
+ We only need the certificate in this case.
|
| |
+
|
| |
+ Secret format:
|
| |
+
|
| |
+ ```yaml
|
| |
+ apiVersion: v1
|
| |
+ kind: Secret
|
| |
+ metadata:
|
| |
+ name: koji-hub-ca-cert
|
| |
+ labels:
|
| |
+ app: koji-hub
|
| |
+ data:
|
| |
+ cert: -|
|
| |
+ cert_in_base64_format
|
| |
+ ```
|
| |
+
|
| |
+ You may also create it from the command line:
|
| |
+
|
| |
+ ```sh
|
| |
+ oc create secret generic koji-hub-ca-cert \
|
| |
+ --from-file=cert=/ca.pem
|
| |
+ ```
|
| |
+
|
| |
+ #### [Koji-Hub] HTTPD Secret
|
| |
+
|
| |
+ Apache HTTPD certificate and private key.
|
| |
+
|
| |
+ **NOTES:**
|
| |
+
|
| |
+ * The certificate CN field must use the same value as the public hostname for koji;
|
| |
+ * The following subject altnames must be used:
|
| |
+ * DNS:$KOJI_PUBLIC_HOSTNAME
|
| |
+ * DNS:koji-hub
|
| |
+ * DNS:koji-hub.$OPENSHIFT_NAMESPACE.svc
|
| |
+ * DNS:koji-hub.$OPENSHIFT_NAMESPACE.svc.cluster
|
| |
+
|
| |
+ Secret format:
|
| |
+
|
| |
+ ```yaml
|
| |
+ apiVersion: v1
|
| |
+ kind: Secret
|
| |
+ metadata:
|
| |
+ name: koji-hub-service-cert
|
| |
+ labels:
|
| |
+ app: koji-hub
|
| |
+ data:
|
| |
+ tls.crt: -|
|
| |
+ cert_in_base64_format
|
| |
+ tls.key: -|
|
| |
+ key_in_base64_format
|
| |
+ ```
|
| |
+
|
| |
+ You may also create it from the command line:
|
| |
+
|
| |
+ ```sh
|
| |
+ oc create secret generic koji-hub-service-cert \
|
| |
+ --from-file=cert=/cert.pem \
|
| |
+ --from-file=key=/key.pem
|
| |
+ ```
|
| |
+
|
| |
+ #### [Koji-Hub] Web Client Secret
|
| |
+
|
| |
+ Koji-Web client secret for koji-hub API authentication.
|
| |
+
|
| |
+ **NOTES:**
|
| |
+
|
| |
+ * The `client.pem` file must contain BOTH private key and certificate;
|
| |
+ * The certificate `CN` field must use the same value used in `web_client_username`
|
| |
+
|
| |
+ Secret format:
|
| |
+
|
| |
+ ```yaml
|
| |
+ apiVersion: v1
|
| |
+ kind: Secret
|
| |
+ metadata:
|
| |
+ name: koji-hub-web-client-cert
|
| |
+ labels:
|
| |
+ app: koji-hub
|
| |
+ data:
|
| |
+ client.pem: -|
|
| |
+ key_and_cert_in_base64_format
|
| |
+ ```
|
| |
+
|
| |
+ You may also create it from the command line:
|
| |
+
|
| |
+ ```sh
|
| |
+ oc create secret generic koji-hub-web-client-cert \
|
| |
+ --from-file=client.pem=/key_and_cert.pem
|
| |
+ ```
|
| |
+
|
| |
+ #### [Koji-Hub] Admin Client Secret
|
| |
+
|
| |
+ Koji-Hub admin client secret for koji-hub API authentication.
|
| |
+
|
| |
+ This certificate is used in several admin level operations by the operator.
|
| |
+
|
| |
+ **NOTES:**
|
| |
+
|
| |
+ * The `client.pem` file must contain BOTH private key and certificate;
|
| |
+ * The certificate `CN` field must use the same value used in `admin_username`
|
| |
+
|
| |
+ Secret format:
|
| |
+
|
| |
+ ```yaml
|
| |
+ apiVersion: v1
|
| |
+ kind: Secret
|
| |
+ metadata:
|
| |
+ name: koji-hub-admin-cert
|
| |
+ labels:
|
| |
+ app: koji-hub
|
| |
+ data:
|
| |
+ client.pem: -|
|
| |
+ key_and_cert_in_base64_format
|
| |
+ ```
|
| |
+
|
| |
+ You may also create it from the command line:
|
| |
+
|
| |
+ ```sh
|
| |
+ oc create secret generic koji-hub-admin-cert \
|
| |
+ --from-file=client.pem=/key_and_cert.pem
|
| |
+ ```
|
| |
+
|
| |
+ #### [Koji-Builder] Client Secret
|
| |
+
|
| |
+ Koji-Builder client secret for koji-hub API authentication.
|
| |
+
|
| |
+ **NOTES:**
|
| |
+
|
| |
+ * The `client.pem` file must contain BOTH private key and certificate;
|
| |
+ * The certificate `CN` field must use the same value used in `host_name`
|
| |
+
|
| |
+ Secret format:
|
| |
+
|
| |
+ ```yaml
|
| |
+ apiVersion: v1
|
| |
+ kind: Secret
|
| |
+ metadata:
|
| |
+ name: koji-builder-client-cert
|
| |
+ labels:
|
| |
+ app: koji-builder
|
| |
+ data:
|
| |
+ client.pem: -|
|
| |
+ key_and_cert_in_base64_format
|
| |
+ ```
|
| |
+
|
| |
+ You may also create it from the command line:
|
| |
+
|
| |
+ ```sh
|
| |
+ oc create secret generic koji-builder-client-cert \
|
| |
+ --from-file=client.pem=/key_and_cert.pem
|
| |
+ ```
|
| |
+
|
| |
+ #### [Kojira] Client Secret
|
| |
+
|
| |
+ Kojira client secret for koji-hub API authentication.
|
| |
+
|
| |
+ **NOTES:**
|
| |
+
|
| |
+ * The `client.pem` file must contain BOTH private key and certificate;
|
| |
+ * The certificate `CN` field must use the same value used in `hub_username`
|
| |
+
|
| |
+ Secret format:
|
| |
+
|
| |
+ ```yaml
|
| |
+ apiVersion: v1
|
| |
+ kind: Secret
|
| |
+ metadata:
|
| |
+ name: kojira-client-cert
|
| |
+ labels:
|
| |
+ app: kojira
|
| |
+ data:
|
| |
+ client.pem: -|
|
| |
+ key_and_cert_in_base64_format
|
| |
+ ```
|
| |
+
|
| |
+ You may also create it from the command line:
|
| |
+
|
| |
+ ```sh
|
| |
+ oc create secret generic kojira-client-cert \
|
| |
+ --from-file=client.pem=/key_and_cert.pem
|
| |
+ ```
|
| |
+
|
| |
+ #### [MBS-Backend] Client Secret
|
| |
+
|
| |
+ MBS Backend client secret for koji-hub API authentication.
|
| |
+
|
| |
+ **NOTES:**
|
| |
+
|
| |
+ * The `client.pem` file must contain BOTH private key and certificate;
|
| |
+ * The certificate `CN` field must use the same value used in `hub_username`
|
| |
+
|
| |
+ Secret format:
|
| |
+
|
| |
+ ```yaml
|
| |
+ apiVersion: v1
|
| |
+ kind: Secret
|
| |
+ metadata:
|
| |
+ name: mbs-client-cert
|
| |
+ labels:
|
| |
+ app: mbs-backend
|
| |
+ data:
|
| |
+ client.pem: -|
|
| |
+ key_and_cert_in_base64_format
|
| |
+ ```
|
| |
+
|
| |
+ You may also create it from the command line:
|
| |
+
|
| |
+ ```sh
|
| |
+ oc create secret generic mbs-client-cert \
|
| |
+ --from-file=client.pem=/key_and_cert.pem
|
| |
+ ```
|
| |
+
|
| |
+ #### [MBS-Frontend] Client Secret
|
| |
+
|
| |
+ MBS Frontend client secret for koji-hub API authentication.
|
| |
+
|
| |
+ **NOTES:**
|
| |
+
|
| |
+ * The `client.pem` file must contain BOTH private key and certificate;
|
| |
+ * The certificate `CN` field must use the same value used in `hub_username`
|
| |
+
|
| |
+ Secret format:
|
| |
+
|
| |
+ ```yaml
|
| |
+ apiVersion: v1
|
| |
+ kind: Secret
|
| |
+ metadata:
|
| |
+ name: mbs-frontend-client-cert
|
| |
+ labels:
|
| |
+ app: mbs-frontend
|
| |
+ data:
|
| |
+ client.pem: -|
|
| |
+ key_and_cert_in_base64_format
|
| |
+ ```
|
| |
+
|
| |
+ You may also create it from the command line:
|
| |
+
|
| |
+ ```sh
|
| |
+ oc create secret generic mbs-frontend-client-cert \
|
| |
+ --from-file=client.pem=/key_and_cert.pem
|
| |
+ ```
|
| |
+
|
| |
+ #### [MBS-Frontend] HTTPD Secret
|
| |
+
|
| |
+ Apache HTTPD certificate and private key.
|
| |
+
|
| |
+ **NOTES:**
|
| |
+
|
| |
+ * The certificate CN field must use the same value as the public hostname for mbs-frontend;
|
| |
+ * The following subject altnames must be used:
|
| |
+ * DNS:$MBS_FRONTEND_PUBLIC_HOSTNAME
|
| |
+ * DNS:mbs-frontend
|
| |
+ * DNS:mbs-frontend.$OPENSHIFT_NAMESPACE.svc
|
| |
+ * DNS:mbs-frontend.$OPENSHIFT_NAMESPACE.svc.cluster
|
| |
+
|
| |
+ Secret format:
|
| |
+
|
| |
+ ```yaml
|
| |
+ apiVersion: v1
|
| |
+ kind: Secret
|
| |
+ metadata:
|
| |
+ name: mbs-frontend-service-cert
|
| |
+ labels:
|
| |
+ app: koji-hub
|
| |
+ data:
|
| |
+ tls.crt: -|
|
| |
+ cert_in_base64_format
|
| |
+ tls.key: -|
|
| |
+ key_in_base64_format
|
| |
+ ```
|
| |
+
|
| |
+ You may also create it from the command line:
|
| |
+
|
| |
+ ```sh
|
| |
+ oc create secret generic mbs-frontend-service-cert \
|
| |
+ --from-file=cert=/cert.pem \
|
| |
+ --from-file=key=/key.pem
|
| |
+ ```
|
| |
lrossett
commented 3 years ago
adding MBBox deployment docs.
spoiler: may contain typos :-)