diff --git a/docs/administrator.md b/docs/administrator.md index 78e4f3a34a9b672b13e836be3258e6355e73542f..ddd5287f6c2374e864a3621b724663b220d76a30 100644 --- a/docs/administrator.md +++ b/docs/administrator.md @@ -68,11 +68,11 @@ the upgrade script for you after the manifest is updated and pods are rotated. [CustomResourceDefinitions](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#customresourcedefinitions) will be registered with schema validation by default when the operator is deployed. The `OperatorConfiguration` CRD will only get created if the -`POSTGRES_OPERATOR_CONFIGURATION_OBJECT` [environment variable](../manifests/postgres-operator.yaml#L36) +`POSTGRES_OPERATOR_CONFIGURATION_OBJECT` [environment variable](https://github.com/zalando/postgres-operator/blob/master/manifests/postgres-operator.yaml#L36) in the deployment yaml is set and not empty. -When submitting manifests of [`postgresql`](../manifests/postgresql.crd.yaml) or -[`OperatorConfiguration`](../manifests/operatorconfiguration.crd.yaml) custom +When submitting manifests of [`postgresql`](https://github.com/zalando/postgres-operator/blob/master/manifests/postgresql.crd.yaml) or +[`OperatorConfiguration`](https://github.com/zalando/postgres-operator/blob/master/manifests/operatorconfiguration.crd.yaml) custom resources with kubectl, validation can be bypassed with `--validate=false`. The operator can also be configured to not register CRDs with validation on `ADD` or `UPDATE` events. Running instances are not affected when enabling the validation @@ -109,7 +109,7 @@ kubectl config set-context $(kubectl config current-context) --namespace=test All subsequent `kubectl` commands will work with the `test` namespace. The operator will run in this namespace and look up needed resources - such as its ConfigMap - there. Please note that the namespace for service accounts and -cluster role bindings in [operator RBAC rules](../manifests/operator-service-account-rbac.yaml) +cluster role bindings in [operator RBAC rules](https://github.com/zalando/postgres-operator/blob/master/manifests/operator-service-account-rbac.yaml) needs to be adjusted to the non-default value. ### Specify the namespace to watch @@ -120,9 +120,9 @@ clusters in the namespace such as "increase the number of Postgres replicas to By default, the operator watches the namespace it is deployed to. You can change this by setting the `WATCHED_NAMESPACE` var in the `env` section of the -[operator deployment](../manifests/postgres-operator.yaml) manifest or by +[operator deployment](https://github.com/zalando/postgres-operator/blob/master/manifests/postgres-operator.yaml) manifest or by altering the `watched_namespace` field in the operator -[configuration](../manifests/postgresql-operator-default-configuration.yaml#L49). +[configuration](https://github.com/zalando/postgres-operator/blob/master/manifests/postgresql-operator-default-configuration.yaml#L49). In the case both are set, the env var takes the precedence. To make the operator listen to all namespaces, explicitly set the field/env var to "`*`". @@ -143,7 +143,7 @@ But, it is also possible to define ownership between operator instances and Postgres clusters running all in the same namespace or K8s cluster without interfering. -First, define the [`CONTROLLER_ID`](../../manifests/postgres-operator.yaml#L38) +First, define the [`CONTROLLER_ID`](https://github.com/zalando/postgres-operator/blob/master/manifests/postgres-operator.yaml#L38) environment variable in the operator deployment manifest. Then specify the ID in every Postgres cluster manifest you want this operator to watch using the `"acid.zalan.do/controller"` annotation: @@ -256,7 +256,7 @@ update of the pods because the UID is used as part of backup path to S3. ## Role-based access control for the operator -The manifest [`operator-service-account-rbac.yaml`](../manifests/operator-service-account-rbac.yaml) +The manifest [`operator-service-account-rbac.yaml`](https://github.com/zalando/postgres-operator/blob/master/manifests/operator-service-account-rbac.yaml) defines the service account, cluster roles and bindings needed for the operator to function under access control restrictions. The file also includes a cluster role `postgres-pod` with privileges for Patroni to watch and manage pods and @@ -965,7 +965,7 @@ but only snapshots of your data. In its current state, see logical backups as a way to quickly create SQL dumps that you can easily restore in an empty test cluster. -2. The [example image](../docker/logical-backup/Dockerfile) implements the backup +2. The [example image](https://github.com/zalando/postgres-operator/blob/master/docker/logical-backup/Dockerfile) implements the backup via `pg_dumpall` and upload of compressed and encrypted results to an S3 bucket. `pg_dumpall` requires a `superuser` access to a DB and runs on the replica when possible. @@ -984,7 +984,7 @@ of the backup cron job. 6. For that feature to work, your RBAC policy must enable operations on the `cronjobs` resource from the `batch` API group for the operator service account. -See [example RBAC](../manifests/operator-service-account-rbac.yaml) +See [example RBAC](https://github.com/zalando/postgres-operator/blob/master/manifests/operator-service-account-rbac.yaml) ## Sidecars for Postgres clusters @@ -1038,8 +1038,8 @@ default. Alternatively, a list can also be passed when starting the Python application with the `--cluster` option. The Operator API endpoint can be configured via the `OPERATOR_API_URL` -environment variables in the [deployment manifest](../ui/manifests/deployment.yaml#L40). -You can also expose the operator API through a [service](../manifests/api-service.yaml). +environment variables in the [deployment manifest](https://github.com/zalando/postgres-operator/blob/master/ui/manifests/deployment.yaml#L40). +You can also expose the operator API through a [service](https://github.com/zalando/postgres-operator/blob/master/manifests/api-service.yaml). Some displayed options can be disabled from UI using simple flags under the `OPERATOR_UI_CONFIG` field in the deployment. diff --git a/docs/developer.md b/docs/developer.md index 03224e971111a4ac105dddcca1c43c5ef29dbc13..09fa5f527a5d073ea72890620166c15445c23d20 100644 --- a/docs/developer.md +++ b/docs/developer.md @@ -267,7 +267,7 @@ the standard Docker `bridge` network. The kind cluster is deleted if tests finish successfully or on each new run in case it still exists. End-to-end tests are executed automatically during builds (for more details, -see the [README](../e2e/README.md) in the `e2e` folder): +see the [README](https://github.com/zalando/postgres-operator/blob/master/e2e/README.md) in the `e2e` folder): ```bash make e2e @@ -291,35 +291,35 @@ parameters (with exceptions for certain Patroni/Postgres options) and variables if you feel a per-cluster configuration is necessary. Note: If one option is defined in the operator configuration and in the cluster -[manifest](../manifests/complete-postgres-manifest.yaml), the latter takes +[manifest](https://github.com/zalando/postgres-operator/blob/master/manifests/complete-postgres-manifest.yaml), the latter takes precedence. ### Go code Update the following Go files that obtain the configuration parameter from the manifest files: -* [operator_configuration_type.go](../pkg/apis/acid.zalan.do/v1/operator_configuration_type.go) -* [operator_config.go](../pkg/controller/operator_config.go) -* [config.go](../pkg/util/config/config.go) +* [operator_configuration_type.go](https://github.com/zalando/postgres-operator/blob/master/pkg/apis/acid.zalan.do/v1/operator_configuration_type.go) +* [operator_config.go](https://github.com/zalando/postgres-operator/blob/master/pkg/controller/operator_config.go) +* [config.go](https://github.com/zalando/postgres-operator/blob/master/pkg/util/config/config.go) -Postgres manifest parameters are defined in the [api package](../pkg/apis/acid.zalan.do/v1/postgresql_type.go). -The operator behavior has to be implemented at least in [k8sres.go](../pkg/cluster/k8sres.go). -Validation of CRD parameters is controlled in [crds.go](../pkg/apis/acid.zalan.do/v1/crds.go). +Postgres manifest parameters are defined in the [api package](https://github.com/zalando/postgres-operator/blob/master/pkg/apis/acid.zalan.do/v1/postgresql_type.go). +The operator behavior has to be implemented at least in [k8sres.go](https://github.com/zalando/postgres-operator/blob/master/pkg/cluster/k8sres.go). +Validation of CRD parameters is controlled in [crds.go](https://github.com/zalando/postgres-operator/blob/master/pkg/apis/acid.zalan.do/v1/crds.go). Please, reflect your changes in tests, for example in: -* [config_test.go](../pkg/util/config/config_test.go) -* [k8sres_test.go](../pkg/cluster/k8sres_test.go) -* [util_test.go](../pkg/apis/acid.zalan.do/v1/util_test.go) +* [config_test.go](https://github.com/zalando/postgres-operator/blob/master/pkg/util/config/config_test.go) +* [k8sres_test.go](https://github.com/zalando/postgres-operator/blob/master/pkg/cluster/k8sres_test.go) +* [util_test.go](https://github.com/zalando/postgres-operator/blob/master/pkg/apis/acid.zalan.do/v1/util_test.go) ### Updating manifest files For the CRD-based configuration, please update the following files: -* the default [OperatorConfiguration](../manifests/postgresql-operator-default-configuration.yaml) -* the CRD's [validation](../manifests/operatorconfiguration.crd.yaml) -* the CRD's validation in the [Helm chart](../charts/postgres-operator/crds/operatorconfigurations.yaml) +* the default [OperatorConfiguration](https://github.com/zalando/postgres-operator/blob/master/manifests/postgresql-operator-default-configuration.yaml) +* the CRD's [validation](https://github.com/zalando/postgres-operator/blob/master/manifests/operatorconfiguration.crd.yaml) +* the CRD's validation in the [Helm chart](https://github.com/zalando/postgres-operator/blob/master/charts/postgres-operator/crds/operatorconfigurations.yaml) -Add new options also to the Helm chart's [values file](../charts/postgres-operator/values.yaml) file. +Add new options also to the Helm chart's [values file](https://github.com/zalando/postgres-operator/blob/master/charts/postgres-operator/values.yaml) file. It follows the OperatorConfiguration CRD layout. Nested values will be flattened for the ConfigMap. -Last but no least, update the [ConfigMap](../manifests/configmap.yaml) manifest example as well. +Last but no least, update the [ConfigMap](https://github.com/zalando/postgres-operator/blob/master/manifests/configmap.yaml) manifest example as well. ### Updating documentation diff --git a/docs/index.md b/docs/index.md index d0b4e4940f4ba941212d132a213e760e470ef3d6..10c90e0b7f6b2ce8a9c99768ec4c8ef8a5df1592 100644 --- a/docs/index.md +++ b/docs/index.md @@ -8,10 +8,10 @@ manages PostgreSQL clusters on Kubernetes (K8s): user submits a new manifest, the operator fetches that manifest and spawns a new Postgres cluster along with all necessary entities such as K8s StatefulSets and Postgres roles. See this - [Postgres cluster manifest](../manifests/complete-postgres-manifest.yaml) + [Postgres cluster manifest](https://github.com/zalando/postgres-operator/blob/master/manifests/complete-postgres-manifest.yaml) for settings that a manifest may contain. -2. The operator also watches updates to [its own configuration](../manifests/configmap.yaml) +2. The operator also watches updates to [its own configuration](https://github.com/zalando/postgres-operator/blob/master/manifests/configmap.yaml) and alters running Postgres clusters if necessary. For instance, if the Docker image in a pod is changed, the operator carries out the rolling update, which means it re-spawns pods of each managed StatefulSet one-by-one diff --git a/docs/operator-ui.md b/docs/operator-ui.md index 7912e82649ff1f63ab0f844ed12367f8882d020e..69da2d23160a7700ee6126ec02644404b9a313f2 100644 --- a/docs/operator-ui.md +++ b/docs/operator-ui.md @@ -33,8 +33,8 @@ status page. Usually, the startup should only take up to 1 minute. If you feel the process got stuck click on the "Logs" button to inspect the operator logs. If the logs look fine, but the UI seems to got stuck, check if you are have configured the -same [cluster name label](../ui/manifests/deployment.yaml#L45) like for the -[operator](../manifests/configmap.yaml#L13). +same [cluster name label](https://github.com/zalando/postgres-operator/blob/master/ui/manifests/deployment.yaml#L45) like for the +[operator](https://github.com/zalando/postgres-operator/blob/master/manifests/configmap.yaml#L13). From the "Status" field in the top menu you can also retrieve the logs and queue of each worker the operator is using. The number of concurrent workers can be diff --git a/docs/quickstart.md b/docs/quickstart.md index bdcc6dfad1b5838d73357ba3f3e066062f82e755..ed01367b73792c4006d9432a01bc921a00a7353e 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -56,7 +56,7 @@ kubectl create -f manifests/api-service.yaml # operator API to be used by UI ``` There is a [Kustomization](https://github.com/kubernetes-sigs/kustomize) -manifest that [combines the mentioned resources](../manifests/kustomization.yaml) +manifest that [combines the mentioned resources](https://github.com/zalando/postgres-operator/blob/master/manifests/kustomization.yaml) (except for the CRD) - it can be used with kubectl 1.14 or newer as easy as: ```bash @@ -64,7 +64,7 @@ kubectl apply -k github.com/zalando/postgres-operator/manifests ``` For convenience, we have automated starting the operator with minikube using the -`run_operator_locally` script. It applies the [`acid-minimal-cluster`](../manifests/minimal-postgres-manifest.yaml). +`run_operator_locally` script. It applies the [`acid-minimal-cluster`](https://github.com/zalando/postgres-operator/blob/master/manifests/minimal-postgres-manifest.yaml). manifest. ```bash @@ -114,8 +114,8 @@ In the following paragraphs we describe how to access and manage PostgreSQL clusters from the command line with kubectl. But it can also be done from the browser-based [Postgres Operator UI](operator-ui.md). Before deploying the UI make sure the operator is running and its REST API is reachable through a -[K8s service](../manifests/api-service.yaml). The URL to this API must be -configured in the [deployment manifest](../ui/manifests/deployment.yaml#L43) +[K8s service](https://github.com/zalando/postgres-operator/blob/master/manifests/api-service.yaml). The URL to this API must be +configured in the [deployment manifest](https://github.com/zalando/postgres-operator/blob/master/ui/manifests/deployment.yaml#L43) of the UI. To deploy the UI simply apply all its manifests files or use the UI helm chart: diff --git a/docs/reference/cluster_manifest.md b/docs/reference/cluster_manifest.md index 616949abf4492efdc016ed7e498e291b2bf32474..3da660939ab0434b6cfe168b59f88c993c41862e 100644 --- a/docs/reference/cluster_manifest.md +++ b/docs/reference/cluster_manifest.md @@ -4,9 +4,9 @@ Individual Postgres clusters are described by the Kubernetes *cluster manifest* that has the structure defined by the `postgresql` CRD (custom resource definition). The following section describes the structure of the manifest and the purpose of individual keys. You can take a look at the examples of the -[minimal](../../manifests/minimal-postgres-manifest.yaml) +[minimal](https://github.com/zalando/postgres-operator/blob/master/manifests/minimal-postgres-manifest.yaml) and the -[complete](../../manifests/complete-postgres-manifest.yaml) +[complete](https://github.com/zalando/postgres-operator/blob/master/manifests/complete-postgres-manifest.yaml) cluster manifests. When Kubernetes resources, such as memory, CPU or volumes, are configured, diff --git a/docs/reference/operator_parameters.md b/docs/reference/operator_parameters.md index 6fb2cd1236b651ca3de514e70c27e2443c5b2937..3305b0764284978d20d63ac466202927e594d386 100644 --- a/docs/reference/operator_parameters.md +++ b/docs/reference/operator_parameters.md @@ -10,12 +10,12 @@ configuration. maps. String values containing ':' should be enclosed in quotes. The configuration is flat, parameter group names below are not reflected in the configuration structure. There is an - [example](../../manifests/configmap.yaml) + [example](https://github.com/zalando/postgres-operator/blob/master/manifests/configmap.yaml) * CRD-based configuration. The configuration is stored in a custom YAML manifest. The manifest is an instance of the custom resource definition (CRD) called `OperatorConfiguration`. The operator registers this CRD during the - start and uses it for configuration if the [operator deployment manifest](../../manifests/postgres-operator.yaml#L36) + start and uses it for configuration if the [operator deployment manifest](https://github.com/zalando/postgres-operator/blob/master/manifests/postgres-operator.yaml#L36) sets the `POSTGRES_OPERATOR_CONFIGURATION_OBJECT` env variable to a non-empty value. The variable should point to the `postgresql-operator-configuration` object in the operator's namespace. @@ -24,7 +24,7 @@ configuration. simply represented in the usual YAML way. There are no default values built-in in the operator, each parameter that is not supplied in the configuration receives an empty value. In order to create your own configuration just copy - the [default one](../../manifests/postgresql-operator-default-configuration.yaml) + the [default one](https://github.com/zalando/postgres-operator/blob/master/manifests/postgresql-operator-default-configuration.yaml) and change it. To test the CRD-based configuration locally, use the following @@ -57,11 +57,11 @@ parameters, those parameters have no effect and are replaced by the `CRD_READY_WAIT_INTERVAL` and `CRD_READY_WAIT_TIMEOUT` environment variables. They will be deprecated and removed in the future. -For the configmap configuration, the [default parameter values](../../pkg/util/config/config.go#L14) +For the configmap configuration, the [default parameter values](https://github.com/zalando/postgres-operator/blob/master/pkg/util/config/config.go#L14) mentioned here are likely to be overwritten in your local operator installation via your local version of the operator configmap. In the case you use the operator CRD, all the CRD defaults are provided in the -[operator's default configuration manifest](../../manifests/postgresql-operator-default-configuration.yaml) +[operator's default configuration manifest](https://github.com/zalando/postgres-operator/blob/master/manifests/postgresql-operator-default-configuration.yaml) Variable names are underscore-separated words. @@ -154,7 +154,7 @@ Those are top-level keys, containing both leaf keys and groups. cluster nodes. This affects all containers created by the operator (Postgres, Scalyr sidecar, and other sidecars except **sidecars** defined in the operator configuration); to set resources for the operator's own container, change the - [operator deployment manually](../../manifests/postgres-operator.yaml#L20). + [operator deployment manually](https://github.com/zalando/postgres-operator/blob/master/manifests/postgres-operator.yaml#L20). The default is `false`. ## Postgres users @@ -217,7 +217,7 @@ configuration they are grouped under the `kubernetes` key. sufficient for the pods to start and for Patroni to access K8s endpoints; service account on its own lacks any such rights starting with K8s v1.8. If not explicitly defined by the user, a simple definition that binds the - account to the 'postgres-pod' [cluster role](../../manifests/operator-service-account-rbac.yaml#L198) + account to the 'postgres-pod' [cluster role](https://github.com/zalando/postgres-operator/blob/master/manifests/operator-service-account-rbac.yaml#L198) will be used. The default is empty. * **pod_terminate_grace_period** @@ -602,7 +602,7 @@ Postgres logical backups. In the CRD-based configuration those parameters are grouped under the `logical_backup` key. * **logical_backup_docker_image** - An image for pods of the logical backup job. The [example image](../../docker/logical-backup/Dockerfile) + An image for pods of the logical backup job. The [example image](https://github.com/zalando/postgres-operator/blob/master/docker/logical-backup/Dockerfile) runs `pg_dumpall` on a replica if possible and uploads compressed results to an S3 bucket under the key `/spilo/pg_cluster_name/cluster_k8s_uuid/logical_backups`. The default image is the same image built with the Zalando-internal CI diff --git a/docs/user.md b/docs/user.md index 64fdd34e43bbf1981d710796a8516a0ad954c66c..d8783c42f290c989f1a3c1bc6853a330ed7831e5 100644 --- a/docs/user.md +++ b/docs/user.md @@ -5,7 +5,7 @@ Learn how to work with the Postgres Operator in a Kubernetes (K8s) environment. ## Create a manifest for a new PostgreSQL cluster Make sure you have [set up](quickstart.md) the operator. Then you can create a -new Postgres cluster by applying manifest like this [minimal example](../manifests/minimal-postgres-manifest.yaml): +new Postgres cluster by applying manifest like this [minimal example](https://github.com/zalando/postgres-operator/blob/master/manifests/minimal-postgres-manifest.yaml): ```yaml apiVersion: "acid.zalan.do/v1" @@ -113,7 +113,7 @@ chapter. ### Manifest roles Manifest roles are defined directly in the cluster manifest. See -[minimal postgres manifest](../manifests/minimal-postgres-manifest.yaml) +[minimal postgres manifest](https://github.com/zalando/postgres-operator/blob/master/manifests/minimal-postgres-manifest.yaml) for an example of `zalando` role, defined with `superuser` and `createdb` flags. Manifest roles are defined as a dictionary, with a role name as a key and a @@ -218,7 +218,7 @@ Note, only the CRD-based configuration allows for referencing multiple secrets. As of now, the ConfigMap is restricted to either one or the existing template option with `infrastructure_roles_secret_name`. Please, refer to the example manifests to understand how `infrastructure_roles_secrets` has to be configured -for the [configmap](../manifests/configmap.yaml) or [CRD configuration](../manifests/postgresql-operator-default-configuration.yaml). +for the [configmap](https://github.com/zalando/postgres-operator/blob/master/manifests/configmap.yaml) or [CRD configuration](https://github.com/zalando/postgres-operator/blob/master/manifests/postgresql-operator-default-configuration.yaml). If both `infrastructure_roles_secret_name` and `infrastructure_roles_secrets` are defined the operator will create roles for both of them. So make sure, @@ -263,8 +263,8 @@ Since an infrastructure role is created uniformly on all clusters managed by the operator, it makes no sense to define it without the password. Such definitions will be ignored with a prior warning. -See [infrastructure roles secret](../manifests/infrastructure-roles.yaml) -and [infrastructure roles configmap](../manifests/infrastructure-roles-configmap.yaml) +See [infrastructure roles secret](https://github.com/zalando/postgres-operator/blob/master/manifests/infrastructure-roles.yaml) +and [infrastructure roles configmap](https://github.com/zalando/postgres-operator/blob/master/manifests/infrastructure-roles-configmap.yaml) for the examples. ### Teams API roles @@ -280,7 +280,7 @@ returns usernames. A minimal Teams API should work like this: /.../<teamname> -> ["name","anothername"] ``` -A ["fake" Teams API](../manifests/fake-teams-api.yaml) deployment is provided +A ["fake" Teams API](https://github.com/zalando/postgres-operator/blob/master/manifests/fake-teams-api.yaml) deployment is provided in the manifests folder to set up a basic API around whatever services is used for user management. The Teams API's URL is set in the operator's [configuration](reference/operator_parameters.md#automatic-creation-of-human-users-in-the-database) @@ -295,12 +295,12 @@ Postgres clusters are associated with one team by providing the `teamID` in the manifest. Additional superuser teams can be configured as mentioned in the previous paragraph. However, this is a global setting. To assign additional teams, superuser teams and single users to clusters of a given -team, use the [PostgresTeam CRD](../manifests/postgresteam.crd.yaml). +team, use the [PostgresTeam CRD](https://github.com/zalando/postgres-operator/blob/master/manifests/postgresteam.crd.yaml). Note, by default the `PostgresTeam` support is disabled in the configuration. Switch `enable_postgres_team_crd` flag to `true` and the operator will start to watch for this CRD. Make sure, the cluster role is up to date and contains a -section for [PostgresTeam](../manifests/operator-service-account-rbac.yaml#L30). +section for [PostgresTeam](https://github.com/zalando/postgres-operator/blob/master/manifests/operator-service-account-rbac.yaml#L30). #### Additional teams @@ -804,7 +804,7 @@ spec: ``` At the moment, the operator only allows to stream from the WAL archive of the -master. Thus, it is recommended to deploy standby clusters with only [one pod](../manifests/standby-manifest.yaml#L10). +master. Thus, it is recommended to deploy standby clusters with only [one pod](https://github.com/zalando/postgres-operator/blob/master/manifests/standby-manifest.yaml#L10). You can raise the instance count when detaching. Note, that the same pod role labels like for normal clusters are used: The standby leader is labeled as `master`.