diff --git a/docs/_files/commit-status-flow.png b/docs/_files/commit-status-flow.png new file mode 100644 index 0000000000000000000000000000000000000000..e30fc6fc80028fd8261af44708c3aa79307a0fe5 Binary files /dev/null and b/docs/_files/commit-status-flow.png differ diff --git a/docs/_files/commit-status-github-failure.png b/docs/_files/commit-status-github-failure.png new file mode 100644 index 0000000000000000000000000000000000000000..09e92464ae957ea4cac4b462116c3fc70ecf9909 Binary files /dev/null and b/docs/_files/commit-status-github-failure.png differ diff --git a/docs/_files/commit-status-github-overview.png b/docs/_files/commit-status-github-overview.png new file mode 100644 index 0000000000000000000000000000000000000000..898d86b56e2d9316a95eb37716e376bc1711127d Binary files /dev/null and b/docs/_files/commit-status-github-overview.png differ diff --git a/docs/_files/commit-status-github-success.png b/docs/_files/commit-status-github-success.png new file mode 100644 index 0000000000000000000000000000000000000000..17cf5fd10f177b5619cafb8444c634364ec0fc1c Binary files /dev/null and b/docs/_files/commit-status-github-success.png differ diff --git a/docs/_files/github-commit-status.png b/docs/_files/github-commit-status.png deleted file mode 100644 index 65129f352918bcf9d0c100bd75608a0d7c9f926c..0000000000000000000000000000000000000000 Binary files a/docs/_files/github-commit-status.png and /dev/null differ diff --git a/docs/_files/gitlab-commit-status.png b/docs/_files/gitlab-commit-status.png deleted file mode 100644 index 9319725f63fac04f912d0e9f052fe70013202398..0000000000000000000000000000000000000000 Binary files a/docs/_files/gitlab-commit-status.png and /dev/null differ diff --git a/docs/guides/notifications.md b/docs/guides/notifications.md index e7e2090ac54c9280f78df70df01812cfe7dab13d..35123025252ddc47292591c0f2cfa400a25fd23b 100644 --- a/docs/guides/notifications.md +++ b/docs/guides/notifications.md @@ -105,47 +105,104 @@ When the verbosity is set to `info`, the controller will alert if: ## Git commit status -The `github` and `gitlab` provider are slightly different to the other chat providers. These providers will -link an event back to its source by writing a commit status event to the repository. For more information about how a -commit status works, refer to the [GitHub](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/about-status-checks) -or [GitLab](https://docs.gitlab.com/ee/api/commits.html) documentation. - -The first image is an example of how it may look like in GitHub while the one below is an example for GitLab. - - - -Currently the provider will only work with Alerts for Kustomization resources as the events have to be linked with a -specific git commit. Any other event that does not contain a commit reference will be ignored by the provider. -Each status will contain some additional information from the event which includes the resource kind, name and reason for the event. -It will be displayed in the format of `{{ .Kind }}/{{ .Name }} - {{ .Reason }}`. - -To get started the git provider require an authentication token to communicate with the API. -Follow the [GitHub](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) -or [Gitlab](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) for a detailed guide how to create a token. -Store the generated token in a Secret with the following data format. +The GitHub, GitLab, Bitbucket, and Azure DevOps providers are slightly different to the other providers. Instead of +a stateless stream of events, the git notification providers will link the event with accompanying git commit which +triggered the event. The linking is done by updating the commit status of a specific commit. + + - [GitHub](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/about-status-checks) + - [GitLab](https://docs.gitlab.com/ee/api/commits.html) + - [Bitbucket](https://developer.atlassian.com/server/bitbucket/how-tos/updating-build-status-for-commits/) + - [Azure DevOps](https://docs.microsoft.com/en-us/rest/api/azure/devops/git/statuses?view=azure-devops-rest-6.0) + +In GitHub the commit status set by notification-controller will result in a green checkmark or red cross next to the commit hash. +Clicking the icon will show more detailed information about the status. + + +Receiving an event in the form of a commit status rather than a message in a chat conversation has the benefit +that it closes the deployment loop giving quick and visible feedback if a commit has reconciled and if it succeeded. +This means that a deployment will work in a similar manner that people are used to with "traditional" push based CD pipelines. +Additionally the status can be fetched from the git providers API for a specific commit. Allowing for custom automation tools +that can automatically promote, commit to a new directory, after receiving a successful commit status. This can all be +done without requiring any access to the Kubernetes cluster. + +As stated before the provider works by referencing the same git repository as the Kustomization controller does. +When a new commit is pushed to the repository, source-controller will sync the commit, triggering the kustomize-controller +to reconcile the new commit. After this is done the kustomize-controller sends an event to the notification-controller +with the result and the commit hash it reconciled. Then notification-controller can update the correct commit and repository +when receiving the event. + + +!!! hint "Limitations" + The git notification providers require that a commit hash present in the meta data + of the event. There for the the providers will only work with `Kustomization` as an + event source, as it is the only resource which includes this data. + +First follow the [get started guide](../../get-started) if you do not have a Kubernetes cluster with Flux installed in it. +You will need a authentication token to communicate with the API. The authentication method depends on +the git provider used, refer to the [Provider CRD](../../components/notification/provider/#git-commit-status) +for details about how to get the correct token. The guide will use GitHub, but the other providers will work in a very similar manner. +The token will need to have write access to the repository it is going to update the commit status in. +Store the generated token in a Secret with the following data format in the cluster. ```yaml apiVersion: v1 kind: Secret metadata: name: github - namespace: gitops-system + namespace: flux-system data: token: <token> ``` +When sending notification events the kustomization-controller will include the commit hash related to the event. +Note that the commit hash in the event does not come from the git repository the `Kustomization` resource +comes from but rather the kustomization source ref. This mean that commit status notifications will not work +if the manifests comes from a repository which the API token is not allowed to write to. + +Copy the manifest content in the "[kustomize](https://github.com/stefanprodan/podinfo/tree/master/kustomize)" directory +into the directory "staging-cluster/flux-system/podinfo" in your fleet-infra repository. Make sure that you also add the +namepsace podinfo. +```yaml +apiVersion: v1 +kind: Namespace +metadata: + name: podinfo +``` + +Then create a Kustomization to deploy podinfo. +```yaml +apiVersion: kustomize.toolkit.fluxcd.io/v1beta1 +kind: Kustomization +metadata: + name: podinfo + namespace: flux-system +spec: + interval: 5m + targetNamespace: podinfo + path: ./staging-cluster/podinfo + prune: true + sourceRef: + kind: GitRepository + name: flux-system + healthChecks: + - apiVersion: apps/v1 + kind: Deployment + name: podinfo + namespace: podinfo + timeout: 1m +``` + Creating a git provider is very similar to creating other types of providers. The only caveat being that the provider address needs to point to the same -git repository as the Kustomization resource refers to. +git repository as the event source originates from. ```yaml apiVersion: notification.toolkit.fluxcd.io/v1beta1 kind: Provider metadata: - name: podinfo + name: flux-system namespace: flux-system spec: type: github - channel: general - address: https://github.com/stefanprodan/podinfo + address: https://github.com/<username>/fleet-infra secretRef: name: github --- @@ -156,7 +213,7 @@ metadata: namespace: flux-system spec: providerRef: - name: podinfo + name: flux-system eventSeverity: info eventSources: - kind: Kustomization @@ -164,14 +221,135 @@ spec: namespace: flux-system ``` -The secret referenced in the provider is expected to contain a [personal access token](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) -to authenticate with the GitHub API. +By now the fleet-infra repository should have a similar directory structure. +``` +fleet-infra +└── staging-cluster/ + ├── flux-system/ + │ ├── gotk-components.yaml + │ ├── gotk-sync.yaml + │ └── kustomization.yaml + ├── podinfo/ + │ ├── namespace.yaml + │ ├── deployment.yaml + │ ├── hpa.yaml + │ ├── service.yaml + │ └── kustomization.yaml + ├── podinfo-kustomization.yaml + └── podinfo-notification.yaml +``` + +If podinfo is deployed and the health checks pass you should get a successful status in +your forked podinfo repository. + +If everything is setup correctly there should now be a green checkmark next to the lastest commit. +Clicking the checkmark should show a detailed view. + + + +Generate error + +A deployment failure can be fored by setting an invalid image tag in the podinfo deployment. ```yaml -apiVersion: v1 -kind: Secret +apiVersion: apps/v1 +kind: Deployment metadata: - name: github - namespace: flux-system -data: - token: <token> + name: podinfo +spec: + minReadySeconds: 3 + revisionHistoryLimit: 5 + progressDeadlineSeconds: 60 + strategy: + rollingUpdate: + maxUnavailable: 0 + type: RollingUpdate + selector: + matchLabels: + app: podinfo + template: + metadata: + annotations: + prometheus.io/scrape: "true" + prometheus.io/port: "9797" + labels: + app: podinfo + spec: + containers: + - name: podinfod + image: ghcr.io/stefanprodan/podinfo:fake + imagePullPolicy: IfNotPresent + ports: + - name: http + containerPort: 9898 + protocol: TCP + - name: http-metrics + containerPort: 9797 + protocol: TCP + - name: grpc + containerPort: 9999 + protocol: TCP + command: + - ./podinfo + - --port=9898 + - --port-metrics=9797 + - --grpc-port=9999 + - --grpc-service-name=podinfo + - --level=info + - --random-delay=false + - --random-error=false + env: + - name: PODINFO_UI_COLOR + value: "#34577c" + livenessProbe: + exec: + command: + - podcli + - check + - http + - localhost:9898/healthz + initialDelaySeconds: 5 + timeoutSeconds: 5 + readinessProbe: + exec: + command: + - podcli + - check + - http + - localhost:9898/readyz + initialDelaySeconds: 5 + timeoutSeconds: 5 + resources: + limits: + cpu: 2000m + memory: 512Mi + requests: + cpu: 100m + memory: 64Mi ``` + +After the commit has been reconciled it should return a failed commit status. +This is where the health check in the Kustomization comes into play together +with the timeout. The health check is used to asses the health of the Kustomization. +A failed commit status will not be sent until the health check timesout. Setting +a lower timeout will give feedback faster, but may sometimes not allow enough time +for a new application to deploy. + + + +### Status changes + +The provider will continuously receive events as they happen, and multiple events may +be received for the same commit hash. The git providers are configured to only update +the status if the status has changed. This is to avoid spamming the commit status +history with the same status over and over again. + +There is an aspect of state fullness that needs to be considered, compared to the other +notification providers, as the events are stored by the git provider. This means that +the status of a commit can change over time. Initially a deployment may be healthy, resulting +in a successful status. Down the line the application, and the health check, may start failing +due to the amount of traffic it receives or external dependencies no longer being available. +The change in the health check would cause the status to go from successful to failed. +It is important to keep this in mind when building any automation tools that deals with the +status, and consider the fact that receiving a successful status once does not mean it will +always be successful. +