Merge pull request #256 from aquasecurity/fix-235

Rationalize and document config

docs/README.md

+# Test and config files
+
+`kube-bench` runs checks specified in `controls` files that are a YAML 
+representation of the CIS Kubernetes Benchmark checks. There is a 
+`controls` file per kubernetes version and node type.
+
+kube-bench automatically selects which `controls` to use based on the detected
+node type and the version of kubernetes a cluster is running. This behaviour
+can be overridden by specifying the `master` or `node` subcommand and the
+`--version` flag on the command line.
+
+For example:
+run kube-bench against a master with version  auto-detection:
+
+```
+kube-bench master
+```
+
+or run kube-bench against a node with the node `controls` for kubernetes 
+version 1.12:
+```
+kube-bench node --version 1.12
+```
+
+`controls` for the various versions of kubernetes can be found in directories
+with same name as the kubernetes versions under `cfg/`, for example `cfg/1.12`.
+`controls` are also organized by distribution under the `cfg` directory for
+example `cfg/ocp-3.10`.
+
+
+## Controls
+
+`controls` is a YAML document that contains checks that must be run against a 
+specific kubernetes node type, master or node and version.
+
+`controls` is the fundamental input to `kube-bench`. The following is an example 
+of a basic `controls`:
+
+```
+---
+controls:
+id: 1
+text: "Master Node Security Configuration"
+type: "master"
+groups:
+- id: 1.1
+  text: API Server
+  checks:
+    - id: 1.1.1
+      text: "Ensure that the --allow-privileged argument is set (Scored)"
+      audit: "ps -ef | grep kube-apiserver | grep -v grep"
+      tests:
+      bin_op: or
+      test_items:
+      - flag: "--allow-privileged"
+        set: true
+      - flag: "--some-other-flag"
+        set: false
+      remediation: "Edit the /etc/kubernetes/config file on the master node and
+        set the KUBE_ALLOW_PRIV parameter to '--allow-privileged=false'"
+      scored: true
+- id: 1.2
+  text: Scheduler
+  checks:
+    - id: 1.2.1
+      text: "Ensure that the --profiling argument is set to false (Scored)"
+      audit: "ps -ef | grep kube-scheduler | grep -v grep"
+      tests:
+        bin_op: or
+        test_items:
+          - flag: "--profiling"
+            set: true
+          - flag: "--some-other-flag"
+            set: false
+      remediation: "Edit the /etc/kubernetes/config file on the master node and
+        set the KUBE_ALLOW_PRIV parameter to '--allow-privileged=false'"
+      scored: true
+```
+
+`controls` is composed of a hierachy of groups, sub-groups and checks. Each of
+the `controls` components have an id and a text description which are displayed 
+in the `kube-bench` output.
+
+`type` specifies what kubernetes node type a `controls` is for. Possible  values
+for `type` are `master` and `node`.
+
+## Groups
+
+`groups` is list of subgroups which test the various kubernetes components
+that run on the node type specified in the `controls`. 
+
+For example one subgroup checks parameters passed to the apiserver binary, while 
+another subgroup checks parameters passed to the controller-manager binary.
+
+```
+groups:
+- id: 1.1
+  text: API Server
+  ...
+- id: 1.2
+  text: Scheduler
+  ...
+```
+
+These subgroups have `id`, `text` fields which serve the same purposes described
+in the previous paragraphs. The most important part of the subgroup is the
+`checks` field which is the collection of actual `check`s that form the subgroup.
+
+This is an example of a subgroup and checks in the subgroup.
+
+```
+id: 1.1
+text: API Server
+checks:
+  - id: 1.1.1
+    text: "Ensure that the --allow-privileged argument is set (Scored)"
+    audit: "ps -ef | grep kube-apiserver | grep -v grep"
+    tests:
+    ...
+  - id: 1.1.2
+    text: "Ensure that the --anonymous-auth argument is set to false (Not Scored)"
+    audit: "ps -ef | grep kube-apiserver | grep -v grep"
+    tests:
+    ...
+``` 
+
+`kube-bench` supports running a subgroup by specifying the subgroup `id` on the
+command line, with the flag `--group` or `-g`.
+
+## Check
+
+The CIS Kubernetes Benchmark recommends configurations to harden kubernetes 
+components. These recommendations are usually configuration options, and can be 
+specified by flags to kubernetes binaries, or in configuration files.
+
+The Benchmark also provides commands to audit a kubernetes installation, identify
+places where the cluster security can be improved, and steps to remediate these
+identified problems.
+
+In `kube-bench`, `check` objects embody these recommendations.  This an example
+`check` object:
+
+```
+id: 1.1.1
+text: "Ensure that the --anonymous-auth argument is set to false (Not Scored)"
+audit: "ps -ef | grep kube-apiserver | grep -v grep"
+tests:
+  test_items:
+  - flag: "--anonymous-auth"
+    compare:
+      op: eq
+      value: false
+    set: true
+remediation: |
+  Edit the API server pod specification file kube-apiserver
+  on the master node and set the below parameter.
+  --anonymous-auth=false
+scored: false
+```
+
+A `check` object has an `id`, a `text`, an `audit` , a `tests`,`remediation`
+and `scored` fields.
+
+`kube-bench` supports running individual checks by specifying the check's `id`
+as a comma-delimited list on the command line with the `--check` flag.
+
+The `audit` field specifies the command to run for a check. The output of this
+command is then evaluated for conformance with the CIS Kubernetes Benchmark
+recommendation.
+
+The audit is evaluated against a criteria specified by the `tests`
+object. `tests` contain `bin_op` and `test_items`.
+
+`test_items` specify the criteria(s) the `audit` command's output should meet to
+pass a check. This criteria is made up of keywords extracted from the output of
+the `audit` command and operations that compare the these keywords against
+values expected by the CIS Kubernetes Benchmark. 
+
+The are two ways to extract keywords from the output of the `audit` command,
+`flag` and `path`.
+
+`flag` is used when the keyword is a command line flag. The associated `audit`
+command is usually a `ps` command and a `grep` for the binary whose flag we are
+checking:
+
+```
+ps -ef | grep somebinary | grep -v grep
+``` 
+
+Here is an example usage of the `flag` option:
+
+```
+...
+audit: "ps -ef | grep kube-apiserver | grep -v grep"
+tests:
+  test_items:
+  - flag: "--anonymous-auth"
+  ...
+```
+
+`path` is used when the keyword is an option set in a JSON or YAML config file.
+The associated `audit` command is usually `cat /path/to/config-yaml-or-json`.
+For example:
+
+```
+...
+
+text: "Ensure that the --anonymous-auth argument is set to false (Not Scored)"
+audit: "cat /path/to/some/config"
+tests:
+  test_items:
+  - path: "{.someoption.value}"
+    ...
+```
+
+`test_item` compares the output of the audit command and keywords using the
+`set` and `compare` fields.
+
+```
+  test_items:
+  - flag: "--anonymous-auth"
+    compare:
+      op: eq
+      value: false
+    set: true
+```
+
+`set` checks if a keyword is present in the output of the audit command or in
+a config file. The possible values for `set` are true and false.
+
+If `set` is true, the check passes only if the keyword is present in the output
+of the audit command, or config file. If `set` is false, the check passes only
+if the keyword is not present in the output of the audit command, or config file.
+
+`compare` has two fields `op` and `value` to compare keywords with expected
+value. `op` specifies which operation is used for the comparison , and `value`
+specifies the value to compare against.
+
+> To use `compare`, `set` must true. The comparison will be ignored if `set` is
+> false
+
+The `op` (operations) currently supported in `kube-bench` are:
+- `eq`: tests if the keyword is equal to the compared value.
+- `noteq`: tests if the keyword is unequal to the compared value.
+- `gt`: tests if the keyword is greater than the compared value.
+- `gte`: tests if the keyword is greater than or equal to the compared value.
+- `lt`: tests if the keyword is less than the compared value.
+- `lte`: tests if the keyword is less than or equal to the compared value.
+- `has`: tests if the keyword contains the compared value.
+- `nothave`: tests if the keyword does not contain the compared value.
+
+## Configuration and Variables
+
+Kubernetes component configuration and binary file locations and names 
+vary based on cluster deployment methods and kubernetes distribution used.
+For this reason, the locations of these binaries and config files are configurable
+by editing the `cfg/config.yaml` file and these binaries and files can be
+referenced in a `controls` file via variables.
+
+The `cfg/config.yaml` file is a global configuration file. Configuration files
+can be created for specific Kubernetes versions (distributions). Values in the
+version specific config overwrite similar values in `cfg/config.yaml`.
+
+For example, the kube-apiserver in Redhat OCP distribution is run as 
+`hypershift openshift-kube-apiserver` instead of the default `kube-apiserver`.
+This difference can be specified by editing the `master.apiserver.defaultbin`
+entry `cfg/ocp-3.10/config.yaml`.
+
+Below is the structure of `cfg/config.yaml`:
+
+```
+nodetype
+  |-- components
+    |-- component1
+  |-- component1
+    |-- bins
+    |-- defaultbin (optional)
+    |-- confs
+    |-- defaultconf (optional)
+    |-- svcs
+    |-- defaultsvc (optional)
+    |-- kubeconfig
+    |-- defaultkubeconfig (optional)
+```
+
+Every node type has a subsection that specifies the main configurations items.
+
+- `components`: A list of components for the node type. For example master 
+  will have an entry for **apiserver**, **scheduler** and **controllermanager**.
+  
+  Each component has the following entries:
+
+- `bins`: A list of candidate binaries for a component. `kube-bench` checks this
+   list and selects the first binary that is running on the node, if none is
+   running, `kube-bench` terminates.
+
+   If `defaultbin` is specified, `kube-bench` ignores the `bins` list (if it is
+   specified) and verifies the binary specified with `defaultbin` is running on
+   the node. `kube-bench` terminates if this binary is not running.
+   
+   The selected binary for a component can be referenced in `controls` using a 
+   variable in the form `$<component>bin`. In the example below, we reference 
+   the selected API server binary with the variable `$apiserverbin` in an `audit`
+   command.
+   
+   ```
+   id: 1.1.1
+    text: "Ensure that the --anonymous-auth argument is set to false (Scored)"
+    audit: "ps -ef | grep $apiserverbin | grep -v grep"
+    ...
+   ```
+   
+- `confs`: A list of candidate configuration files for a component. `kube-bench`
+  checks this list and selects the first config fille that is found on the node,
+  if none of the config files exists `kube-bench` terminates.
+  
+  If `defaultconf`is specified for a component, `kube-bench` ignores the `confs`
+  list (if it is specified) and verifies the config specified by `defaultconf`
+  exists on the node. `kube-bench` terminates if this file does not exist.
+  
+  The selected config for a component can be referenced in `controls` using a
+  variable in the form `$<component>conf`. In the example below we reference the 
+  selected API server config file with the variable `$apiserverconf` in an `audit`
+  command.
+  
+  ```
+  id: 1.4.1
+    text: "Ensure that the API server pod specification file permissions are
+    set to 644 or more restrictive (Scored)"
+    audit: "/bin/sh -c 'if test -e $apiserverconf; then stat -c %a $apiserverconf; fi'"
+
+  ```
+  
+- `svcs`:  A list of candidates unitfiles for a component. `kube-bench` checks this 
+  list and selects the first unitfile that is found on the node, if none of the
+  unitfiles exists `kube-bench` terminates.
+  
+  If `defaultsvc`is specified for a component, `kube-bench` ignores the `svcs`
+  list (if it is specified) and verifies the unitfile specified by `defaultsvc`
+  exists on the node. `kube-bench` terminates if this file does not exist.
+  
+  The selected unitfile for a component can be referenced in `controls` via a
+  variable in the form `$<component>svc`. In the example below, the selected 
+  kubelet unitfile is referenced with `$kubeletsvc` in the `remediation` of the 
+  `check`.
+  
+  ```
+  id: 2.1.1
+    ...
+    remediation: |
+      Edit the kubelet service file $kubeletsvc
+      on each worker node and set the below parameter in KUBELET_SYSTEM_PODS_ARGS variable.
+      --allow-privileged=false
+      Based on your system, restart the kubelet service. For example:
+      systemctl daemon-reload
+      systemctl restart kubelet.service
+    ...
+  ```
+  
+  - `kubeconfig`: A list of candidate kubeconfig files for a component. `kube-bench`
+    checks this list and selects the first file that is found on the node, if none
+    of the files exists `kube-bench` terminates.
+    
+    If `defaultkubeconfig` is specified for a component, `kube-bench` ignores the
+    `kubeconfig` list (if it is specified) and verifies the kubeconfig file exists on
+    the node. `kube-bench` terminates if this file does not exist.
+    
+    The selected kubeconfig for a component can be referenced in `controls` with
+    a variable in the form `$<component>kubeconfig`. In the example below, the
+    selected kubelet kubeconfig is referenced with `$kubeletkubeconfig` in the
+    `audit` command.
+    
+    ```
+    id: 2.2.1
+      text: "Ensure that the kubelet.conf file permissions are set to 644 or
+      more restrictive (Scored)"
+      audit: "/bin/sh -c 'if test -e $kubeletkubeconfig; then stat -c %a $kubeletkubeconfig; fi'"
+      ...
+    ```