Merge pull request #4670 from gjtempleton/CA-AWS-IAM-Permissions-Rework

CA - AWS - Rework IAM Docs/Snippets

cluster-autoscaler/cloudprovider/aws/CA_with_AWS_IAM_OIDC.md

@@ -53,9 +53,11 @@ D) Set up [Cluster Autoscaler Auto-Discovery] using the [tutorial] .
 
 Note: The keys for the tags that you entered don't have values. Cluster Autoscaler ignores any value set for the keys.
 
-- Create an IAM Policy for cluster autoscaler and to enable AutoDiscovery. 
+__NOTE:__ Please see [the README](README.md#IAM-Policy) for more information on best practices with this IAM role.
 
-```sh
+- Create an IAM Policy for cluster autoscaler and to enable AutoDiscovery as well as discovery of instance types.
+
+```json
 {
     "Version": "2012-10-17",
     "Statement": [
@@ -66,17 +68,23 @@ Note: The keys for the tags that you entered don't have values. Cluster Autoscal
                 "autoscaling:DescribeAutoScalingInstances",
                 "autoscaling:DescribeLaunchConfigurations",
                 "autoscaling:DescribeTags",
+                "ec2:DescribeInstanceTypes",
+                "ec2:DescribeLaunchTemplateVersions"
+            ],
+            "Resource": ["*"]
+        },
+        {
+            "Effect": "Allow",
+            "Action": [
                 "autoscaling:SetDesiredCapacity",
                 "autoscaling:TerminateInstanceInAutoScalingGroup"
             ],
-            "Resource": "*"
+            "Resource": ["*"]
         }
     ]
 }
 ```
 
-NOTE: ``` autoscaling:DescribeTags ``` is very important if you are making use of the AutoDiscovery feature of the Cluster AutoScaler. 
-
 - Attach the above created policy to the *instance role* that's attached to your Amazon EKS worker nodes.
 - Download a deployment example file provided by the Cluster Autoscaler project on GitHub, run the following command:
 

cluster-autoscaler/cloudprovider/aws/README.md

@@ -19,7 +19,23 @@ the EC2 instance on which the Cluster Autoscaler pod runs.
 
 ### IAM Policy
 
-The following policy provides the minimum privileges necessary for Cluster Autoscaler to run:
+There are a number of ways to run the autoscaler in AWS, which can significantly
+impact the range of IAM permissions required for the Cluster Autoscaler to function
+properly. Two options are provided below, one which will allow use of all of the
+features of the Cluster Autoscaler, the second with a more limited range of IAM
+actions enabled, which enforces using certain configuration options in the
+Cluster Autoscaler binary.
+
+It is strongly recommended to restrict the target resources for the autoscaling actions
+by either [specifying Auto Scaling Group ARNs](https://docs.aws.amazon.com/autoscaling/latest/userguide/control-access-using-iam.html#policy-auto-scaling-resources) in the `Resource` list of the policy or
+[using tag based conditionals](https://docs.aws.amazon.com/autoscaling/ec2/userguide/control-access-using-iam.html#security_iam_service-with-iam-tags). The [minimal policy](#minimal-iam-permissions-policy)
+includes an example of restricting by ASG ARN.
+
+#### Full Cluster Autoscaler Features Policy (Recommended)
+
+Permissions required when using [ASG Autodiscovery](#Auto-discovery-setup) and
+Dynamic EC2 List Generation (the default behaviour). In this example, only the second block of actions
+should be updated to restrict the resources/add conditionals:
 
 ```json
 {
@@ -31,9 +47,17 @@ The following policy provides the minimum privileges necessary for Cluster Autos
         "autoscaling:DescribeAutoScalingGroups",
         "autoscaling:DescribeAutoScalingInstances",
         "autoscaling:DescribeLaunchConfigurations",
+        "autoscaling:DescribeTags",
+        "ec2:DescribeInstanceTypes",
+        "ec2:DescribeLaunchTemplateVersions"
+      ],
+      "Resource": ["*"]
+    },
+    {
+      "Effect": "Allow",
+      "Action": [
         "autoscaling:SetDesiredCapacity",
-        "autoscaling:TerminateInstanceInAutoScalingGroup",
-        "ec2:DescribeInstanceTypes"
+        "autoscaling:TerminateInstanceInAutoScalingGroup"
       ],
       "Resource": ["*"]
     }
@@ -41,17 +65,42 @@ The following policy provides the minimum privileges necessary for Cluster Autos
 }
 ```
 
-If you'd like Cluster Autoscaler to [automatically
-discover](#auto-discovery-setup) EC2 Auto Scaling Groups **(recommended)**, add
-`autoscaling:DescribeTags` to the `Action` list. Also add
-`autoscaling:DescribeLaunchConfigurations` (if you created your ASG using a
-Launch Configuration) and/or `ec2:DescribeLaunchTemplateVersions` (if you
-created your ASG using a Launch Template) to the `Action` list.
-
-If you prefer, you can restrict the target resources for the autoscaling actions
-by specifying Auto Scaling Group ARNs in the `Resource` list of the policy. More
-information can be found
-[here](https://docs.aws.amazon.com/autoscaling/latest/userguide/control-access-using-iam.html#policy-auto-scaling-resources).
+#### Minimal IAM Permissions Policy
+
+*NOTE:* The below policies/arguments to the Cluster Autoscaler need to be modified as appropriate
+for the names of your ASGs, as well as account ID and AWS region before being used.
+
+The following policy provides the minimum privileges necessary for Cluster Autoscaler to run.
+When using this policy, you cannot use autodiscovery of ASGs. In addition, it restricts the
+IAM permissions to the node groups the Cluster Autoscaler is configured to scale.
+
+This in turn means that you must pass the following arguments to the Cluster Autoscaler
+binary, replacing min and max node counts and the ASG:
+
+```bash
+--aws-use-static-instance-list=false
+--nodes=1:100:exampleASG1
+--nodes=1:100:exampleASG2
+```
+
+```json
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Effect": "Allow",
+      "Action": [
+        "autoscaling:DescribeAutoScalingGroups",
+        "autoscaling:DescribeAutoScalingInstances",
+        "autoscaling:DescribeLaunchConfigurations",
+        "autoscaling:SetDesiredCapacity",
+        "autoscaling:TerminateInstanceInAutoScalingGroup"
+      ],
+      "Resource": ["arn:aws:autoscaling:${YOUR_CLUSTER_AWS_REGION}:${YOUR_AWS_ACCOUNT_ID}:autoScalingGroup:*:autoScalingGroupName/${YOUR_ASG_NAME}"]
+    }
+  ]
+}
+```
 
 ### Using OIDC Federated Authentication