Skip to content
Snippets Groups Projects
  • Damien Grisonnet's avatar
    4c6a06cf
    jsonnet: make prometheus-adapter highly-available · 4c6a06cf
    Damien Grisonnet authored
    
    Prometheus-adapter is a component of the monitoring stack that in most
    cases require to be highly available. For instance, we most likely
    always want the autoscaling pipeline to be available and we also want to
    avoid having no available backends serving the metrics API apiservices
    has it would result in both the AggregatedAPIDown alert firing and the
    kubectl top command not working anymore.
    
    In order to make the adapter highly-avaible, we need to increase its
    replica count to 2 and come up with a rolling update strategy and a
    pod anti-affinity rule based on the kubernetes hostname to prevent the
    adapters to be scheduled on the same node. The default rolling update
    strategy for deployments isn't enough as the default maxUnavaible value
    is 25% and is rounded down to 0. This means that during rolling-updates
    scheduling will fail if there isn't more nodes than the number of
    replicas. As for the maxSurge, the default should be fine as it is
    rounded up to 1, but for clarity it might be better to just set it to 1.
    For the pod anti-affinity constraints, it would be best if it was hard,
    but having it soft should be good enough and fit most use-cases.
    
    Signed-off-by: default avatarDamien Grisonnet <dgrisonn@redhat.com>
    4c6a06cf
    History
    jsonnet: make prometheus-adapter highly-available
    Damien Grisonnet authored
    
    Prometheus-adapter is a component of the monitoring stack that in most
    cases require to be highly available. For instance, we most likely
    always want the autoscaling pipeline to be available and we also want to
    avoid having no available backends serving the metrics API apiservices
    has it would result in both the AggregatedAPIDown alert firing and the
    kubectl top command not working anymore.
    
    In order to make the adapter highly-avaible, we need to increase its
    replica count to 2 and come up with a rolling update strategy and a
    pod anti-affinity rule based on the kubernetes hostname to prevent the
    adapters to be scheduled on the same node. The default rolling update
    strategy for deployments isn't enough as the default maxUnavaible value
    is 25% and is rounded down to 0. This means that during rolling-updates
    scheduling will fail if there isn't more nodes than the number of
    replicas. As for the maxSurge, the default should be fine as it is
    rounded up to 1, but for clarity it might be better to just set it to 1.
    For the pod anti-affinity constraints, it would be best if it was hard,
    but having it soft should be good enough and fit most use-cases.
    
    Signed-off-by: default avatarDamien Grisonnet <dgrisonn@redhat.com>