The AWS Certified DevOps Engineer - Professional (DOP-C02)

To scale microservice Pods based on CPU utilization, the Kubernetes Horizontal Pod Autoscaler (HPA) uses the Kubernetes Metrics Server to monitor resource usage and automatically adjusts the number of Pods. However, scaling Pods may require additional nodes if the current node capacity is insufficient.

The Cluster Autoscaler works with EKS managed node groups to add or remove worker nodes based on pending Pod requirements and resource usage.

By deploying both HPA and Cluster Autoscaler, the system can automatically scale Pods and add nodes as necessary, ensuring efficient resource utilization and availability.

Configuring the Cluster Autoscaler with auto-discovery allows it to manage node groups without manual intervention, reducing operational effort.

Option A only scales nodes based on node CPU utilization, not Pods.

Option B uses VPA recommender mode, which only suggests resource changes and does not scale automatically.

Option C involves manual updates and is not automated scaling. Therefore, option D provides the most operationally efficient, fully automated scaling solution.

Reference from AWS Official Documentation:

Kubernetes Horizontal Pod Autoscaler: " HPA automatically scales the number of Pods based on observed CPU utilization or other metrics. " (Kubernetes HPA)

Cluster Autoscaler on Amazon EKS: " The Cluster Autoscaler automatically adjusts the size of the Kubernetes cluster when there are Pods that fail to run due to insufficient resources or when nodes in the cluster are underutilized. " (AWS EKS Cluster Autoscaler)

The requirement is to update the infrastructure to ensure that only the Lambda function’s execution role can access the values in Secrets Manager. The solution must apply the principle of least privilege, which means granting the minimum permissions necessary to perform a task.

To do this, the DevOps engineer needs to use the following steps:

Create a KMS customer managed key that trusts Secrets Manager and allows the Lambda function’s execution role to decrypt. A customer managed key is a symmetric encryption key that is fully managed by the customer. The customer can define the key policy, which specifies who can use and manage the key. By creating a customer managed key, the DevOps engineer can restrict the decryption permission to only the Lambda function’s execution role, and prevent other principals from accessing the secret values. The customer managed key also needs to trust Secrets Manager, which means allowing Secrets Manager to use the key to encrypt and decrypt secrets on behalf of the customer.

Update Secrets Manager to use the new customer managed key. Secrets Manager allows customers to choose which KMS key to use for encrypting each secret. By default, Secrets Manager uses the default KMS key for Secrets Manager, which is a service-managed key that is shared by all customers in the same AWS Region. By updating Secrets Manager to use the new customer managed key, the DevOps engineer can ensure that only the Lambda function’s execution role can decrypt the secret values using that key.

Ensure that the Lambda function’s execution role has the KMS permissions scoped on the resource level. The Lambda function’s execution role is an IAM role that grants permissions to the Lambda function to access AWS services and resources. The role needs to have KMS permissions to use the customer managed key for decryption. However, to apply the principle of least privilege, the role should have the permissions scoped on the resource level, which means specifying the ARN of the customer managed key as a condition in the IAM policy statement. This way, the role can only use that specific key and not any other KMS keys in the account.

Container restart failures require detailed observability into pod-level, node-level, and container-level metrics and logs. CloudWatch Container Insights is purpose-built for Kubernetes operational diagnostics and provides granular visibility into CPU, memory, network I/O, disk I/O, container restarts, OOM kills, throttling, pod lifecycle issues, and Kubernetes control plane behaviors.

The CloudWatch Observability add-on deploys Fluent Bit and the CloudWatch Agent directly into the EKS cluster as DaemonSets. These components automatically collect:

Container logs

Pod metrics

Node metrics

Cluster events

OOM errors

CrashLoopBackOff restart cycles

Control plane request anomalies

With this data, the DevOps engineer can easily identify misconfigurations, resource bottlenecks, unhealthy nodes, failing containers, or image pull issues.

Option A (dashboards only) lacks per-container diagnostic data.

Option B (CloudTrail) only logs API calls — not useful for restart debugging.

Option C (CloudTrail Insights) only detects anomalous API usage, not container failures.

Therefore, CloudWatch Container Insights is the correct and AWS-recommended solution for diagnosing container restart failures in EKS.

Step 1: Sending AWS WAF Logs to CloudWatch LogsAWS WAF allows you to log requests that are evaluated against your web ACLs. These logs can be sent directly to CloudWatch Logs, which enables real-time monitoring and analysis.

Action: Configure the AWS WAF web ACL to send log messages to a CloudWatch Logs log group.

Why: This allows the operations team to view the logs in real time and analyze patterns using CloudWatch metric filters.

Option A is incorrect because increasing the memory of the Lambda function does not address the root cause of the problem, which is that the Lambda function is not triggered by the S3 event source. Increasing the memory of the Lambda function might improve its performance or reduce its execution time, but it does not affect its invocation. Moreover, increasing the memory of the Lambda function might incur higher costs, as Lambda charges based on the amount of memory allocated to the function.

Option B is correct because creating a resource policy on the Lambda function to grant Amazon S3 the permission to invoke the Lambda function for the S3 bucket is a necessary step to configure an S3 event source. A resource policy is a JSON document that defines who can access a Lambda resource and under what conditions. By granting Amazon S3 permission to invoke the Lambda function, the company ensures that the Lambda function runs when a new object is created or an existing object is modified in the S3 bucket1.

Option C is incorrect because configuring an Amazon Simple Queue Service (Amazon SQS) queue as an On-Failure destination for the Lambda function does not help with triggering the Lambda function. An On-Failure destination is a feature that allows Lambda to send events to another service, such as SQS or Amazon Simple Notification Service (Amazon SNS), when a function invocation fails. However, this feature only applies to asynchronous invocations, and S3 event sources use synchronous invocations. Therefore, configuring an SQS queue as an On-Failure destination would have no effect on the problem.

Option D is incorrect because provisioning space in the /tmp folder of the Lambda function does not address the root cause of the problem, which is that the Lambda function is not triggered by the S3 event source. Provisioning space in the /tmp folder of the Lambda function might help with processing large files from the S3 bucket, as it provides temporary storage for up to 512 MB of data. However, it does not affect the invocation of the Lambda function.

This solution will meet the requirements in the most operationally efficient manner because it will use CloudWatch Logs destination to aggregate the log groups from all the accounts to a single S3 bucket in the logging account. However, unlike option A, this solution will create a CloudWatch Logs destination for each region, instead of a single destination for all regions. This will improve the performance and reliability of the log delivery, as it will avoid cross-region data transfer and latency issues. Moreover, this solution will use an Amazon Kinesis data stream and an Amazon Kinesis Data Firehose delivery stream for each region, instead of a single stream for all regions. This will also improve the scalability and throughput of the log delivery, as it will avoid bottlenecks and throttling issues that may occur with a single stream.