Skip to content

CloudWatch

LogGroups

In CloudWatch, log groups settings can contribute to unnecessary resource wastage, especially when they are never expiring.

It's important to regularly audit these log groups, as retaining logs indefinitely can lead to excessive and unneeded storage costs.

The best practice is to set appropriate retention policies for most log groups, tailoring them to the specific needs of each application or service.

However, there are exceptions, such as CloudTrail log groups and VPC Flow Logs, which are often retained for long-term due to their critical role in security and compliance.

By selectively applying long-term retention to these key log groups and setting expiration policies for others, organizations can effectively manage their log storage, ensuring cost-efficiency while maintaining essential data for security and compliance purposes.

Custom Metrics

CloudWatch custom metrics can become a significant cost driver, especially when cardinality explosion occurs. Each unique combination of Namespace, MetricName, and Dimension name/value pairs is billed as a separate metric.

Pricing Tiers

CloudWatch custom metrics are billed based on the number of unique metrics:

  • First 10,000 metrics: $0.30 per metric per month
  • Next 240,000 metrics: $0.10 per metric per month
  • Next 750,000 metrics: $0.05 per metric per month
  • Over 1,000,000 metrics: $0.02 per metric per month

Cardinality Explosion

The most common cause of unexpected CloudWatch costs is cardinality explosion. This happens when high-cardinality dimensions are used, such as:

  • UserId or UserID
  • RequestId or RequestID
  • PodName or ContainerId
  • SessionId or TraceId
  • IP addresses
  • Path or URI values

For example, if you emit 5 custom metrics per user across 10,000 users, you'll have 50,000 unique metrics costing around $7,000 per month.

What We Detect

unusd.cloud scans for custom metric namespaces (those not starting with AWS/) and analyzes:

  • Metric count per namespace: Identifies namespaces with high metric counts
  • Average cardinality: Detects potential cardinality issues by analyzing dimension patterns
  • High-cardinality dimensions: Flags namespaces using risky dimension names
  • Estimated monthly cost: Calculates the cost based on AWS pricing tiers

Best Practices

  1. Avoid high-cardinality dimensions: Use fixed values instead of dynamic IDs
  2. Use Embedded Metric Format (EMF): Aggregate metrics before publishing to reduce metric count
  3. Review unused namespaces: Custom metrics cannot be deleted, but they expire after 15 months of no data
  4. Consider CloudWatch Metric Streams: For high-volume metrics, streaming to S3 can be more cost-effective

Important Note

Custom CloudWatch metrics cannot be manually deleted. They automatically expire after 15 months of receiving no new data. The best strategy is to prevent high-cardinality metrics from being published in the first place.