Auto-Orphan-Volume-Cleanup-Automation

Auto-Orphan-Volume-Cleanup-Automation

Unused Amazon EBS volumes silently accumulate in cloud environments, driving up costs and creating operational inefficiencies. This project addresses this challenge with an automated, secure, and auditable workflow for managing and cleaning up these orphaned volumes.

The traditional approach of manual scripts and periodic audits for identifying unused volumes is prone to errors, lacks cross-account visibility, and lacks a centralized approval mechanism – potentially leading to accidental data loss and significant financial waste, costing organizations hundreds or even thousands of dollars monthly.

Why This Matters

Ideal cloud resource management envisions a self-healing, cost-optimized environment. In reality, dynamic workloads often leave behind orphaned EBS volumes, a common source of wasted spend. Manual cleanup is unsustainable at scale, and relying on infrequent audits results in continued, unnecessary costs. This automation addresses a critical gap in cloud operational efficiency.

Key Insights

• AWS EBS volume costs can exceed $1,250/month for high-performance volumes (io2, 500GB, 20,000 IOPS).
• Sagas are a pattern for managing distributed transactions, offering resilience compared to traditional ACID properties in microservice architectures.
• Temporal, a workflow orchestration platform, is used by companies like Stripe and Coinbase to manage complex stateful operations.

Working Example

import boto3
import os
import sys
import pandas as pd
from datetime import datetime

# Create EC2 client and describe volumes
client = boto3.client('ec2')
sns_client = boto3.client('sns')

# Define a blank variable to store ebs volume info
volume_data = []

# Loop through the response and extract relevant information
def vol_discovery():
    response = client.describe_volumes()
    for vol in response['Volumes']:
        if vol['State'] == 'available':
            Volume_ID = vol['VolumeId']
            Size = vol['Size']
            State = vol['State']
            Creation_time = vol['CreateTime']
            Creation_time = Creation_time.replace(tzinfo=None) if Creation_time.tzinfo is not None else None
            Creation_time = Creation_time.strftime("%Y-%m-%d %H:%M:%S")
            Vol_Type = vol['VolumeType']
            Disk_Type = [ tag['Value'] for tag in vol['Tags'] if tag['Key'] == 'Type'][0]
            Owner = [ tag['Value'] for tag in vol['Tags'] if tag['Key'] == 'Owner'][0]
            data = {
                "VolumeID": Volume_ID,
                "Size": Size,
                "State": State,
                "Created": Creation_time,
                "VolumeType": Vol_Type,
                "DiskType": Disk_Type,
                "Owner": Owner,
                "DeleteConfirmation": "Pending"
            }
            volume_data.append(data)

    # Create a Excel file with extracted volume information
    time = datetime.now().strftime("%H%M%S")
    df = pd.DataFrame(volume_data)
    output_file = f'discovery_available_ebsvol-{time}.xlsx'
    df.to_excel(f'/tmp/{output_file}', index=False)

    # Upload the Excel file to S3
    s3 = boto3.client('s3')
    bucket_name = os.environ['BUCKET_NAME']
    file_path = f'/tmp/{output_file}'
    s3_object_key = f'OrphanEBSReport/{output_file}' # Desired object key in S3
    try:
        s3.upload_file(file_path, bucket_name, s3_object_key)
        s3_url = f"https://{bucket_name}.s3.amazonaws.com/{s3_object_key}"
        print(f"File uploaded to S3: {s3_url}")
    except Exception as e:
        print(f"Error uploading file to S3: {e}")
        exit()

    # Send an email with the Excel file as an attachment
    snsarn = os.environ['SNS_ARN']
    body = f"Hi Team, \n\nPlease be informed that the following EBS volumes are in 'available' state and not attached to any EC2 instances. Kindly review the excel report from below link.\n\nLink: {s3_url}\n\nPlease click on http://ALB-External-364496655.us-east-1.elb.amazonaws.com to provide an approval.\n\nBest Regards,\nSystems Management Team."
    res = sns_client.publish(
        TopicArn = snsarn,
        Subject = f'Orphan EBS Volume Discovery Report',
        Message = str(body)
    )
    return volume_data

Practical Applications

• Stripe: Could utilize this system to automatically identify and clean up unused EBS volumes associated with testing or development environments, optimizing infrastructure spending.
• Pitfall: Relying solely on tagging for identifying orphaned volumes can be unreliable if tagging conventions are not consistently enforced across all teams and resources.

References:

• https://dev.to/susseta/auto-orphan-volume-cleanup-automation-33ej