Optimizing Google Cloud Storage and Secret Manager for Zero-Cost Architectures

The Zero-Cost Cloud Engineer Part 4: Cloud Storage, Secret Manager, and the Legacy Access Trap

This guide integrates Google Cloud Storage and Secret Manager into a Spring Boot application running on an internet-less Compute Engine VM. It addresses the 30GB local disk limit by utilizing the 5GB-month Always Free tier for object storage.

Why This Matters

Engineers often assume IAM roles are sufficient for VM permissions, but legacy Access Scopes on Google Compute Engine can explicitly block Secret Manager and throttle storage even with Admin roles attached. Furthermore, the default Soft Delete policy on GCS buckets can lead to unexpected billing by retaining deleted files for 7 days, consuming the 5GB free quota without visible warnings. Transitioning to the cloud-platform scope and disabling retention policies is critical for maintaining a strictly free, production-grade environment.

Key Insights

• GCS Always Free Tier provides 5 GB-months of Standard Storage in specific regions like us-east1, avoiding data egress charges when matched with VM location.
• The Soft Delete feature enabled by default retains files for 7 days, which can trigger billing alerts by exceeding the 5GB free quota even after file deletion.
• GCE Default access scopes override IAM privileges, limiting storage to read-only and blocking Secret Manager access regardless of attached IAM roles.
• GCP Secret Manager offers 6 free secret versions per month, allowing for secure fetching of bucket names and connection properties on boot.
• The cloud-platform scope is the modern standard for GCE VMs, delegating authentication authority entirely to IAM for seamless service integration.

Working Examples

Commands to strip legacy GCE Access Scopes and delegate authority to IAM.

# 1. Stop the instance
gcloud compute instances stop free-tier-vm --zone=us-east1-b
# 2. Grant full API access (Delegating authority completely to IAM)
gcloud compute instances set-service-account free-tier-vm \
--zone=us-east1-b \
--scopes=https://www.googleapis.com/auth/cloud-platform
# 3. Restart the instance
gcloud compute instances start free-tier-vm --zone=us-east1-b

Spring Boot configuration to import secrets from GCP Secret Manager, guarded by profile.

spring:
  application:
    name: hello-gcp
---
spring:
  config:
    activate:
      on-profile: "!test"
    import: sm://

Spring Boot Controller for direct GCS file uploads using Secret Manager for configuration.

@RestController
@RequestMapping("/api/files")
public class GcsUploadController {
  private final Storage storage;
  private final String bucketName;

  public GcsUploadController(Storage storage, @Value("${sm://GCS_BUCKET_NAME}") String bucketName) {
    this.storage = storage;
    this.bucketName = bucketName;
  }

  @PostMapping("/upload")
  public String uploadFile(@RequestParam("file") MultipartFile file) throws IOException {
    BlobInfo blobInfo = BlobInfo.newBuilder(BlobId.of(bucketName, file.getOriginalFilename()))
      .setContentType(file.getContentType())
      .build();
    storage.create(blobInfo, file.getBytes());
    return "Uploaded " + file.getOriginalFilename() + " directly to secure bucket!";
  }
}

Practical Applications

• Offloading user uploads from VM block storage to GCS buckets. Pitfall: Forgetting to set Soft Delete retention to 0 days, leading to hidden quota consumption.
• Injecting runtime configuration via Secret Manager using sm:// prefixes. Pitfall: Receiving PERMISSION_DENIED due to legacy VM Access Scopes overriding IAM permissions.
• Establishing local tunnels via IAP for secure testing of private services. Pitfall: Hardcoding bucket names in code which complicates multi-environment deployments.

References:

• https://dev.to/mohamadawwaad/the-zero-cost-cloud-engineer-part-4-cloud-storage-secret-manager-and-the-legacy-access-trap-32hi