Will the output of this Storage Transfer Service builder pass `gcloud ... --validate-only` or `terraform validate`?

It produces structurally valid output for the GCP schemas it supports. We still recommend running provider validation locally before applying — schemas evolve and a recently-released property may not yet be reflected. When validation does fail, the message points at the exact attribute the schema rejected.

How does this Storage Transfer Service builder keep up with new GCP features?

The Storage Transfer Service options surface what is currently documented in the Google Cloud reference for that service. When Google adds a new property or value, we add it here after verifying the schema in a real project. If a recently-announced feature is not yet selectable, treat that as a 'not yet supported' signal rather than an opinion that it should not be used.

GCP Storage Transfer Service Builder

StorageGCP

Build Storage Transfer Service job configurations for cross-cloud and cross-bucket data migration.

Last verified: May 2026

Storage Transfer Service Configuration

Build Storage Transfer Service job configurations for cross-cloud and cross-bucket data migration with scheduling and filters.

Required Fields

descriptionprojectIdtransferSpecstatus

{
  "description": "Daily S3 to GCS data sync for analytics pipeline",
  "projectId": "my-project",
  "transferSpec": {
    "awsS3DataSource": {
      "bucketName": "source-data-bucket",
      "awsAccessKey": {
        "accessKeyId": "AKIAIOSFODNN7EXAMPLE",
        "secretAccessKey": "<REDACTED>"
      },
      "path": "exports/daily/"
    },
    "gcsDataSink": {
      "bucketName": "dest-analytics-bucket",
      "path": "imports/raw/"
    },
    "objectConditions": {
      "minTimeElapsedSinceLastModification": "3600s",
      "maxTimeElapsedSinceLastModification": "86400s",
      "includePrefixes": ["data/", "reports/"],
      "excludePrefixes": ["temp/", "logs/"]
    },
    "transferOptions": {
      "overwriteObjectsAlreadyExistingInSink": false,
      "deleteObjectsFromSourceAfterTransfer": false,
      "deleteObjectsUniqueInSink": false,
      "overwriteWhen": "DIFFERENT",
      "metadataOptions": {
        "storageClass": "STORAGE_CLASS_PRESERVE"
      }
    }
  },
  "schedule": {
    "scheduleStartDate": {
      "year": 2026,
      "month": 3,
      "day": 15
    },
    "scheduleEndDate": {
      "year": 2027,
      "month": 3,
      "day": 15
    },
    "startTimeOfDay": {
      "hours": 2,
      "minutes": 0,
      "seconds": 0
    },
    "repeatInterval": "86400s"
  },
  "notificationConfig": {
    "pubsubTopic": "projects/my-project/topics/transfer-notifications",
    "payloadFormat": "JSON",
    "eventTypes": ["TRANSFER_OPERATION_SUCCESS", "TRANSFER_OPERATION_FAILED"]
  },
  "status": "ENABLED"
}

Generated Output

Output will appear here...

About This Tool

Build Storage Transfer Service job configurations for cross-cloud and cross-bucket data migration. This tool helps GCP engineers generate valid configurations quickly without consulting documentation, reducing errors and accelerating infrastructure deployment. All processing runs in your browser with no data sent to external servers.

Real-World Scenario

Your team needs to migrate 50 TB from AWS S3 to GCS as part of a multi-cloud strategy shift. Custom gsutil scripts on a Compute Engine VM = unstable, slow, no resumability. The builder generates an STS job: source S3 bucket with credentials, destination GCS bucket, includes all prefixes, schedule one-time. STS handles the migration at multi-Gbps throughput with built-in retry. Transfer logs validate completeness. Total time: ~24 hours for 50 TB; total operational involvement: ~2 hours of setup + monitoring.

When to Use This Tool

•Producing a Storage Transfer Service starting point for a new service that follows the Google Cloud reference architecture rather than copy-paste from another team's repo.
•Producing a Storage Transfer Service snapshot for a compliance audit where the auditor wants the canonical config alongside the rendered policy.
•Walking a security reviewer through a proposed Storage Transfer Service configuration without forcing them through the Google Cloud console.
•Onboarding a new engineer by giving them a Storage Transfer Service sandbox to iterate on without touching live projects.

Pro Tips

TIP

Storage Transfer Service is dramatically faster and more reliable than rolling-your-own gsutil-based migration scripts. STS handles bandwidth throttling, retry on transient failures, and parallel transfer optimization. For >1 TB migrations, this is the right tool — not custom scripts.

TIP

Cross-cloud transfers (S3 → GCS, Azure Blob → GCS) are STS's killer feature. Native APIs at both ends, no intermediate compute needed, transfer happens at Google-scale bandwidth. For one-time migrations or scheduled cross-cloud sync, STS is dramatically simpler than custom solutions.

TIP

Always enable transfer logs to GCS. The logs detail every successful and failed file transfer — essential for verifying migration completeness and troubleshooting failures. Without logs, large migrations (millions of files) become impossible to validate.

How It Works Under the Hood

The builder constructs Storage Transfer Service job configurations: transfer job resource (description, schedule for one-time or recurring transfers, source: GCS bucket / S3 bucket / HTTPS URL list / Azure Blob, destination: GCS bucket, transfer specification: include/exclude prefix lists, file conditions like minimum age, deletion behavior, transfer logging configuration). Output is generated as gcloud transfer jobs create commands and Terraform google_storage_transfer_job resources.

Frequently Asked Questions

Will the output of this Storage Transfer Service builder pass `gcloud ... --validate-only` or `terraform validate`?: It produces structurally valid output for the GCP schemas it supports. We still recommend running provider validation locally before applying — schemas evolve and a recently-released property may not yet be reflected. When validation does fail, the message points at the exact attribute the schema rejected.
How does this Storage Transfer Service builder keep up with new GCP features?: The Storage Transfer Service options surface what is currently documented in the Google Cloud reference for that service. When Google adds a new property or value, we add it here after verifying the schema in a real project. If a recently-announced feature is not yet selectable, treat that as a 'not yet supported' signal rather than an opinion that it should not be used.

Related Learning Guides

GCS Storage Classes & Lifecycle22 min read
Memorystore for Redis Guide24 min read

Was this tool helpful?

Disclaimer: This tool runs entirely in your browser. No data is sent to our servers. Always verify outputs before using them in production. AWS, Azure, and GCP are trademarks of their respective owners.