S3 migration is the process of moving objects between storage systems that expose an S3 API. A safe migration preserves the data and the behavior applications depend on, including access permissions, metadata, version history and retention controls. Copying every object is one part of the job. The migration is complete when applications can use the destination correctly and the source can be retired without losing required data or protection.
The difficulty depends on what is moving. A static archive with one version of each object is relatively straightforward. A live repository with frequent updates, immutable backups and applications that reference specific object versions requires a more deliberate approach.
The most useful starting question is: What must still work exactly as it does today after the endpoint changes?
Define what the migration must preserve
An object inventory should capture more than total capacity. Record the number of objects, their size distribution, the buckets involved and how frequently data changes. Identify versioned buckets, retention requirements, encryption dependencies and applications that read or write to each location.
Object count matters because it changes the work involved. A repository containing billions of small objects can spend much of its migration time on listing, requests and metadata operations. A repository of large media files may be more constrained by sustained transfer bandwidth. Both can hold the same capacity and require very different migration schedules.
Separate the migration scope into three areas:
| Area | What to account for |
|---|---|
| Object content | Object keys, data, required versions and integrity checks |
| Object attributes | Content type, custom metadata, tags and applicable retention settings |
| Storage configuration | Access policies, identities, lifecycle rules, encryption configuration, notifications and logging |
These areas need separate validation. A transfer can preserve object content while omitting the configuration that determines who can retrieve it or when it expires.
Also decide what should stay behind. Expired data, abandoned test buckets and incomplete multipart uploads can increase the scope without improving the destination. Apply existing retention and deletion policies before excluding anything; migration is a poor time to make undocumented decisions about which records still matter.
Test S3 compatibility against the application
Support for the S3 API provides a common interface, but it does not make every implementation interchangeable. The relevant question is whether the destination supports the operations and behavior your applications actually use.
Start with a representative application and a small test bucket. Exercise uploads, downloads, listing, overwrites and deletion. Then test any dependencies beyond those basic operations, such as multipart uploads, presigned URLs, object tagging, version-specific reads and Object Lock.
Use the production client or SDK configuration where possible. A successful transfer from an administrator’s workstation does not prove that an application using different credentials, addressing settings or request headers will work.
Check endpoint and identity dependencies
Applications may depend on bucket names, endpoint URLs, certificate trust, signing settings or a particular addressing style. Some have configurable endpoints; others contain assumptions about the original service. Identify those assumptions before scheduling the transfer, and do not infer behavior from API compatibility alone.
Access policies also need their own review. Recreate the intended permissions using the destination’s supported identity and policy model, then test them with application credentials. Include negative tests: an account that should only read data must still be unable to overwrite or delete it.
Encryption requires similar preparation. Confirm that the migration process can read the source objects and that destination applications can decrypt the new copies. Source encryption keys and key-management integrations do not automatically become available on another platform.
Treat version history and immutability as separate work
A migration of current objects is different from a migration of all retained versions. Listing the current contents of a bucket may omit older versions and objects whose latest state is a delete marker.
Decide whether the destination must retain historical versions, the current application-visible state, or both. That decision affects the inventory, transfer method, capacity estimate and acceptance criteria. It also determines how you handle objects that users deleted during the migration.
Do not assume source version IDs will survive a copy. If an application stores version IDs in a database or uses them to retrieve historical content, validate how those references will work on the destination. Copying the underlying bytes does not resolve that dependency.
Preserve protection throughout the move
For locked objects, identify the retention mode, retain-until date and legal-hold status that must apply at the destination. Confirm that the selected tools can read and apply the required settings. Test the destination’s behavior before moving protected production data.
A critical failure would be creating a readable destination copy that lacks the source’s deletion protection. Another would be applying an unintended retention period that prevents legitimate future deletion. The acceptance check should verify the effective protection on destination objects, not simply that an Object Lock option was enabled.
Backup repositories deserve additional care. A backup application may maintain catalogs, indexes or references that make a generic bucket copy insufficient for a supported move. Use the backup vendor’s supported migration or repository transition procedure and prove that a restore works from the destination.
Choose a migration method that can handle change
The right method depends on the available downtime and how much the source changes while data is moving. Three patterns cover many migrations.
Stop writes and copy
For a small repository or a workload with an acceptable maintenance window, pause writes, transfer the data, validate it and change the application endpoint. This reduces the difficulty of reconciling concurrent changes.
The maintenance window must include validation and application testing, not just copying. A transfer that fits the window leaves little room for investigation if the destination fails an access or integrity check.
Copy first, then reconcile changes
For larger live repositories, perform an initial copy while the source remains active. Follow it with additional passes or a supported change-tracking process to capture new objects, updates and deletions.
The challenge is determining what changed and applying those changes correctly. A job that copies new objects but ignores deletions can leave data visible at the destination that should no longer be there. A job that propagates deletions without adequate controls can remove data prematurely.
Define how the process identifies changes, handles retries and reconciles conflicting operations. Before the final cutover, pause writes or use another proven method to establish a consistent boundary.
Use supported replication or migration tooling
Platform replication and specialist migration tools can reduce manual work, particularly at large object counts. Their suitability depends on the exact source, destination and features involved.
Confirm whether the chosen mechanism handles existing objects, subsequent updates, historical versions and deletions. Document which attributes it preserves and which settings require a separate operation. A feature described as replication should not be assumed to reproduce the entire bucket and its configuration.
Estimate the transfer window from a pilot
A useful first estimate divides the data volume by the sustained transfer rate. The important word is sustained: network link speed alone does not describe how quickly the migration can list, read, transmit, write and verify objects.
Consider an illustrative migration of 500 TB at a measured end-to-end rate of 2 GB per second. Using decimal units, the initial transfer would take approximately 69 hours, or just under three days.
That estimate covers one pass through the data. It does not include retries, validation, newly written objects or the final cutover.
Run the pilot with a representative mix of object sizes and attributes. Observe the effect on production latency as well as transfer speed. Raising concurrency can improve throughput until a source, destination, network or client limit becomes the bottleneck.
The migration must also catch up with ongoing writes. If new and changed data accumulate faster than the process can reconcile them, the final backlog will keep growing. You may need more transfer capacity, a longer write pause or smaller migration groups.
Estimate costs alongside time. Depending on the source, the move may incur retrieval charges, request charges, network egress and temporary duplicate-storage costs. Data in an archive tier may also require a restore step before it can be read.
Validate content, configuration and application behavior
A successful transfer report is useful evidence, but it is not sufficient acceptance criteria. Validation should answer three distinct questions.
Did the required objects arrive intact?
Compare the source inventory with the destination using the agreed migration scope. Reconcile missing objects, unexpected objects, size differences and failed transfers. For a versioned migration, compare the required versions rather than relying only on current-object counts.
Use compatible checksums to verify content where possible. An S3 ETag is not a universal content checksum: multipart uploads and some encryption methods affect its meaning. Different ETags do not necessarily indicate different data, and an ETag-only comparison is not a reliable general migration test.
Where directly comparable checksums are unavailable, define another verification method, such as reading objects and calculating the same checksum on both sides. Make any sampling limits explicit; a sample does not establish the integrity of every migrated object.
Are the required controls in place?
Check permissions, metadata, tags, encryption and retention against the agreed requirements. Validate lifecycle rules before enabling them so that newly migrated data does not expire unexpectedly.
Test denied operations as well as allowed ones. A migration should not be accepted because the administrator can retrieve everything if the production service account has excessive access or cannot read its own objects.
Can the application complete its work?
Run actual workflows against the destination. For a media application, that might include uploading a file, retrieving a byte range and serving it with the expected content type. For a data pipeline, it may mean listing a prefix, processing new objects and writing results.
For backup storage, perform a restore through the backup application. The ability to download an individual backup object does not prove that the application can reconstruct the protected workload, or that it can do so at the speed recovery targets assume.
Make cutover a controlled change
Cutover is the point at which the destination becomes authoritative. Give that decision explicit entry criteria: the required data has arrived, remaining exceptions are resolved or accepted, application tests pass and the final change backlog is understood.
A practical sequence is:
- Pause writes or establish the tested consistency boundary.
- Reconcile the final source changes and validate the resulting state.
- Switch application endpoints and credentials.
- Test reads and controlled writes through the production application.
- Resume normal activity and monitor errors, latency and access failures.
Changing DNS may be only one part of this process. Clients can cache endpoint information, retain connections or require a restart. Confirm that writers have moved to the destination before allowing normal operation to resume.
Plan rollback before accepting new writes
Rollback is relatively straightforward while the source remains authoritative and the destination has received no unique writes. Once applications start changing destination data, switching back can lose those changes or create conflicting histories.
Define the rollback procedure for both situations. After destination writes begin, recovery may require pausing the application and reconciling those writes back to the source.
Keeping the old storage available provides options. It does not, by itself, provide a complete rollback plan.
Retire the source only after acceptance
Keep the source available for an agreed validation period, subject to cost and retention requirements. Restrict access so that it cannot quietly become an active repository again, and investigate any remaining application requests reaching it.
Retirement should depend on evidence: application acceptance, completed integrity checks, verified protection settings and a resolved rollback window. Retain the migration inventory, exception records and validation results so the team can explain what moved and how it was checked.
What changes when the destination is Scality RING
For organizations moving to Scality RING, the destination decision also includes the operating model. RING provides software-defined file and object storage for infrastructure under the organization’s control, so the migration plan needs to cover destination capacity, data protection, monitoring and operational ownership alongside S3 application testing.
Those responsibilities are worth naming explicitly in the plan. Capacity and durability settings are chosen rather than inherited from a service tier, and the team that will operate the platform after cutover should be part of the validation work rather than receiving the result of it.
Accept the migration on evidence, not on a transfer report
The acceptance standard remains practical: the required data is present, applications behave correctly, protection controls work and the old system can be removed without breaking a dependency.
Start with one application and one bucket, run it through inventory, compatibility testing, transfer, validation and cutover, and record what each step proved. The rest of the estate is easier to schedule once the first migration has produced that evidence.