Nearly every storage product now advertises an S3 interface, and most applications connect to any of them on the first try. The trouble surfaces later: a lifecycle rule that never fires, a listing that takes minutes once a bucket holds a few hundred million objects, or an object lock retention that holds only until someone gains administrative access to the layer underneath.
The S3 API has become the de facto standard for object storage, and vendors have taken two very different paths to supporting it. Some implement S3 semantics natively in the data path. Others place an S3 gateway in front of an existing file system or block layer and translate each request. Both are commonly labeled “S3-compatible,” and the label alone says little about how a system will behave once real applications and real data volumes arrive.
What does “S3-native” actually mean?
An S3-native object store is one in which the object and the bucket are the primary constructs of the storage system itself, not an abstraction layered over something else. A request such as PutObject, CompleteMultipartUpload or PutObjectRetention is handled by the storage engine directly, and the engine’s model of data, metadata, permissions and durability was designed around S3 semantics from the start. Several properties follow:
- Object and bucket as first-class constructs. Keys are flat identifiers in a namespace. A key such as
2026/09/report.pdfis one string, not a path through three directories, and prefix listing is an index query rather than a directory walk. - Native multipart upload. Parts are stored independently and the object becomes visible atomically on completion. Aborted uploads leave no half-written object behind.
- Versioning in the metadata model. Each version is a distinct record with its own ID, checksum and retention state. Delete markers are real metadata, not files with special names.
- Object lock enforced by the engine. Retention and legal holds are checked by the same code that performs deletes and overwrites. There is no lower layer where a privileged user can bypass the rule.
- IAM-style policies evaluated at the object layer. Bucket and user policies, with conditions such as source IP or prefix, follow S3’s allow-and-deny evaluation logic rather than being mapped onto POSIX permissions.
- Lifecycle as a native process. Expiration and abort-incomplete-multipart rules run against the object index rather than a periodic filesystem crawl.
- A defined consistency model, with stated guarantees for read-after-write and list-after-write behavior.
- Metadata handled as data. Tags and user metadata are stored and indexed with the object, not squeezed into extended attributes with size limits.
- Performance under mixed load: many concurrent clients writing millions of small objects and multi-terabyte objects, without one starving the other.
How does a gateway-based S3 implementation work?
A gateway-based S3 implementation accepts S3 requests and translates them into operations on another storage system, most often a scale-out file system. The gateway maps a bucket to a directory, a key to a file path, and object metadata to extended attributes or a sidecar database.
This approach has real advantages: it exposes existing file data to S3-speaking applications, reuses a mature data path, and can be built quickly. Many gateways handle core PUT, GET, DELETE and LIST well. The limitations appear at the edges of the API, in mismatches between a hierarchical model and a flat one, and under load.
Partial API coverage
The S3 API has well over a hundred operations. Gateways typically implement the subset their first customers needed, and applications discover the gaps only when they call them: an object lock header returns an error, or a lifecycle configuration is accepted and silently ignored.
Semantic mismatches
File systems have rename and directories that must exist before a file is created; S3 has neither. A gateway must create and remove intermediate directories, decide what a zero-byte key ending in a slash means, and reconcile a key that looks like both a file and a directory. Listing is the most visible case: a prefix-and-delimiter LIST is an index query in a native store but a directory traversal in a gateway, and millions of keys under one prefix can produce listings that take minutes or time out.
Versioning emulated on files
A file system has one inode per path. Gateways emulate versioning by copying old files to a hidden location, appending suffixes, or keeping a separate version database. Under concurrent writes to the same key, version ordering and delete-marker correctness depend on how carefully that emulation was built.
Object lock that is not WORM below the gateway
This is the mismatch with the highest stakes. The gateway can refuse any S3 request that violates a compliance-mode retention, but if the underlying file system still lets an administrator delete the file, the data is not immutable. Ransomware operators with privileged access do not use the S3 API; they use the shell. Object lock is only as strong as the lowest layer that enforces it.
Inconsistent behavior under load, and dual metadata layers
Gateways add their own caching and their own view of directory state, so a client may see an object in one listing and not the next. They also keep object metadata (ETags, version IDs, policies) separately from the file system’s inodes and timestamps. After a crash, a failed migration or an out-of-band change to the file system, the two can disagree, and repairing that state is usually manual.
Where does each approach stand, capability by capability?
The table summarizes typical implementations; individual products vary.
| Capability | S3-native implementation | Gateway-based S3-compatible implementation | Why it matters |
|---|---|---|---|
| Object and bucket model | Flat key namespace, indexed directly | Keys mapped to directory paths and files | Listing cost, key naming limits, slash-terminated keys |
| Multipart upload | Independent parts, atomic completion | Parts staged as temp files, then concatenated or renamed | Large backups and media depend on parallel parts and clean aborts |
| Versioning | Every version a distinct metadata record | Emulated with copies, suffixes or a side database | Point-in-time recovery relies on correct version history |
| Object lock (compliance mode) | Enforced by the engine at every layer | Enforced at the gateway; underlying files may remain deletable | Immutability is meaningful only if no privileged path bypasses it |
| IAM and bucket policies | S3 policy language with condition keys | Often mapped to POSIX users and groups | Least privilege and tenant isolation depend on policy fidelity |
| Lifecycle rules | Executed against the object index | Periodic crawl of the file tree, or unsupported | Retention enforcement and capacity reclamation at scale |
| Listing at scale | Indexed prefix queries, predictable latency | Directory traversal, latency grows with tree size | Backup catalogs and analytics engines list constantly |
| Consistency model | Documented and designed for | Inherited from file system plus gateway cache | Read-after-write needs a guarantee, not a hope |
| Error codes | S3 status and error codes per specification | File system errors translated, sometimes generically | Applications branch on codes; a wrong one turns a retry into a silent failure |
It is worth stating plainly that S3-compatible describes a spectrum, not a defect. Some gateway-based products implement a large share of the API faithfully and serve their intended workloads well. The label simply does not say where a given product sits on that spectrum, and the only reliable way to find out is to test.
What should a practical compatibility test cover?
A meaningful evaluation uses the organization’s actual applications and pushes the system into the conditions where translation is most likely to fail.
API coverage for the real applications
- Capture the S3 calls each application makes and confirm every one is implemented, not merely accepted, including tagging, lifecycle, presigned URLs and conditional requests.
Object lock compliance mode
- Lock an object, then attempt deletion through the S3 API, the administrative console and direct privileged access to the underlying layer. Verify retention cannot be shortened.
Multipart edge cases
- Upload parts out of order from several clients, abort mid-upload and confirm no partial object is visible or consuming capacity. Check ETag calculation against what the client expects.
Listing performance at scale
- Populate one bucket with tens of millions of keys under a few prefixes, measure list latency with and without a delimiter, and repeat while writes are in progress.
IAM policy semantics
- Write a policy with an explicit deny, a source IP condition and a prefix restriction, and confirm each is enforced exactly as S3 evaluation logic specifies.
Error codes
- Trigger missing key, precondition failed, access denied, invalid part and bucket not empty, and verify the HTTP status and S3 error code match the specification.
How Scality RING and ARTESCA implement S3 natively
Scality RING is software-defined, scale-out object and file storage used at petabyte to exabyte scale. Its S3 API is implemented directly against the object store: buckets, keys, versions, multipart parts, policies and lifecycle rules are constructs of the storage engine rather than translations onto a file hierarchy, alongside file protocols for workloads that need them.
Scality ARTESCA is S3 object storage designed for backup, with object lock immutability enforced in the data path and validation with major backup applications. It is intended as an immutable backup target whose retention guarantees hold at every layer of the system, not only at the API.
Whatever the vendor, the practical takeaway is the same: choose an object storage platform by observing how it behaves under the organization’s own applications, data volumes and threat model, not by the presence of the letters S3 on a datasheet.
