Object storage as a service is usually something bought from a cloud provider. A growing number of internal IT teams are building the same thing for their own organization: a shared object storage platform, run centrally, that application teams consume through an API without owning any hardware. The motivation is sound. Dozens of teams each buying and running their own storage is wasteful, and sending every workload to a public cloud is not always possible or affordable. Running storage as a service is a different job from running storage, though, and the difference is where most internal platforms struggle.
This post is a guide for internal IT teams: what turns a storage system into a service, what has to be defined before the first tenant arrives and what consumers will expect.
A storage system is not yet a service
A storage system is hardware and software that stores data. A service is a promise made to consumers about what they get, how they get it and what it costs. The gap between the two is everything a public cloud provider wraps around its storage and an internal team has to supply itself.
| Element | What consumers expect | What the platform team has to build |
|---|---|---|
| Onboarding | An account and credentials within hours | A request process, automated account creation |
| Self-service | Create buckets and keys without a ticket | Delegated administration per tenant, guardrails |
| Service classes | A small menu with clear differences | Protection policies, locations and rates behind each class |
| Service levels | Published availability, durability and support hours | Monitoring that measures them, reporting |
| Isolation | No other tenant can see or affect their data | Multi-tenant accounts, quotas, policy boundaries |
| Usage and cost | A monthly figure they can reconcile | Metering per tenant, showback or chargeback |
| Documentation | Endpoint, supported API operations, examples | A maintained service description |
| Exit | A way to get data out | Standard API, no proprietary lock on the data |
Start with the service catalog
The first deliverable is a short description of what is on offer. Two or three service classes are enough to begin with, such as a standard class at one site, a resilient class across sites and an immutable class for backup and records. Each class states its protection, its location, any retention behavior and its rate. Consumers choose a class. They do not choose erasure coding parameters.
Keeping the catalog small is deliberate. Every additional class is something to explain, to monitor and to price. The post on storage service levels covers how to define levels that can be measured and met.
Tenancy and self-service
The unit of consumption should be a tenant account with its own administrators, users, keys and buckets. Within that account, the consuming team manages itself: it creates buckets, issues and rotates access keys and writes its own bucket policies. The platform team sets the outer limits, including the service classes available to the tenant and its quota, and stays out of day-to-day requests.
Self-service is what lets a small platform team support many consumers. It also requires guardrails, since tenants will make mistakes. Sensible defaults, such as blocking public access unless explicitly requested and enforcing encryption in transit, prevent the most common ones. The posts on S3 access policies and S3 bucket design give consumers a starting point that the service documentation can link to.
Metering and money
A shared service without cost visibility fills up. Metering per tenant, with capacity, object count and noncurrent versions reported monthly, is the minimum. Whether that becomes a real charge or a showback report depends on how the organization funds shared infrastructure. Either way, the rate should cover the full cost of the platform, including headroom and staff, and hold steady for a budget year. The post on storage quotas and chargeback goes through the options.
Capacity is the provider's problem now
Consumers of a service do not forecast for the platform. They arrive with a project and expect space to exist. The platform team therefore has to plan capacity across all tenants, hold enough headroom to absorb an unexpected onboarding and have a procurement path that can add capacity in the time available. Hardware lead times make this the part of the job with the least slack.
Two practices help. Asking each tenant for a rough twelve-month estimate at onboarding, and comparing actual growth with it quarterly, gives an aggregate forecast that is good enough. And choosing a platform that expands by adding nodes to a running system means capacity can follow demand in increments. The post on object storage expansion describes what an expansion involves.
Operating as a provider
Running a service means changes are announced, incidents are communicated and maintenance does not interrupt consumers. A platform that can be upgraded node by node while serving requests, as covered in object storage rolling upgrades, makes that achievable without negotiated outage windows. Monitoring has to look at the service from the consumer's side as well: error rates and latency at the S3 endpoint, per tenant where possible, in addition to the health of drives and nodes.
Support needs a front door. A named channel, stated hours and a clear line between what the platform team handles and what belongs to the tenant, such as application-side S3 errors, saves a great deal of misdirected effort.
Where internal services fall short
Internal storage services tend to fail in recognizable ways. Onboarding takes weeks, so teams go around it. The catalog is vague, so every tenant negotiates a special case. There is no metering, so nobody cleans up. Or the platform cannot grow fast enough, so the service is closed to new tenants at the moment it has proven useful. Each of these is an organizational gap more than a technical one, and each can be addressed before launch.
Where Scality RING fits
Scality RING is software-defined object and file storage that exposes an S3 interface and is multi-tenant, with accounts, buckets and policy-based isolation, which provides the tenant boundary an internal service is built on. Erasure coding and replication are selectable per policy, and RING can span multiple sites, so service classes with different protection and locations can be offered from one platform.
RING runs on standard x86 servers, and nodes and servers can be added to a live system without taking the namespace offline, so capacity can follow tenant demand. It emits operational metrics and logs that the operator keeps locally, which supplies the per-account usage data that metering and service reporting need.
Before the first tenant
- Is there a one-page catalog with two or three classes and a rate for each?
- Can a new tenant be onboarded in a day, with self-service from then on?
- Is usage metered per tenant and reported monthly?
- How much headroom is held, and how long does adding capacity take?
- Who do consumers contact, and what is the platform team responsible for?
An internal object storage service succeeds when consuming it is easier than the alternatives. The technology has to be sound. What makes it a service is the catalog, the self-service, the metering and the reliability of the team behind it.