Scality & Splunk
Big Data Indexing and Search
Splunk is a software solution for monitoring and searching machine-generated data via a web interface.
Generally the way it works is data is slurped and indexed. Traditionally it’s categorized into Hot, Warm, Cold, Frozen. Hot data is typically on SSD, Warm and Cold is somewhere between, most likely on a NAS. All of this is on mountable file systems, typically SSD or fast spinning drives. Frozen is historic and not indexed and typically resides on tape for ultra low cost, but also value is lost since it’s not searchable and not easily retrievable. Our approach was to support an NFS based solution but the world has moved towards S3 and so has Splunk, thus enabling use of a scale-out on-prem object storage solution.
Product: Splunk Enterprise
Category: Big Data Indexing and Search
Software Version: 8.0
An indexer capability that provides a way to use remote object stores, such as Amazon S3, to store indexed data
As the amount of data in local storage continues to grow. While local storage costs might not be a significant issue for a small deployment, you should reconsider your use of local storage as your deployment scales over time.
If you are using indexer clusters to take advantage of features such as data recovery and disaster recovery. Through SmartStore, you can achieve these aims through the native capabilities of the remote store, without the need to store large amounts of redundant data on local storage.
Through SmartStore, you can eliminate much of the cluster management overhead. In particular, you can greatly reduce the scale of time-consuming activities such as offlining peer nodes, data rebalancing, and bucket fixup, because most of the data no longer resides on the peer nodes.
ABOUT SCALITY RING
Scality’s principal product is a scale-out object storage software platform known as the RING. Scality’s RING delivers petabyte-scale software-defined storage designed to use commodity hardware and characterized by cost-effective scaling, performance, and auto-recovery. Scality’s RING is a multitier architecture and can scale up to thousands of servers and many hundreds of petabytes under a single global namespace. It allows customers to deploy both performance-optimized and capacity-optimized storage, varied data durability levels, and small to large object and file support in a single global namespace.