Graylog: Understanding indices

This documentation is part of the Handle retention guide. View the full guide here: How to configure log retention.

👋 Welcome to Stackhero documentation

Stackhero provides a fully managed Graylog cloud solution designed for speed and simplicity. You can:

• Rely on an unlimited, dedicated SMTP email server included with your service.
• Apply updates effortlessly with a single click, with no manual intervention needed.
• Use a custom domain name secured by HTTPS (for example, https://logs.your-company.com), giving your team secure, direct access.
• Experience strong performance and security on a private, dedicated infrastructure with no shared resources or noisy neighbors.

Focus on your data, not your tooling: you can get started with Stackhero's Graylog cloud hosting solution in just a few minutes.

Before defining your retention policy, it is important to understand how indices used by Graylog and OpenSearch work. Think of indices as physical containers. Graylog "opens" a container (an index) and places incoming messages inside it. When the quota assigned to that container is exceeded, the container is closed, stored on a shelf, and a new container is started for subsequent messages.

You can set this quota using different criteria:

1. A number of messages: "Keep 20 million messages per container, then start a new one."
2. A time limitation: "Use a container for 10 days, then switch to a new one."
3. A size limitation: "Store 20 GB per container, then move on to a new one."

A maximum number of containers that can be stored on the shelf is also defined. If the number is exceeded, the oldest containers are automatically deleted. For instance, if you set a maximum of 20 containers and have 22 on the shelf, the 2 oldest containers will be removed.

In this analogy, the containers represent the indices, the shelf is OpenSearch, and the maximum number represents the permitted number of indices.