Graylog: Understanding indices

This documentation is part of the Manage retention guide. You can view the complete guide here: How to configure log retention.

👋 Welcome to Stackhero documentation

Stackhero offers a fully managed Graylog cloud solution designed for speed and simplicity. You can:

• Rely on an unlimited, dedicated SMTP email server included with your service.
• Apply updates effortlessly with a single click, with no manual intervention required.
• Use a custom domain name secured by HTTPS (for example, https://logs.your-company.com), providing your team with secure, direct access.
• Benefit from strong performance and security on a private, dedicated infrastructure, with no shared resources or noisy neighbours.

Focus on your data, not your tools: you can get started with Stackhero's Graylog cloud hosting solution in just a few minutes.

Before setting your retention policy, it is important to understand how indices used by Graylog and OpenSearch function. Think of indices as physical containers. Graylog "opens" a container (an index) and places incoming messages inside it. When the quota assigned to that container is exceeded, the container is closed, placed on a shelf, and a new container is started for subsequent messages.

You can set this quota using different criteria:

1. A number of messages: "Keep 20 million messages per container, then start a new one."
2. A time limit: "Use a container for 10 days, then switch to a new one."
3. A size limit: "Store 20 GB per container, then move on to a new one."

A maximum number of containers that can be stored on the shelf is also defined. If this number is exceeded, the oldest containers are automatically deleted. For example, if you set a maximum of 20 containers and have 22 on the shelf, the 2 oldest containers will be removed.

In this analogy, the containers represent the indices, the shelf is OpenSearch, and the maximum number represents the permitted number of indices.