Graylog: Handling errors related to OpenSearch read-only indices

This documentation is part of the Manage retention guide. You can view the complete guide here: How to configure log retention.

👋 Welcome to Stackhero documentation

Stackhero offers a fully managed Graylog cloud solution designed for speed and simplicity. You can:

• Count on an unlimited, dedicated SMTP email server included with your service.
• Apply updates easily with a single click, with no manual intervention required.
• Use a custom domain name secured by HTTPS (for example, https://logs.your-company.com), providing your team with secure, direct access.
• Benefit from strong performance and security on a private, dedicated infrastructure with no shared resources or noisy neighbours.

Focus on your data, not your tools: you can get started with Stackhero's Graylog cloud hosting solution in just a few minutes.

Sometimes, OpenSearch may switch to read-only mode and you may encounter errors such as:

1. "Flood stage disk watermark exceeded, all indices on this node will be marked read-only"
2. "FORBIDDEN/12/index read-only / allow delete (api)"

These errors occur as part of OpenSearch's protection mechanism when disk space becomes critically low. When available disk space drops below 7 GB, OpenSearch sets indices to read-only as a precaution to prevent data corruption.

If you encounter these errors, you have two options:

1. Reconfigure your retention policy to keep fewer logs. After adjusting the policy, delete the oldest index to free up disk space and allow OpenSearch to return to read-write mode. Please note that deleting an index will permanently erase all data it contains.
2. Upgrade to an instance with a larger disk. With a single click from your Stackhero dashboard, the instance will restart with more disk space and OpenSearch will automatically return to read-write mode.