Graylog: Handle errors about OpenSearch's read-only indices

This documentation is part of the Handle retention guide. View the full guide here: How to configure log retention.

👋 Welcome to Stackhero documentation

Stackhero provides a fully managed Graylog cloud solution designed for speed and simplicity. You can:

• Rely on an unlimited, dedicated SMTP email server included with your service.
• Apply updates effortlessly with a single click, with no manual intervention needed.
• Use a custom domain name secured by HTTPS (for example, https://logs.your-company.com), giving your team secure, direct access.
• Experience strong performance and security on a private, dedicated infrastructure with no shared resources or noisy neighbors.

Focus on your data, not your tooling: you can get started with Stackhero's Graylog cloud hosting solution in just a few minutes.

Occasionally, OpenSearch may switch to read-only mode and you might encounter errors such as:

1. "Flood stage disk watermark exceeded, all indices on this node will be marked read-only"
2. "FORBIDDEN/12/index read-only / allow delete (api)"

These errors occur as part of OpenSearch's protection mechanism when disk space is critically low. When available disk space drops below 7 GB, OpenSearch sets indices to read-only as a precautionary measure to prevent data corruption.

If you encounter these errors, you have two options:

1. Reconfigure your retention policy to keep fewer logs. After adjusting the policy, delete the oldest index to free up disk space and allow OpenSearch to switch back to read-write mode. Please note that deleting an index means that all data in that index will be lost.
2. Upgrade your instance to one with a larger disk. With a single click in your Stackhero dashboard, the instance will restart with additional disk space and OpenSearch will automatically return to read-write mode.