Graylog: Handle error "failed to parse field [XXXX] of type [YYYY]"

This documentation is part of the Getting started guide. View the full guide here: How to get started with Graylog.

👋 Welcome to Stackhero documentation

Stackhero provides a fully managed Graylog cloud solution designed for speed and simplicity. You can:

• Rely on an unlimited, dedicated SMTP email server included with your service.
• Apply updates effortlessly with a single click, with no manual intervention needed.
• Use a custom domain name secured by HTTPS (for example, https://logs.your-company.com), giving your team secure, direct access.
• Experience strong performance and security on a private, dedicated infrastructure with no shared resources or noisy neighbors.

Focus on your data, not your tooling: you can get started with Stackhero's Graylog cloud hosting solution in just a few minutes.

You might encounter an error such as:

org.opensearch.index.mapper.MapperParsingException: failed to parse field [time] of type [long] in document with id 'xxxx'

You can view this error in the logs provided in the Stackhero dashboard or in the Graylog admin panel under System > Overview > Indexer failures.

This error indicates that a log was sent with a value for the field time that does not match the expected type (in this case, a numeric value of type "long"). Graylog leverages the dynamic mapping feature of OpenSearch. When a log is sent for the first time, OpenSearch attempts to guess the field types. For example, if a log includes the field time with the numeric value 1234, OpenSearch defines it as a numeric field. If another log is sent with the field time set as "abcd", a string, OpenSearch will reject it because it expects a numeric value.

Remember that the field name time is used for illustration only. It can be any field name and type.

To resolve this issue, you need to redefine the type that OpenSearch expects. For more information, please refer to the official Graylog documentation.

Update the OpenSearch mapping

Before proceeding, enable OpenSearch access in the Stackhero dashboard. Navigate to your Graylog service and click on the "Configure" button to activate OpenSearch access.

Be careful with these changes as incorrect configurations can block your OpenSearch cluster and potentially result in data loss. If you are uncertain, do not proceed.

1. Define your new mapping. In this example, we redefine the field time to be of type string. You can find available types in the OpenSearch field datatypes documentation.

2. Save the following content to a file named graylog-custom-mapping.json:

   {
     "template": "graylog_*",
     "mappings": {
       "message": {
         "properties": {
           "time": {
             "type": "string",
             "index": "not_analyzed"
           }
         }
       }
     }
   }
   

3. Post this file using the following curl command (replace <XXXXXX>.stackhero-network.com with your instance domain name):

   curl -u 'admin' -X PUT -d @'graylog-custom-mapping.json' -H 'Content-Type: application/json' 'https://<XXXXXX>.stackhero-network.com/opensearch/_template/graylog-custom-mapping?pretty'
   

   You should receive a response like:

   {
     "acknowledged": true
   }
   

4. Finally, verify that the mapping has been updated with this command (replace the domain as needed):

   curl -u 'admin' -X GET 'https://<XXXXXX>.stackhero-network.com/opensearch/graylog_deflector/_mapping?pretty'