Loading

source

The source from which to derive the index or snapshot age. Can be one of name, creation_date, or field_stats.

Note

This setting is only used with the age filtertype, or
with the space filtertype when use_age is set to True.

Note

When using the age filtertype, source requires
direction, unit, unit_count,
and additionally, the optional setting, epoch.

name-based ages

Using name as the source tells Curator to look for a timestring within the index or snapshot name, and convert that into an epoch timestamp (epoch implies UTC).

- filtertype: age
  source: name
  direction: older
  timestring: '%Y.%m.%d'
  unit: days
  unit_count: 3
A word about regular expression matching with timestrings

Timestrings are parsed from strftime patterns, like %Y.%m.%d, into regular expressions. For example, %Y is 4 digits, so the regular expression for that looks like \d{{4}}, and %m is 2 digits, so the regular expression is \d{{2}}.

What this means is that a simple timestring to match year and month, %Y.%m will result in a regular expression like this: ^.*\d{{4}}\.\d{{2}}.*$. This pattern will match any 4 digits, followed by a period ., followed by 2 digits, occurring anywhere in the index name. This means it will match monthly indices, like index-2016.12, as well as daily indices, like index-2017.04.01, which may not be the intended behavior.

To compensate for this, when selecting indices matching a subset of another pattern, use a second filter with exclude set to True

- filtertype: pattern
 kind: timestring
 value: '%Y.%m'
- filtertype: pattern
 kind: timestring
 value: '%Y.%m.%d'
 exclude: True

This will prevent the %Y.%m pattern from matching the %Y.%m part of the daily indices.

This applies whether using timestring as a mere pattern match, or as part of date calculations.

creation_date-based ages

creation_date extracts the epoch time of index or snapshot creation.

- filtertype: age
  source: creation_date
  direction: older
  unit: days
  unit_count: 3

field_stats-based ages

Note

source can only be field_stats when filtering indices.

In Curator 5.3 and older, source field_stats uses the Field Stats API to calculate either the min_value or the max_value of the field as the stats_result, and then use that value for age comparisons. In 5.4 and above, even though it is still called field_stats, it uses an aggregation to calculate the same values, as the field_stats API is no longer used in Elasticsearch 6.x and up.

field must be of type date in Elasticsearch.

- filtertype: age
  source: field_stats
  direction: older
  unit: days
  unit_count: 3
  field: '@timestamp'
  stats_result: min_value