Run Discover and Dashboards queries in the background

Serverless Unavailable Stack Preview 9.2.0

Sometimes you might need to search through large amounts of data, no matter how long the search takes. Consider a threat hunting scenario, where you need to search through years of data.

You can send your long-running searches to the background from Discover or Dashboards and let them run while you continue your work.

You can access your list of background searches at any time to check their status and manage them from the Background searches button in the toolbar.

This feature is disabled by default. You can enable background searches in versions 9.2 and later, or search sessions in versions 9.1 and earlier, by setting data.search.sessions.enabled to true in the kibana.yml configuration file.

The background searches that you run are personal and only visible by you. To use this feature, you must have the following minimum permissions:

You can send a search to the background only after it starts running. Until then, the Send to background button is disabled.

1. Write or edit the query.

2. Select Run (or Refresh if you already ran the query at least once) to start executing the query. At this point, the Send to background button becomes available.

3. Select Send to background. The search is sent to the background and added to the queue of background searches.

You can resume your other tasks, for example start a new search, navigate to a different application, or close the browser. Once the search has completed, a notification informs you and lets you access the search to view its results.

Background searches expire after 7 days. Beyond that period, you must run the search again. You can change this default value by editing the data.search.sessions.defaultExpiration setting.

From the list of background searches, you can reopen and edit any searches, but also extend their validity period or delete them to keep only searches that you care about.

1. Open your list of background searches using one of the following methods:

   • Once a background search is sent to the background, a notification appears to inform you, with a link to open the list of background searches.

   • If you miss the notification or need to access this list at any time, go to Discover or Dashboards and select the Background searches button in the toolbar. This option is only available from version 9.2.

     Tip

     From Discover, you can only view Discover background searches. And from Dashboards, you can only see Dashboards background searches.

   • Open the Background Search management page in Kibana.

2. Find the background search that you want to interact with using the search or status filter options.

   • To open it to view its results and continue your explorations, select its name. Relative dates are converted to absolute dates.
   • To rename it, select the More actions button, then select Edit name. By default, background searches get default names that indicate their execution date and time.
   • To extend its current expiration date by another 7 days, select the More actions button, then select Extend.
   • To delete it, select the More actions button, then select Delete.

Some visualization features do not fully support background searches. When you restore a dashboard, panels with unsupported features won’t load immediately, but instead send out additional data requests, which can take a while to complete. The Your background search is still running warning appears. You can either wait for these additional requests to complete or come back to the dashboard later when all data requests have finished.

A panel on a dashboard can behave like this if one of the following features is used:

Lens

• A top values dimension with an enabled Group other values as "Other" setting. This is configurable in the Advanced section of the dimension.
• An intervals dimension.

Aggregation-based visualizations

• A terms aggregation with an enabled Group other values in separate bucket setting.
• A histogram aggregation.

Maps

• Layers using joins, blended layers, or tracks layers.