Run a scrolling search
- POST also GET /_search/scroll/{scroll_id}
- POST also GET /_search/scroll
The scroll API is no longer recommend for deep pagination. If you need to preserve the index state while paging through more than 10,000 hits, use the search_after parameter with a point in time (PIT).
The scroll API gets large sets of results from a single scrolling search request.
To get the necessary scroll ID, submit a search API request that includes an argument for the scroll query parameter.
The scroll parameter indicates how long Elasticsearch should retain the search context for the request.
The search response returns a scroll ID in the _scroll_id response body parameter.
You can then use the scroll ID with the scroll API to retrieve the next batch of results for the request.
If the Elasticsearch security features are enabled, the access to the results of a specific scroll ID is restricted to the user or API key that submitted the search.
You can also use the scroll API to specify a new scroll parameter that extends or shortens the retention period for the search context.
Results from a scrolling search reflect the state of the index at the time of the initial search request. Subsequent indexing or document changes only affect later search and scroll requests.
Required privileges
- Index privileges read
Response
-
tooknumberrequired -
The number of milliseconds it took Elasticsearch to run the request. This value is calculated by measuring the time elapsed between receipt of a request on the coordinating node and the time at which the coordinating node is ready to send the response. It includes:
- Communication time between the coordinating node and data nodes
- Time the request spends in the search thread pool, queued for execution
- Actual run time
It does not include:
- Time needed to send the request to Elasticsearch
- Time needed to serialize the JSON response
- Time needed to send the response to a client
-
timed_outbooleanrequired -
If
true, the request timed out before completion; returned results may be partial or empty. -
hitsobjectrequired -
The returned documents and metadata.
-
aggregationsmap string to {} Aggregation result -
profileobject -
shardsobject[]required -
aggregationsobject[]required -
childrenobject[] -
Same fields as
aggregationsinshards -
dfsobject -
rescoreunion oneOf[] -
One entry per rescorer, in the order they ran. Omitted when the request has no rescorer.
- One of:
-
time_in_nanosnumberrequired -
The total time spent in the rescorer, including the time spent running its queries.
-
timeunion oneOf -
The time spent in the rescorer, in human-readable units. Only returned when
humanistrue. -
Values:
-10 -
debugmap string to object -
Extra details about the rescorer.
Every rescorer reports
window_size, the number of top documents it was configured to rescore.timed_outis set totrueif the search timed out while the rescorer was running. Rescorers can add their own keys. -
searchesobject[]required -
queryobject[]required -
debugmap string to object -
Extra details about the query.
Queries run by a rescorer include a
rescorerkey with the position of that rescorer in the shard'srescorearray. -
childrenobject[] -
Same fields as
queryinsearches -
requestobject -
When profiling is enabled, the original query source and target indices from the coordinating request.
-
sourceobject -
Original query source from the search request (
SearchSourceBuilderas JSON). -
aggregationsmap string to {} Aggregation -
Defines the aggregations that are run as part of the search request.
-
collapseobject -
Collapses search results the values of the specified field.
-
inner_hitsunion oneOf -
The number of inner hits and their sort order
-
objectorobject[] -
namestring -
The name for the particular inner hit definition in the response. Useful when a search request contains multiple inner hits.
-
collapseobject -
Same fields as
collapseinsource -
highlightobject -
typeunion anyOf -
Supported values include:
plain: Theplainhighlighter uses the standard Lucene highlighterfvh: The fvh highlighter uses the Lucene Fast Vector highlighter.unified: The unified highlighter uses the Lucene Unified Highlighter.
-
Values:
plainfvhunified -
boundary_scannerenum -
Specifies how to break the highlighted fragments: chars, sentence, or word. Only valid for the unified and fvh highlighters. Defaults to
sentencefor theunifiedhighlighter. Defaults tocharsfor thefvhhighlighter.Supported values include:
chars: Use the characters specified byboundary_charsas highlighting boundaries. Theboundary_max_scansetting controls how far to scan for boundary characters. Only valid for thefvhhighlighter.sentence: Break highlighted fragments at the next sentence boundary, as determined by Java’sBreakIterator. You can specify the locale to use withboundary_scanner_locale. When used with theunifiedhighlighter, thesentencescanner splits sentences bigger thanfragment_sizeat the first word boundary next to fragment_size. You can setfragment_sizeto0to never split any sentence.word: Break highlighted fragments at the next word boundary, as determined by Java’sBreakIterator. You can specify the locale to use withboundary_scanner_locale.
-
Values:
charssentenceword -
boundary_scanner_localestring · default: Locale.ROOT -
Controls which locale is used to search for sentence and word boundaries. This parameter takes a form of a language tag, for example:
"en-US","fr-FR","ja-JP". -
fragmenterenum · default: span -
Specifies how text should be broken up in highlight snippets:
simpleorspan. Only valid for theplainhighlighter. -
Values:
simplespan -
highlight_query{} Query -
Highlight matches for a query other than the search query. This is especially useful if you use a rescore query because those are not taken into account by highlighting by default.
-
max_analyzed_offsetnumber -
If set to a non-negative value, highlighting stops at this defined maximum limit. The rest of the text is not processed, thus not highlighted and no error is returned The
max_analyzed_offsetquery setting does not override theindex.highlight.max_analyzed_offsetsetting, which prevails when it’s set to lower value than the query setting. -
no_match_sizenumber · default: 0 -
The amount of text you want to return from the beginning of the field if there are no matching fragments to highlight.
-
number_of_fragmentsnumber · default: 5 -
The maximum number of fragments to return. If the number of fragments is set to
0, no fragments are returned. Instead, the entire field contents are highlighted and returned. This can be handy when you need to highlight short texts such as a title or address, but fragmentation is not required. Ifnumber_of_fragmentsis0,fragment_sizeis ignored. -
orderenum · default: none -
Sorts highlighted fragments by score when set to
score. By default, fragments will be output in the order they appear in the field (order:none). Setting this option toscorewill output the most relevant fragments first. Each highlighter applies its own logic to compute relevancy scores. -
Values:
score -
phrase_limitnumber · default: 256 -
Controls the number of matching phrases in a document that are considered. Prevents the
fvhhighlighter from analyzing too many phrases and consuming too much memory. When usingmatched_fields,phrase_limitphrases per matched field are considered. Raising the limit increases query time and consumes more memory. Only supported by thefvhhighlighter. -
post_tagsstring[] -
Use in conjunction with
pre_tagsto define the HTML tags to use for the highlighted text. By default, highlighted text is wrapped in<em>and</em>tags. -
pre_tagsstring[] -
Use in conjunction with
post_tagsto define the HTML tags to use for the highlighted text. By default, highlighted text is wrapped in<em>and</em>tags. -
sortunion oneOf -
How the inner hits should be sorted per
inner_hits. By default, inner hits are sorted by score. -
_sourceunion oneOf - One of:
-
explainboolean · default: false -
If
true, the request returns detailed information about score computation as part of a hit. -
fromnumber · default: 0 -
The starting document offset, which must be non-negative. By default, you cannot page through more than 10,000 hits using the
fromandsizeparameters. To page through more hits, use thesearch_afterparameter. -
highlightobject -
Specifies the highlighter to use for retrieving highlighted snippets from one or more fields in your search results.
-
Same fields as
highlightininner_hits - Reference
-
track_total_hitsboolean | number · default: 10000 -
Number of hits matching the query to count accurately. If
true, the exact number of hits is returned at the cost of some performance. Iffalse, the response does not include the total number of hits matching the query. -
indices_booststring to number[] -
Boost the
_scoreof documents from specified indices. The boost value is the factor by which scores are multiplied. A boost value greater than1.0increases the score. A boost value between0and1.0decreases the score. - Reference
-
docvalue_fieldsobject[] -
An array of wildcard (
*) field patterns. The request returns doc values for field names matching these patterns in thehits.fieldsproperty of the response. - Reference
-
knnunion oneOf -
The approximate kNN search to run.
-
objectorobject[] - Reference
-
query_vector_builderobject -
The query vector builder. You must provide a query_vector_builder or query_vector, but not both.
-
visit_percentagenumber -
The percentage of vectors to explore per shard while doing knn search with bbq_disk
-
inner_hitsobject -
If defined, each search hit will contain inner hits.
- Reference
-
namestring -
The name for the particular inner hit definition in the response. Useful when a search request contains multiple inner hits.
-
highlightobject -
Same fields as
highlightininner_hits -
sortunion oneOf -
How the inner hits should be sorted per
inner_hits. By default, inner hits are sorted by score. -
Same options as
sortininner_hits -
_sourceunion oneOf - One of:
-
rankobject -
The Reciprocal Rank Fusion (RRF) to use.
-
min_scorenumber -
The minimum
_scorefor matching documents. Documents with a lower_scoreare not included in search results or results collected by aggregations. -
post_filter{} Query -
Use the
post_filterparameter to filter search results. The search hits are filtered after the aggregations are calculated. A post filter has no impact on the aggregation results. -
profileboolean · default: false -
Set to
trueto return detailed timing information about the execution of individual components in a search request. NOTE: This is a debugging tool and adds significant overhead to search execution. - Reference
-
query{} Query -
The search definition using the Query DSL.
-
rescoreunion oneOf -
Can be used to improve precision by reordering just the top (for example 100 - 500) documents returned by the
queryandpost_filterphases. -
objectorobject[] - Reference
-
queryobject -
rescore_query{} Queryrequired -
The query to use for rescoring. This query is only run on the Top-K results returned by the
queryandpost_filterphases. -
query_weightnumber · default: 1 -
Relative importance of the original query versus the rescore query.
-
rescore_query_weightnumber · default: 1 -
Relative importance of the rescore query versus the original query.
-
score_modeenum · default: total -
Determines how scores are combined.
Supported values include:
avg: Average the original score and the rescore query score.max: Take the max of original score and the rescore query score.min: Take the min of the original score and the rescore query score.multiply: Multiply the original score by the rescore query score. Useful forfunctionquery rescores.total: Add the original score and the rescore query score.
-
Values:
avgmaxminmultiplytotal -
retrieverobject -
A retriever is a specification to describe top documents returned from a search. A retriever replaces other elements of the search API that also return top documents such as
queryandknn. - Reference
-
standardobject -
A retriever that replaces the functionality of a traditional query.
-
min_scorenumber -
Minimum _score for matching documents. Documents with a lower _score are not included in the top documents.
-
query{} Query -
Defines a query to retrieve a set of top documents.
-
sortunion oneOf -
A sort object that that specifies the order of matching documents.
-
Same options as
sortininner_hits -
collapseobject -
Collapses the top documents by a specified key into a single top document per key.
-
Same fields as
collapseinsource -
knnobject -
A retriever that replaces the functionality of a knn search.
-
min_scorenumber -
Minimum _score for matching documents. Documents with a lower _score are not included in the top documents.
-
query_vectorunion oneOf -
Query vector. Must have the same number of dimensions as the vector field you are searching against. You must provide a query_vector_builder or query_vector, but not both.
-
One of:
number[]orstring -
query_vector_builderobject -
Defines a model to build a query vector.
-
Same fields as
query_vector_builderinknn -
rrfobject -
A retriever that produces top documents from reciprocal rank fusion (RRF).
-
min_scorenumber -
Minimum _score for matching documents. Documents with a lower _score are not included in the top documents.
-
retrieversunion oneOf[]required -
A list of child retrievers to specify which sets of returned top documents will have the RRF formula applied to them. Each retriever can optionally include a weight parameter.
- One of:
-
text_similarity_rerankerobject -
A retriever that reranks the top documents based on a reranking model using the InferenceAPI
-
min_scorenumber -
Minimum _score for matching documents. Documents with a lower _score are not included in the top documents.
-
retrieverobjectrequired -
The nested retriever which will produce the first-level results, that will later be used for reranking.
-
Same fields as
retrieverinsource -
rank_window_sizenumber -
This value determines how many documents we will consider from the nested retriever.
-
fieldstringrequired -
The document field to be used for text similarity comparisons. This field should contain the text that will be evaluated against the inference_text.
-
chunk_rescorerobject -
Whether to rescore on only the best matching chunks.
-
chunking_settingsobject -
Chunking settings to apply
-
max_chunk_sizenumber · default: 250required -
The maximum size of a chunk in words. This value cannot be lower than
20(forsentencestrategy) or10(forwordstrategy). This value should not exceed the window size for the associated model. -
overlapnumber · default: 100 -
The number of overlapping words for chunks. It is applicable only to a
wordchunking strategy. This value cannot be higher than half themax_chunk_sizevalue. -
sentence_overlapnumber · default: 1 -
The number of overlapping sentences for chunks. It is applicable only for a
sentencechunking strategy. It can be either1or0. -
separator_groupstring -
Only applicable to the
recursivestrategy and required when using it.Sets a predefined list of separators in the saved chunking settings based on the selected text type. Values can be
markdownorplaintext.Using this parameter is an alternative to manually specifying a custom
separatorslist. -
separatorsstring[] -
Only applicable to the
recursivestrategy and required when using it.A list of strings used as possible split points when chunking text.
Each string can be a plain string or a regular expression (regex) pattern. The system tries each separator in order to split the text, starting from the first item in the list.
After splitting, it attempts to recombine smaller pieces into larger chunks that stay within the
max_chunk_sizelimit, to reduce the total number of chunks generated. -
strategystring · default: sentence -
The chunking strategy:
sentence,word,noneorrecursive.- If
strategyis set torecursive, you must also specify:
max_chunk_size- either
separatorsorseparator_group
Learn more about different chunking strategies in the linked documentation.
- If
- Reference
-
ruleobject -
A retriever that replaces the functionality of a rule query.
-
min_scorenumber -
Minimum _score for matching documents. Documents with a lower _score are not included in the top documents.
-
ruleset_idsunion string | string[]required -
The ruleset IDs containing the rules this retriever is evaluating against.
-
match_criteriaobjectrequired -
The match criteria that will determine if a rule in the provided rulesets should be applied.
-
retrieverobjectrequired -
The retriever whose results rules should be applied to.
-
Same fields as
retrieverinsource -
linearobject -
A retriever that supports the combination of different retrievers through a weighted linear combination.
-
pinnedobject -
A pinned retriever applies pinned documents to the underlying retriever. This retriever will rewrite to a PinnedQueryBuilder.
-
diversifyobject -
A retriever that diversifies the results from its child retriever.
-
min_scorenumber -
Minimum _score for matching documents. Documents with a lower _score are not included in the top documents.
-
retrieverobjectrequired -
The nested retriever whose results will be diversified.
-
Same fields as
retrieverinsource -
rank_window_sizenumber -
The number of top documents from the nested retriever to consider for diversification.
-
query_vector_builderobject -
a dense vector query vector builder to use instead of a static query_vector
-
Same fields as
query_vector_builderinknn -
script_fieldsmap string to object -
Retrieve a script evaluation (based on different fields) for each hit.
-
search_afterunion[] -
Used to retrieve the next page of hits using a set of sort values from the previous page.
- Reference
-
sizenumber · default: 10 -
The number of hits to return, which must not be negative. By default, you cannot page through more than 10,000 hits using the
fromandsizeparameters. To page through more hits, use thesearch_afterproperty. -
sliceobject -
Split a scrolled search into multiple slices that can be consumed independently.
- Reference
-
_sourceunion oneOf -
The source fields that are returned for matching documents. These fields are returned in the
hits._sourceproperty of the search response. If thestored_fieldsproperty is specified, the_sourceproperty defaults tofalse. Otherwise, it defaults totrue. - One of:
- Reference
-
fieldsobject[] -
An array of wildcard (
*) field patterns. The request returns values for field names matching these patterns in thehits.fieldsproperty of the response. -
suggestobject -
Defines a suggester that provides similar looking terms based on a provided text.
- Reference
-
terminate_afternumber · default: 0 -
The maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting.
ImportantUse with caution. Elasticsearch applies this property to each shard handling the request. When possible, let Elasticsearch perform early termination automatically. Avoid specifying this property for requests that target data streams with backing indices across multiple data tiers.
If set to
0(default), the query does not terminate early. -
timeoutstring -
The period of time to wait for a response from each shard. If no response is received before the timeout expires, the request fails and returns an error. Defaults to no timeout.
-
track_scoresboolean · default: false -
If
true, calculate and return document scores, even if the scores are not used for sorting. -
versionboolean · default: false -
If
true, the request returns the document version as part of a hit. -
seq_no_primary_termboolean -
If
true, the request returns sequence number and primary term of the last modification of each hit. - Reference
-
stored_fieldsunion oneOf -
A comma-separated list of stored fields to return as part of a hit. If no fields are specified, no stored fields are included in the response. If this field is specified, the
_sourceproperty defaults tofalse. You can pass_source: trueto return both source fields and stored fields in the search response. -
One of:
string[]orstring - Reference
-
pitobject -
Limit the search to a point in time (PIT). If you provide a PIT, you cannot specify an
<index>in the request path. -
runtime_mappingsobject -
One or more runtime fields in the search request. These fields take precedence over mapped fields with the same name.
- Reference
-
pit_idstring -
An updated identifier for the point-in-time that was searched.
ImportantEach search request against a PIT returns in its response a
pit_idfield which may be different from the identifier you originally supplied. Always use the most recently-received PIT identifier for the next request. If you make concurrent search requests against the same PIT, Elasticsearch can return several differentpit_idvalues in its responses. In that case, use any of these values for later requests, preferring more recently-received values whenever possible. -
_scroll_idstring -
The identifier for the search and its search context. You can use this scroll ID with the scroll API to retrieve the next batch of search results for the request. This property is returned only if the
scrollquery parameter is specified in the request. - Reference