Pagination Strategies

The Symptom

The documentation platform’s search results page supports “Load More” pagination. The first page loads in 15ms. Page 10 loads in 80ms. Page 100 loads in 900ms. Page 500 times out. The infrastructure team increases timeouts. The product team asks why page 500 is slow. The answer is that from=4990, size=10 on a 5-shard index requires each shard to return its top 5,000 results to the coordinating node, which merges 25,000 results to extract 10.

The Internals

from/size

The default pagination mechanism. from skips the first N results. size returns the next M results. The coordinating node requests from + size results from each shard and merges (from + size) * number_of_shards results.

The cost is $O((from + size) \times shards)$ in coordinating node memory and $O(from + size)$ in per-shard computation.

OpenSearch enforces a hard limit: index.max_result_window defaults to 10,000. Requesting from + size > 10,000 returns an error. This limit exists because the resource cost beyond this point is prohibitive for user-facing search.

search_after

Keyset pagination. Instead of skipping N results, the query starts after a specific sort value. Each shard only processes results beyond the cursor position, returning exactly size results. The coordinating node merges size * number_of_shards results regardless of page depth.

The cost is $O(size \times shards)$ for every page, including page 1,000.

Scroll

Creates a snapshot of the search results at a point in time and provides a cursor for iterating through all matching documents. Designed for batch processing (exporting all matching documents), not for user-facing pagination.

The cost is an open search context per scroll, which consumes resources on data nodes (segment files cannot be merged away while a scroll holds a reference to them).

Point in Time (PIT)

PIT creates a lightweight snapshot of the index state and allows search_after queries to paginate consistently against that snapshot, even if documents are added or deleted between pages. PIT is the recommended approach for user-facing deep pagination.

The Implementation

// HARDENED: search_after with Point in Time for consistent deep pagination

public class PaginatedSearch {

    private final OpenSearchClient client;

    public PaginatedSearch(OpenSearchClient client) {
        this.client = client;
    }

    public record SearchPage(
        List<Hit<DocPage>> hits,
        List<FieldValue> searchAfter,
        String pitId,
        boolean hasMore
    ) {}

    public SearchPage firstPage(String index, String tenantId,
            String query, int pageSize) throws IOException {

        // Create a Point in Time
        CreatePitResponse pit = client.createPit(p -> p
            .targetIndexes(List.of(index))
            .keepAlive(Time.of(t -> t.time("5m")))
        );

        return searchWithPit(pit.pitId(), tenantId, query, pageSize, null);
    }

    public SearchPage nextPage(String pitId, String tenantId,
            String query, int pageSize, List<FieldValue> searchAfter)
            throws IOException {

        return searchWithPit(pitId, tenantId, query, pageSize, searchAfter);
    }

    private SearchPage searchWithPit(String pitId, String tenantId,
            String query, int pageSize, List<FieldValue> searchAfter)
            throws IOException {

        SearchRequest.Builder builder = new SearchRequest.Builder()
            .pit(p -> p.id(pitId).keepAlive(Time.of(t -> t.time("5m"))))
            .size(pageSize)
            .sort(SortOptions.of(so -> so.score(sc -> sc.order(SortOrder.Desc))))
            .sort(SortOptions.of(so -> so.field(f -> f.field("_id").order(SortOrder.Asc))))
            .query(q -> q
                .bool(b -> b
                    .filter(f -> f.term(t -> t
                        .field("tenant_id").value(tenantId)))
                    .must(mu -> mu.match(m -> m
                        .field("body").query(query)))
                )
            );

        if (searchAfter != null) {
            builder.searchAfter(searchAfter);
        }

        SearchResponse<DocPage> response = client.search(
            builder.build(), DocPage.class);

        List<Hit<DocPage>> hits = response.hits().hits();
        boolean hasMore = hits.size() == pageSize;

        List<FieldValue> lastSort = hasMore
            ? hits.getLast().sort()
            : null;

        return new SearchPage(hits, lastSort, pitId, hasMore);
    }

    public void closePit(String pitId) throws IOException {
        client.deletePit(d -> d.pitId(List.of(pitId)));
    }
}

// FRAGILE: from/size pagination for a "load all results" feature
// Page 100 with 10 results per page on a 5-shard index:
// Each shard returns 1,000 results. Coordinating node merges 5,000.

for (int page = 0; page < totalPages; page++) {
    SearchRequest request = SearchRequest.of(s -> s
        .index("docs-v1")
        .from(page * PAGE_SIZE)
        .size(PAGE_SIZE)
        .query(q -> q.match(m -> m.field("body").query(userQuery)))
    );
    // Latency grows linearly with page number
}

The Measurement

Page Number	from/size Latency	search_after Latency	search_after + PIT Latency
1	12ms	12ms	14ms
10	18ms	13ms	15ms
50	55ms	12ms	14ms
100	95ms	13ms	15ms
500	480ms	12ms	14ms
1000	Error (exceeds max_result_window)	13ms	15ms

The search_after latency is constant regardless of page depth. The PIT adds ~2ms of overhead for maintaining the consistent snapshot.

The Decision Rule

Use from/size for the first 10 pages of user-facing search results (up to from=100). The overhead is negligible, and the implementation is simpler than search_after.

Use search_after with PIT for any pagination beyond page 10 or for any use case that requires consistent results across pages (no documents appearing on two pages or missing between pages).

Use the Scroll API only for batch export operations that iterate through all matching documents. Never use Scroll for user-facing pagination. Scroll contexts hold segment references and prevent segment merges, consuming resources proportionally to the number of open scrolls.