Lazy Sequences vs Eager Collections — Performance Analysis

Eager vs Lazy: Seeing the Difference

Consider a concrete pipeline — finding the first three engineers with salaries above 100K, and extracting their names:

data class Employee(val name: String, val department: String, val salary: Double)

// Eager: collection pipeline
val result = employees
    .filter { it.department == "Engineering" }
    .map { it.salary }
    .filter { it > 100_000.0 }
    .take(3)

Every step here creates a new List. If employees contains 50,000 entries and 10,000 are engineers, the first filter allocates a List<Employee> of 10,000 elements. The map allocates another List<Double> of 10,000 elements. The second filter allocates yet another list. Finally, take(3) extracts three elements — after three full passes over the data.

Now the lazy version:

// Lazy: sequence pipeline
val result = employees.asSequence()
    .filter { it.department == "Engineering" }
    .map { it.salary }
    .filter { it > 100_000.0 }
    .take(3)
    .toList()  // terminal operation

No intermediate lists are created. Each employee flows through the entire chain before the next one enters. Once three qualifying employees are found, processing stops — even if 49,000 employees remain unexamined.

The evaluation model is fundamentally different:

• Eager (Collections): Horizontal processing. Complete step 1 for ALL elements, then complete step 2 for ALL remaining elements, and so on.
• Lazy (Sequences): Vertical processing. Complete ALL steps for element 1, then ALL steps for element 2, and so on.

How Sequences Work Internally

A Sequence<T> is a remarkably minimal interface:

public interface Sequence<out T> {
    public operator fun iterator(): Iterator<T>
}

That’s it. One method. The power comes from how intermediate operations compose. When you call .filter() on a sequence, it doesn’t execute anything — it returns a new Sequence that wraps the original:

// Simplified from kotlin-stdlib source
public fun <T> Sequence<T>.filter(
    predicate: (T) -> Boolean
): Sequence<T> {
    return FilteringSequence(this, sendWhen = true, predicate)
}

internal class FilteringSequence<T>(
    private val sequence: Sequence<T>,
    private val sendWhen: Boolean,
    private val predicate: (T) -> Boolean
) : Sequence<T> {
    override fun iterator(): Iterator<T> = object : Iterator<T> {
        val iterator = sequence.iterator()
        var nextState: Int = -1  // -1 = unknown, 0 = done, 1 = ready
        var nextItem: T? = null

        private fun calcNext() {
            while (iterator.hasNext()) {
                val item = iterator.next()
                if (predicate(item) == sendWhen) {
                    nextItem = item
                    nextState = 1
                    return
                }
            }
            nextState = 0
        }

        override fun next(): T {
            if (nextState == -1) calcNext()
            if (nextState == 0) throw NoSuchElementException()
            val result = nextItem as T
            nextItem = null
            nextState = -1
            return result
        }

        override fun hasNext(): Boolean {
            if (nextState == -1) calcNext()
            return nextState == 1
        }
    }
}

Each intermediate operation (filter, map, flatMap, etc.) creates a wrapper Sequence that delegates to the previous one. The chain forms a linked structure of sequence wrappers. Nothing executes until a terminal operation (toList(), first(), count(), forEach(), etc.) calls iterator() on the outermost wrapper, which cascades down through the chain.

This means employees.asSequence().filter { ... }.map { ... }.filter { ... } is three nested objects — no data processing has occurred.

Benchmark Results

Measured using JMH (Java Microbenchmark Harness) on JDK 17, with a pipeline of filter → map → filter → take(10):

Collection Size	Eager Collection	Sequence	Speedup
10	0.08 μs	0.12 μs	0.67× (slower)
100	0.9 μs	0.7 μs	1.3×
1,000	8.5 μs	2.1 μs	4.0×
10,000	85 μs	3.8 μs	22×
100,000	1.2 ms	4.2 μs	285×
1,000,000	14 ms	4.5 μs	3,111×

The sequence time barely grows because take(10) short-circuits after finding 10 matching elements, regardless of input size. The collection time scales linearly because every step must process all elements.

Without take() — processing all elements through filter → map → filter → toList():

Collection Size	Eager Collection	Sequence	Speedup
10	0.08 μs	0.15 μs	0.53× (slower)
100	0.9 μs	1.1 μs	0.82× (slower)
1,000	8.5 μs	7.8 μs	1.09×
10,000	85 μs	52 μs	1.6×
100,000	1.2 ms	0.6 ms	2.0×

Without short-circuiting, the advantage comes purely from reduced memory allocation. At small sizes, the overhead of sequence wrapper objects and virtual dispatch exceeds the allocation savings.

The sorted() Problem

Not all sequence operations maintain laziness. sorted() is a stateful intermediate operation — it must consume the entire upstream sequence before emitting any element:

val result = hugeList.asSequence()
    .map { transform(it) }
    .sorted()              // Buffers ALL elements internally
    .take(5)
    .toList()

When the pipeline reaches sorted(), it collects all mapped elements into an internal list, sorts it, then emits elements lazily to downstream operations. You get lazy evaluation before sorted() and after it, but sorted() itself is a full materialization point.

Other operations that break laziness: sortedBy(), sortedWith(), sortedDescending(), shuffled(), distinct() (needs to track seen elements, though it doesn’t buffer all at once), and chunked()/windowed() (partial buffering).

Building Sequences from Scratch

Beyond converting existing collections, you can create sequences directly:

sequenceOf() — Literal Elements

val seq = sequenceOf(1, 2, 3, 4, 5)

generateSequence() — Seed + Next Function

// Fibonacci sequence — infinite, evaluated on demand
val fibonacci = generateSequence(Pair(0L, 1L)) { (a, b) -> Pair(b, a + b) }
    .map { it.first }

println(fibonacci.take(10).toList())  // [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]

// File system traversal — stops when parent is null
fun File.ancestors(): Sequence<File> = generateSequence(this) { it.parentFile }

generateSequence terminates when the lambda returns null. For truly infinite sequences, ensure your downstream pipeline has a terminal condition like take() or first().

sequence { } Builder with yield() and yieldAll()

The sequence builder uses a restricted coroutine (a suspension mechanism limited to the sequence context) to produce values on demand:

val primes = sequence {
    yield(2)
    var candidate = 3
    while (true) {
        if ((2 until candidate).none { candidate % it == 0 }) {
            yield(candidate)
        }
        candidate += 2
    }
}

println(primes.take(10).toList())  // [2, 3, 5, 7, 11, 13, 17, 19, 23, 29]

yield() suspends the builder, emits the value, and resumes only when the consumer requests the next element. yieldAll() emits all elements from an iterator, iterable, or another sequence:

val combined = sequence {
    yieldAll(listOf(1, 2, 3))
    yield(4)
    yieldAll(generateSequence(5) { it + 1 })  // infinite tail
}

println(combined.take(8).toList())  // [1, 2, 3, 4, 5, 6, 7, 8]

The sequence builder is restricted — you cannot call arbitrary suspending functions inside it. Only yield() and yieldAll() are allowed. For general-purpose asynchronous lazy streams, that’s what Flow is for (covered in CH7).

Comparison with Java Streams

Java introduced the Stream API in Java 8 with similar goals. Here’s how they compare:

// Java Stream
List<String> result = employees.stream()
    .filter(e -> e.getDepartment().equals("Engineering"))
    .map(Employee::getName)
    .sorted()
    .limit(10)
    .collect(Collectors.toList());

// Kotlin Sequence
val result = employees.asSequence()
    .filter { it.department == "Engineering" }
    .map { it.name }
    .sorted()
    .take(10)
    .toList()

Key Differences

Single-use vs reusable. Java Streams are single-use — calling a terminal operation consumes the stream, and attempting to reuse it throws IllegalStateException. Kotlin sequences can be iterated multiple times (unless the underlying source is single-use, like reading from a file).

val seq = listOf(1, 2, 3).asSequence().filter { it > 1 }
println(seq.toList())  // [2, 3]
println(seq.toList())  // [2, 3] — works fine

Stream<Integer> stream = List.of(1, 2, 3).stream().filter(i -> i > 1);
stream.collect(Collectors.toList());  // [2, 3]
stream.collect(Collectors.toList());  // IllegalStateException!

Parallel execution. Java Streams offer parallelStream() and .parallel() out of the box. Kotlin sequences have no built-in parallel processing — they are strictly sequential. If you need parallelism in Kotlin, you use coroutines with Flow or Java’s parallel streams directly (Kotlin interoperates with them seamlessly).

API surface. Kotlin’s collection/sequence API is significantly larger — operations like groupBy, associate, zip, windowed, chunked, partition, flatMapIndexed, and many others are available without importing external collectors. In Java, complex aggregations often require writing custom Collector implementations.

Collector vs terminal operations. Java Streams require Collectors for most terminal operations. Kotlin sequences use direct terminal functions: toList(), toSet(), toMap(), first(), count(), sum(), joinToString(). The Kotlin approach is less verbose but also less extensible — Java’s Collector interface allows arbitrary mutable reduction strategies.

Primitive specialization. Java provides IntStream, LongStream, DoubleStream to avoid boxing overhead. Kotlin has no primitive sequence specialization — all elements are boxed. For performance-critical numeric pipelines, consider using Java streams directly or working with arrays via IntArray.asSequence().

Decision Framework: When to Use What

Use eager collection operations when:

• The collection has fewer than ~100 elements
• You apply only one or two transformations
• You need the intermediate results for other computations
• You need random access (get(index)) on intermediate results
• You’re inside a non-performance-critical code path

Use sequences when:

• The pipeline has three or more chained operations
• You use short-circuiting terminals: first(), find(), take(), any(), none()
• The source collection is large (thousands+ elements)
• Memory allocation pressure is a concern (tight loops, high-throughput paths)
• You’re building an infinite or very large computed series

Use Java Streams when:

• You need parallel processing without coroutines
• You need primitive specialization for numeric computation
• You’re working at a Java interop boundary and the consuming code expects a Stream

The common mistake is reaching for sequences everywhere. For a list.filter { ... }.map { ... } with 20 elements, the collection version is both faster and more readable. Sequences are an optimization tool — apply them where measurement shows they matter.