# Splitting sequence in chunks based on predicate **URL:** \ **Category:** Libraries\ **Created:** [September 2, 2020, 11:39am UTC](https://discuss.kotlinlang.org/t/splitting-sequence-in-chunks-based-on-predicate/19005 "2020-09-02T11:39:04Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![medium](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.kotlinlang.org/medium/32/10404_2.png) [@medium](https://discuss.kotlinlang.org/u/medium)\ **Post date:** [September 2, 2020, 11:39am UTC](https://discuss.kotlinlang.org/t/splitting-sequence-in-chunks-based-on-predicate/19005/1 "2020-09-02T11:39:04Z")
I want to split a sequence in chunks, but not depending on the number of elements, but on a certain criterion. Think of a sorted list of names, which should be splitted into sub lists depending on the first character. After some research how to write an extension for sequences I came to the following solution: ```auto fun Sequence.chunked(predicate: (T, T) -> Boolean): Sequence> { val underlyingSequence = this if (!underlyingSequence.iterator().hasNext()) return emptySequence() return sequence { val buffer = mutableListOf() var last: T? = null underlyingSequence.forEach { current -> val shouldSplit = last?.let { predicate(it, current) } ?: false last = current if (shouldSplit) { yield(buffer.toList()) buffer.clear() } buffer.add(current) } yield(buffer) } } sequenceOf("Adelheid", "Anna", "Annika", "Berthold", "Bertram", "Borat", "Carl") .chunked { last, current -> last[0] != current[0] } .forEach { println(it) } // prints: // [Adelheid, Anna, Annika] // [Berthold, Bertram, Borat] // [Carl] ``` Do you see anything that could be improved? Since I [doubt](https://discuss.kotlinlang.org/t/sequence-chunk-or-window-with-predicate/7339) that I’m the only one with such a requirement, I could imagine that such a method would be a good addition to the standard library. What do you think? ---
**Author:** ![kyay10](https://avatars.discourse-cdn.com/v4/letter/k/c57346/32.png) [@kyay10](https://discuss.kotlinlang.org/u/kyay10)\ **Post date:** [September 2, 2020, 1:23pm UTC](https://discuss.kotlinlang.org/t/splitting-sequence-in-chunks-based-on-predicate/19005/2 "2020-09-02T13:23:26Z")
Maybe try `groupBy` with a map afterwards? ---
**Author:** ![medium](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.kotlinlang.org/medium/32/10404_2.png) [@medium](https://discuss.kotlinlang.org/u/medium)\ **Post date:** [September 2, 2020, 3:23pm UTC](https://discuss.kotlinlang.org/t/splitting-sequence-in-chunks-based-on-predicate/19005/3 "2020-09-02T15:23:07Z")
I’ve thought about [`groupBy`](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.sequences/group-by.html), but I don’t think it would be a good solution for a large collection, since it needs to hold all elements from the sequence in memory. The whole purpose of sequences in my case is to avoid holding all elements in memory, because the server would crash with an `OutOfMemoryError` then. ---
**Author:** ![medium](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.kotlinlang.org/medium/32/10404_2.png) [@medium](https://discuss.kotlinlang.org/u/medium)\ **Post date:** [September 3, 2020, 11:23am UTC](https://discuss.kotlinlang.org/t/splitting-sequence-in-chunks-based-on-predicate/19005/4 "2020-09-03T11:23:14Z")
I’ve created an issue to promote this function for the standard library. [https://youtrack.jetbrains.com/issue/KT-41648](https://youtrack.jetbrains.com/issue/KT-41648) ---
**Author:** ![medium](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.kotlinlang.org/medium/32/10404_2.png) [@medium](https://discuss.kotlinlang.org/u/medium)\ **Post date:** [September 3, 2020, 3:42pm UTC](https://discuss.kotlinlang.org/t/splitting-sequence-in-chunks-based-on-predicate/19005/5 "2020-09-03T15:42:25Z")
The code above works in my artificial test but not in practice. I get the following exception: > java.lang.IllegalStateException: stream has already been operated upon or closed The problem is the `hasNext` check, since that already starts the iteration and then the `foreach` iterates a second time, but the concrete sequence returned from a JPA query can be iterated only once. The improved version: ```kotlin fun Sequence.chunked(predicate: (T, T) -> Boolean): Sequence> { val underlyingSequence = this return sequence { val buffer = mutableListOf() var last: T? = null for (current in underlyingSequence) { val shouldSplit = last?.let { predicate(it, current) } ?: false if (shouldSplit) { yield(buffer.toList()) buffer.clear() } buffer.add(current) last = current } if (buffer.isNotEmpty()) { yield(buffer) } } } ``` The last `if` suppresses yielding an empty list. If the underlying sequence was empty it should stay empty and `chunked` shouldn’t create an artificial and superfluous empty list. ---
**Author:** ![ilya.gorbunov](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.kotlinlang.org/ilya.gorbunov/32/1645_2.png) [@ilya.gorbunov](https://discuss.kotlinlang.org/u/ilya.gorbunov)\ **Post date:** [September 3, 2020, 9:25pm UTC](https://discuss.kotlinlang.org/t/splitting-sequence-in-chunks-based-on-predicate/19005/6 "2020-09-03T21:25:34Z")
What concerns me in the proposed signature is that it covers only that particular use case when you can determine in the predicate whether or not to start a new chunk based on the current and previous elements. I think other cases of custom splitting should be considered in order to decide on the signature. One example that comes to mind is when you want to split weighted items in buckets so that total weight of items in one bucket is not greater than the given value: [Kotlin - How to split list of strings based on the total length of characters - Stack Overflow](https://stackoverflow.com/questions/63633977/kotlin-how-to-split-list-of-strings-based-on-the-total-length-of-characters/63634737#63634737) ---
**Author:** ![medium](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.kotlinlang.org/medium/32/10404_2.png) [@medium](https://discuss.kotlinlang.org/u/medium)\ **Post date:** [September 4, 2020, 6:00am UTC](https://discuss.kotlinlang.org/t/splitting-sequence-in-chunks-based-on-predicate/19005/7 "2020-09-04T06:00:36Z")
@ilya.gorbunov Yes, my use case might be to specific for a general purpose function in the standard library. What do you think of a signature like the following? ```kotlin fun Sequence.chunked( windowSize: Int, chunkMaxSize: Int, predicate: (List) -> Boolean ): Sequence> ``` The `predicate` would receive a window that would have a maximum of elements given by `windowSize`, and the resulting chunks would have a maximum size of `chunkMaxSize` where `-1` (or maybe `null`) would mean “unlimited”. ---
**Author:** ![medium](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.kotlinlang.org/medium/32/10404_2.png) [@medium](https://discuss.kotlinlang.org/u/medium)\ **Post date:** [September 8, 2020, 9:51am UTC](https://discuss.kotlinlang.org/t/splitting-sequence-in-chunks-based-on-predicate/19005/8 "2020-09-08T09:51:35Z")
@ilya.gorbunov Reactor provides the method [`bufferUntilChanged`](https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Flux.html#bufferUntilChanged-java.util.function.Function-) for this use case. That would be even more restricted than my first version, but seems to be good enough for many use cases. Reactor is, by the way, a great source of inspiration for methods that would be useful for Kotlin Flow. ---
**Author:** ![fvasco](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.kotlinlang.org/fvasco/32/1961_2.png) [@fvasco](https://discuss.kotlinlang.org/u/fvasco)\ **Post date:** [September 8, 2020, 12:14pm UTC](https://discuss.kotlinlang.org/t/splitting-sequence-in-chunks-based-on-predicate/19005/9 "2020-09-08T12:14:16Z")
Please file an issue here: [Issues · Kotlin/kotlinx.coroutines · GitHub](https://github.com/Kotlin/kotlinx.coroutines/issues) ---
**Author:** ![medium](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.kotlinlang.org/medium/32/10404_2.png) [@medium](https://discuss.kotlinlang.org/u/medium)\ **Post date:** [September 8, 2020, 2:36pm UTC](https://discuss.kotlinlang.org/t/splitting-sequence-in-chunks-based-on-predicate/19005/10 "2020-09-08T14:36:36Z")
I’ve actually thought of filing an issue for each operator of `Flux`, or at least for each operator name (to not become overwhelmed by all signature variants). What do you think? Flow seems to be a great tool in principle. Remembering a benchmark it should be significantly faster than Reactor or RxJava, but it has very few operators at the moment. ---
**Author:** ![fvasco](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.kotlinlang.org/fvasco/32/1961_2.png) [@fvasco](https://discuss.kotlinlang.org/u/fvasco)\ **Post date:** [September 8, 2020, 2:47pm UTC](https://discuss.kotlinlang.org/t/splitting-sequence-in-chunks-based-on-predicate/19005/11 "2020-09-08T14:47:34Z")
See **[qwwdfsad](https://github.com/qwwdfsad)**’s comment: > > > All the currently provided channel abstractions in \`kotlinx.coroutines\` are \_hot…\_. The data is being produced regardless of the presence of subscriber. This is good for data sources and applications that are inherently hot, like incoming network and UI-events. > > However, hot streams are not an ideal solution for cases where data stream is produced on demand. Consider, for example, the following simple code that produces \`ReceiveChannel\\`: > > > \`\`\` > produce\ { > while (true) { > val x = computeNextValue() > send(x) > } > } > \`\`\` > > One obvious downside is the \`computeNextValue()\` is invoked before \`send\`, so even when receiver is not ready, the next value gets computed. Of course, it gets suspended in \`send\` if there is no receiver, but it is not as lazy as you get with cold reactive Publisher/Observable/Flowable/Flux/Flow. > > We need the abstraction for cold streams in \`kotlinx.coroutines\` that is going to be just as lazy, computing data in "push" mode versus "pull" mode of hot channels that we have now. > > There are the following related discussions: > \* https://github.com/reactor/reactor-core/issues/979#issuecomment-351770494 describes preliminary performance test that indicates that "push" mode is much faster for same-thread cases. > \* https://github.com/Kotlin/kotlinx.coroutines/issues/113 (SPSC channels) seems to get superseded by the support of cold streams. Keep care of already filed issues. ---
**Author:** ![medium](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.kotlinlang.org/medium/32/10404_2.png) [@medium](https://discuss.kotlinlang.org/u/medium)\ **Post date:** [September 10, 2020, 4:16pm UTC](https://discuss.kotlinlang.org/t/splitting-sequence-in-chunks-based-on-predicate/19005/12 "2020-09-10T16:16:14Z")
My idea to file an issue for (more or less) every Flux operator is against the goals of the Flow developers. Flow should stay small and should only have very relevant operators. Maybe it is better to create a separate operator lib, like the creator or RxJava did: > **[GitHub - akarnokd/kotlin-flow-extensions: Extensions to the Kotlin Flow library.](https://github.com/akarnokd/kotlin-flow-extensions)** > > Extensions to the Kotlin Flow library. Contribute to akarnokd/kotlin-flow-extensions development by creating an account on GitHub.