arrayOf unnatural

The argument that “it’s a known pattern” is valid. Lots of other languages do have array literals and they work well in them. Some languages are passing up collection literals and adding in functions to create them (e.x. Map.of in Java).

At the same time we have the opportunity to do something better than simple array literals. Like replacing array literals with some new way of using arbitrary collection literals. Just like how Kotlin decided to not add the await keyword and instead do something better by leaving it out-- any feature we add carries a cost. This is part of the minus 100 point rule. Another example is how Kotlin left out the ternary operator and instead turned if-statements into if-expressions.

Languages that heavily rely on collections like Python or Groovy often have a goal is to be concise and therefore easier to read. Java takes the opposite approach to Python and seeks readability with wordiness. Kotlin’s goal is to be readable first–to strike a balance. Sometimes this means fewer characters and other times it means more.
From KotlinConf:

You ask what the benefit of functions to declare arrays over literals, here are a few:

  • It saves an entire language concept and complexity and uses the already existing ones instead
  • It’s more readable (you would argue it’s the opposite, which is also true… This point is the most vague IMHO)
  • It’s a well fit solution when your design starts without literals.

If I understand correctly, the main issue you have with lacking array literals is the readability. What’s your use case-- is it common for you to want a literal array? And my follow up questions would be:
Why are you using arrays over lists?
What case are you needing to declare and fill with specific elements (I find this rare outside of learning to code via a tutorial)?

1 Like

Not sure if this is helpful but I have done some playing around with operator overloading and using [] operator in unconventional ways and it occurred to me that you could achieve something like this:

val myArray = array["A", "B"]
val myIntArray = array[1, 2, 3]
val myList = list[1, "a"]

Using definitions like this:

class ArrayGenerator {
    inline operator fun <reified T: Any> get(vararg items:T): Array<out T> = items
    operator fun get(vararg items: Int): IntArray = items
    operator fun get(vararg items: Long): LongArray = items
    // Plus the other builtin types
}

class ListGenerator {
    inline operator fun <reified T: Any> get(vararg items:T): Array<out T> = items
}

val array = ArrayGenerator()
val list = ListGenerator()

Is this better that arrayOf, etc.? That is a judgement call, but if you do find it preferable there is nothing stopping you from using it now. Be aware that while it uses vararg, spread operator will not work

I also find that the desire to create collections like this can be a small indicator of using imperative programming instead of functional. Not entirely, I’m certainly not saying that you won’t be creating collections. But if you are creating an array to mutate later that is imperative.

1 Like

Other languages like Groovy and Python build their APIs heavily on collection manipulation, but with Kotlin it’s rare to use an Array and even rarer to manually fill an array with elements at declaration.

Do you think maybe that’s because collectiond are so un-ergonomic in Kotlin that the language discourages it? In Java and Kotlin, arrays are often discouraged in favor of Lists, so even Java’s array syntax is often an obscure trivia fact.

Even if it’s just handling literal collections (not just a literal Array but a literal Set, List, Map, or maybe any arbitrary collection). It’s worth playing with some of these ideas before giving up a valuable syntax forever just because we’re familiar with the Array literal.

I addressed that well in my proposal.

How impactful or widespread the pain on lacking literal array syntax is or is not.

That’s what you’re saying shouldn’t be in this thread - folks are so burdened by this heavy syntax that they took the time to both seek out this discussion and leave a full comment on it. Such effort should not go unnoticed, nor minimized.

Hey! Thanks for the link to the proposal. There’s plenty of discussions there as well :slight_smile:

There may be some miscommunication: When I said the following:

^ I meant it to be something that should be included, not excluded from the discussion. Anyways, I felt I should clarify that since from your reply it looks like I wasn’t very clear on what I meant there. I did not mean that it shouldn’t be in this thread.

Off-Topic: Comparing to heavily collection-based languages

I hid this block since it goes a bit off-topic with how collections literals are often used by other languages to detrimental of the code.

Comparing to heavily collection-based languages

You ask a really good question: “Does Kotlin’s handling of collection literals discourage creating APIs that heavily really on collections like in Groovy/Python?” (heavily paraphrasing so please let me know if I’m miss interpreting… Although it might be best to take that discussion outside of this thread).

I would say “yes”. But there’s a few follow up questions I feel are prompted:

  1. What specifically about the average Groovy/Python API are we considering to have a “heavy focus on collections”?
  2. Is the lack of collection literals the primary discouragement to heavy collection based APIs?
  3. Is discouragement of this of API approach a negative or positive thing?

For #1, when I say heavily collection-based APIs I’m mainly referring to things like Groovy’s named params which is really just a collection literal. As well as the common practice in Groovy to return a collection as if it’s a tuple.
Unlike the average Kotlin code, It leads to method params becoming unstructured maps all over the code. (Side note, Kotlin used to have tuples but since Kotlin is a typed language “tuples” are not the same concept as a dynamic language’s “tuple”).

Kotlin encourages you to pass structured data in classes. This lack of tuples, first-class named params, and lack of collection literals all contribute to Kotlin code more commonly using named data containers (structured data in data classes) over collection-based APIs.

For example, while one might be tempted to use a collection literal for Groovy like so, [name: "Bob", city: "NY"], in Kotlin you would stray towards Contact(name = "Bob", city = "NY").

For #2: Is the lack of literals the primary discouragement for this kind of tuple/collection heavy API style? I don’t know. You can always make the same errors in Kotlin–especially with Pair instead of creating your own named container for that data.
For #3: Is this kind of discouragement good? IMO discouraging this type of API is positive. It’s an awkward change coming from some fields and languages where you never structure your data other than layering collections on collections. It’s a good habit to break.

At the risk of ending up with so many comments that we overflow the capacity of any feasible collection type implementation, I’ve just posted an alternative proposal to that of @BenLeggiero at https://github.com/Kotlin/KEEP/pull/229, with discussion at https://github.com/Kotlin/KEEP/issues/230. I hatched this idea ages ago, before I read your proposal, Ben, but I got some useful inspiration from it, so thanks for that. I’m approaching it in a somewhat different way.

2 Likes

I posted this on YouTrack, but I wanted to put it here in case folks visit this discussion and not the others:

I would like to add that there are features missing from Kotlin because it doesn’t have ergonomic collections.

My favorite example is in Swift, with their OptionSet type. This type abstracts and enhances the concept of bitmasking. In Kotlin, bitmasking might look like this:

data class ShippingOptions(val rawValue: Int) {
    companion object {
        val none      = ShippingOptions(0)
        val nextDay   = ShippingOptions(1 shl 0)
        val secondDay = ShippingOptions(1 shl 1)
        val priority  = ShippingOptions(1 shl 2)
        val standard  = ShippingOptions(1 shl 3)
        
        val express = ShippingOptions(nextDay.rawValue or secondDay.rawValue)
        val all = ShippingOptions(express.rawValue or priority.rawValue or standard.rawValue)
    }
}

fun takeShippingOptions(shippingOptions: ShippingOptions) { ... }

takeShippingOptions(ShippingOptions.priority) // Just one option
takeShippingOptions(ShippingOptions(ShippingOptions.nextDay.rawValue or ShippingOptions.secondDay.rawValue or ShippingOptions.priority.rawValue)) // Three options
takeShippingOptions(ShippingOptions.none) // No options
takeShippingOptions(ShippingOptions.express) // Semantically-combined options

fun hasNextDayShippingOption(shippingOptions: ShippingOptions) = 0 != (shippingOptions.rawValue and ShippingOptions.nextDay.rawValue)

whereas in Swift, it’s much more ergonomic and easier at the use-site:

struct ShippingOptions: OptionSet {
    let rawValue: Int

    static let nextDay    = ShippingOptions(rawValue: 1 << 0)
    static let secondDay  = ShippingOptions(rawValue: 1 << 1)
    static let priority   = ShippingOptions(rawValue: 1 << 2)
    static let standard   = ShippingOptions(rawValue: 1 << 3)

    static let express: ShippingOptions = [.nextDay, .secondDay]
    static let all: ShippingOptions = [.express, .priority, .standard]
}

func takeShippingOptions(_ shippingOptions: ShippingOptions) { ... }

takeShippingOptions(.priority) // Just one option
takeShippingOptions([.nextDay, .secondDay, .priority]) // Three options
takeShippingOptions([]) // No options
takeShippingOptions(.express) // Semantically-combined options

func hasNextDayShippingOption(in shippingOptions: ShippingOptions) { shippingOptions.contains(.nextDay) }

This is because Swift’s OptionSet is declared as ExpressibleByArrayLiteral, meaning that the arrays which are passed to the takeShippingOptions function are automatically converted to ShippingOptions by the synthesized init(arrayLiteral:) initializer.

The end result is that runtime performance ends up being the same in both languages, but clarity is greatly improved when people write and read the Swift code.


Of course, OptionSet further takes advantage of other Swift features such as: Implicit Member Expressions, RawRepresentable, and SetAlgebra, but these are out-of-scope of this discussion.

1 Like

EDIT, since I let myself type for too long :stuck_out_tongue_winking_eye:

OptionSet looks nice, but it’s just another collection that brackets could be used with if they were implemented to support various collection types.
OptionSet could be implemented separately and created with optionSetOf(...) and followed up by proposing creating an OptionSet literal with brackets or set notation. Overall I think it shows the importance of the issue’s “concerns” section where the design of collection literals should not be done separately from concise object literals.


Original reply:

Thanks for posting here (I don’t often check youtrack issue updates so I’d probably not have seen this). OptionSet looks pretty cool :slight_smile: Although I would lump it in as just another collection type that brackets could or could not support.

While I like the example, I feel like the other feature changes (Implicit Member Expressions, RawRepresentable, and SetAlgebra) distract from the smaller difference of being able to use sets and bracket notation.
This topic is already very long and introducing those discussions here may take away from the discussion of collection literals.

I get the feeling the example was first written in Swift and ported to Kotlin. Here’s how I feel the example could be implemented more naturally in Kotlin with a few changes (NOTE, I do break the OptionSet niceness of masking all of the bits together. You could always do that in a library solution):

enum class ShippingOptions(val rawValue: Int) {
    NONE(0),
    NEXT_DAY(1 shl 0),
    SECOND_DAY(1 shl 1),
    PRIORITY(1 shl 2),
    STANDARD(1 shl 3),
    
    EXPRESS(NEXT_DAY.rawValue or SECOND_DAY.rawValue),
    ALL(NEXT_DAY.rawValue or SECOND_DAY.rawValue or PRIORITY.rawValue or STANDARD.rawValue)
}

fun takeShippingOptions(vararg shippingOptions: ShippingOptions) {
    // Note: I'm not doing the OptionSet combining stuff here, just accepting a raw list. So not quite the same but it's not related to brackets so I just print it.
    shippingOptions.forEach { print(it) }
    println()
}

fun main() {
    takeShippingOptions(ShippingOptions.PRIORITY)
    takeShippingOptions(ShippingOptions.NEXT_DAY, ShippingOptions.SECOND_DAY, ShippingOptions.PRIORITY)
    takeShippingOptions(ShippingOptions.NONE)
    takeShippingOptions(ShippingOptions.EXPRESS)
    
    // Other things to note: I think ShippingOptions.NONE is better than [] as it's far clearer to the reader that is an okay thing to do.
    takeShippingOptions(*emptyArray()) // If for some reason, we actually want a different equivalent of takeShippingOptions([]) we could do this.
    //takeShippingOptions(null) // If for some reason, we actually want a different equivalent of takeShippingOptions([]) we could allow null.
}

A quick note: I added rawValue to the ors of your example since it didn’t compile.


My main point is that the biggest gain the Swift implementation has over the Kotlin one in readability is probably the Implicit Member Expressions. The bracket notation for collections doesn’t give you much over vararg–in fact, I bet someone would propose dropping the brackets.

IMHO, this code example does not present a strong case for allowing you to use set literals. The use in the code example is being able to pass a variable number of arguments.

Even if there are stronger cases, the bracket notation is separate. An OptionSet proposal for Kotlin could just ignore the bracket notation since, in this example, are just used as boilerplate and the outer parens would be enough.

If OptionSet is just a container of bitwise flags representing an enumerated list of options, I think this example is a stretch for bracket notation. I see the convenience in being able to use brackets in Swift to be detected and transformed as a bitmask, but IMO that’s the same kind of detection that would allow Kotlin brackets to be passed in for a method expecting a Set, List, or any other collection implementation.

If we want to create a strong argument for collection literals, OptionSet doesn’t help since all it takes is for someone to say optionSetOf(x, y, z) and we’re back at square one.

1 Like

tl;dr - Sequence literals allow APIs to avoid leaky abstractions.

I was merely giving an example of expressive syntax that can be achieved through well-designed features of a language and its standard library. Such good design would not necessitate something so specialized as “an OptionSet literal”, as it would be generally-useful enough to enable things like OptionSet and other out-of-the-box usages of syntax without any proposals.
I was not intending to introduce discussion around implicit member expressions, nor RawRepresentable, nor SetAlgebra.

The entire point of my example was to show how the low-level and often ugly task of creating and using a bitmask can be semantically described in an elegant and human-friendly way, thanks to features of the language and standard library harmoniously working together. It was not to say “here’s something that is impossible in Kotlin,” but instead “here’s something that is so clunky in Kotlin that devs avoid doing it entirely.”

I also chose OptionSet specifically because it’s most-often used to hard-code configure something, such as a UI control or network request, rather than arrays and dictionaries which can commonly be read from a file.

The bracket notation for collections doesn’t give you much over vararg–in fact, I bet someone would propose dropping the brackets.

Odd you bring this up - my original proposal had the bracket notation be syntactic sugar for a operator fun with vararg.

Additionally, the brackets are there to say “here is one singular object,” not “here are three separate objects.” It extends to all places where an OptionSet might go:

struct Example {
    var options: ShippingOptions = [.priority, .standard]

    func evaluate(against options: ShippingOptions = [.nextDay, .secondDay]) -> Bool {
        self.options.contains(options)
    }

    mutating func reset() {
        options = [.priority, .standard]
    }

    static func suggestedDefault() -> ShippingOptions {
        return [.priority, .standard]
    }
}

Let’s look at this example without any syntactic sugar, using only what we can do in Kotlin today, assuming all necessary functions are well-implemented somewhere else:

data class Example(var options: ShippingOptions = ShippingOptions(ShippingOptions.priority, ShippingOptions.standard)) {

    fun evaluateAgainst(options: ShippingOptions = ShippingOptions(ShippingOptions.nextDay, ShippingOptions.secondDay))
        = 0 != this.options.rawValue and options.rawValue

    fun reset() {
        options = ShippingOptions(ShippingOptions.priority, ShippingOptions.standard)
    }

    companion object {
        fun suggestedDefault() = ShippingOptions(ShippingOptions.priority, ShippingOptions.standard)
    }
}

And once again pretending all these discussed language and stdlib features are implemented except collection literals:

data class Example(var options: ShippingOptions = ShippingOptions(.priority, .standard)) {

    fun evaluateAgainst(options: ShippingOptions = ShippingOptions(.nextDay, .secondDay))
        = this.options.contains(options)

    fun reset() {
        options = ShippingOptions(.priority, .standard)
    }

    companion object {
        fun suggestedDefault() = ShippingOptions(.priority, .standard)
    }
}

And finally, pretending only collection literals were adopted:

data class Example(var options: ShippingOptions = [ShippingOptions.priority, ShippingOptions.standard]) {

    fun evaluateAgainst(options: ShippingOptions = [ShippingOptions.nextDay, ShippingOptions.secondDay])
        = 0 != this.options.rawValue and options.rawValue

    fun reset() {
        options = [ShippingOptions.priority, ShippingOptions.standard]
    }

    companion object {
        fun suggestedDefault() = [ShippingOptions.priority, ShippingOptions.standard]
    }
}

As you can see, whether or not Kotlin has IMEs, SetAlgebra, RawRepresentable, etc., it still becomes more readable and more elegant with collection literals. It communicates that it deals with a collection of some sort, rather than performing some operation on the given shipping options (regardless of how things work under the good). The user doesn’t have to know what a ShippingOptions is under the hood.


Your counter-example of creating an optionSetOf(...) function goes back to the original gripe that arrayOf is unnatural, and that API users must memorize far too many someCollectionOf functions just to do what other languages let you do with two characters: [ and ]. That’s why my original proposal included getting rid of these entirely.

This also means that, if something becomes implemented differently under the hood, the syntax is not portable: If an SDK goes from taking a List to taking a Set, the APIs must then either break by requiring setOf, or duplicate new APIs must be made to marshal lists to sets, which bloats the API space and runtime. That must be done for these APIs since general-purpose sequence literals can’t be passed - at best it must be a listOf or setOf or mutableListOf or mutableSetOf or or or or or.

Thanks to first-class support for collection literals, Swift programmers generally do not need to know (and I bet many beginners do not know at all) that the Array and Dictionary types are actually called that. They don’t need to, since the language syntax covers that. They also don’t need to know what an OptionSet is unless they’re making their own, nor how it uses array literal syntax; all they need to know is that they can use this familiar tool in this familiar place.

I figured I’d rather just let myself write a long reply on the forum since this is more opinion and I don’t feel it has merit being raw on the issue tracker.

The OptionSet example

I’d like to backpedal a bit on my last post when I said that “these ideas distract from the collection literal solution.” My reply misunderstood the goal carried over from the YouTrack issue. Collecting information on use-cases should help discover related concerns–which is not a distraction like I said it was. I was still focused on evaluating the specific solutions already proposed.

My issue with the example is that it doesn’t create focus well on collection literals. But I see now how the issue is focused on collecting use cases and that it does provide a good general use case :slight_smile:

The OptionSet doesn’t show collection literals playing the primary part in the example. Especially when considering the limits on collection literals proposed and possible small focused steps forward–the example covers more ground and collection literals are too easy to factor out. But that was my thinking it was focused on the arguments framed in the solution instead of general use cases.

I know I said OptionSet is basically just another collection, but in this case it starts to bleed into more general object usage.

It’s not a trivial proposal

There’s been a lot of debate on collection literals. One of the major issues in discussing the topic is the design requires more consideration than most newcomers to the topic expect.

There are those who enter the discussion believing we need raw Java-style literals for arrays without having considered any of the better alternatives or evolutions of the concept.
Concerns have to be given for limitations, different collections, how many, multiple or unified syntax, object literals, type inference, custom implementations.

Even proposals outside of collection literals have to be considered: syntax overlap for things like multi-receivers, tuples, pattern matching, etc. Things like how little syntax changes would be sufficient, and the impact to existing parts of the proposal (like the current annotation array literal).

Meta considerations have to be made for how much work it will take, what kinds of stdlib changes will need to be made. Maintenance and compiler performance need consideration.

The working proposal

The current proposal and discussion has a lot going in its favor–even simple benefits like pushing away from Java-style raw array literals and other poor solutions (unless there are no better solutions to be found). We even have collection literals in Kotlin now in a limited way (in annotations).

Progress seem slow for collection literals

The direction to discuss use-cases is a great idea–it’s easy to get stuck on solutions instead of the problem. Like getting stuck on raw array literals, which is a very different thing than arbitrary collection literals.

Waiting for things to refine itself

Some would argue that literal collections aren’t beneficial in a meaningful way due to being so rarely used (I’m in this group). Another debate is if being able to use brackets doesn’t really improve the readability in the common use-case (I’m also in this group). Even if some small steps would add a small benefit, I’d rather focus on something else.

But, things have changed since the initial requests for collection literals. Thankfully, no one pushed forward with the Java-style raw array solution since there are better ideas brewing (like your proposal, and variations of it).

There are still several ideas brewing that impact the proposal. For those with an opinion that the benefit isn’t motivating enough, they are happy to defer while you and others refine the proposal (and other related features).

For me, I’m happy to wait on a proposal that, in my opinion, doesn’t add more than the opportunity cost is worth.

Maybe things have changed

Maybe now the Kotlin team will have a good enough idea for the direction of related features to move forward another step with collection literals. The new roadmap helps us know what’s in the queue (I wish we had more scripting support planned but oh well. Another time). I’d bet we’ll start to see small changes to enable more aspects of collection literals as higher priority features are ironed out.

The movement towards collecting use-cases shows signs of interest (e.x. the second proposal, being closed, sorry HughG :stuck_out_tongue: ). It could end up as another one of Kotlin’s classic “You thought you wanted X but here’s something much better that solves the real problem!” like structured concurrency :crossed_fingers: :partying_face:

or it may end up being a small targeted step lifting some of the limitations on collection literals.

Now that Roman is the project lead, we may have new interest in addressing the pain-cases that relate to literals and bracket syntax.

1 Like

this is more opinion

That’s all a programming language is - a set of opinions. Kotlin shares a lot of its opinions with Java, C, Swift, JavaScript, Python, Ruby, etc. (BTW, all of those have collection literals). Embrace your opinion! It might be assimilated into Kotlin’s opinions someday :grin:

It’s not a trivial proposal

Agreed! Which is why I put my proposal on my résumé - I think I took everything you mentioned into consideration, coming up with a good general solution which can cover all use cases, feels native to the language, and does not break any existing code. I feel like the opportunity cost is negligible if not nonexistant.

Sorry if it sounds like I’m a broken record who’s tooting my own horn, but honestly I really think my proposal is the best path forward for Kotlin. The only thing holding it back from a technical implementation perspective is that Kotlin doesn’t distinguish functions based on their return type, which is a feature that’s easy enough to implement and also won’t need breaking changes.

Progress seem slow for collection literals

Yeah… and it hurts me. There’s so much community support for the concept (even if there’s a lot of debate over implementation details), but nobody from JetBrains seems interested in shepherding any proposals, nor making their own. Kinda makes me wanna fork Kotlin just to make this happen.

Maybe things have changed

Gosh I hope so. This stagnating is what absolutely killed my ambition for Kotlin. Now it’s just another language I know, like Ruby or C++. I can write it well enough, just have no passion for it anymore. :pensive:

a bit off-topic, but you kinda can today actually with a compiler plugin. Check out Arrow Meta maybe even to make your life a bit easier.

2 Likes