Unfortunately I'm quitting Kotlin after 4.5 years because the Binary support is just unacceptable

Language Design
1

So, I hate to do this. Make a big “about me” negative fuss. But the issues that are making me quit, I feel like they are direct and explicit choices that team has made (both about what they should and shouldn’t focus on and about what to and not to carry over from more Classical languages) and I think they were extremely wrong-minded. And I feel it’s important to give this feedback that these issues are causing such friction with my use of the language on a day to day level, that despite having over 100k lines of code I would love to be able to push forward, I just have to burn that bridge and rip off the band-aid. Maybe people sympathize or maybe I’m alone in my issues.

Brief background: I’m a professional software engineer, going on seven years. Hobbyist going on twenty. In my professional life I mostly live in the C# .NET environment. But as a hobbyist, for the last four and a half years I’ve been using Kotlin in my personal projects. And in the past I’ve really been rooting for it as a language, even recommended it a few times. It’s full of special flavor syntax that just made sense and with a deep integration into a compiler that enabled really robust pre-compile-time code analysis and language functionality that depended on this.

There was one tremendously glaring weakness that I personally had trouble stomaching, but thought that they just haven’t got around to. But when they did “get around to it” it just got. SO. MUCH. WORSE. And they refuse to fix it.
I mean look at this.

    val byte : Byte = -111
    println(byte.toUByte()) // 145
    println(byte.toUShort()) // 65425
    println(byte.toUByte().toUShort()) // 145
    println(byte.toInt()) // -111
    println((byte.toInt() shl 24) and 0x0F000000) // 16777216
    println(byte.toInt() shr 24) // -1
    println(byte.toUInt()) // 4294967185
    println((byte.toUInt() shl 24) and 0x0F000000u) // 16777216
    println(byte.toUInt() shr 24) // 255

    val ubyte : UByte = 0u
    //val isZero = ubyte == 0u // compile error : cannot compare UByte with UInt

    //val byte2 : Byte = 0xf3 // compile error

    val ubyteArray = UByteArray(256) {0u} // Experimental because arrays of native types?  Why would we have that as a first class citizen?
    val x = ubyteArray[10] xor ubyteArray[30] // Enjoy your entire code base getting colored yellow.

I’m sorry. To me, this is worse than the oft-repeated Javascript casting memes. These aren’t “accidental misunderstandings when doing weird things”, this is “trying to do basic tools that I’ve always had access to and there literally is no correct way to do it.” And I absolutely shudder to think what’s going on once it gets converted to machine code. Just for trying to re-interpret data into something that you can action on.

Maybe this is just me being a weird fossil, but sometimes as a person who writes text that makes computers do things, every once in a while when typing up code, I run into data. And I like to manipulate that data. See, I get the feeling that the people in charge of this language think that bitwise operators are only ever used by strange embedded programmers who should be using Rust or by programmers who think they’re being clever and that all good-hearted developers deal in more traditional mathematical concepts like + and - rather than, you know, mathematics relevant to computer science. But (a) that makes the supposed commitment to native a joke because I can’t even imagine a use case that needs native access and accepts abysmal binary manipulation. And (b) they’re dead wrong. Data streams bombard the program from every angle. Even if you don’t deal with novel file types or data streams in your particular area because everything is just out-the-box, who has never used BINARY type in SQL? I get the idea behind it. “Don’t blindly copy over everything that previous languages did because maybe they’re not all relevant anymore.” But if they haven’t realized a mistake so fundamental as this in the time they’ve had to, then I just, yeah I’m 100% abandoning the language. There IS NO SUBSTITION for bitwise functions. These are op codes. This is how computers work. You can’t abstract it away. You can’t pretend it’s irrelevant.

2 Likes
2

I get the feeling that most of the said inconsistencies are due to a confusion between static casts (which convert the data) and reinterpret casts (which don’t touch the binary representation). Who is doing the confusion here? You or the language? The methods you are using all provide static casting. Static casting of negative values to a wider unsigned type always follow some convention.

It turns out that kotlin convention mimics c++ convention. The following c++ program:

#include <iostream>

using namespace std;

int main() {
  char b { -111 };
  cout << "byte = " << static_cast<int>(b) << endl;
  unsigned char c = static_cast<unsigned char>(b);
  cout << "ubyte = " << static_cast<int>(c) << endl;
  unsigned short s = static_cast<unsigned short>(b);
  cout << "uhort = " << s << endl;
  unsigned short ss = static_cast<unsigned short>(static_cast<unsigned char>(b));
  cout << "ushort(ubyte) = " << ss << endl;
  int i = static_cast<int>(b);
  cout << "int = " << i << endl;
  cout << "int<<24 & 0x0F000000 = " << ((i << 24) & 0x0F000000) << endl;
  cout << "int>>24 = " << (i >> 24) << endl;
  unsigned int ui = static_cast<unsigned int>(b);
  cout << "uint = " << ui << endl;
  cout << "uint<<24 & 0x0F000000 = " << ((ui << 24) & 0x0F000000) << endl;
  cout << "uint>>24 = " << (ui >> 24) << endl;  
  return 0;
}

produces:

byte = -111
ubyte = 145
uhort = 65425
ushort(ubyte) = 145
int = -111
int<<24 & 0x0F000000 = 16777216
int>>24 = -1
uint = 4294967185
uint<<24 & 0x0F000000 = 16777216
uint>>24 = 255

So, I’m not sure your opinion is very relevant…

12 Likes
3

It’s great to hear the pain points that exist in Kotlin, especially when examples are provided! But I do think the method of this message obscures the value of providing feedback with a personal tone and a focus on big picture language design instead of on specific pain points, use-cases, or possible improvements.

I do think that your perspective is relevant, which is why it would be beneficial to the language if you used your experience with Kotlin to contribute with use-cases and feedback as you are hoping to do now.

some meta comments :slight_smile:

Saying this paints things as one broad stroke, and when I read this I hear it in a harsher tone than it was likely intended to be. Reading the post makes me feel a hint of ad hominem that may take away from the feedback you hope to provide.

Your background is nice but it’s less important than the investigation into the details so I’m glad you provided an example :slight_smile:

the feedback

From a quick pass, I’d break out the following specific pain points:

  1. Confusing or inconsistent converting between types
  2. Shifting behavior (that I can’t define right now but I did find in the Kotlin course in Jetbrains Academy)
  3. Lack of a standard library comparison with == for comparing Ubyte with UInt
  4. Experimental status of arrays and value types (or other native types)
  5. ? An issue I didn’t understand with ubyte array xor’ed with another ubyte array ?

I’d also separately start to talk about the pain of feeling the language choices have not prioritized these issues and a few assumptions about why that is the case and the motives of the others. I’d consider a lot of this to be distracting from the effort to identify and understand problems–it’s more valuable to stick to concrete specifics :slight_smile:

I can’t speak to all of the points but here are a few quick comments:

@arkanovicz had a good example for #1. I would maybe add conversion is complex. See this other conversion example of complexity. It often feels like there is an obvious way to do conversions when in reality it’s a more complex problem in many cases.

Thankfully, Kotlin enables us to reinvent these “language level” features as libraries!
Kotlin has the tools to allow you to create powerful library-based tools (see for example, GitHub - kunalsheth/units-of-measure: Type-safe dimensional analysis and unit conversion in Kotlin.).

A few enhancements that may be relevant:
Value classes is one of the enhancements that will even further enable you to tune Kotlin using standard language features by defining your own “primitives”, just as the unsigned numbers are defined. Especially valuable for number stuff :slight_smile:
Context clauses will also allow for more powerful extension of existing types greater than the current extension function ability.

Another note on bitwise operations from my experience: The JetBrains Academy course in Kotlin spends a maaaaasive amount of time doing bit and byte stuff… I was pretty surprised that it was so heavily focused on. It’s definitely beginner material but the focus is there from the start for performing bit operations and working with bytes–at least it is there way more than I expected.

6 Likes
4

+1 to what @arkanovicz said. This is how we always handled integers in most languages that use fixed-width integers. Kotlin is not at all an exception here.

The only thing that I personally don’t like is the function naming. My impression is that in Kotlin toSomething() usually means the data conversion while asSomething() means data casting/wrapping. In the case of integers it is definitely casting/wrapping, but it uses to* naming. I think it would be more consistent to name these functions e.g. asUInt() and optionally provide toUInt() which enforces the bounds and then returns the value of asUInt().

8 Likes
5

I wonder if it’s possible to start a round of depreciation for the next major version, and delete them in the one after that, or if it’s considered too ‘core’ to even be touched.

I completely agree that Int.asUInt() since much clearer on what it does than Int.toUInt.

1 Like
6

In the case of integers it is definitely casting/wrapping

The “as” prefix implies that the result represents same instance.

Since value types like Int have no concept of an instance, it’d be really bizarre for them to ever have any “as” prefixed methods. Even though UInt is implemented as an inline class wrapping an Int, the result of toUInt() is not intended to be treated as a reference to the original instance.

Consider val x = myInt on JVM. The value of myInt will be copied to x. No reference to myInt is kept. It’s not any different for val x = myInt.toUInt()

4 Likes
7

Yes, I know toUInt() copies the data and if this is the main differentiator between “as” and “to” then it makes sense.

For me “as” is conceptually for operations that keep the same data (or almost the same), but represents/interprets it differently, adapts it to needs of some API or somehow affects its behavior. While “to” is for operations that transforms the data in order to adapt it to new needs, often validating and copying it in the process. toUInt is somewhere in between. Technically speaking, it copies the data, it could shrink/widen it, but conceptually it re-interprets the existing data as a new integer type, possibly changing its logical value.

Even if thinking in English and reading “as” as “represent as”, “interpret as”, “use as”, etc. and “to” as “convert to”, “transform to”, “change to”, I think “as” fits much better to the case where -111 becomes 145. Honestly, I suspect the naming itself might be the main reason why OP misunderstood their results and assumed Kotlin is broken.

Anyway, the decision was made long ago and it can’t be easily changed right now. I’m pretty sure Kotlin team discussed this at designing phase and “to” has been chosen intentionally. Also, this decision was made long before unsigned integers, because it is the same for converting between signed integers of different widths (yes, I just used the word: “converting”!).

2 Likes
9

I think posts about other issues should probably live in their own threads.

The title reads as if it’s a place to post any and every complaint or struggle related to Kotlin. But at the heart of OP’s post we find that the issues they are bringing to discuss is working with bytes.

Other topics, such as yours with compose, get less visibility than it deserves and takes away from OPs issues.

I recommend we avoid using this unfortunately named thread to combine any Kotlin issue together as it’s less effective for all of the issues. Instead, focus on the binary support.

2 Likes
10

This thread comes to me as a complete surprise.

I’ve been using Kotlin professionally since about 2018. And I’m doing a lot of binary manipulations. I never even tried to make sense of them, neither in Kotlin nor in any other languages. I don’t think there’s one “right” way to do it anyway.

Whenever I need to do some binary manipulations, I just write a bunch of functions tailored for that specific task, and Kotlin provides just the right tools with its inline, infix and extension functions, so I often end up writing stuff like word.getField(lsb = 1, msb = 8), or b.toWord(), or word.parityBit (here, Word stands not for uint16, but for a domain-specific thing). Pretty much the only complaint I have here is that IDEA bugs me about using inline functions when there’s no lambda argument to inline.

It’s basically a very specific application of the Dependency Inversion principle: instead of using (and depend on) the tool that is given to you, use it to make a better tool that depends on your domain-specific code.

8 Likes