Best practices on loading and deserializing files? (Amiga Protracker)

Hi everyone,

I have a small side-project I’m working on - I want to create a library to load mod music files. These were various file formats created mostly in the late 1980’s and in the 1990’s. They were largely written by amateurs and enthusiasts, and aren’t very well-supported by modern software. Learn more about mod files here.

I’m not new to Kotlin, but I don’t have much experience in loading files from disk since most of my career has been focused on web services.

To start with, I want to be able to load one of the simplest formats, the Amiga Protracker format. A file descriptor is available here. I just want to be able to load it from disk and store it as an object. The object will contain song data including title, song metadata, samples (the audio used for the instruments), and instructions for how to play the samples (patterns, effect commands, etc).

For now, I just want to be able to deserialize the file into an object. However, I’m not sure what the best practices are for doing this in Kotlin. I’ve started by just using File.inputStream.readBytes and then traversing through the byte array at the specified borders. However, I’m sure there’s a better way of doing it than this. Any suggestions?

A few notes:

  • I will occasionally need to take two bytes and store them as a short (or word). Is there a simple way of creating a short from two bytes in Kotlin? Big-endian in this case.
  • In another case, I’ll only be looking at the lower four bits of a byte. Any recommendations on handling this? I was probably going to look at my options with bitwise operators.
  • Are there any recommended ways of storing the audio data from the samples themselves? I could just keep them as a byte array, but I’m sure there’s better classes for storing audio. I’m not sure what format / codec is needed, but I’m not planning on actually playing the audio until later.

It’s been a while since I did anything with file formats directly like that so I might not be up to date on the current libs. I’m guessing that you are using kotlin jvm and not build for multiplatform so most of the java libraries won’t work if you want to go MPP.
Anyways unless your goal is to implement a reader for the file format(s) yourself I would look for java libraries that can already read them. You can use any java library as if it was written in kotlin so you might get lucky.

You should take a look at ByteBuffer. ByteArray is great for simple stuff but if you really want to read binary files ByteBuffer should give you more options.

Binary operators are a bit of a struggle in kotlin. The problem is that unlike other programming languages kotlin does not have any “real” binary operators. There are infix functions that you can use, but they don’t follow the same precedence as real operators. Therefor you should try to keep binary operations to 1 operator per statement, maybe 2 and use parentheses to define precedence when necessary. If all you care about is a simple bitmask to get the lower bits you should not have any problems though someByte or lowerByteMask.

Not sure. I’m guessing it depends on the library you use to play back the audio in the end. My guess is that storing the raw byte data is fine for now. You could create a wrapper class so it’s easier to switch it out with a different class later but I’m not sure this is really necesary.

There is kotlinx-io that provides for multiplatform I/O and should make much of this easier

kotlinx-io will handle this with no problem. It has methods to read various sizes in big or little endian (think it defaults to big) and supports signed vs. unsigned.

Basically you would probably want to not store it yourself until you need it and kotlinx-io supports buffers sou you can let it store it and then you can copy it to an output when needed.

1 Like

If you need help with the formats, have a look at mikmod: https://github.com/sezero/mikmod . There are a lot of formats supported and at some point there was even a java port of it.

Thanks, will check it out. I don’t really know C and I haven’t used C++ since college, so my low-level programming skills are pretty rusty / non-existent, but I should still be able to get something from it.

It is pretty surprising to see how low-level these formats go (I’ve found myself having to split bytes in half and store the data from those nibbles into different fields), but that was kind of the nature of these module formats: squeezing as much performance as they could out of limited resources.

Thanks for the tip, I will take a look at this.