How to render offscreen using the GPU

Support
1

My goal is to use Skiko in a Kotlin Compose Multiplatform app which requires offscreen rendering utilizing the GPU in background threads. The app is currently targeting the desktop and Android platforms (and hopefully will also target iOS in the future).

In the CommonMain code I have a factory which produces a Skia surface that is declared as ‘expected’. In the desktopMain and Android main implementations the factory is implemented via ‘actual’ declarations using native calls in each platform.

I am new to OpenGL, Skiko and Kotlin, so the learning curve has been steep but I am excited about the technology and its promise.

So far I have been successful in implementing the desktop implementation by using the LWJGL library to provide OpenGL support which binds nicely into Skia via the Skiko library.

I’m not having much luck though, binding the Skiko library into the OpenGL-ES native functionality provided by Android itself. I first tried the approach given here: GLESOffScreenRenderingDemo/app/src/main/java/com/android/gl2jni/PixelBuffer.java at master · ragnraok/GLESOffScreenRenderingDemo · GitHub. While the functionality worked in terms of creating a display, a context, and a surface and making the combination current, I couldn’t find a way to make Skiko bind via its DirectContext method. I would have thought that if a thread has a context which is current, then this method would just pick it up?? At some point in my research I read that Skiko wants a ‘native’ surface, so that pbuffers would not provide that and what was required was a window surface.

I then followed the approach given here, using a window surface instead: opengl es - Android initialise openGL2.0 context with EGL - Stack Overflow. Note that this advice was given nearly a decade ago, and is possibly / probably out of date. Here is the overall strategy:

The factory has an initGraphicsLibrary method which is called first and does the following:

  1. Gets an instance of the egl library
  2. Uses the instance to get the characteristics of the default display
  3. Initialize the egl library with those display characteristics
  4. bind the OpenGL API to the instance of the library

The factory has a getOffscreenSurface method which is then called once in each thread, to provide a Skia surface to render upon in that thread. Here is the approach taken in this method:

  1. A config spec array of ints is created to specify the desired config
  2. eglChooseConfig is used to get the closest match to the display and requested config spec
  3. An eglContext is then generated with eglCreateContext method, based on the display and the config returned from eglChooseConfig
  4. A window surface is then created using egl.CreateWindowSurface, passing in the display and config
  5. The settings are made current in the thread using eglMakeCurrent, given the display, the surface, and the context
  6. Then in the Skiko world I request a renderTarget using BackendRenderTarget.makeGL
  7. I ask Skiko to create a context, using DirectContext.makeGL()
  8. I ask Skiko to make the backend Surface using the Surface.makeFromBackendRenderTarget method. The surface is returned and its associated Canvas is used to render upon.

I’ve attached the code I use:
Creating a Skia Android Surface.pdf (4.7 KB)

The first thing I note is the code relies on GL10. I read that I need to bind the API to OpenGL not to OpenGL-ES, but GL10 does not support the eglBindAPI method. So I used GL14 instead. Even when I used the GL14 version, the call failed, so I commented it out, just to see how far I could get, knowing there would be consequences.

The code given in the stack overflow question, provides a SurfaceTexture which is passed to the eglCreateWindowSurface. I don’t know how to create a SurfaceTexture. I have seen examples of the call being made in other approaches by passing in a null or EGL10.EGL_NO_CONTEXT for the native window which is meant to hold the surfaceWindow being created. As a result, when this call is made, AndroidStudio detaches from emulator with no explanation. I am not even sure if after the WindowSurface is created (which should satisfy Skiko with a native window) and with the graphics context being current, why Skiko will not give back a DirectContext in its makeGL method. I tried to understand this a bit more by looking into the Skiko library itself but it seems to use JNI to get contexts in some way.

I would be beyond grateful if someone could point me in the right direction with all of this. I have spent weeks trying to research what needs to be done, but there are too many gaps to fully understand what I need to do. If I can get this working, I am sure that there are many people who could make use of this approach, particularly when capturing live feeds of information that must be processed quickly for quick display updates.

Any insights would be greatly appreciated.

2 Likes