Stay in Touch with Us

Pages

Saturday, August 10, 2013

Developer's notes II. ETC2 texture compression in OpenGL ES 3.0

The most recent Android version 4.3 has been released a few weeks ago. Even long before final release there has been a few leaked ROMs around - first for Samsung Galaxy S4, and then for Nexus 4. In both of these ROMs I've found libraries for OpenGL ES 3.0 which was extremely good news - rumors about March demo of OpenGL ES 3.0 on HTC One working on native libraries were true. As well as rumors for support of Bluetooth Low Energy were true, too.
So after receiving OTA updates on Nexus 10 and Nexus 4 I've decided to try this goodness out. (Nexus 7 received update with almost one week delay but I didn't care about this, I'll explain later why).


Introduction. What's new in OpenGL ES 3.0

There are quite a lot of new cool features in latest version of OpenGL ES. I won't enlist all of them here, you can read about them here: http://www.khronos.org/opengles/3_X and in short press-release here: https://www.khronos.org/news/press/khronos-releases-opengl-es-3.0-specification . In this article I describe the easiest to implement new feature of OpenGL ES 3.0 - new standard texture compression format ETC2.

ETC2

ETC2 is based on ETC1 and is implemented using invalid bit combinations of ETC1. To understand how ingeniously it is implemented you should read these documents describing compression algorithm here and description of ETC2 format here.
ETC2 is based on ETC1 - it works with the same blocks of 4x4 pixels, but for each block is used certain compression algorithm - usual ETC1 or one of three additional algorithms. So it achieves the same 4x compression ratio as ETC1 but with significantly improved image quality.
This results in quite balanced compression which can preserve sharp edges in one blocks as well as introduce minimal distortion of gradients in another blocks.

Initialization of OpenGL ES 3.0 context

To initialize OpenGL ES 3.0 context in Android 4.3 you don't need to perform any special actions in your code. All you need to do is to initialize usual OpenGL ES 2.0 context and if GPU supports GL ES 3.0 Android will automatically create OpenGL ES 3.0 context which is fully backwards compatible with OpenGL ES 2.0. So actually, on Android 4.3 all OpenGL ES 2.0 apps actually work with OpenGL ES 3.0. To ensure that created context is 3.0 you should check GL_VERSION variable. Example of source code from Android 4.3: https://android.googlesource.com/platform/frameworks/base/+/android-4.3_r0.9/libs/hwui/Extensions.cpp 
Thanks Romain Guy for this and other notes about using OpenGL ES 3.0 in release day of Android 4.3: https://plus.google.com/u/0/+RomainGuy/posts/iJmTjpUfR5E
Example of Java code:

    protected int mOpenGLVersionMajor;
    protected int mOpenGLVersionMinor;


    String strGLVersion = GLES20.glGetString(GLES20.GL_VERSION);
       if (strGLVersion != null) {
         Scanner scanner = new Scanner(strGLVersion);
           scanner.useDelimiter("[^\\w']+");


           int i = 0;
           while (scanner.hasNext()) {
               if (scanner.hasNextInt()) {
                   if (i == 0) {
                       mOpenGLVersionMajor = scanner.nextInt();
                       i++;
                   }
                   if (i == 1) {
                       mOpenGLVersionMinor = scanner.nextInt();
                       i++;
                   }
               }
               if (scanner.hasNext()) {
                   scanner.next();
               }
           }
       }


   protected Boolean isES2() {
       return mOpenGLVersionMajor == 2;
   }


   protected Boolean isES3() {
       return mOpenGLVersionMajor == 3;
   }

nVidia

Here I will explain why I was absolutely not worried about delayed 4.3 update of Nexus 7. It is because Tegra 2 and Tegra 3 GPUs don't support OpenGL ES 3.0. What is worse is that even Tegra 4 doesn't support it. nVidia just keeps to put their desktop solutions into mobile chips and their marketing successfully advertises this as solution for high-end devices. Tegra 4 whitepaper has quite awkward explanation of why GPU lacks such features on page 11: "We do not expect applications/games to use ES 3.0 for quite some time." Surprise, nVidia - Android 4.3 itself uses OpenGL ES 3.0 in certain parts of UI rendering pipeline.
It is obvious that nVidia won't expand features of current mobile GPUs lineup, however Tegra K1 chip will support OpenGL ES 3.0: http://www.ubergizmo.com/2013/07/nvidia-tegra-5-release-date-specs-news/ as well as full "desktop" OpenGL 4.4 and DirectX 11. For example, this video showcases Unreal Engine 4 demo (with Tim Sweeney comments): http://www.youtube.com/watch?v=t3M6dh1f6Dw.

Creating compressed textures

To create ETC2 compressed textures Mali Texture Compression Tool was used: http://malideveloper.arm.com/develop-for-mali/mali-gpu-texture-compression-tool/.
To achieve the best visual quality of images was used "Slow" compression method and Error Metric "Perceptual".

Here I provide image to compare visual quality of ETC2 and ETC1 textures and difference between compressed and original image in 4x scale to make compression artifacts more noticeable.


On ETC1 texture you can notice artifacts like horizontal (more visible on sample image) and vertical lines. With ETC2 compression such distortions are minimal.
I our live wallpapers we use uncompressed textures in parts where visual quality of images are critical. As seen on comparison image ETC1 adds significant artifacts into gradients which makes it inapplicable for textures of sky. But uncompressed textures take up a lot of video memory because they typically have size 2048x512. PVRTC compression also provides good image quality but it is available only on chips with PowerVR GPUs. Using standard OpenGL ES 3.0 texture compression ETC2 allows us to use 4x less memory with good enough visual quality.
For 2048x512 texture:
Uncompressed (16-bit color 565 - 2 bytes per pixel): 2*2048*512 = 2097152 // 2 MB
ETC2 (16 bytes - PKM header): 524304-16 = 524288 // 512 kB.

Loading ETC2 texture

Textures are loaded from PKM files. The same format is used to store ETC1 textures. Header structure is described here: http://forums.arm.com/index.php?/topic/15835-pkm-header-format/. I decided not to load ETC2 textures with ETC1Util class because it has header validation code, and there is no ETC2 support in it.
Code for loading ETC2 texture:


    private static int PKM_HEADER_SIZE = 16;
    private static int PKM_HEADER_WIDTH_OFFSET = 8;
    private static int PKM_HEADER_HEIGHT_OFFSET = 10;

    protected int loadETC2Texture(String filename, int compression, Boolean bClamp, Boolean bHighQuality) {
       int[] textures = new int[1];
       GLES20.glGenTextures(1, textures, 0);


       int textureID = textures[0];
       GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureID);


       if (bHighQuality) {
           GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
           GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
       } else {
           GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST);
           GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
       }


       if (bClamp) {
           GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
           GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
       } else {
           GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_REPEAT);
           GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_REPEAT);
       }


       InputStream is = null;
       try {
           is = mWallpaper.getContext().getAssets().open(filename);
       } catch (IOException e1) {
           e1.printStackTrace();
       }


       try {
           byte[] data = readFile(is);


           ByteBuffer buffer = ByteBuffer.allocateDirect(data.length).order(ByteOrder.LITTLE_ENDIAN);
           buffer.put(data).position(PKM_HEADER_SIZE);


           ByteBuffer header = ByteBuffer.allocateDirect(PKM_HEADER_SIZE).order(ByteOrder.BIG_ENDIAN);
           header.put(data, 0, PKM_HEADER_SIZE).position(0);


           int width = header.getShort(PKM_HEADER_WIDTH_OFFSET);
           int height = header.getShort(PKM_HEADER_HEIGHT_OFFSET);


           GLES20.glCompressedTexImage2D(GLES20.GL_TEXTURE_2D, 0, compression, width, height, 0, data.length - PKM_HEADER_SIZE, buffer);
           checkGlError("Loading of ETC2 texture; call to glCompressedTexImage2D()");
       } catch (Exception e) {
           Log.w(TAG, "Could not load ETC2 texture: " + e);
       } finally {
           try {
               is.close();
           } catch (IOException e) {
               // ignore exception thrown from close.
           }
       }


       return textureID;
   }


...
if (isES3()) {
   textureID = loadETC2Texture("textures/etc2/sky1.pkm", GLES30.GL_COMPRESSED_RGB8_ETC2, false, false);
} else {
   textureID = loadTexture("textures/sky1.png");
}
...


So if OpenGL ES 3.0 context is available app will use ETC2 texture and will fall back to uncompressed image in case of OpenGL ES 2.0 context.
Of course, to use OpenGL ES 3.0 you need to specify android:targetSdkVersion="18" in manifest file and target=android-18 in project.properties.

Result

In app difference between ETC2-compressed and uncompressed image is not visible (however, you can notice some color banding, this is because of 16-bit color used in app).

Conclusion

We hope this small article will help somebody to improve their apps with help of our code and compression recommendations.

6 comments:

  1. This comment has been removed by a blog administrator.

    ReplyDelete
  2. What are the values of
    PKM_HEADER_WIDTH_OFFSET & PKM_HEADER_HEIGHT_OFFSET

    ReplyDelete
    Replies
    1. Hey, thank you for pointing this out.

      PKM_HEADER_SIZE = 16
      PKM_HEADER_WIDTH_OFFSET = 8
      PKM_HEADER_HEIGHT_OFFSET = 10


      I've updated code excerpt in post too.

      Delete
    2. I have used these values and now I am getting proper value for width & height but the texture showing is only black. Do we need any other change in code or shader? simple bitmap texture is loading with my code. I have only changed the texture load function.

      Delete
    3. Are you using correct `compression` parameter? It should be `GLES30.GL_COMPRESSED_RGB8_ETC2`, for example:

      loadETC2Texture("textures/etc2/sky1.pkm", GLES30.GL_COMPRESSED_RGB8_ETC2, false, false);

      Delete
    4. Thanks Popov.
      It was the compression format issue. Would like to share it with others.

      In RGB8_ETC2 format the compress size of each pixel is 8 bit so image size would be (width/4)*(height/4)*8

      But I was using RGBA8_ETC2_EAC which uses 16 bit per pixels to store the alpha. Thats why the image size should be:
      (width/4)*(height/4)*16

      Delete