Friday, December 28, 2012

Using JNI_OnLoad() in Adroid NDK development




Previous parts

  Last time I wrote how to combine C and Java for Android app to load textures. I also mentioned I will write some short article on how and why to use JNI_OnLOad() method. So, here it is.

JNI_OnLoad()  - am I forced to use it?

 The answer is: no. If you do not want you do not have to implement this method. But if you do so you can  gain some benefits from it. These benefits includes java class instance caching and native method registration.
 The JNI_OnLoad is called when the native library is loaded. If you do not implement this method then you can see "No JNI_OnLoad found in ..." message in your Logcat view. This is not error just debug message. So if your code does not work the reason is not because of this message...

 If you did not implement JNI_OnLoad() and did not register your native methods, you have probably something like this in your code:

jstring Java_com_example_hellojni_HelloJni_stringFromJNI(JNIEnv* env, jobject thiz)
{
    return (*env)->NewStringUTF(env, "Hello from JNI !");
}

 The example was taken from Android NDK example hello-jni.c. In the name of the method you have to put in "Java_". Then the whole package name "com.example.hellojni" but with dots replaced with underscores. Then class name "HelloJni" and finally the name of the native method. Of course, you can use javah.exe to generate this for you but it would be better to bypass it somehow.

Implementation

 Here is the implementation of JNI_OnLoad() as I have it in my engine:

extern "C"
{
JavaVM* gJavaVM = NULL;
jobject gJavaActivityClass;
const char* kJavActivityClassPath = "com/sbcgames/sbcengine/SBCEngine";

static JNINativeMethod methodTable[] = {
  {"engine_tick", "()V", (void *) engine_tick},
  {"engine_start", "(Landroid/content/res/AssetManager;)V", (void *) engine_start},
  {"engine_stop", "(Z)V", (void *) engine_stop},
  {"engine_pause", "()V", (void *) engine_pause},
  {"engine_resume", "()V", (void *) engine_resume},
  {"engine_message", "(III)V", (void *) engine_message},
  {"engine_set_screen_size", "(II)V", (void *) engine_set_screen_size},
  {"engine_on_touch", "(III)V", (void *) engine_on_touch},
};

//------------------------------------------------------------------------
jint JNI_OnLoad(JavaVM* aVm, void* aReserved)
{
 // cache java VM
 gJavaVM = aVm;

 JNIEnv* env;
 if (aVm->GetEnv(reinterpret_cast<void**>(&env), JNI_VERSION_1_6) != JNI_OK)
 {
  LOGE("Failed to get the environment");
  return -1;
 }

 // Get SBCEngine activity class
 jclass activityClass = env->FindClass(kJavActivityClassPath);
 if (!activityClass)
 {
  LOGE("failed to get %s class reference", kJavActivityClassPath);
  return -1;
 }
 gJavaActivityClass = env->NewGlobalRef(activityClass);

 // Register methods with env->RegisterNatives.
 env->RegisterNatives(activityClass, methodTable, sizeof(methodTable) / sizeof(methodTable[0]));

 return JNI_VERSION_1_6;
}

} // extern "C"
 
 On the top there are two variables: gJavaVM and gJavaActivityClass (g means these methods are global). In these variables I will cache object that I use during game in it (remember using it in previous article on loading textures).

 Next is constant string that contains Java package name and Java class that contains native methods. The dots are replaced with slashes.

 It is followed with table of native methods. It is not important what each of these methods do in the engine. What is important is the structure. For each method there are three attributes:
  • the method name - the same as in Java
  • the method signature - the signature follows the rules for JNI Types and Data structures (see detailed doc at Oracle)
  • the method function pointer - the void* pointer to C implementation of the method
 The method itself first caches java virtual machine and engine activity class. Then it simply registers all the native methods with RegisterNatives() call. For example the engine_stop method that returns void and takes bool does not need to look like this more:
void Java_com_sbcgames_sbcengine_SBCEngine_engine_stop(
        JNIEnv* aEnv, jobject aObj, jboolean aTerminating)
{
  :
  :
}

 but it can look as friendly as this:

void engine_stop(JNIEnv* aEnv, jobject aObj, jboolean aTerminating)
{
  :
  :
}

 However, note that every single native method takes as its first two arguments "JNIEnv* aEnv" and  "jobject aObj". These arguments are not part of method signature in registration table but you have to include them.



Friday, December 7, 2012

Load images under Android with NDK and JNI




Previous parts
  This snippet will focus on loading images on Android NDK using JNI. Our cross-platform mobile engine runs on Android, bada and desktop Windows. In last part I described two ways how to load images on Windows platform. Similar to that I again wanted solution that will load .png image as well as .jpg image.

 The solution that works good for me is split into C and Java part. The actual image loading and texture creating takes part in Java part. It also works either in fully native app with its C main loop and using native_app_glue or in Java app with GLSurfaceView.Renderer calling native methods in onDrawFrame method.

 The java part not only loads image but also creates openGL texture for me. If you are interested only in loading the image you will have to adjust it a little for your needs.

Native part

  The initial part of the method is split depending on whether you are using native_app_glue or not. If yes then first call to getApplication is actually returning android_app* type. This structure is part of parameter list of native main method. In my engine it is typedefed to SBC::System::Application (typedef struct android_app Application;). If you are not writing fully native app then you will have to use some variables previously cached (gJavaVM, gJavaActivityClass). I am caching it in JNI_OnLoad which is called when native library is loaded (see another article on JNI_OnLoad).
void Texture::construct(u8* aFileName, u32 aIdx)
{
 JNIEnv *env;

#ifdef USE_NATIVE_APP_GLUE
 SBC::System::Application* app = &Game::getGame().getApplication();

 JavaVM* vm = app->activity->vm;
 vm->AttachCurrentThread (&env, NULL);
 jclass activityClass = env->GetObjectClass(app->activity->clazz);
#else
 bool shouldDetach = false;
 JavaVM* vm = gJavaVM;
 jint rc = vm->GetEnv((void **)&env, JNI_VERSION_1_6);
 if (rc != JNI_OK)
 {
  shouldDetach = true;
  vm->AttachCurrentThread(&env, NULL);
 }
 jclass& activityClass = gJavaActivityClass;
#endif

  Then main part of the method follows:
 jmethodID mid = env->GetStaticMethodID(activityClass, "loadTexture", "(Ljava/lang/String;I)I");
 jstring mystr = env->NewStringUTF(aFileName);
 jint ret = env->CallStaticIntMethod(activityClass, mid, mystr, aIdx);
 env->DeleteLocalRef(mystr);

 // store information on ID, width and height of texture
 mTextureID = ret;

 mid = env->GetStaticMethodID(activityClass, "getTextureWidth", "()I");
 mWidth = env->CallStaticIntMethod(activityClass, mid);
 mid = env->GetStaticMethodID(activityClass, "getTextureHeight", "()I");
 mHeight = env->CallStaticIntMethod(activityClass, mid);

 mTextureID, mWidth and mHeight are member variables of the Texture class. The ID is OpenGL ID of the texture. So I am simply calling java methods that will do all the work. After that I just "clean" with detaching if necessary:

#ifdef USE_NATIVE_APP_GLUE
 vm->DetachCurrentThread();
#else
 if (shouldDetach)
  vm->DetachCurrentThread();
#endif

 LOGI("texture ID %i, width %i, height %i", mTextureID, mWidth, mHeight);
} 

Java part

 The main things are going on in java class. I call it Tools as the image/texture loading is not its only purpose. But only this is subject of this article. In the beginning I just open file fname for reading. The second parameter beyond the file name (int id) is used when using compound files (single file made from multiple other files with some offsets header in the beginning). If using single file then this parameter contains -1.
package com.sbcgames.sbcengine;

import java.io.DataInputStream;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.util.Locale;

import android.content.res.AssetManager;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.graphics.Canvas;
import android.opengl.GLES20;
import android.opengl.GLUtils;
import android.util.Log;

/*
 * Help methods for doing things I cannot do from my native code now...
 * Hope, this class will get smaller and smaller when it finally disappears...
 */

public class Tools
{
 //-----------------------------------------------------
 // BITMAP
 //-----------------------------------------------------
 
 // last loaded texture parameters
 static int txtID, width, height;
 
 //-----------------------------------------------------
 /* fname = asset file name
  * id >= 0 ... position in compound file
  * id < 0 ... single file (no compound)
  */
 public static int loadTexture(String fname, int id)
 {
  // clear last texture parameters
  txtID = width = height = -1;
  
  Log.d("Helper", "Loading texture from asset file " + fname + " with id " + id);

  final BitmapFactory.Options options = new BitmapFactory.Options();
  options.inScaled = false;    // No pre-scaling
  AssetManager am = SBCEngine.getSBCEngine().getAssets();
  Bitmap bitmap = null;

  try
  {
   InputStream stream = am.open(fname);

 After the file is opened I can start loading the image. Again there is split depending on whether the file is compound or not.
   // loading from compound file?
   if (id >= 0)
   {
    DataInputStream input = new DataInputStream(stream);
    
    // skip header
    input.skip(3);
    // skip to entry offset
    input.skip(id * 4);
    // read entry beginning
    int dataStart = input.readInt();
    // read data length
    int dataLen = input.readInt() - dataStart;
    // skip to start of subfile
    // offsets are without header (3) bytes
    // we already skipped id * 4 bytes
    // we already have read 2 offset by 4 bytes = 8 in total
    input.skip(dataStart - (id * 4) - 8);

    // get data from correct position
    byte[] data = new byte[dataLen];
    input.read(data);

    bitmap = BitmapFactory.decodeByteArray(data, 0, data.length, options);
   }
   else // no compound
   {
    Log.d("Helper", "Loading from stream");
    bitmap = BitmapFactory.decodeStream(stream, null, options);
   }

 Here we can test whether we are succesful and have bitmap loaded

   // test returned bitmap for success
   if (bitmap == null)
   {
    Log.e("Helper", "Failed to load texture " + fname + " with id " + id);
   }

 If yes, we can continue to creating OpenGL texture from it. First we also check whether the image height and width is power of 2. If not we put message into log and then we convert it to nearest power of two image.
   // check whether the loaded bitmap has width and height equal to power of 2
   int w = bitmap.getWidth();
   int h = bitmap.getHeight();
   if (getNearestPOT(w) != w || getNearestPOT(h) != h)
   {
    Log.w("Helper", "Texture " + fname + " with id " + id +
      " has not either width or height power of 2");
    
    // new dimensions
    w = getNearestPOT(w);
    h = getNearestPOT(h);
    
    // redraw bitmap into POT bitmap
    Bitmap newBitmap = Bitmap.createBitmap(w, h, Bitmap.Config.ARGB_8888);
    Canvas canvas = new Canvas(newBitmap);
    canvas.drawBitmap(bitmap, 0.0f, 0.0f, null);
    bitmap.recycle();
    canvas = null;
    bitmap = newBitmap;
    
    Log.w("Helper", "Texture " + fname + " rebuilded into texture with POT");
   }

 From the bitmap (possibly rebuilt into new power of two image) we create OpenGL texture like this:
   // generate textureID
   int[] textures = new int[1];
   GLES20.glGenTextures(1, textures, 0);
   int textureID = textures[0];
   
   // create texture
   GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureID);
   GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
   GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
   GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
   GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
   GLUtils.texImage2D(GLES20.GL_TEXTURE_2D, 0, bitmap, 0);

 And finally we do some clean up:
   // destroy bitmap
   bitmap.recycle();
   
   txtID = textureID;
   width = w;
   height = h;
   
   Log.d("Helper", "Loaded texture ID:" + textureID + ", width:" + w + ", height:" + h);
   return textureID;
  }
  catch (IOException e)
  {
   Log.e("Helper", "Failed to load texture " + fname + " with id " + id);
   return 0;
  }
 }

 The other Java methods called from C or from within the Java loadTexture method are these:
        //------------------------------------------------------------------------
 public static int getTextureWidth()
 {
  return width;
 }
 
 //------------------------------------------------------------------------
 public static int getTextureHeight()
 {
  return height;
 }
 
 //------------------------------------------------------------------------
 private static int getNearestPOT(int val)
 {
  int newDim = 1;
  while(val > newDim) newDim *= 2;
  return newDim;
 }
 

 It is probably pretty stupid to call three methods to get texture first and then to get its width and height. Better way would be to return some kind of object with three members. But it works and as I will get deeper into NDK/JNI I will adjust it in future.

 I hope this article helped you to write your own image loading class. The final class can load OpenGL textures from .png, .jpg files in your asset directory and it can also adjust images that are not power of two to make OpenGL happy.



Saturday, December 1, 2012

Loading images under Windows



Previous parts
  Today I will describe how I managed loading of images for Win platform in my small cross-platform mobile engine. If you wonder why I support desktop Win in mobile engine read this: Simple cross-platform game engine - Introduction. But shortly: Win with OpenGL ES emulation is not primary target for me. It is convenient way how to get the game run fast, easy way how to share progress with team mates, comfortable debugging, ...

 Before you can create texture in OpenGL you have to get the texture data somehow. As Android platform in my engine can handle both .png and .jpg I wanted to have the same in Win. Finally I got 2 ways working on my PC through:

 Microsoft WIC

 WIC was the first way. The following routine will load image (.png, .jpg, ...) and return unsigned char* to raw RGBA data. You can take this data and create texture from it. You will also have to add library windowscodecs.lib.

1. convert char* file name to  WCHAR
u8* AssetLoader::loadImage(s32 aIdx, u32& aWidth, u32& aHeight)
{
 // convert string to wchar
 int len = strlen(mActualFileName);
 if (len + 1 > 256)
  LOGE("ERROR loading file - filename is loner than 256 characters");
 // long enough
 WCHAR wFileName[256];
 MultiByteToWideChar(0, 0, mActualFileName, -1, wFileName, 256);

2. prepare variables and initialize WIC Imaging Factory
 HRESULT hr = S_OK;
 IWICImagingFactory* pImagingFactory = NULL;
 IWICBitmapDecoder* pIDecoder = NULL;
 IWICBitmapFrameDecode* pIDecoderFrame  = NULL;
 IWICFormatConverter* pIFormatConverter = NULL;

 hr = CoCreateInstance(
  CLSID_WICImagingFactory,
  NULL,
  CLSCTX_INPROC_SERVER,
  IID_IWICImagingFactory,
  (LPVOID*) &pImagingFactory);

3. create decoder and decode first frame of the image
 // Create the decoder.
 if (SUCCEEDED(hr))
  hr = pImagingFactory->CreateDecoderFromFilename(
   wFileName,                      // previously created filename
   NULL,                           // Do not prefer a particular vendor
   GENERIC_READ,                   // Desired read access to the file
   WICDecodeMetadataCacheOnDemand, // Cache metadata when needed
   &pIDecoder                      // pointer to created decoder
   );

 // Retrieve the first bitmap frame.
 if (SUCCEEDED(hr))
  hr = pIDecoder->GetFrame(0, &pIDecoderFrame);

4. create and initialize the convertor. We will use it to convert the image into RGBA format
 // Create convertor.
 if (SUCCEEDED(hr))
  hr = pImagingFactory->CreateFormatConverter(&pIFormatConverter);

 // Initialize the format converter.
 if (SUCCEEDED(hr))
  hr = pIFormatConverter->Initialize(
  pIDecoderFrame,                  // Input source to convert
  GUID_WICPixelFormat32bppPRGBA,   // Destination pixel format
  WICBitmapDitherTypeNone,         // Specified dither pattern
  NULL,                            // Specify a particular palette 
  0.f,                             // Alpha threshold
  WICBitmapPaletteTypeCustom       // Palette translation type
  );

5. convert the image and copy its pixels
 // get pixels in RGBA format
 u8* buffer = NULL;
 if (SUCCEEDED(hr))
 {
  pIFormatConverter->GetSize(&aWidth, &aHeight);
  buffer = new u8[aWidth * aHeight * 4];
  pIFormatConverter->CopyPixels(0, aWidth * 4, aWidth * aHeight * 4, buffer);
 }

6. clean and return
 pIFormatConverter->Release();
 pIDecoderFrame->Release();
 pIDecoder->Release();

 return buffer;
}

 In this way the loading of images worked fine for me. But our graphician got only black screen. And you will agree that black screen for someone whou should see how his graphics looks in game like is not good at all. It seems that WIC is supported from Windows XP SP3. And "something missing" is probably the reason why it returned only black screen. As I had no opportunity to test on his computer I started to look for something that is less dependant on windows versions. I found open source library FreeImage.

FreeImage

 FreeImage has very clearly and well written documentation with lot of examples. The library is capable of many things but my only target was to load image (regardless the format at the best). So, to make FreeImage work download the header, .lib file and .dll. Set path to header and to FreeImage.lib file.

1. add header and create variables
#include <FreeImage.h>

u8* AssetLoader::loadImage(s32 aIdx, u32& aWidth, u32& aHeight)
{
 FIBITMAP* bitmap = NULL;
 FREE_IMAGE_FORMAT fif = FIF_UNKNOWN;

2. load the image. This part was directly taken from documentation with copy&paste (just very slightly adjusted). You can see it is very simple to load image while checking lot of situations
 // check the file signature and deduce its format
 // (the second argument is currently not used by FreeImage)
 fif = FreeImage_GetFileType(mActualFileName, 0);
 if(fif == FIF_UNKNOWN)
 {
  // no signature ?
  // try to guess the file format from the file extension
  fif = FreeImage_GetFIFFromFilename(mActualFileName);
 }
 
 // check that the plugin has reading capabilities ...
 if((fif != FIF_UNKNOWN) && FreeImage_FIFSupportsReading(fif))
 {
  // ok, let's load the file
  bitmap = FreeImage_Load(fif, mActualFileName, 0);
  // unless a bad file format, we are done !
  if (!bitmap)
  {
   LOGE("loading bitmap %s failed", mActualFileName);
   return NULL;
  }
 }

3. convert image into 32bpp RGBA format
 // convert to 32bpp
 FIBITMAP* bitmap32 = FreeImage_ConvertTo32Bits(bitmap);
 // FreeImage bitmaps are always upside down
 FreeImage_FlipVertical(bitmap32);
 //retrieve the image data and get the image width and height
 RGBQUAD* bits = (RGBQUAD*) FreeImage_GetBits(bitmap32);
 aWidth = FreeImage_GetWidth(bitmap32);
 aHeight = FreeImage_GetHeight(bitmap32);
 // check results
 if((bits == NULL) || (aWidth <= 0) || (aHeight <= 0))
 {
  LOGE("bitmap is somehow corrupted (width=%i, height=%i, bits=%i)", aWidth, aHeight, bits);
  return NULL;
 }

4. switch red and blue color
 u8* buffer = new u8[aWidth * aHeight * 4];
 RGBQUAD* dest = (RGBQUAD*) buffer;

 for(u32 i = 0; i < aHeight; i++)
 {
  for (u32 j = 0; j < aWidth; j++)
  {
   RGBQUAD colorQuad = *(bits ++);

   // swap red and blue
   BYTE rgbTemp = colorQuad.rgbBlue;
   colorQuad.rgbBlue = colorQuad.rgbRed;
   colorQuad.rgbRed = rgbTemp;

   *(dest ++) = colorQuad;
  }
 }

5. clean and return data
 // clean
 FreeImage_Unload(bitmap);
 FreeImage_Unload(bitmap32);

 return buffer;
}

 When first compiling I had problem saying that RGBQUAD is not defined. RGBQUAD is structure defined in wingdi.h which is included in windows.h. As I have NOGDI defined in my project (for some reasons) this structure was undefined. I had to alter slightly the FreeImage.h header file to overcome this. This part ...
 :
 : 
typedef unsigned __int64 UINT64;
#endif // _MSC_VER

#if (defined(_WIN32) || defined(__WIN32__))
#pragma pack(push, 1)
#else
#pragma pack(1)
#endif // WIN32

typedef struct tagRGBQUAD {
#if FREEIMAGE_COLORORDER == FREEIMAGE_COLORORDER_BGR 
 :
 : 

 ... was changed to this:
 :
 :
typedef unsigned __int64 UINT64;
#endif // _MSC_VER
#endif // _WINDOWS_                                                *******

#if (defined(_WIN32) || defined(__WIN32__))
#pragma pack(push, 1)
#else
#pragma pack(1)
#endif // WIN32

#if !defined(_WINDOWS_) || (defined(_WINDOWS_) && defined(NOGDI))  *******
typedef struct tagRGBQUAD {
#if FREEIMAGE_COLORORDER == FREEIMAGE_COLORORDER_BGR
 :
 :

 So, here are two ways how you can get image data for your textures. Next time I will write how is this achieved for Android platform in my engine using NDK and JNI.