Advanced Testing Tips for Android, Part 1

I’m currently developing an Android app and have been reading up on Android testing. Here are some tips that I’ve come across and found useful, as well as some ideas of my own.

Emulator vs Device

Testing on a real device is generally recommended over using the Android emulator for various reasons:

  • You should test as close as possible to the user environment.
  • A device is a more realistic environment for acceptance testing.
  • The emulator is slow, and often buggy. I’ve encountered issues with various versions of the emulator, when trying to test things like orientation changes, location services, etc.

While I agree that you should do most of your testing on real devices, sometimes testing on the emulator can be a useful addition.

Advantages of Testing on the Emulator

1. You can increase memory and heap for an AVD (Android Virtual Device).

This may be necessary if you have a lot of tests to run together or have complicated tests (e.g. if using recursion). Of course devices are limited to how much memory they have installed and could run out of memory when running test suites.

2. For testing with particular configurations, e.g system settings such as GPS.

You can setup an AVD with GPS on and another exactly the same except with GPS off (once configured, they would be saved as snapshots). Then you could have annotations in the test cases for which tests to run in which environment.

If you were testing this with a device, then you would need to remember to configure the settings before the tests are run. This may be alright for manual testing, but a major hassle for automated testing (for instance, if you’re using Continuous Integration practices).

In fact, a tool like the Jenkins Android Emulator plugin could be configured to start the appropriate AVD in the emulator when required (or even create one based on a set of configurations).

3. The emulator is free.

Of course using the emulator is cheaper than buying a truckload of devices, especially with so many Android version and screen sizes out there. You can use the emulator to fill in the gaps for whichever configurations you don’t have devices for (is it worth buying separate devices to test Android version 2.3 vs 2.33, for instance).

Testing on a Rooted Device

Most advice about using rooted devices for testing seems to be, DON’T. Most of the time this is good advice, as you want your test environment to be as close to what your customers are using as much as possible.

However you might find it sometimes useful to run some of your tests on a rooted device. In particular, being able to have read and write access to otherwise secure directories can be useful.

Some examples of this would be:

  • to copy a test database onto the device
  • copying junit report files off the device for archiving
  • doing a full backup of various device configurations, to restore later for testing

Some more tips later in part 2 of this article.

Flash – don’t shoot the messenger

I have been reading that Adobe is stopping development of Flash for mobile devices (official announcement). If fact there have been many recent articles lately about the demise of Flash in general, they seem to argue that Flash is a ‘bad’ web technology and should be dumped in favour of HTML 5.

One of the reasons cited for Flash being bad was that it is used to create those annoying ads and banners. I think this is confusing the Flash technology itself with its usage – in other words blaming Flash when the real culprits are the purposes to which people have used it (or abused it).

As a former web developer, I remember that before Flash came along there were equally annoying popups and banners being produced using animated GIF’s. I suppose if Flash were to go away we would still be getting those annoyances using whichever is the current web animation technology of the day

Don’t get me wrong, I know that Flash technology does have real issues regarding stability, performance, etc – but then just about every web technology has its problems (including HTML when being used to create interactive web applications).

Luckily there are also other articles with a more balanced view, e.g. “Adobe Flash: Still the 800-Pound Gorilla”.

Using load-time AOP to help testing and debugging

After reading this blog ‘Practical Introduction into Code Injection with AspectJ, Javassist, and Java Proxy‘, it reminded me how handy AOP tools like AspectJ are, in testing and debugging applications – particularly on servers.

How many times have you encountered a problem or bug in staging (or common test or reference) servers that you can’t reproduce in your own dev server? Ideally deployments should be consistent and reproducable so that this should never happen, but unfortunately in real projects that is not always the case.

Sometimes various issues can cause these differences:

  • manual configurations to the build process
  • continuous integration not used rigorously
  • different environmental settings on different server instances
  • different external services used in development, as opposed to a staging environment

Factors like these may lead to inconsistent builds or differing behaviour in different server environments.

I’ve found that using load-time weaving with AspectJ can be useful testing and debugging the application code on a staging server, without requiring any source code changes. The good thing is that because it doesn’t change the actual build artifacts on the file system (of course the code is changed when the affected classes are loaded!), you can remove the load-time weaving any time and revert back to the original code without having to do another build.

Here are some examples of what can be done to assist testing and debugging on a server that is not directly controlled by the developer. I’m assuming here that you are able to implement load-time weaving on the server.

Also if you are not familiar with AOP and AspectJ, else please read the AspectJ documentation or a book on AspectJ, such as the excellent AspectJ in Action.


The generic introductory example for demonstrating how useful AOP is, seems to be tracing. The ability to add logging code to trace (program flow, method parameters and return value, etc) is very handy for debugging, particularly when using pointcuts to narrow the scope of the tracing to the sections of code that are problematic.

Here is a simple example with some test classes:

public class Customer {
  private String name;
  private String phone;
  private String address;

  public String getName() {
    return name;
  public void setName(String name) { = name;
  public String getPhone() {
    return phone;
  public void setPhone(String phone) { = phone;
  public String getAddress() {
    return address;
  public void setAddress(String address) {
    this.address = address;

public class AccountService {

  public boolean addCustomer(Customer customer, String accountId, boolean isNewCustomer)
    // real code to add customer to an account would be added here

    return true;

For the tracing code, you can use the examples from the AspectJ programming guide or the book AspectJ in Action.

Then in a trace aspect class, use a pointcut to trace the code we’re interested in (in this case AccountService.addCustomer()).

pointcut traceOperations() : execution(* trace.blogexample.AccountService.addCustomer(..));

When the code is run, you might get some tracing, showing that the code was run and the parameters that were passed to the method in the pointcut.

Enter  [trace.blogexample.AccountService.addCustomer]
  [ This: trace.blogexample.AccountService@6262937c]
  [Args: (trace.blogexample.Customer@35c0e45a,ABC123,true)]
Exit  [trace.blogexample.AccountService.addCustomer]

Even more useful is if you combine the logging code with reflection to trace not only simple values, but also the contents of objects that you are interested in.
For the above example you might get tracing that shows the fields of objects passed to the method as parameters.
Hopefully the info from the tracing may be helpful in fixing a bug or problem.

Enter  [trace.blogexample.AccountService.addCustomer]
  [ This: trace.blogexample.AccountService@1d5a0305]
  [Args: (trace.blogexample.Customer@377653ae[
  name=John Smith
  address=1 Test Street
Exit  [trace.blogexample.AccountService.addCustomer]

As I’ve mentioned, you can target where the want the tracing to occur:

  • by setting the pointcut to trace in details the areas of code that are (or that you think are) causing problems,
  • where exceptions are being thrown (using an After throwing advice)

You can also build on the AOP tracing to do other useful things, for example I have an open source project (TestDataCaptureJ) that uses tracing to capture data to use in unit testing.

Data Validation

Instead of just tracing, you can go further by doing some data validation as well.

For instance you can insert some code to validate the parameters for a business method, and log any validation errors.

Another way is to use tools, such as Contract4J, to test compliance (checking for preconditions, postconditions and invariants in the business requirements). Use load-time weaving to hook in the Contract4J configuration for business methods that you want to test.

Method Replacement

Another technique I sometimes use is to replace methods invokations in the app with test code.
This involves creating an Around advice for a pointcut that includes a method we want to replace.
Then inside the Around advice you can:
1. run another method altogether

So instead of just invoking the original method …

Object returnValue = joinPoint.proceed(args);

invoke another method with the same return type.

Object returnValue = newMethod(args);

2. run with different parameters

Invoke the original method, but pass in different parameter values (of course the parameter types must match the method signature)

Object returnValue = joinPoint.proceed(newArgs);

So using method replacement techniques you can do things like:

  • inject your own data to test a particular scenario
  • proxy external calls, e.g. for a method that uses an external service, replace with a method that returns some test data we can use for diagnosis


These are just some ideas that I have found useful when testing and debugging issues in app development. They can be easily applied to your development environment, but using load-time weaving with AOP also makes them useful when you encounter problems in a server that is not directly under your control.

Of course these techniques do not replace your standard tools and debuggers, it is just another tool in your toolbox.

Using a custom ServiceLoader in Android

For those looking for a way to create modular and extendible code using some sort of ‘plugin’ architecture, one of the possibilities is to use the ServiceLoader class.

A service provider is a factory for creating all known implementations of a particular class or interface S. The known implementations are read from a configuration file in META-INF/services/.

Here are some examples of it’s usage in Java:
Creating Extensible Applications With the Java Platform

What I was after was use it in Android, and to be able to put the service implementation classes in their own libray project or jar file, so they could be used independently in a modular fashion. A possible use of this structure is to have different ‘plugins’ for different versions of an app.

There are other similar java plugin frameworks out there, but I thought I’d stick with something that already exists in Java rather than learning yet another API.

However there are some issues with using ServiceLoader in Android, mostly having to do with where to put the configuration file that contains the list of plugin implementation classes. Unfortunately ServiceLoader is hard-coded to look for the file at ‘META-INF/services/’, which makes it hard if we want to make it work in Android.

This article ‘Using serviceloader on android‘ gives some ideas about how use it with Ant. However I wanted a solution that could also be used with Eclipse.

Customizing ServiceLoader

Because ServiceLoader is a final class, it cannot be subclassed but we can still customize it by making our own copy from the Java SDK source code. For example, let’s get a copy of from JDK 1.6 source, and rename it to CustomServiceLoader.

Looking at the source, the first thing we notice is that the location for the config file is hard-coded:

private static final String PREFIX = "META-INF/services/";

Then here is where that location is used (it is only used when hasNext() is called on the iterator due to the lazy initialization pattern):

public boolean hasNext() {
  if (nextName != null) {
  return true;
  if (configs == null) {
    try {
      String fullName = PREFIX + service.getName();
      if (loader == null)
        configs = ClassLoader.getSystemResources(fullName);
        configs = loader.getResources(fullName);
    } catch (IOException x) {
      fail(service, "Error locating configuration files", x);
  } ...

There are several ways we could customize this class to make it easier to use with Android.

Option 1: Replace the hard-code PREFIX field with your own location

So in our custom class, we could just change the location for the PREFIX constant to point to somewhere else in our classpath. This is the quickest way, but suffers from the same inflexibility as the original ServiceLoader.

private static final String PREFIX = "";

Since the location is being looked up internally using getResources() or getSystemResources() from the classloader, all we have to do is to make sure the configuration file is in the classpath somewhere, and then tell CustomServiceLoader where to search for it.

Another idea for this option was to do it the ‘Android’ way and use the AssetManager to find the configuration file in the assets directory. This would mean passing the AssetManager as an extra parameter in the public API methods for CustomServiceLoader.
However the Android build process will only take into account the assets directory in the main project and ignore the assets directory in library projects, so the file would have to go in the main project. I prefer the configuration to exist in the library along with it’s code so that it could function as an independent unit, so I did not use the assets folder.

Option 2: Pass the location

Rather than hard-coding the location, we could just pass the location as an additional parameter when you want to load the services. This would mean changing the method signatures to pass the location parameter through (or store it) until it is needed. Hence instead of calling …


you would call …

CustomServiceLoader.load(MyService.class, "")

I did not end up using this option, since it would have required more substantial changes to the original class.

This is one of the caveats regarding copying source code and changing it, and why I generally dislike doing this. If you make any significant changes to it, then there is always the chance you will introduce bugs into the code (therefore requires more testing!). Also if the original code base changes, your code will not include those changes.

Making it Modular for Multiple Plugins

The above solutions would work if you only had one plugin (i.e. a jar or project containing some service implementation classes and a configuration file), but what if you wanted to be able to handle any number of plugins. My idea was to have each plugin as a independent unit, each with it’s own configuration file.
Then the app would load all the plugin classes it found and aggregate them to make all the service implementations available.

Why would you want to do this? Well, many Android apps have multiple versions, i.e. a paid vs a free version, or a lite vs a pro version. So you could have 1 plugin for the lite version offering a limited number of services.
Then for the pro/paid version, you could either replace the plugin with another version that offers more services or just add those extra services as addition plugins.

It would require a bit more work on CustomServiceLoader to allow it to handle this situation.

This means that, for instance, if the location you want is at ‘’, then if you have 2 plugin libraries or jars with their configuration files at that same location, then the apk will build since it would consider them to be duplicates:

Error generating final archive: Found duplicate file for APK:

Let’s have a look at the code again:

private static final String PREFIX = "";


public boolean hasNext() {
  if (nextName != null) {
  return true;
  if (configs == null) {
    try {
      String fullName = PREFIX + service.getName();
      if (loader == null)
        configs = ClassLoader.getSystemResources(fullName);
        configs = loader.getResources(fullName);
    } catch (IOException x) {
      fail(service, "Error locating configuration files", x);
  } ...

The problem is that the code is only looking at 1 location as specified by the PREFIX constant, but having the same location in multiple plugin libraries will break the build process due to duplicate file problem.
We can put the configuration files in different locations in each plugin library, but then how does CustomServiceLoader find them?

I’m still working on this, but got some ideas:
1. Add some code to be able to handle wildcards in the location.
Plugin1 would have the location of it’s configuration file at ‘com.yourdomain.services1/’.
Plugin2 would have the location at ‘com.yourdomain.services2/’.
We could pass is a wildcard location, such as ‘*’.

In the Spring Framework, there are classes such as ResourcePatternResolver and PathMatchingResourcePatternResolver that does something like this. This is what enables Spring to find ApplicationContext.xml in different locations. This seems to be the most flexible way of doing it, but requires more work.

2. Another way would be to use option 2 above (passing the location as a parameter), but instead of just 1 location, pass in multiple locations (in a Collection or array) and aggregate the results of the getResources() calls.
This is easier, but far from ideal, as you have to hard-code the locations of the configuration file in each plugin library somewhere.

Tips on handling orientation changes in Android 2

Here are some tips on how to handle and test orientation changes in Android 2

This post applies to Android 2.x, since onRetainNonConfigurationInstance() is deprecated in Android 3.x.

In my previous post I showed one method I use to test orientation changes. The standard way of handling orientation change is to save the results of expensive operations
in onRetainNonConfiguration(), and then when the activity is recreated that data can be re-used.

From the Android developer documentation

Finally, remember that onRetainNonConfigurationChange() should be used only to retain data that is expensive to load. Otherwise, keep it simple and let Android do everything.

And here.

This function is called purely as an optimization, and you must not rely on it being called

This means that onRetainNonConfigurationChange() is supposed to be an optimization, so the activity should be able to recreate the data if required

Now here is a list some things that I will usually save in onRetainNonConfigurationChange().

Expensive operation results – just like the documention says. After all you don’t want your app to be re-doing things like database queries, network calls or complex
calculations every time the user flips the phone.

AsyncTask – there are some situations where a running AsyncTask will be saved, so that it can continue it’s processing after the orientation change (rather than starting the task again). Makes sense since one of reasons to use an AsyncTask is to handle long-running operations.

Intermediate data – I was using some fields in the activity to hold some (non-expensive) data, so I didn’t bother to save it because I thought it was trivial. The result was NullPointerExceptions because that data was required either in the onCreate() or later on, e.g. onDialogCreate()when I wanted to display that data in a message. So you can either save this data, re-create it in onCreate() or at least do a null check before using it.

Maps – if the activity displays a map (e.g. using a MapActivity), then it might be more user friendly to save things like the zoom level, satellite view option, and the centre point of the map.  This is so that the user has a similar map view after the orientation changes.

Some other tips not related to saving state

Dialogs– if you are doing your own dialog handling, instead of letting the activity do it by using onCreateDialog() and showDialog(), then you need to handle the situation
where the dialog is displayed when the orientation change occurs. I generally let the activity do the hard lifting, except maybe in cases where the life cycle of the dialog doesn’t
match the activity, e.g. a progress dialog for an AsyncTask.
Generally in my unit tests, I will follow each dialog display with an orientation change to check that afterwards:
1. the dialog is still displayed
2. if the dialog shows data, that the data hasn’t changed

Layouts – if you have alternate layouts for portrait and landscape, goes without saying that you need to test both! This includes things like testing the order of focus for the views, etc. since the views may be in different positions in the different layouts.
This is the reason that in my previous post: Test Android orientation changes with Robotium I ran the test methods multiple time starting in both portrait and landscape mode.

Some more testing tips

  • Some versions of the emulator may have bugs that show up when your test invokes an orientation change, e.g. the 2.3 version will allow your app to change normally from portrait to landscape, but then it has problems when you try to change back to portrait again. The test will still run OK, the will not look right in the emulator window.
  • If you are using the emulator and have some Robotium or instrumentation test code to click on an item in a list by index, e.g. solo.clickInList(1). Then the orientation change may cause some test failures when going into landscape mode.
    This is because fewer items being displayed in the list in landscape than in portrait mode, of course this is only a problem is there are more items in the list than can be displayed in the view or ‘page’.
    See if you can use solo.clickOnText() instead with the option to allow scrolling if required.

If you can think of anything else that might be useful, please let me know!

Test Android orientation changes with Robotium

One of the most painful things I found while learning Android development was handling orientation changes (who thought flipping your phone would cause so much trouble).

This is because the activities are being destroyed and recreated every time the phone is changing between portrait and  landscape mode. Here I’m assuming that it is being handled ‘properly’ by saving state data (e.g. in onRetainNonConfigurationInstance() ) and then reloading it when the activity is created.

After this saving and reloading code is in place, then it needs to be unit tested, and here I’ve found Robotium to be very useful in running instrumentation tests with orientation changes.

An example

This is just one way that I’ve found useful, here is an example – take this simple test to simulate the user entering some text and clicking some buttons:

public void testValidateText()
  EditText targetText = solo.getEditText(0);
  solo.enterText(targetText, "hello");

  // click on button to bring up dialog

  // wait to dialog to pop up

  // click on button in dialog

For the sake of simplicity, I’ve left out the Robotium setup code and the asserts (which would normally be the point of the testing!).

Ring the (orientation) changes

In the next version of this test, I will:

  • rename the test method to something that doesn’t start with ‘test’, because Android used JUnit 3, this will stop this method from being run in the test case
  • add some parameters to the test method as flags for doing orientation change
  • add to code in the method to change the orientation
  • write some wrapper methods to call our test method
public void testValidateText()
  doTestValidateText(ActivityInfo.SCREEN_ORIENTATION_PORTRAIT, false);

public void testValidateTextWithOrientationChange()
  doTestValidateText(ActivityInfo.SCREEN_ORIENTATION_PORTRAIT, true);

public void testValidateTextInLandscapeWithOrientationChange()
  doTestValidateText(ActivityInfo.SCREEN_ORIENTATION_LANDSCAPE, true);

public void doTestValidateText(int currentOrientation, boolean changeOrientation)
  int orientation = currentOrientation;

  EditText targetText = solo.getEditText(0);
  solo.enterText(targetText, "hello");

  orientation = changeOrientation(solo, changeOrientation, orientation);

  // click on button to bring up dialog

  orientation = changeOrientation(solo, changeOrientation, orientation);

  // wait to dialog to pop up

  orientation = changeOrientation(solo, changeOrientation, orientation);

  // click on button in dialog

So now when the test case is run, the original test method will be run in 3 ways:

  • testValidateText() will run the test as before
  • testValidateTextWithOrientationChange() will run the test, but after every user action will change the orientation with the line
    orientation = changeOrientation(solo, changeOrientation, orientation);
  • testValidateTextInLandscapeWithOrientationChange() will also run the test with orientation changes, but starting off in landscape mode

Here is the code for the orientation change.

public int getNextOrientation(int currentOrientation)
  if (currentOrientation == ActivityInfo.SCREEN_ORIENTATION_PORTRAIT)

public int changeOrientation(Solo solo, int currentOrientation)
  int nextOrientation = getNextOrientation(currentOrientation);

  return nextOrientation;

public int changeOrientation(Solo solo, boolean changeOrientation, int orientation)
  int newOrientation = orientation;
  if (changeOrientation)
    newOrientation = changeOrientation(solo, orientation);
  return newOrientation;

So basically the wrapper methods just invoke the original test method with a flag for the orientation that it starts in, and whether to do orientation changes during the test or not.

In the next post, I will list some problems in my code that I exposed with this testing, and other issues with coding and testing for orientation changes.

TestDataCaptureJ – Capture data for your unit tests

When you are writing unit tests or doing Test Driven Development (TDD) , one of tasks you may need to do is to get some data to run your tests with. If the data that you need is fairly simple then there are various options, such as manually creating the data classes, create some mocks, stick some data in a database, etc.

However if you need a large amount of complex data, then these options can become very time consuming. This was the situation that I’ve come across myself in various projects, in particular where I need to test a the checkout stage in a online shop using data from the basket. The development environment was running on a snapshot of the production database (therefore ‘real’ data) and for the test scenarios I need the data from the shopping basket with various combinations of basket items. However the basket items were quite complex: the basket item class had about 35 fields, some of which were other objects each containing maybe 5 – 15 fields themselves.

To get all the test data I needed to test all the scenarios, I created a tool which evolved into the TestDataCaptureJ open source project that is hosted on GitHub. This is a Java development tool that can be used in java (web) applications to capture data as you run through the test scenarios. The data is logged to file, but in a format that you can just cut and paste into your unit tests (or better still, into classes that are used by your unit tests).

As an example, in the tutorial I used the jpetstore sample app from the Spring Framework. This just meant configuring the jpetstore web app to run with TestDataCaptureJ, and then going through the checkout process with some items in the basket. Then I used the generated log in some unit tests that I wrote to test part of the checkout process. This is an example of the code that was generated:

public org.springframework.samples.jpetstore.domain.Account createParam1Account_org_springframework_samples_jpetstore_domain_Order_initOrder() {

org.springframework.samples.jpetstore.domain.Account account0 = new org.springframework.samples.jpetstore.domain.Account();
account0.setAddress1("901 San Antonio Road");
account0.setAddress2("MS UCUP02-206");
account0.setCity("Palo Alto");

return account0;

The tutorial also demonstrates some of the limitations of the tool as well.


TestDataCaptureJ was really meant to handle data objects which are designed to hold data and follow JavaBean conventions, e.g.

  • objects are created using constructors
  • fields has setter methods using standard naming convention
    e.g. a field named ‘userAccountName’ would have a public setter method ‘setUserAccountName()’

Also to intercept the processing of the application in order to log the data, the test data required must be an object that is either passed to a method as a parameter, or returned from a method as a return value.

Therefore it can’t currently handle object where this isn’t the case, e.g.

  • object that are not created with constructors, e.g. if they are created using factory methods instead
  • fields without setter methods or setter methods that don’t follow the standard naming convention
  • static fields (just not implemented)
  • objects pass into methods as varargs

There is some configuration that

How it works

Basically this is just a glorified version of the common AspectJ tracing example, using weave time loading to intercept the data objects that we are interested in. Only instead of just logging the contents, there is a 2 stage process:

  1. use java reflection to store access the field data recursively and store it in some metadata classes
  2. log the data as java code to file

There is a explanation page in the documentation that goes into more detail.

Please have a look at the code (or better still, fork it and play around with it) or read the documentation if you think this might be useful to you.