Pluggable Tools with Docker Data Containers

There are some apps that have a simple installation process. When using them with other applications in Docker, they may be able to be installed in their own data volume container and used in a pluggable way.

The kind of apps I’m talking about are some Java apps (and in fact, Java itself) which follow this installation process:

  1. Install the contents of the app into a single directory
  2. Set an environmental variable to point to the installation directory, e.g. XXX_HOME
  3. Add the executables of the app to the PATH environmental

 

That’s it.

An example of an app installation that follows this pattern is Gradle:

  1. Uncompress the Gradle files from an archive to a directory.
  2. Set the enviromental variable GRADLE_HOME to point to the gradle installation directory
  3. Add GRADLE_HOME/bin to the PATH

 

Docker

Using Gradle as an example, here is a Dockerfile that installs it in a data volume container:

# Install Gradle as a data volume container. 
#
# The app container that uses this container will need to set the Gradle environmental variables.
# - set GRADLE_HOME to the gradle installation directory
# - add the /bin directory under the gradle directory to the PATH

FROM mini/base

MAINTAINER David Wong

# setup location for installation
ENV INSTALL_LOCATION /opt

# install Gradle version required
ENV GRADLE_VERSION 2.2.1

WORKDIR ${INSTALL_LOCATION}
RUN curl -L -O http://services.gradle.org/distributions/gradle-${GRADLE_VERSION}-bin.zip && \
    unzip -qo gradle-${GRADLE_VERSION}-bin.zip && \
    rm -rf gradle-${GRADLE_VERSION}-bin.zip
    
# to make the container more portable, the installation directory name is changed from the default
# gradle-${GRADLE_VERSION} to just gradle, with the version number stored in a text file for reference
# e.g. instead of /opt/gradle-2.2.1, the directory will be /opt/gradle

RUN mv gradle-${GRADLE_VERSION} gradle && \
    echo ${GRADLE_VERSION} > gradle/version
    
VOLUME ${INSTALL_LOCATION}/gradle

# echo to make it easy to grep
CMD /bin/sh -c '/bin/echo Data container for Gradle'

(From github https://github.com/davidwong/docker/blob/master/gradle/Dockerfile)

Build the image and container from the Dockerfile. Here I’ve tagged the image with the version number of the Gradle installation, and named the container gradle-2.2.1.


docker build -t yourrepo/gradle:2.2.1 .

docker run -i -t --name gradle-2.2.1 yourrepo/gradle:2.2.1

A few things to note about this installation:

  • I have changed the directory name where Gradle is installed from the default, by removing the version number in order to make it generic.
  • no environmental variables have been set, that will be done later
  • you can use any minimal image as the basis for the container, it justs need curl or wget in order to download the Gradle archive file

Now we have the Gradle installation in a docker data volume that can be persisted and shared by other containers.

You can then repeat this process with difference versions of Gradle to create separate data containers for each version (of course giving the containers different names, e.g. gradle-2.2.1, gradle-1.9, etc).

Use Case

I originally got this idea when I was running my Jenkins CI docker container. Some of the Jenkins builds required Gradle 2.x while others were using Gradle 1.x.

So instead of building multiple Jenkins + Gradle images for the different versions of Gradle required, I can now just run the Jenkins container with the appropriate Gradle data container. This is done by using –volumes-from to get access to the Gradle installation directory and setting the require environmental variables.

To use the data container with Gradle 2.2.1 installed:

docker run -i -t --volumes-from gradle-2.2.1 -e GRADLE_HOME=/opt/gradle -e PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/gradle/bin myjenkins</pre>

To use the one with Gradle 1.9:

docker run -i -t --volumes-from gradle-1.9 -e GRADLE_HOME=/opt/gradle -e PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/gradle/bin myjenkins</pre>

Of course there are limitations to this technique since Docker data volume containers were designed to share persistant data rather than application installs. In particular they do not allow sharing of environmental variables.

However this work around that can be useful in some circumstances.

Backup a Docker Data Container with Fig

I have been using data volume containers to persist data in docker containers.  There are various reasons why this tends to be a better option than just using data volumes, but probably the most important is portability.

Of course now we have to backup the data in the data containers. This can be for archiving, or when the containers that use the data need to be upgraded or recreated. If your backup requirements are simple you can simply use the docker cp command or something like tar.

A Jenkins example

As a simple example, let’s run a Jenkins server in a docker container and use a data volume container to persist its data.

1. Pull or build a Jenkins image from the official repository.

http://jenkins-ci.org/content/official-jenkins-lts-docker-image

2. The Jenkins images uses the directory /var/jenkins_home as the volume to store it’s data, so we need a data volume container for that volume. Here is a sample of a Dockerfile to build the data container:

Build and tag the image from the Dockerfile.

docker build -t your_repository:jenkins-data .

You can now create the data container, giving it a name for convenience. Optionally we can run the docker ps command afterwards to check that the container has been created, it should be in a stopped state.

docker run -i -t --name jenkins-data your_repository:jenkins-data
docker ps -a

3. Run the Jenkins server with the data container attached and make some changes, e.g. create a job, etc. The Jenkins data volume should have your changes in it now.

docker run --name=jenkins-sample -p 8080:8080 --volumes-from=jenkins-data jenkins

4. For this example we will use tar to backup the data container, using this command to create a temporary container to access the data container.

docker run -rm --volumes-from jenkins-data -v $(pwd):/backup busybox tar cvf /backup/jenkins_backup.tar /var/jenkins_home

There should now be a file jenkins_backup.tar in the current directory. Of course for real usage, we would probably run this command from a script and make it generic to be able to backup any data volume container.

I do give a fig …

Something else I use for development with Docker is the orchestration tool Fig (this has saved me a lot of typing!). So here is an example of  doing the same backup on the Jenkins data container using Fig.

1. Create Fig YAML file, using the same information that we used in the backup command.

2. Run Fig, that’s it!

fig up

This is a simple example that has only scratched the surface of what can be done with Docker (and Fig). If the backup requirements for the data is more complex, then you could also consider creating a dedicated container just for doing backups, with all the required tools installed in it.

The great thing about Docker is that once everything has been setup, you can get applications such as Jenkins up and running very quickly.

Another Defection to Android Studio

Like many other developers out there, I have been using Eclipse as my main IDE for many years now. However for Android development I have decided to take the plunge and migrate to Android Studio (especially since it has finally been released).

Here is a blog post I found that closely echoes what I have long thought regarding the issues with Eclipse:

http://engineering.meetme.com/2014/02/a-tale-of-migrating-from-eclipse-to-android-studio/

Build, build, build

For me, another reason was that the Ant build files I was using to handle building different versions (free vs paid, dev vs release, etc) were getting too complicated to manage easily. So I can now change over to Gradle at the same time, since that’s what Android Studio uses by default.

Gradle has the concept of build variants to handle building different versions of an Android app.

The Recurring Eclipse Re-install

Here are some other problems that I personally have had with using Eclipse.

  • Plugins, well not the plugins themselves, but having too many plugins. I’ve found that having lots of plugins in one Eclipse installation can cause Eclipse to misbehave , especially after several updates. There are several ways I use to get around this:
    • Keep separate Eclipse installations for different types of development, e.g one for Java, one for Android, one for Cloud, etc. Therefore each installation will only have a few plugins relevant to the type of development. However this is not always convenient if a project does require multiple types of development.
    • Every so often, when Eclipse starts to play up, do a fresh re-install of Eclipse (along with the latest version of the plugins required).
  • Intermittent miscellaneous bugs, e.g. cut and paste stops working, builds not alway done automatically, etc. A lot of these issues are more of a nuisance rather than being a serious problem, but all the same it tends to kill your productivity (and isn’t that why we use IDE’s in the first place?).

No Pain, No …

Make no mistake, despite what the Android Studio documentation might try to tell you, migrating a non-trivial project will take some time and probably involve some pain. But worth the effort I think.

User changes for Address Location Finder

I’m currently working on upgrading my app Address Location Finder. While most of the changes are internal improvements or bug fixes, there are 2 major changes for users.

1. The map will be dropped from the app.

The simple built-in map screen will be removed from the app for the next version. In the future it will come back as an optional add-on.

Why?

I was getting quite a few error reports from users trying to run the app on devices that did not have the mapping requirements.

One of the requirements stated in the Google Play app store for the app was:

– device that supports the standard Google Mapping API (not the same as having the Google Maps app installed)

This means the manufacturers need to have licensed the Google Mapping API v1 for the device in order for the map in Address Location Finder to work. Unfortunately it may be difficult for users to know whether their device has this requirement or not, possibly resulting in the app crashing.

So, removing the map from the app will remove this requirement and allow it to run on more devices (as well as reducing app crashes).

The map will become an optional add-on app for a future version of Address Location Finder.

2. The app will require Android 4.0.3+ to run.

The minimum API level to run the app will be raised to 15, which means it will now require Android version 4.0.3+ on the device to run.

Why?

The original minimum API level required for Address Location Finder was 8 (Android 2.2).

According to the Google dashboard for platform version, devices running Android 2.x now account for only about 10%.

Supporting old versions requires a fair bit of additional work

  • needs more testing
  • requires additional code, e.g. Android support library, to implement newer Android functionality
  • may require internal version checks for functions that would not work on old versions, possibly with alternate code

This all means it takes longer for updates of the app to come out. The diminishing returns in supporting the old versions is not really worth the extra effort and time.

Users with Android older than 4.0.3 who have already installed the app can just continue to use the old version of the app. They should not even see the new version of the app when it comes to the Google Play app store, since the store applies various filters to determine which apps to display (include the minimum Android version require to run).

Help! my Android USB Connection is dropping out

After updating Android on a phone that I use for testing, I started having problems when I connected it up to my PC. The USB connection would be there when I connected the cable, but then would seem to drop out after a short time. Sometimes it would disappear after 10 seconds, sometimes a couple of minutes, sometimes when I started up DDMS or Eclipse.

Very frustrating, but I hoped it would be a quick fix. But an hour later …

I guess I’m writing this as a cautionary tale for myself about going through proper procedures for problem solving instead of just guessing.

Firstly I tried to isolate the problem:

  • use a different USB port
  • use a different USB cable
  • try another Android device

From this I determined that the problem was with the phone, an old Nexus S that I use for testing older hardware and Android versions. Since I had just updated the Android version on the phone, I just jumped to the conclusion that it must have been a USB driver problem.

So next I made sure my Android SDK installation was up-to-date, and then updated the Android USB driver from it. Made no difference.

Then I tried uninstalling and re-installing the USB driver. Still not working.

At this point I was out of ideas and just went away for a bit.

Tip: When you hit a brick wall when trying to fix a problem, sometimes it can be helpful just to go away from it for a bit. Work on something else, go for a walk, have a coffee break, whatever. Let your subconscious do the work.

When I came back later, I remembered that I’ve had other intermittent issues with the phone since it was quite old. So I did what I should have done in the first place, the golden rule about fixing tech equipment.

Turn it off, wait, then turn it on again. Working now!

Guess this shows that it is easy sometimes to forget the basics.

Spring Profiles and Reusable Tests, Part 2

In the 1st part of the post, we were able to use Spring profiles to inject a specific implementation of a class to test into some test cases. Now we will try to create a reusable test case by using Spring Dependency Injection and Spring profiles to inject the appropriate expected results when testing a particular implementation.

Firstly we need to get the expected test results into a format that be be injected by Spring DI into a test case.

public interface FormatResults {

  public String getExpectedResult(String testMethodName);
}

public abstract class BaseFormatResults implements FormatResults {

  private Map<String, String>  results;

  public BaseFormatResults()
  {
    results = new HashMap<String, String>();

    setUpResults();
  }

  protected abstract void setUpResults();

  protected void addResult(String testMethodName, String result)
  {
    results.put(testMethodName, result);
  }

  @Override
  public String getExpectedResult(String testMethodName)
  {
    return results.get(testMethodName);
  }
}

public class HelloResults extends BaseFormatResults {

  @Override
  protected void setUpResults()
  {
    addResult("testDave", "Hello Dave");
  }
}

public class GoodByeResults extends BaseFormatResults {

  @Override
  protected void setUpResults()
  {
    addResult("testDave", "Good Bye Dave");
  }
}

For the sample code, a class is created as a wrapper around a map, which stores the expected test results for a test case. The expected result for each test method is keyed by the test method name in the map. Then some methods are included for adding and retrieving these expected results.

I have also made this wrapper class implement an interface and be abstract so that it can be subclassed to add the expected results for a particular test class.

Next we refactor the Spring JavaConfig classes to include these result classes, so that they can be injected into the test case along with the implementation of the class to test.

@Configuration
public class CommonTestConfig {

  @Autowired
  LogTestConfiguration logTestConfiguration;

  @Bean
  public Formatter formatter()
  {
    return logTestConfiguration.formatter();
  }

  @Bean
  public FormatResults results()
  {
    return logTestConfiguration.results();
  }

  @Bean
  public String testData()
  {
    return "Dave";
  }
}

public interface LogTestConfiguration {

  public Formatter formatter();

  public FormatResults results();
}

@Configuration
@Profile("hello")
public class HelloConfig implements LogTestConfiguration {

  @Bean
  public Formatter formatter()
  {
    return new HelloFormatter();
  }

  @Bean
  public FormatResults results()
  {
    return new HelloResults();
  }
}

@Configuration
@Profile("goodbye")
public class GoodByeConfig implements LogTestConfiguration {

  @Bean
  public Formatter formatter()
  {
    return new GoodByeFormatter();
  }

  @Bean
  public FormatResults results()
  {
    return new GoodByeResults();
  }
}

An extra method is added to the ‘LogTestConfiguration’ interface to retrieve a result class. Then the extra @Bean method to the configuration classes for getting the appropriate result class based on the active Spring profile.

Lastly we again refactor the test case to use the updated configuration files.

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes={CommonTestConfig.class, HelloConfig.class, GoodByeConfig.class})
public class SpringProfileTest {

  @Autowired
  private Formatter formatter;

  @Autowired
  private FormatResults results;

  @Autowired
  private String testData;

  @Test
  public void testDave()
  {
    String result = formatter.format(testData);
    System.out.println("Result = " + result);

    String expected = results.getExpectedResult("testDave");

    assertEquals(expected, result);
  }

  // getters and setters left out for brevity ...
}

Here are the changes from the test case examples in the previous post:

  • there is now only one test case required to test either test class implemenation ‘HelloFormatter’ or ‘GoodByeFormatter’ (previously a separate test case was required for each)
  •  the results for the test is also now injected into the test case via Spring DI
  • all of the Spring configuration files are included in the @ContextConfiguration annotation
  • the @ActiveProfiles annotation has been left out, now we need to specify the active Spring profile to use external to the test case, this can be done by setting the ‘spring.profiles.active’ property before running the test case, and there are various ways to do this.

For the sample code, if you are running the test case using the Eclipse IDE, then this could be set in the Run Configuration for the JUnit test class.

Eclipse JUnit run configuration

If you are using Gradle, then just set the property in the build script in the ‘test’ task.

test {
  systemProperties 'spring.profiles.active': 'hello'
}

The Sample Code

You can download the sample code for this example, SpringProfileTestingExample2.zip from GitHub.

Note that I have kept the sample code very simple for the sake of brevity, and also to more clearly illustrate the point. For instance, you may need to add the @DirtiesContext annotation to the test classes if you don’t want the injected beans to be cached between tests.

Conclusion

Using Spring bean definition profiles, we can make test cases reusable in the particular scenario where we are testing different implementations of a class.

If you only have a few tests to run, then the this setup with the Spring profiles would be overkill.

However for my project, where I had many test cases, it allowed me to reuse them without having to duplicate them for each implementation of a class under test. Of course, it would also allow me to use those same test cases to test any future implementations of the class too.

Reusable Aspect Library for Android, Part 2

In part 1 of this post, we created an Android library containing aspects that could be used to intercept code in an application. However the library would require code changes in order to work with other application, and so was not really reusable as it stood.

In this part, we will modify the sample code to make the library more reusable across multiple applications.

Scenario 2 – The Pointcut in the Application

In this example, the aspect tracing code is kept in the aspect library project, but we will put the pointcut in the application.

Create the Projects

Create an Android application project and library project, and configure them as AspectJ projects in Eclipse, same as for part 1. Then add the library project to the application project, both as an Android library and to the application aspectpath.

Code Changes

1. Aspect Library

Once again put the AspectJ tracing code in the library project,  but this time the pointcut has been made abstract.

public abstract aspect BaseTraceAspect {

    /** Pointcut for tracing method flow tracing */
    protected abstract pointcut traceOperations();

    before() : traceOperations() {
        Signature sig = thisJoinPointStaticPart.getSignature();
        log("Entering [" + sig.toShortString() + "]");
    }

    /** Implement in subaspect for actual logging */
    protected abstract void log(String msg);
}

Notice I have also made the log method abstract as well, this leaves it up to the application to determine how it wants to log the tracing output. In general any funcationality that is specific to the application should be made abstract here, and be implemented in the application.

2. Test Application

In the application project, add some code that can be intercepted by the aspect, as for part 1.

Then create another aspect file which subclasses the aspect in the library project. This aspect just implements the pointcut (and in this sample, the logging method as well).

public aspect TraceAspect extends BaseTraceAspect {

    private final static String TAG = "Aspect-Test";

    protected pointcut traceOperations(): execution(* au.com.example.test.**.testMethod(..));

    protected void log(String msg)
    {
        Log.i(TAG, msg);
    }
}

Run and Verify

Once again run the application and verify in LogCat that the aspect tracing has worked.

Sample app 2

LogCat tracing

The Sample Code

You can download the sample code for this example, aspect-lib-example_2.zip from GitHub. Follow the same instructions as for the previous example for running the sample app.

Conclusion

Now the aspect library project is reusable in other applications. An application would just need to include the library and then override the abstract pointcut to implement a concrete pointcut that is specific to it.

In the last part of this article, we will add another library to the application and show how to apply the aspects to it as well.

Spring Profiles and Reusable Tests, Part 1

When writing test cases for an update to an open source project I was working on, I came across the issue of how to make test classes reusable.

I already had dozens of test cases written for testing a particular class. Now I had written another implementation of that class, and I wanted to reuse the test cases I already had, to avoid duplication.

I decided to see if I could use Spring profiles to do this.

Spring Profiles

Spring bean definition profiles allows the wiring of different beans in different environments.

One use of Spring profiles in testing is where different test data or application configurations can be injected into test cases depending on the profile. There are various examples of this:

However my requirements were a bit different.

  • more specifically I had various implementations of a class that logged an object graph and produce some output in differing formats.
  • I had many test cases written that would test a particular implementation of the logging class and then compare the generated logging with some expected output.
  • I wanted to run the same set of tests with the same test data, but testing different implementations of the logging class

So rather than changing the test data, what I wanted to change was the class under test and the expected results using Spring DI. Using profiles would allow me to reuse the same tests and test data.

Of course if there are only a few tests to run, this could also be achieved using parameterized tests. Then for each test you could use JUnit 4 to pass parameters for the test data and expected test results.

Here are some simple code examples, showing one way to use Spring profiles in tests.

Step 1: A Simple Test with Spring

Note: Here I’m also using Spring’s JavaConfig instead of XML for the application contexts, but either would work.

Firstly let’s create some classes to test. Here are 2 classes that do some string formatting, both of which have a common interface:

public interface Formatter {

  public String format(String name);
}

public class HelloFormatter implements Formatter {

  @Override
  public String format(String name) {
    return "Hello " + name;
  }
}

public class GoodByeFormatter implements Formatter {

  @Override
  public String format(String name) {
    return "Good Bye " + name;
  }
}

Now let’s write a simple JUnit test case for to test ‘HelloFormatter’, using Spring to inject the test class and test data.

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes={SimpleTestConfig.class})
public class SimpleTest {

	@Autowired
	private Formatter formatter;

	@Autowired
	private String testData;

	@Test
	public void testDave() {
		String result = formatter.format(testData);
		System.out.println("Result = " + result);

		String expected = "Hello Dave";

		assertEquals(expected, result);
	}

	// getters and setters left out for brevity ...
}

And here’s the JavaConfig used to wire the beans for the test.

@Configuration
public class SimpleTestConfig {

	@Bean
	public Formatter formatter()
	{
		return new HelloFormatter();
	}

	@Bean
	public String testData()
	{
		return "Dave";
	}
}

Now if we want to test the ‘GoodbyeFormatter’ implementation class as well, we could just write another test case and another Spring JavaConfig class, but this would mean code duplication.

So instead let’s try using Spring profiles to inject the appropriate test class into the test cases.

Step 2 : Using Spring Profiles

Firstly let’s refactor the Spring JavaConfig classes.

@Configuration
public class CommonTestConfig {

	@Autowired
	LogTestConfiguration logTestConfiguration;

	@Bean
	public Formatter formatter()
	{
		return logTestConfiguration.formatter();
	}

	@Bean
	public String testData()
	{
		return "Dave";
	}
}

public interface LogTestConfiguration {

	public Formatter formatter();

}

@Configuration
@Profile("hello")
public class HelloConfig implements LogTestConfiguration {

	@Bean
	public Formatter formatter()
	{
		return new HelloFormatter();
	}
}

@Configuration
@Profile("goodbye")
public class GoodByeConfig implements LogTestConfiguration {

	@Bean
	public Formatter formatter()
	{
		return new GoodByeFormatter();
	}
}

Notice we have split the configuration between 3 classes now:

  • there is one class that configures the beans required for all the test cases
  • there is one configuration file specifically for testing the ‘HelloFormatter’ class, and it has the annotation ‘@Profile(“hello”)’.
  • there is one configuration file specifically for testing the ‘GoodByeFormatter’ class, and it has the annotation ‘@Profile(“goodbye”)’.
  • the 2 configuration files that have profile annotations share a common interface so that they can, in turn, be injected into the common configuration file

This setup is base on the blog, SPRING 3.1 M1: INTRODUCING @PROFILE, which contains a more detailed explanation.

So here are the refactored test cases that use the new configuration files, one test class to test each formatter implementation class.

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes={CommonTestConfig.class, HelloConfig.class})
@ActiveProfiles("hello")
public class SpringProfileHelloTest {

	@Autowired
	private Formatter formatter;

	@Autowired
	private String testData;

	@Test
	public void testDave() {
		String result = formatter.format(testData);
		System.out.println("Result = " + result);

		String expected = "Hello Dave";

		assertEquals(expected, result);
	}

	// getters and setters left out for brevity ...
}

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes={CommonTestConfig.class, GoodByeConfig.class})
@ActiveProfiles("goodbye")
public class SpringProfileGoodbyeTest {

	@Autowired
	private Formatter formatter;

	@Autowired
	private String testData;

	@Test
	public void testDave() {
		String result = formatter.format(testData);
		System.out.println("Result = " + result);

		String expected = "Good Bye Dave";

		assertEquals(expected, result);
	}

	// getters and setters left out for brevity ...
}

Note that each test case class has the @ActiveProfiles annotation, so that only the appropriate test class is injected for testing. Also we have added the required configuration files to the @ContextConfiguration annotation.

The Sample Code

You can download the sample code for this example, SpringProfileTestingExample1.zip from GitHub.

The requirements to run the sample code are:

Next

We have achieved half of what we set out to do, by using Spring profiles to determine which implementation of a test class is used for each test case.

In the next part of this post, we will make further improvements so that we have one reusable test case able to test both test class implementations by injecting the expected test results as well.

Reusable Aspect Library for Android, Part 1

There are various projects / tutorials / blogs out there about how to integrate AspectJ into the Android development process. They generally show you how to setup an application project to use AspectJ, with the aspects in the project itself.

However the way I tend to use AspectJ in Java projects is to try to put them into reusable libraries so that they can be applied to multiple projects. After all they are supposed to be used for cross-cutting concerns which are common to many applications. Searching the internet, I did not manage to find much information about how to do this in Android.

So here are some examples of one solution I found when using Eclipse (Windows version). If  you are not already familiar with creating and setting up Android projects in the Eclipse IDE, and with using AspectJ, have a look at the documentation for Android and AspectJ.

These are the requirements for the samples:

Note that the Android tools tend to be a bit brittle, so different versions may have changes (and bugs). So the examples may not work if you are using a different version.

Also I had the following design goals in mind:

  • keep it as simple as possible
  • stick to using Android libraries


With Eclipse you can put code in separate Java projects and either reference them from an Android application either as a project or a jar, e.g. create a ‘glue’ Android library that references a Java library.
While this might work in Eclipse, it would make things much more complicated if I wanted to use another build system (e.g. Ant) to build the application.

Scenario 1: Self-Contained Aspect Library

In the first example, we will create an android library that contains all the aspect code, and use it to apply the aspects to some code in an application.

Here are some instructions on setting up an Android library with aspects, along with some sample code.

The Test Application

This is just a very simple app to demonstrate the aspects working.

1. Create a new Android application project and add some code that the aspects can intercept.


public class MainActivity extends Activity {

private Button testButton;

 @Override
 protected void onCreate(Bundle savedInstanceState) {
 super.onCreate(savedInstanceState);
 setContentView(R.layout.activity_main);

 testButton = (Button)findViewById(R.id.test_button);
 testButton.setOnClickListener(testButtonListener);
 }

private OnClickListener testButtonListener = new OnClickListener() {
 public void onClick(View v)
 {
 // invoke the method to be traced
 testMethod();
 }
 };

 public void testMethod()
 {
 Toast.makeText(this, "Inside test method", Toast.LENGTH_SHORT).show();
 }

2. Add the AspectJ nature to the application project.

If you already know how to setup AspectJ in an Eclipse project, then you can skip the next bit …

To add the AspectJ nature to the application project, there are 2 ways of doing this:

a. Using the AJDT plugin
Right click on the project in the Package Explorer to bring up the context menu.

  • Click ‘Configure’
  • Click ‘Convert to AspectJ project’

Menu to convert to AspectJ project

b. Do it manually

  • Edit the ‘.projects’ file for the project.

Replace the java builder …

<buildCommand>
  <name>org.eclipse.jdt.core.javabuilder</name>
  <arguments>
  </arguments>
</buildCommand>

with the AspectJ builder …

<buildCommand>
  <name>org.eclipse.ajdt.core.ajbuilder</name>
  <arguments>
  </arguments>
</buildCommand>

and then add the AspectJ nature …

<nature>org.eclipse.ajdt.ui.ajnature</nature>

The edited file should look something like this:

<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
	<name>ATest</name>
	<comment></comment>
	<projects>
	</projects>
	<buildSpec>
		<buildCommand>
			<name>com.android.ide.eclipse.adt.ResourceManagerBuilder</name>
			<arguments>
			</arguments>
		</buildCommand>
		<buildCommand>
			<name>com.android.ide.eclipse.adt.PreCompilerBuilder</name>
			<arguments>
			</arguments>
		</buildCommand>
		<buildCommand>
			<name>org.eclipse.ajdt.core.ajbuilder</name>
			<arguments>
			</arguments>
		</buildCommand>
		<buildCommand>
			<name>com.android.ide.eclipse.adt.ApkBuilder</name>
			<arguments>
			</arguments>
		</buildCommand>
	</buildSpec>
	<natures>
		<nature>org.eclipse.ajdt.ui.ajnature</nature>
		<nature>com.android.ide.eclipse.adt.AndroidNature</nature>
		<nature>org.eclipse.jdt.core.javanature</nature>
	</natures>
</projectDescription>
  • Add the AspectJ runtime jar to the classpath for the project.

The jar is named ‘aspectjrt.jar’ and it is probably a good idea to use the one that came with the AJDT plugin, which should be located under your Eclipse install at ‘plugin/org.aspectj.runtime_x.x.x’.

There are various ways of doing this, but if you have AJDT installed, the easiest way would be to add it as a library in your ‘Java Build Path’ dialog. In the ‘Java Build Path’ dialog, select the ‘Libraries’ tab, and then click the ‘Add Variable’ button. Then in the ‘New Variable Classpath Enty’ dialog that comes up, select the ‘ASPECTRT_LIB’ entry and click the ‘OK’ button.

Add aspectrt.jar as variable

To make the sample code a bit more portable, rather than doing it this way,  I’ve just copied ‘aspectjrt.jar’ to a directory named ‘lib’ in the project and added it to the classpath from there.

The Aspect library

This is the Android library project that will contain the aspects.

1. Create an Android library project

2. Add the AspectJ nature to the library project (the same as for the application project)

3. For these samples I am going to show a very common usage of AOP – tracing. So add some an AspectJ tracing code to an aspect file in the library project, with the pointcut configured to intercept some code in the application project.

public aspect TraceAspect {

    private final static String TAG = "Aspect-Trace";

    // pointcut to intercept a method in the application project
    protected pointcut traceOperations(): execution(* au.com.example.test.**.testMethod(..));

    before() : traceOperations() {
        Signature sig = thisJoinPointStaticPart.getSignature();
        log("Entering [" + sig.toShortString() + "]");
    }

    protected void log(String msg)
    {
	Log.i(TAG, msg);
    }
}

4. Add the aspect library project as an Android library for the application project.

Add aspect library project to application

5. Also add the aspect library project to the AspectPath for the application. Do this in the ‘AspectJ Build’ dialog for the application project.

For the application project, right click the project in the Package Explorer to bring up the context menu.

Then either …

  • Click ‘Build Path -> Configure Build Path …’ to bring up the project properties dialog
  • Click ‘AspectJ Build’ in the side menu

or …

  • Click ‘AspectJ Tools -> Configure AspectJ Build Path …’

In the ‘AspectJ Build’ dialog, click the ‘Aspect Path’ tab. Then click the ‘Add Project…’ button and select the aspect library project.

Add aspect library to aspectpath of application

Run the application

Now run the application in your device or emulator, making sure that the code that you want to trace is invoked.

Sample app 1

For the sample app, just click the button ‘Click to test tracing’, and then check in LogCat that the aspect tracing has occurred.

LogCat tracing 1

The Sample Code

You can download the sample code for this example, aspect-lib-example_1.zip from GitHub.

Running the sample code:

  • Unzip the zip file containing the Eclipse projects.
  • Import the projects into your Eclipse IDE.
  • The sample projects have Android 4.0 (API level 14) as the Android build target. So you need to either have API 14 present in your Android SDK installation or change the build target for all the projects to another API version.
  • Optionally you may have to adjust any application or class paths to suit your development setup.
  • Build and run the sample application project.
  • Check LogCat to see if the aspect tracing is present.

Conclusion

The problem with this example is that the aspect library isn’t really all that reusable.

If you wanted to use the library in another application, chances are you would have to changed the pointcuts to match the code you want to intercept in the new application.

In the next part of this blog, we will make some improvements to it to make it more reusable.

Path Problems: Java (Android), Windows 64 and 2 Drives

I recently updated my installation of the Android SDK and ran into a problem that I’ve alway had on my current computer – the SDK Manager would not run after the update.

This is due to a Java path problem for a particular setup:

  • a computer running 64-bit Windows (Windows 7 in my case)
  • a computer with more than one disk drive installed

I also have Java 6 64-bit installed on the computer (not Java 7 as I use it for Android development).

Android SDK Manager – it no work

To re-iterate the issue: after installing or updating the Android SDK on a system with this setup, the SDK Manager may not be able run.

When trying to start the SDK Manager, in the ‘tools’ directory of the SDK installation, the batch file ‘android.bat’ is run. Inside the batch file it sets various path variables that point to libraries or to your Java installation that is required to run the SDK Manager.

This is where I run into problems.

One issue is this line in ‘android.bat’:

for /f %%a in ('%java_exe% -jar lib\archquery.jar') do set swt_path=lib\%%a

Here it is looking for the file ‘swt.jar’ which is located in the SDK installation under ‘tools\lib\x86’ or ‘tools\lib\x86_64’ depending on whether you are running a 64-bit system.

If you have only 1 drive in your system, then this should work. But if, like me, you have two drives then it may not work and the variable swt_path is not set.

Then there is this line where it tries to find Java:

call lib\find_java.bat

This runs the batch file ‘find_java.bat’ to set the variable ‘java_exe’ to point to the path of java.exe in your Java installation.

for /f %%a in ('%~dps0\find_java.exe -s') do set java_exe=%%a

Once again in my system this doesn’t work and the batch file can’t find my Java executables.

By the way when you try to do a build with Ant, it runs ‘platform-tools\dx.bat’ which also calls ‘find_java.bat’, so it runs into the same problem of not finding Java.

Why Does This Happen?

It seems this all has to do with the way Windows x64 handles the long and short format of file (and directory) names when working out paths.

Of course, this is for my particular setup. On your system it may all work fine – but you only have to do a quick google to see that many others have similar problems.

If you have this issue, then it’s worth checking for more common errors first, e.g.

  • the appropriate version of Java is installed correctly
  • JAVA_HOME and PATH environmental variables have been configured

Other Java Programs – GMote

I was recently reminded of all this when I was setting up my HTPC. This is a Windows 7 64-bit PC, once again with 2 drives (one for applications and the other for storing media files).

I was trying out GMote (an Android remote control app for playing media on a computer). This has a server component which runs on the computer and is a Java program.

After doing the default installation, I tried to run the GMote server and kept getting an error “Unable to load library ‘libvlc'”. After checking that the ‘libvlc’ library was in a folder in the PATH environmental variable, this error still kept occurring.
It took me a little while to find the problem with getting the GMote server to run:

  • I had the 64 bit version of Java installed, so I installed the 32 bit Java and pointed the JAVA_HOME environmental variable to it.
  • Since the path to the support library was under the default installation of GMote server at “C:\Program Files (x86)\…”, I re-installed the GMote server to another directory name that did not contain any spaces “C:\media\…”.

Finally it worked.

Why Two Drives?

At this point you might ask why do I have 2 drives on my computers?

I tend to install the system and application files on one drive, and data files on the other, I find this convenient for various reasons:
Backup – I can use different backup strategies on the different drives, e.g. do an disc image backup on the system and application drive and some other type of backup on the data that I want on the other drive. I also point the TEMP environmental variable to the data drive so that the temporary files don’t clog up my image backups.
Performance – Use a SSD for the operating system and applications disk for faster booting and app loading (Of course it would be nice to have a SSD for the data drive as well, but if you have a lot of data then this would be very expensive!).

The Moral of the Story …

So in this particular setup of trying to run Java programs on 64-bit Windows on a computer with multiple disk drives, I follow these guidelines:

  • check which version of Java is required by your program (64-bit or 32-bit)
  • check whether it requires the JDK or JRE
  • install Java programs in a path that doesn’t have spaces in the directory names, as you might do on a UNIX or LINUX system
  • point your JAVA_HOME environment to the appropriate Java installation (you can do this in a batch file before launching the Java program so that it is only local and doesn’t affect other Java apps)

So how did I resolve Android SDK issue on my system? If you google this issue, there are various suggestions as to how to fix it. These include things like digging into the Windows and Java internals, registry hacks, etc.

For me, I just took the easy route of editing android.bat to point the variables to the correct paths on my particular system. I know this is not the best way to deal with it, but after many frustrating hours of research, configuration and testing, I wanted to get things up and running quickly.
It’s a shame you have to waste so much time digging into Android issues to find the causes (and if you’re lucky, the solutions) of these problems. It’s not like running Windows x64 or having multiple drives on a computer is all that unusual.

But then the Android build tools have always been a bit flaky, so I guess it’s part of being an Android developer.