chrisport.ch

Amortizing Software Rewrites: An Evolutionary Approach

Fri, 16 Apr 2021 00:00:00 +0000

This post I have written and published with a colleague during my time at Thoughtworks. Please find it here: thoughtworks.com/en-de/insights/blog/amortizing-software-rewrites-evolutionary-approach

Bringing vision into a code base

Sat, 26 Dec 2020 00:00:00 +0000

This post is about how to inject and persist architectural vision in a code base, based on a recent experience.

High-level approach to restructure and refactor a messy code base
Persist architectural vision to outlive people and team changes and move the code in a single direction.

Preface: The tale of my own legacy

Some code comes with a long history and reflects the opinion and knowledge of all the people that have worked on it.
Recently I had the honor to work on a rather small, but messy code base which has been passed around between teams multiple times over the course of three years. In fact, I had worked on it myself for a while a year before and I remember very well how we were sitting in a small meeting room with stifling air, talking over the code smells and trying to come up with ideas on how to make it shine.
Due to organisational reasons, it was soon to be transferred to another team on another continent. We shared our gained understanding as best we could and explained our half-baked ideas of the next steps.

One year later this code had become a bottleneck to the project and the assumption is that no more than two pairs are able to work on it in parallel. As an experiment we put together a team of two pairs, one from our team and one from the other team, with the mission to make work on this code base more parallelizable - of course with on-going delivery on top. As part of this new team the code base came back to me after over a year. The good news: The other team has made vast improvements and did an excellent job in the time given. But big parts were untouched and moreover our half-baked solutions from one year ago are now just additional complexity and became part of the problem.
It’s great to work in a small and fresh team on a single problem. The scope is narrow, which leaves our brains the energy to understand the technical details in-depth and make the right decisions on what to tackle, and how and when.¹ Let’s see how we deal with the actual task.

Approach the Restructuring

The initial milestone is to reach a clearly structured base, in which we can implement new changes and continue to clean up messy parts as we touch it.
We approach the refactoring in five steps:

Get a shared understanding of the business context
It is essential to first get a common understanding of the business domain and capabilities that the code base provides. A simplified event storming² session helps us to get to know what we are dealing with and have a common and well-defined language to talk about the business.
Later we often have to distinguish between essential complexity, which reflects the complexity of the business capability, and accidental complexity, which is added through the chosen implementation and can be eliminated.
Define a high-level target structure Based on this business understanding we create a very simple first high-level structure that looks something like this. This structure is by no means written in stone. In fact, as we proceed into the next phases, we refine it multiple times.
Name the mess
It is time to collect the low-hanging fruits! This means reading the code, splitting and moving files to reach higher cohesion, and renaming things to reflect their semantics. The terms from the event storming session come in handy here, and the high-level target structure guides where files should live. At this point we don’t really touch any logic, but simply rename and move files, classes and (top-level) functions. Thanks to modern IDEs like IntelliJ, these are quite risk-free refactorings with immediate benefit. One of our measures of success is that a new developer should be able to intuitively navigate to the code related to a feature they are working on. In our case, a new joiner gives her subjective impression and uncovers some more unintuitive names and structures.
Example: we collect a bunch of tools, which are not directly related to business cases, like the swagger setup, in a tools package. ‘tools’ might not be the best name, but the important thing now is to split the mess into manageable and obvious parts that can be worked on further later.
Example: The package api contains a class DataController. Neither “api” nor “data” says what the content of this file is about. We figure out that these are HTTP APIs for customer profile preview and editing, including image upload. We move them to corresponding feature packages. The result looks like this: The entanglement in code is still there, maybe the preview code uses functions from the edit code to get the uploaded image. Maybe it would make sense to split out the image functionalities and move them to the core. But in this phase, demystifying and making them obvious brings more value. The names reflect the content and are close to the right place now, and we are much happier!
Identify and prioritise the hot spots
While a lot of things are tightly coupled, we can identify hot spots that are central to the application logic and particularly complex. We identify these exploratively by using tools like dependency graphs. Some hot spots only become obvious during the feature work. Such discoveries are actually the most valuable ones, because solving them first will have an immediate impact on our feature delivery speed, which makes them cheap improvements.
Restructure This is the most fun phase for the devs! We refactor the code to make it more understandable, maintainable and evolvable. It’s advisable to limit and coordinate the bigger tasks, as some might go deep into the fundamental structure of the application. In an ideal case we do this as part of the feature work, but since we want to significantly improve the code base in a limited time, we also run refactorings as tasks on their own, so that a pair can concentrate on finding the best way. While refactoring per definition excludes changing behaviour, don’t be afraid to question existing logic. Again the question is, if this complexity is accidental or essential. One of the greatest feelings is when you spend half a day understanding what a particularly complex part is doing, just to come to the conclusion that we don’t need it at all.

Persisting Vision

By nature the entropy, or chaos, of a code base increases with the features and changes applied on it. Therefore, refactoring should be part of everyday development work. But changes and refactorings need to go in one direction towards a common vision of how the code base ought to look. A good analogy I’ve read is that “a code base should look like it’s written by a single person”³. The easiest way to achieve this in a team is through pair programming and regular open discussions, but in fact it is shaped by any interaction and casual conversation team members have. With larger teams this becomes increasingly difficult⁴ and when a code base is even passed around between teams, the communication of the vision barely happens.
So considering that this team has an expiry date, a big concern for us is how we can transfer our vision and knowledge to whoever will encounter this codebase. The vision must be translated into the code base. Ideally it is readable and obvious, when a developer wants to do a change, they should feel “this is how I should do it”. But even if the code base still is a bit messy, we want to provide some guide rail and the following tools and techniques have been particularly useful.

ArchUnit
ArchUnit is a testing framework for the architecture of a JVM based application. Our code base is in Kotlin and the framework does a wonderful job. It allows us to express our intentions in small tests that then assert that the rule is not violated. Here is a simple example of an ArchUnit test. ArchUnit is a really amazing tool and I have yet to discover such tools for other technologies.

  @ArchTest
  val `classes in the core package should not use classes in the features package` =
      noClasses().that().resideInAPackage("ch.chrisport.core..").should()
          .dependOnClassesThat().resideInAPackage("ch.chrisport.features..")
          
  @ArchTest
  val `features should be free of circular dependencies` =
      slices().matching("ch.chrisport.features.(*)..").should().beFreeOfCycles()

Manifesto / Code of Conduct
We create a Code of Conduct in the root folder of the code base, requesting any external contributor to read it before writing code. The goal of the code of conduct is to raise awareness, not explain in detail. It describes in short the business scope, the architectural goal of the code base, current state, as well as Do’s and Dont’s.
The document should be short enough for anybody to read it in just five minutes, not an extensive documentation. The points are at least mid-term, so that they don’t need to be updated every week. Some parts might be a bit more short-lived, such as

Don’t write more Enzyme tests, we want to migrate to React Testing Library tests

or

Do not add more properties to the Preview class, we want to move to more specialised interfaces.

Where possible, move such things to ArchUnit tests, where they are much more practical.
Continuous guided change with pre-commit hooks
On a similar line as the ArchUnit tests, we’ve introduced a git pre-commit hook (using husky), which runs through the touched files and verifies that certain things are not present anymore. This supports a continuous refactoring approach to things like renaming or changing tools, in the sense of a fitness function⁵.
For example whenever you have touched a test, all junit4 tests should be junit5 now and the name ‘OutputView’ should not appear anymore in any variable name, but be replaced with ‘View’.
Scanning through the files of a commit, the script searches for appearances of unwanted content and prints the violation to the console. Each violation has an explanatory comment which is printed alongside.

Violation: File Application.test.kt contains org.junit.test. Please migrate Junit4 tests to Junit5.

Summary

We looked at an approach to restructure a messy code base in five steps:

Get a shared understanding of the business context: Bring everything on the table, share knowledge and agree on terms
Define a high-level target structure: Define a first version of the target structure to guide further steps. Revisit it frequently.
Name the mess: Collect low hanging fruits by moving and renaming things according to their semantic.
Identify and prioritise the hot spots: Analyse the code base to identify where complexity lies, prioritise it according to impact on the upcoming work.
Restructure: Limit and coordinate the more complex refactorings to stay focused. Don’t be afraid to question.

We also looked at three tools that support the continuation of a unified vision, even if the people change:

ArchUnit: For expressing intention and structure in form of tests.
Manifesto / Code of Conduct: For sharing the core ideas and direction with any person unfamiliar, explicitly and brief.
Continuous guided change with pre-commit hooks: For verifying that no unwanted content is present in any touched file, this pushes developers to apply the Boy Scout rule.

Send me some feedback on this post via Social Media.

References

¹ In her book Dynamic Reteaming, Heidi Helfand calls this approach of building a new team around a specific problem Isolation Pattern.
² Eventstorming is an approach to discover the domain in form of a workshop. We used Mural to do this remotely.
³ If you know the origin of this analogy, please let me know.
⁴ See also https://www.solutionsiq.com/learning/blog-post/lines-of-communication
⁵ A fitness function, as described in the book Building Evolutionary Architectures produces a statement, which tells us if we move forward in the direction of a desired architectural characteristic.

Bubble up errors in Golang

Wed, 20 Sep 2017 00:00:00 +0000

One problem I often encounter is errors without any context. If an arbitrary method returns an error “invalid character ‘}’ after object key”, you will have to dive down the stack until you find the exact place the error happened. Therefore whenever you receive an error and you decide to not handle it, but instead return it to the caller of your method, you should provide context about the reason the error occurred. The examples below use errors.Wrap from github.com/pkg/errors.

Let’s take the following method as example:

func main() {
	err := process([]byte("some_invalid_input"))
	if err != nil {
		panic(err)
	}
}

func process(input []byte) error {
	r, err := serviceA.Process(input)
	if err != nil {
		return err
	}

	err = serviceB.Process(r)
	if err != nil {
		return err
	}

	return nil
}

If serviceA or serviceB returns an error, we don’t know which one failed and we don’t know whether it is our fault or whether the service is unreachable or misconfigured. The error could look like this:

panic: invalid character '}' looking for beginning of value

goroutine 1 [running]:
main.main()
	/Users/christophportmann/workspace/go/src/github.com/chrisport/go-playground/errorhandling/main.go:10 +0x92
exit status 2

By wrapping the error, we can provide the missing information about what part failed:

func main() {
	err := process([]byte("some_invalid_input"))
	if err != nil {
		panic(err)
	}
}

func process(input []byte) error {
	r, err := serviceA.Process(input)
	if err != nil {
		return errors.Wrap(err,"call to service A failed")
	}

	err = serviceB.Process(r)
	if err != nil {
		return errors.Wrap(err,"call to service B failed")
	}

	return nil
}

The result would be unambiguous:

panic: call to service B failed: invalid character '}' looking for beginning of value

goroutine 1 [running]:
main.main()
	/Users/christophportmann/workspace/go/src/github.com/chrisport/go-playground/errorhandling/main.go:11 +0x92
exit status 2

In this simple example the advantage is not crystal clear. But imagine serviceA is doing a complex task itself and doesn’t wrap its errors either.
You could end up getting generic errors (e.g. a timeout on an http call) and need to debug down the whole stack to find out which call has failed. But if errors are always wrapped with necessary context, an error message could look as nice as this:

panic: call to service B failed: call to backend failed: http: wrote more than the declared Content-Length

Therefore if you decide to bubble up an error, always provide context.

Context Aware Java Executor and Spring's @Async

Wed, 12 Oct 2016 00:00:00 +0000

Recently I came across the problem of ThreadLocal context in multithreaded environment. If the Thread that handled a request uses an Executor to asynchronously execute tasks, the ThreadLocal context is lost and information such as requestId will be missing in the logs.
We can solve this using the Decorator pattern:

Wrap Runnable to preserve caller Thread’s context
Implement ExecutorDecorator that wraps every execution
Additional: Configure Spring’s @Async to use Decorator

Please note:

MDC (Mapped Diagnostic Context) used in the example as the ThreadLocal context
caller Thread, which wants to schedule some task
executor Thread, which can be any Thread (e.g. from a pool) that executes the task

1. Make the runnable context aware

To pass the MDC of the caller Thread to the executor Thread, we get a copy of the ContextMap.
Then we set this context map before execution of the task and reset it after the execution, even if the task fails.

 
// get a copy of the values of calling Thread's MDC
final Map<String, String> callerContextCopy = MDC.getCopyOfContextMap();

Runnable ctxAwareTask = () -> {
  // get a copy of the values of executing-Thread's MDC
  final Map<String, String> executorContextCopy = MDC.getCopyOfContextMap();

  MDC.clear();
  if (callerContextCopy != null) {
    // set the desired context that was present at point of calling execute
    MDC.setContextMap(callerContextCopy);
  }

  try {
    // execute the command
    command.run();
  } finally {
    // reset the context, even if the command throws
    MDC.clear();
    if (executorContextCopy != null) {
      MDC.setContextMap(executorContextCopy);
    }
  }
};

2. Implement custom Executor that decorates every task

  
public class ContextAwareExecutorDecorator implements Executor, TaskExecutor {

    private final Executor executor;

    public ContextAwareExecutorDecorator(Executor executor) {
        this.executor = executor;
    }

    @Override
    public void execute(Runnable command) {
        // decorate the task
        Runnable ctxAwareCommand = wrapContextAware(command);
        // execute the decorated task
        executor.execute(ctxAwareCommand);
    }

  
    private Runnable wrapContextAware(Runnable command) {
        ... see above
        return ctxAwareTask;
    }
}

Please note that I also implement Spring’s TaskExecutor for usage below.

3. Addition: Configure Spring’s @Async to use Decorator

We can configure the default executor easily with the following Configuration. See also Spring’s documentation: EnableAsync.

@Configuration
@EnableAsync
public class AppConfig extends AsyncConfigurerSupport {

    @Override
    public Executor getAsyncExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(7);
        executor.setMaxPoolSize(42);
        executor.setQueueCapacity(11);
        executor.setThreadNamePrefix("ContextAwareExecutor-");
        executor.initialize();
        return new ContextAwareExecutorDecorator(executor);
    }
    
}

Sample Application

There is a small sample application on Github. It has a service, which has an @Async method. After startup, the service is called 10 times with a specific MDC context containing the loop-count and the caller’s threadId. The asynchronous service method prints its MDC to the log, which should contain the originally mentioned properties.

Without our ContextAwareExecutorDecorator, we will encounter the initially described problem: the ThreadLocal context gets lost.
In that case the log looks like this:

13:14:20.643 executorThreadId: 17; ContextMap on execution: null
13:14:20.748 executorThreadId: 18; ContextMap on execution: null
13:14:20.852 executorThreadId: 19; ContextMap on execution: null
...

If we now wrap the Executor into our ContextAwareExecutorDecorator, as described in Step 3, the problem will be solved.
The resulting log looks like this:

13:07:50.634 executorThreadId: 17; ContextMap on execution: {count=0, callerThreadId=16}
13:07:50.737 executorThreadId: 18; ContextMap on execution: {count=1, callerThreadId=16}
13:07:50.839 executorThreadId: 19; ContextMap on execution: {count=2, callerThreadId=16}
13:07:50.943 executorThreadId: 20; ContextMap on execution: {count=3, callerThreadId=16}
13:07:51.045 executorThreadId: 21; ContextMap on execution: {count=4, callerThreadId=16}
13:07:51.149 executorThreadId: 23; ContextMap on execution: {count=5, callerThreadId=16}
13:07:51.254 executorThreadId: 24; ContextMap on execution: {count=6, callerThreadId=16}
13:07:51.359 executorThreadId: 17; ContextMap on execution: {count=7, callerThreadId=16}
13:07:51.462 executorThreadId: 18; ContextMap on execution: {count=8, callerThreadId=16}
13:07:51.565 executorThreadId: 19; ContextMap on execution: {count=9, callerThreadId=16}

Visualize call trees in a microservice architecture

Mon, 12 Jan 2015 10:00:00 +0000

Disclaimer 2020/2026: Distributed tracing is provided by various mature libraries, prominently by the standard OpenTelemetry. Please consider these before implementing anything yourself.

In a microservice architecture, one request can lead to multiple log entries distributed across services. One client request can cause multiple internal requests, which in turn can cause further requests. This can lead to unwanted depth of requests which leads to high latency. I worked in a system with central logging using the ELK stack. It was easy to search for log entries and even to search for all entries by unique ID per request. But the possibilities to analyse request trees were very limited. In this article I will show a solution to analyse the logs and create useful results and graphs. The examples are invented but reflect our experiences.

Request Trees

The DOT graph description language is used to create graphs in plain text, which can be visualized using tools like Graphviz. The language can be understood in few minutes which makes it easy to work with. Here is an example:

    digraph "/pay" {
      "CLIENT" -> "proxy"
      "Proxy" -> "BillService"
      "BillService" -> "UserService"
      "BillService" -> "BankGateway"
      "BankGateway" -> "AuditService"
    }

You can store this in a .dot or .gv file and open it with GraphViz.

Track request trees through chained IDs

As mentioned it is not possible to track a chain if there is only a unique ID per request, since one service can call multiple other services and do this even in parallel. Therefore we need three parts, a Unique Request ID, a Caller ID and a Receiver ID. For a fast implementation I just modified the unique ID to be in the format

<FixedRequestId>::<LastCallerID>::<ReceiverId>

A chain from the example above can now easily be parsed and printed in dot notation. The IDs from the example above would be the following:

Proxy           UID
BillService     UID::BillServiceID
UserService     UID::BillServiceID::UserServiceID
BankGateway    UID::BillServiceID::BankGatewayID
AuditService    UID::BankGatewayID::AuditServiceID

The Go function to create these IDs is as simple as

func transformUniqueId(parentUID string) string {
  ids := strings.Split(parentUID, "::")

  if len(ids) < 3 {
    return parentUID + "::" + getNextInstanceId()
  }

  return ids[0] + "::" + ids[2] + "::" + getNextInstanceId()
}

Since we use a beego filter to write the access logs, it is very easy to apply this to every incoming request. Parallel benchmarking shows that the additional cost of this is in range of 600-800 nanoseconds, including decomposition of the received ID, generation of the new ID and storing the result in the request context. The resulting ID will be sent in the header of every consecutive request to other services.

Parser implementation

After some logs have been produced, I wrote a parser for analysing these logs. A simple Golang interface represents a log entry:

type Log interface {
	// RequestID returns the unique ID of the original request
	RequestID() string

	// String returns a string representation of the log
	String() string

	// SenderID returns the ID of the sender
	SenderID() string

	// ID returns the ID of this log. It is basically the ReceiverId
	ID() string
}

// Default implementation with functionality to parse chained IDs
type DefaultLog struct {
	data      map[string]interface{}
	requestID string
	senderID  string
	id        string
}

The parser basically just takes one entry, decomposes the unique id and creates an instance of DefaultLog. All the parsed logs are stored in a map or slice which can be iterated, sorted and filtered for analysis and graph creation.

Results of analysis

Here are three examples of results that can easily be produced with the parsed logs:

1. Request tree and elapsed time

By changing the String() method, the desired information can easily be printed into dot notation. In this example I executed a request and printed elapsed time per service. This is sort of profiling on system level. The graphic shows which service takes how much time in a clean way.

(numbers do not represent real environment times)

2. Request number between services

This graphic shows system wide calls. It is created over a certain time period and shows absolute numbers of total requests. Please note, one node is one service, not one running instance. We can make the following interpretations:

Service A, C and H call other services through proxy. This is unnecessary and should be corrected.
Service C calls itself, these calls should be refactored to method calls.
Service D is called most by far. These calls are very cheap and caused by a temporary solution. Therefore it can be ignored.
Service F is called directly, the system would be cleaner if we can redirect those calls through the proxy.

3. Outgoing requests per incoming requests Example:

6.5322833   ServiceA
2.5714285   ServiceB
0.99878484  Proxy
0           mailer

This ratio can reveal flaws in the services. Some may have a high number because they basically aggregate data from other services, but they should be investigated. Also it can be seen that Proxy is almost but not exactly 1. This 0.0013 difference consists of invalid requests which the proxy couldn’t resolve and responded with “service not found”. A similar statistic can be made per endpoint.

Simple JSON in Golang

Sun, 14 Sep 2014 22:25:00 +0000

Simplejson is an imitation of Java’s json-simple for the Go language. It allows you to access JSON data without the overhead of predefining a model or casting elements manually. Therefore it is useful, whenever you want to get data from different schemas in a quick and clean way. It additionally provides handy error handling, as well as segmented keys to directly access nested Objects and Arrays. Current Version is 0.3 and its API will change slightly to allow a better error handling in the next version.

Context

While developing a configuration reader, I came across the problem: how to generically access data in nested JSON files in Go. Golang’s native json-package provides methods to access data in json-format by unmarshaling into a predefined struct. Therefore to handle various config-files, every project had to define its own config model that is used to unmarshal data and this was a pain.
If the type is not clear, you are always free to parse it into a map[string]interface{}, but it means you need to cast the elements you access on this map (unless they are all of the same type, e.g. map[string]string). But when you have nested data, it quickly becomes a pain, because in every step you need to cast the elements.
Let’s look at an example. Json config file:

{
    "routes": [
        {
            "path": "profile/id/:id",
            "url": "https://api.eventmanager.com/events",
            "method": "GET"
        },
        {
            "path": "events/id/:id",
            "url": "https://api.eventmanager.com/events",
            "method": "GET"
        }
    ],
    "adminMode": true
}

Built-in possibilities to access

To access the “path”-attribute of the first route, there is the first solution including casting:

// define the base map
var rootMap map[string] interface{}
// unmarshal data into map
json.Unmarshal(data, &rootMap)
// access and cast until desired data is retrieved
routes := rootMap["routes"].([]interface{})
firstRoute := routes[0].(map[string]interface{})
path := firstRoute["path"].(string)

That’s crazy, let’s try another solution by defining a local struct:

var localStruct struct{
  Routes []struct{
    Path string `json:"path"`
  } `json:"routes"`
}

json.Unmarshal(data, &localStruct)
pathOfFirstObject := localStruct.Routes[0].Path

That’s a bit less crazy, but still quite crazy. And this doesn’t make sense at all, if we need to write this for multiple items, then we can just define the model on its own.

Imitation of simple-json

From Java I am familiar with json-simple, which allows you to access json-data like this:

//parse to json
JSONObject rootObject = new JSONObject(data);
//get object
String pathOfFirstObject = rootObject.getJSONArray("routes").getJSONObject(0).getString("path");

The implementation of json-simple in Go allows to do exactly the same:

rootObject, _ := NewJSONObjectFromString(jsonString)
pathOfFirstObject, _ := rootObject.JSONArray("routes").JSONObject(0).String("path")

Direct access through segmented keys

Besides chaining calls, you have also the possibility to directly access nested data by providing a segmented key-string. To access the example data, you would call the key “routes::0::path”:

rootObject, _ := NewJSONObjectFromString(jsonString)
pathOfFirstObject, _ := rootObject.String("routes::0::path")

Version 0.4 preview: Error handling and chained calls

This feature is planned to be included in Version 0.4. For sure Error handling is an important topic. In Java we have the possibility to wrap the whole chain into a Try-Catch block, which prevents the application from crashing, but may hide the source of the error. For this implementation you can do something similar:

pathOfFirstObject, err := NewJSONObjectFromString(jsonString).
                            JSONArray("routes").
                            JSONObject(0).
                            String("path")
if err != nil {
  fmt.Println("Gotcha! Message: " + err.Error())
}

The special thing about this is the customized error that is returned when any error happened. The error provides all methods that JSONObject and JSONArray provide, therefore the chained method-calls can be executed safely and without worrying about panic. All methods of this error return the error itself, so it can be analyzed at the end. Additionally the error includes the key where the casting failed and can be used to track back the source of the error.

Model Binding for Android

Mon, 08 Sep 2014 20:50:00 +0000

There are tons of approaches to update the UI when data changes or the other way round, in short: model binding. In this post we will investigate four possible implementations of a one-way binding and compare their performance in a series of tests. Different code snippets provide an overview of the complexity when applying the approaches to a real example.

Context

While developing the Android Client for a platform that provides near-realtime experience for interactions between users, we had to implement a binding between the Model and the UI. During the development we created and evolved our own system, but we ended up with something very similar to Java Bean’s PropertyChangeListener. To find a better approach, I created a small setup consisting of a model “Profile” and a UI reflecting a profile’s properties (such as name, age, followers). Inside this setup I implemented the different solutions and trialed them in a predefined set of tests.
Unfortunately I didn’t find time to finish a version with an Eventbus such as Square’s Otto; I may add this in the future.

Approaches

Overview

Original: Project’s implementation as-is
PropertyChangeListener: the java.beans solution + a modified version that notifies on UI-Thread
Property Class: Properties are replaced by a Wrapper-class with Observable-feature
Proxy solution: Java’s dynamic proxies are used for notification in an AOP-like manner

Project’s solution

Our own solution is very similar to PropertyChangeListener, a FieldObserver can directly bind to a FieldObservable Model’s Field. FieldObserver and Field are generic and provide type-safety.
Further it provides Thread-safety and updates the Observers directly on the UIThread which allows us to make updates from any thread.
The disadvantages are the huge boilerplate when creating a new model, as well as the high complexity of the implementation. Therefore this implementation needs some serious refactoring, which will probably never happen, since the app is not under active development anymore.

Code example on Gist

Let’s take a look how we implement the Profile:

public class Profile extends AbstractFieldsObservable {
    public static final Field<String> FIRST_NAME = new Field<>("firstName");
    private String firstName;
        
    public void setFirstName(String firstName) {
        this.firstName = firstName;
        notifyFieldObservers(FIRST_NAME, firstName);
    }
    
    public String getFirstName(){
        return firstName;
    }
}

As you can see, we were using an abstract class to inherit the common functions. Refactoring this to a component would make Profile implementation look like the next one, PropertyChangeListener.

PropertyChangeListener

This implementation is based on Java Bean’s PropertyChangeListener. There are basically two versions under test, the original one and a slightly modified one. In the modified version the Listeners are notified on the UI-Thread which allows us to call the model’s update-method from a background thread.

    public void firePropertyChange(final PropertyChangeEvent event) {
        ...
                uiHandler.post(new Runnable() {
                    @Override
                    public void run() {
                        finalP.propertyChange(event);
                    }
                });
      }

Profile implementation looks like this:

public class ProfilePropChangeListenerOwn implements Profile {
    private PropertyChangeSupport changeSupport = new PropertyChangeSupport(this);
    private String name = null;
 
    public String getName() {
        return name;
    }
 
    public void setName(final String name) {
        final String oldName = this.name;
        this.name = name;
        changeSupport.firePropertyChange("name", oldName, name);
    }
 
    public void addPropertyChangeListener(
            PropertyChangeListenerOwn l) {
        changeSupport.addPropertyChangeListener(l);
    }
 
    public void removePropertyChangeListener(
            PropertyChangeListenerOwn l) {
        changeSupport.removePropertyChangeListener(l);
    }
}

Property Class

The property class is a generic class, that holds its own listener. The Observer pattern is therefore moved from the model to its properties.

Code example on Gist

Profile implementation now gets really comfortable:

public class ProfilePropsObserver implements Profile {
    public final Property<String> name = new Property<String>();
}

Done. This is a very solid solution and lets you implement new models lightning fast. The disadvantage of this solution may be the high number of small objects that are produced, which shouldn’t be a problem in most contexts. Further it is hard to acquire information about who is listening to a model, since the listeners are spread over the model’s properties.

Dynamic Proxy solution

The most advanced and interesting solution includes the usage of Java’s dynamic proxy. A similar approach is also used by the widespread Spring framework.
The dynamic proxy stores properties according to the called “set”-method in a Map. Therefore “setFirstName” will store the provided value under the key “firstName”. Further it implements “get” and “is”, as well as addObserver. RemoveObserver is not implemented, since it is not used by the tests. The full code of the Proxy can be found here:

Code example on gist

This is the interface of Profile:

public interface Profile {
    public final static String name = "name";
    public String getName();
    public void setName(String name);
}

Creation of a new Profile will in this case look as follows:

profile = (Profile) PojoProxy.newInstance(new Class[]{Profile.class}, this);

Note: To use Dynamic Proxy on Android, the ClassLoader must be obtained via a context: context.getClassLoader()

For more information see also “Java Reflection - Dynamic Proxies” by Jakob Jenkov

Comparison

All tests are run on an LG Nexus 4. There are two profile models that are modified during the test.
The time measurement ends when all changes have been applied (using Timelatch). The times below represent the average time per setter-call and are calculated by: (Total time) / (number of setter-calls)

UI-Thread: 24’000 setter-calls
Multithread: 3 Threads à 8’000 setter-calls

Approach	UI-Thread	Multithread	GC runs
Project solution	28 μs	40 μs	20
PropertyChangeListener	38 μs	–	23
PropertyChangeListener (mod)	31 μs	44 μs	23
Property Class	33 μs	40 μs	20
Dynamic Proxy	85 μs	93 μs	45

Conclusion

Please note the performance penalty of multithreaded runs, which is caused by the low priority of background threads. The custom implementation seems to have a slight performance advantage. But the differences are small. Only dynamic proxy takes significantly more time and causes more GC-runs than the other solutions. I suspect the overhead of the reflective method lookup causes this performance cost.

From implementation perspective the Property Class is the easiest to implement and use. But the fact that observers are stored for every property can be a big disadvantage.
The Dynamic Proxy provides also an easy-to-use solution, but comes with an overhead at runtime. In our case this is the solution I would go with today, because it gives us further possibilities to intercept interactions with the model.