Why should I use metrics?
It’s almost certain that you are reading this because you think that the use of metrics could improve your development process. You may already be convinced, but you may have to convince yourself or others that using metrics in your development process will result an improvement in the quality of your code and that the value gained from this improvement will be greater than the costs of implementing it. The intention of this page is to give you such information and to assist you in making your judgement. The approaches we suggest are not prescriptive – you should view them as starting points for developing your own approach.
How do I start?
The quick answer here is slowly. If you are going to build metrics into your process you need to do so in a controlled way. You need to understand what you are trying to achieve before you start. Having established what you want to achieve you need to adopt a strategy to realise that aim. Then you have to execute it. Once you have completed this first cycle you then need to refine your process based on how well the approach is meeting your aims. After an iteration of the process you might even change your view of what you are trying to achieve.
Here are a number of aims that you might have –
I have ordered these approaches by their level of complexity. Simple measurement is not that useful as a first step in assessing code quality but it may be a requirement imposed on you (some project managers insist on knowing how much code has been created in a development iteration) and if so you can use it as a useful starting point – perhaps breaking the figures down by component (e.g. Java packages) and class.
- Simple measurement – you might just want to measure the quantity of code generated in an iteration.
- Poor coding practice – you might be looking here at very large classes and methods, very complex classes and methods.
- Poor design – you would be looking here at the relationships between classes and packages, measuring aspects like coupling and cohesion.
- Fault prediction – based on the values of previous metrics you would try to identify areas that might generate above average levels of errors.
- Overall quality – you are using a combination of measurements to assess whether this particular release is of a higher or lower quality than its predecessor and which areas of the code have caused this difference to arise.
I would suggest starting with poor coding practice, identifying large classes and methods. My reasoning here is that this is probably the quickest way to identifying areas that might be poorly coded. Resolving the size issues will probably force you to rethink your designs. When you first start metric measurement you should set your metric limits to a figure that will catch only the very worst of your code then gradually bring them closer to your ultimate target. This will mean that developers won’t be faced with an overwhelming body of code to deal with. Most metrics tools are fairly quick so it might be worth doing a couple of runs to find a level that would be suitable to start with. Another approach is to limit the measurement to a particular area. This can be useful as an illustrative example to persuade developers and management that the effort undertaken is likely to yield dividends.
No matter what your initial approach is the effort will be worthless unless you build in a measure / assess / fix cycle for each development iteration. You also need to look at other project measurements which can show a tangible improvement in your development process – basically the return on your investment. These could be things like the number of bugs reported or the quantity of code changed. Turning round to the person who is funding your development process and saying – ‘We know it’s working because our metrics are approaching the target values’ is useless. If you set up a process that encourages developers to meet certain metric targets then they will try to do so, and will probably succeed. That is not the same as improving the quality of your software unless you can prove a link between the metrics achieved and quality.
It’s important to acknowledge that even if you automate the process you will still have to devote considerable resources to it. It may be possible that after a few iterations your process will be largely automatic – immediately directing you to code that may have quality issues – but in the early iterations someone will have to interpret results and make judgements on the appropriate levels of metrics.
Asking the right questions - Selecting the appropriate metric set
You will only get a useful answer if you ask the right question. In this case asking the right question means selecting the right metrics. The ‘right’ metrics will be different for every project and perhaps even for different stages of the same project.
The first step to being able to ask the right questions is to understand the facts available to you – in this case the metrics. In general metrics fall into the following categories –
You might choose to measure only one aspect – say complexity - and in such a case you might simply choose a single metric – say Cyclomatic Complexity and just use that.
- Quantitative metrics – Metrics that measure the quantity of things – lines of code, number of methods, number of references.
- Complexity metrics – these metrics measure how complex the code is both in terms of its functional complexity but also in terms of its ‘readability’. Both of these can contribute to measuring how easy a piece of code is to maintain and how likely it is to have faults.
- Structural metrics – Measure the contribution of individual code artefacts to the quality of the overall design of the code.
Avoiding duplication by using metric families
Let’s say you want to measure the complexity of the code in your application. There are many measures ofcomplexity. There is little point in using more than one of them unless you have a specific reason. So when you try to decide which metric to use you need to think about what it is that you are measuring. For example you might consider measuring the complexity of a piece of Java code by measuring expression density i.e. the average number of expressions per line of code or you might choose to use Cyclomatic Complexity as your measure.
You can simplify your selection process by deciding which families of metrics you are interested in using – for example you might choose to measure complexity, coupling and cohesion in your code. There are a huge number of metrics that you could choose from just to measure these three factors. Choosing more than one of each will mean that you will have more data to analyse. You might also see conflicting results between different metrics measuring the same code – these will be difficult to explain unless you have a good understanding of the principles behind the metric that you have used. There are a number of ways to decide between metrics that apparently measure the same thing –
You could study the (usually academic) information relating to each of the metrics and use this to determine which metric best meets your needs. This is likely to be a lengthy and difficult process unless you already have expertise in this area.
You can look for comparative studies of some of these metrics where the work in 1 has been done for you. Such studies aren’t available for all metric families but most of them will help you in making a decision.
Base your selection on an existing metric suite. Again this might not cover all the metrics that you might feel that you require but it can provide a starting point. The Chidamber-Kemmerer (C-K) suite is a good starting point as it covers a number of the main metric families.
Carry out an empirical study with a subset of your own code. For example you can visually grade this code, measure the results with the metrics under consideration then choose the metric that offers the closest match to your grading. This might seem to be a good way to start but it does involve a lot of initial labour and a good understanding of the metrics available. I would suggest that this is an approach best left until you have developed some expertise in metrics.
The next step
This part of the process - choosing the tools, selecting the metrics, designing the reports, shaping the overall process - it's a lot of work, it takes time - but all this pales into insignificance when you actual try to implement it and integrate it with your current development process. We'll handle that in the next page.
If you want to find out more about Java metrics you can read our full tutorial here or you can read the individual sections on System and Package Level Metrics, Class Level Metrics and Method Level Metrics.