Software metrics measure various attributes of a piece of software and are becoming essential for a variety of purposes, including software quality evaluation. One type of measurement is based on source code evaluation. Many tools have been developed to perform source code analysis or to measure various metrics, but most use different metrics definitions, leading to inconsistencies in measurement results. The metrics measured by these tools also vary by programming language. We propose a unified framework for measuring source code that supports multiple programming languages. In this paper, we present commonalities of measurable elements from various programming languages as the foundation for developing the framework. We then describe the approach used within the framework and also its preliminary development. We believe that our approach can solve the problems with existing measurement tools.

Yann-Gaël Guéhéneuc, 2014/02/03

This paper make explicit the problem of having consistent measurements across programming languages. It starts by observing three problems with the state-of-the-art and state-of-the-practice:

1. the inconsistent metric definitions across different programming languages;
2. which is aggravated by the inconsistent metrics definitions for the same languages across different tools;
3. the lack of tool support to analyse in the same, consistent framework components written in different languages.

The paper then recalls that several tools exist and name a few of them (but not PADL…) and justifies the lack of multi-language support in these tools by the cost of developing analysers and metrics for different languages. It then proposes a new framework that should have a “lower-cost” although it is not clear how different is the proposed framework from that of the previous tools. The paper implicitly states the need for a unified meta-model, which is then describes in the form of the Unified Code Model (UCM).

The UCM is the result of a thorough and systematic study of 20 popular programming languages and of their measurable elements. Table III in the paper is very interesting because it provides a global view of the elements common and different across programming languages. However, I feel that the elements “method”, “procedure”, and “function” could have been considered together wrt. the measurements. Also, it is not clear why the Ruby programming language is marked as having no modules.

The paper then describes a tool, UNICOEN, that can parse components in various programming languages and generate UCM models. Using UCM, UNICOEN, and a study of some major metrics, focusing “on size and complexity because they are the most important measurements” (McCabe, Halstead's, CK…), the paper shows the metrics values can be computed on components in Java and JavaScript.

The paper does not raise the problem of dealing with the dependencies among heterogeneous components, i.e., components written in different programming languages. Also, it faces the same issues as other works: how to “provide an easy way to measure a metric”, “to evaluate the metrics of software written in more than one programming language”, and to “use the same metric definition standard”. It solves these issues through the UCM and UNICOEN, as we did with PADL and its various parsers. Now, the paper does not say, for example, if each UCO (unified code objects, our constituents) keep track of a line start and end, although they are map to statements. It does not consider the presence of “ghosts”, i.e., UCO known only be references but not analysed. Finally, it brushes the problem of consistency of the analysis and metric computations for different programming languages, which would be a very nice follow-up for this paper.