Jean's Journal

⊕ Jean Tessier on LinkedIn

May 03, 2025

Re-created the "recent" API changes for Dependency Finder using the latest version of JarJarDiff. By "recent", I mean from version 1.2.1-beta3 forward.

May 01, 2025

There is a curious check in DifferencesFactory when dealing with removed methods. It checks that either both oldClass and newClass are classes, or both are interfaces, before marking a removed method as being inherited. I found a note, dated 2001-04-05, that discusses methods that are now being inherited. Back then, interfaces could not provide implementations, so a class could not inherit a method implementation from an interface. I did think, at the time, of an abstract class inheriting a method declaration only. Now, interfaces can supply a default implementation for their methods. So, a class can (or an interface) can inherit declarations as well as implementations from their parents, superclasses or superinterfaces. I went ahead and removed this extra check that is no longer relevant.

I wrote some tests first, this time.

The note from 2001 says that removed features should only be shown if they are not inherited from a parent. The XML output from JarJarDiff will continue to show the declaration, but mark it with inherited="yes". The HTML output from DiffToHTML will drop it, so it does not confuse diff reports.

April 30, 2025

Found another problem with JarJarDiff and bridge methods. Earlier, I switched to processing methods up by unique name instead of signature. I forgot to update the part where it checks if a removed method is still inherited from a superclass or interface. It was comparing signatures to a wanted unique name, and not finding anything.

I need to get better at test-driving these errors. Setting up test fixtures for differences can be fairly complex, and I've become lazy.

I noticed the XSL stylesheet for DiffToHTML does not take the inherited attribute into consideration. If a feature is no longer in a class, but gets inherited from a parent, it shows up as a removed field or as a removed method, with the inherited="yes" attribute. The stylesheet could drop them from the HTML output. Or should that be a separate tool?

April 27, 2025

I think there are two underlying problems with how JarJarDiff deals with bridge methods.

The first problem is that I use parallelStream() in many places, e.g., Classfile.getMethod(filter), and blindly trust the JVM to do the right thing. In most places, I use LinkedList to store lists of things, and setting up a parallel stream on a LinkedList somewhat defeats the purpose See lesson 3-5 in this course on functional programming.. When I call findAny() on a parallel steam over a LinkedList, sometimes it finds the matching thing, and sometimes it does not. The solution, here, is for me to use stream() only. Processing with parallel streams might be more efficient on very long lists, but I don't know that programming elements lists fall in that range. I could profile it to make sure, but my intuition will do for now.

The second problem is that DifferencesFactory looks up method by their signature. Bridge methods share the same signature as the method they are bridging. So, sometimes Classfile.getMethod(filter) returns the actual method, and sometimes it returns the bridge(s). The solution, here, is to look up methods by their unique name instead. The unique name includes the return type, so the factory will get a unique result. It also has the added benefit that I can treat fields A field's unique name is just its name, so going from names to unique names changes nothing for them semantically. There is a notion of synthetic fields, but I don't think they circumvent the restrictions on names the way methods do with their signatures. and methods the same: look them up by their unique name and use their full signature.

Fixing these two problems takes care of all the fuzziness around bridge methods I discussed on 2025-04-25. The method with the more specific return type will either appear as having been removed or added, depending on the change. The inherited signature and return type will appear unchanged (unless you use -code to look at the actual instructions in the method).

One side effect, though, is that changing the return type of a method used to be considered a method modification. Now, it is treated as if the old method has been removed and the new one has been added.

With the fix in place, I pushed out version 1.4.2.

April 26, 2025

Fixed the deep links into the Tools page (and Manual and Developer too). The previous version was very simpleminded. Each heading became a top-level anchor; duplicates were dealt with by appending Xs to them. The new links are hierarchical, building on higher headings and joining them using _. The link to OOMetrics -version went from #-versionXXXXXXXXXXXXXXXX to #ToolsinDependencyFinder_OOMetrics_Switches_-version.

One day, I would like to turn the switches in the various synopses into links to each switch's description.

April 25, 2025

Looking back at my bridge method example from 2024-09-04 and 2024-07-04 to figure out what to do in JarJarDiff. To refresh our mind, the implementation of an interface method can either match the return type from the declaration, or return something that is congruent with it (i.e., a subtype). In the examples, I posited a stylistic choice on the part of the implementer. They could conceivably go from one to the other, in either direction.

Suppose version X uses a congruent return type, the compiler will add a bridge method to "bridge" the gap.

And suppose version Y uses the exact return type, and does not need a bridge method.

The question remains: what, if anything, should JarJarDiff report when going from X to Y? And about the converse, where it's going from Y to X?

My first immediate concern is that when, in version X, I ask the Classfile for the buildFrom(String) method, I randomly get the implementation or the bridge method.

One tentative solution would be to have two Method_info structures to define the signature: one for its declaration and one for its implementation. If there is no bridge method, the same Method_info can serve both purposes. If there is a bridge method, it plays the role of the declaration.

I explored a couple codebases to see some bridge methods in the wild. It turns out a method can have multiple bridge methods. Here is a "simple" example.

Class B inherits a getMe() method from an interface hierarchy I found a read-life example in Saxon-HE where the AtomicValue class inherits multiple declarations of the iterate() method from multiple interfaces., each with a different return type. The interfaces themselves can have their own bridge methods, according to their own superinterfaces. Class B also inherits a getYou() method from an abstract base class.

Perhaps JarJarDiff should spread each signature into declaration-implementation tuples, one for each bridge method.

When looking at the B class, for the getMe() method, JarJarDiff would consider the following tuples:

• B.getMe(): SI — B.getMe(): B
• B.getMe(): I1 — B.getMe(): B
• B.getMe(): I2 — B.getMe(): B
• B.getMe(): I3 — B.getMe(): B
• B.getMe(): A — B.getMe(): B
• B.getMe(): B — B.getMe(): B

For the getYou() method, it would consider these tuples:

• B.getYou(): A1 — B.getYou(): B
• B.getYou(): B — B.getYou(): B

And for the getThem() method, it would consider a single tuple:

• B.getThem(): B — B.getThem(): B

For each tuple, JarJarDiff would look for a matching tuple in the old and the new codebases and use them to build a CodeDifferences. The declaration part of the tuple would determine any changes in return type or visibility. The implementation part of the tuple would determine if the code itself was changed.

In the case of versions X and Y of com.acme.impl above, version X would have the two tuples:

1. buildFrom(name) : org.ietf.foo.Foo — buildFrom(name) : com.acme.impl.Foo
2. buildFrom(name) : com.acme.impl.Foo — buildFrom(name) : com.acme.impl.Foo

And version Y would have a single tuple:

1. buildFrom(name) : org.ietf.foo.Foo — buildFrom(name) : org.ietf.foo.Foo

JarJarDiff would match the version Y tuple with the version X tuple for the bridge method (No. 1 in each list) and find no differences. That would leave the second tuple in version X (No. 2 in that list) as either a removed method if it's going from X to Y, or an added method if it's going from Y to X.

April 24, 2025

Big bug: JarJarDiff can be non-deterministic under certain conditions. I was generating differences between version 1.4.1.1 and the latest code, and the results kept changing from one call to the next. For example, Classfile could have anywhere between 1 and 6 differences, even though I was analyzing the very same files. I ran the same comparison 10 times and compared the results. A few methods kept coming in and out of the report.

• getConstantPool()
• getField()
• getInterface()
• getMethod()
• getRawClass()
• getRawSuperclass()

These methods have one thing in common: the latest version of Classfile has two methods for each signature: one explicit method and one synthetic bridge method.

In DifferencesFactory, when analyzing a class, we collect unique signatures from the old and new versions. If a method is a synthetic variation, it gets treated as one with the explicit method. The factory uses Classfile.getMethod() to get the old and new Method_info that represent the method. But getMethod() uses a stream with findAny(), whose documentation says: "The behavior of this operation is explicitly nondeterministic." Sometimes, it might return the explicit method, and at other times, it might return the corresponding synthetic method.

I will need to revise how DifferencesFactory selects Method_info instances when looking for differences between methods.

April 23, 2025

Fixed the servlet-based webapp so it properly escapes () and [] and \ and $ characters when you click on a name in a dependency graph.

I also updated the copyright notice to 2025.

I added the link to Perl regular expressions in the Manual to the React app. And I copied inputs to and from the URL's search params. I am still figuring out how to trigger a call to the backend when parameters are passed in the URL.

April 22, 2025

The servlet-based webapp lets you click on a name in a dependency graph to move the query to that element. This is useful when navigating a graph and to follow from one dependency to the next. My React app does not do that yet. I added a Name component to render each name. Each has an onClick handler. For now, the handler is in the Query component that represents the page, but I am struggling to make it populate the right input element and trigger a query to the backend. I did add a way to save the query parameters to the URL. Baby steps.

I also noticed a while back that the servlet-based webapp was slightly broken. When I click on a method name, I get a "URL contains illegal characters" error. I believe it is reacting to () and escape \ characters in the regex. I will need to refine how I escape names for inclusion in URLs in HTMLPrinter. This used to work years ago. There must have been some recent development to tighten what is considered a valid URL.

April 21, 2025

I learned that Pivotal Tracker is going away. Here is the official announcement, dated 2024-09-18. I have not maintained the project there for quite some time. I took a quick look, just to see if anything was worth salvaging. I closed one story about bridge methods.

There was another story about getting deprecation status when it is done with the @Deprecated annotation. As I described on 2008-11-13, you can deprecate a programming element with either a @deprecated javadoc tag or with a @Deprecated annotation. In both cases, the compiler adds a Deprecated attribute to the .class structure. In the case of the annotation, there is also an entry in the RuntimeVisibleAnnotations attribute to represent the annotation and its optional elements, if any. Therefore, the current logic in Classfile and in Feature_info are sufficient to detect deprecated elements, irrespective of how that deprecation is actually done. The specification JVMS §4.7.15 "A class, interface, method, or field may be marked using a Deprecated attribute to indicate that the class, interface, method, or field has been superseded." ties deprecation semantics to the attribute and not to the annotation, so I might not need to look explicitly for annotations in this case. I did add tests to cover both means of deprecating programming elements.

The other stories in Pivotal Tracker were less immediately actionable. I exported the project data to a CSV file so I can dig into it later.

April 11, 2025

Loading Classfile instances from files starts with a beginSession event and ends with an endSession event. These events are triggered in the ClassfileLoader.load(filenames) method, which drives the loading process.

The dependency extraction process described in the DependencyListener interface also has beginSession and endSession events, but they are never actually fired. This process is tied to loading Classfile instances via a LoadListenerVisitorAdapter. This adapter receives load-related events, but only has a Visitor to interact with. It takes the Classfile instance from the load endClassfile event and applies the visitor to it. The visitor sees Classfile one after the other, but it has no notion of a session with a beginning and an end.

In the sequence diagram below, I've attached a VerboseListener to both the TransientClassfileLoader and the CodeDependencyCollector to illustrate the events. In the diagram, load-related events are in yellow and the dependency extraction-related events are in red. The adapter only acts on endClassfile events and does nothing on all others.

If, instead, I removed the adapter and I made CodeDependencyCollector implement the LoadListener interface directly, it would then know when the session starts and ends. It would be able to map a dependency extraction session to the Classfile loading session.

The revised sequence diagram below shows how CodeDependencyCollector acts on beginSession, endClassfile, and endSession load-related events, and does nothing on the other ones.

There are other tools that rely on LoadListenerVisitorAdapter, but they have nothing to do with dependency extraction.

April 09, 2025

I have been toying with the idea of listening to dependency events using DependencyListener and feeding the dependencies to a relational database. The schema could separate package, class, and feature names. It could also track the cardinality of each dependency.

For example, take the following class Test:

    public class Test {{
        System.out.println("abc");
        System.out.println("123");
    }}

When extracting dependencies, a listener receives the following dependency events:

    Test --> java.lang.Object *
    Test.Test() --> java.lang.Object.Object() *
    Test.Test() --> java.lang.System.out *
    Test.Test() --> java.io.PrintStream *
    Test.Test() --> java.io.PrintStream.println(java.lang.String): void *
    Test.Test() --> java.lang.String *
    Test.Test() --> java.lang.System.out *
    Test.Test() --> java.io.PrintStream *
    Test.Test() --> java.io.PrintStream.println(java.lang.String): void *
    Test.Test() --> java.lang.String *

The Test.Test() constructor depends on class java.io.PrintStream twice. It depends on method java.io.PrintStream.println(java.lang.String): void twice. Currently, Dependency Finder collapses these multiples into a single edge in the dependency graph. Is there some value to this information? I don't know.

I toyed with a schema that would split each node into its constituents. It can use NULL in feature positions for class-related dependencies. Likewise, it can use NULl in class and feature positions for package-related dependencies.

+--------------+------------+--------------+----------------+------------+-------------+---------------------------------+--------------+-------------+
| package_from | class_from | feature_from | confirmed_from | package_to | class_to    | feature_to                      | confirmed_to | cardinality |
+--------------+------------+--------------+----------------+------------+-------------+---------------------------------+--------------+-------------+
|              | Test       | NULL         | true           | java.lang  | Object      | NULL                            | false        | 1           |
|              | Test       | Test()       | true           | java.lang  | Object      | Object()                        | false        | 1           |
|              | Test       | Test()       | true           | java.lang  | System      | out                             | false        | 2           |
|              | Test       | Test()       | true           | java.io    | PrintStream | NULL                            | false        | 2           |
|              | Test       | Test()       | true           | java.io    | PrintStream | println(java.lang.String): void | false        | 2           |
|              | Test       | Test()       | true           | java.lang  | String      | NULL                            | false        | 2           |
+--------------+------------+--------------+----------------+------------+-------------+---------------------------------+--------------+-------------+

In SQL, we can roll up dependency counts with some GROUP BY statements. Here is an example that sums outbound dependencies by method.

    SELECT
        package_from,
        class_from,
        feature_from,
        SUM(cardinality)
    FROM
        dependencies
    WHERE
        class_from IS NOT NULL AND
        feature_from IS NOT NULL
    GROUP BY
        package_from,
        class_from,
        feature_from
    ;

Here is another example that sums them up by package.

    SELECT
        package_from,
        SUM(cardinality)
    FROM
        dependencies
    GROUP BY
        package_from
    ;

And here is an example for inbound dependencies, summed up by class:

    SELECT
        package_to,
        class_to,
        SUM(cardinality)
    FROM
        dependencies
    WHERE
        class_to IS NOT NULL
    GROUP BY
        package_to,
        class_to
    ;

I struggled to use an ON DUPLICATE KEY UPDATE clause to increment the cardinality column and update the confirmed_* in a single pass. Otherwise, I would have to accumulate all events in memory and write the final status after extracting dependencies. More on this later, maybe.

Another question that arose was how wide should these columns be? I don't want to use CLOB columns, so I have to give them a size large enough to contain all values. Is 1,024 too much and wasteful? Is 256 too small?

I added some measurements in OOMetrics, with a StatisticalMeasurement at the project level to find the largest value. Is it useful enough to warrant adding them to the codebase? I found an alternate way to get mostly the same information with ListSymbols and some creative bash scripting.

    ListSymbols lib/DependencyFinder.jar -txt -methods \
    | perl -n -e 'chomp; print sprintf("%6d\t%s\n", length($_), $_)' \
    | sort -n -r \
    | head

OOMetrics does a better job of separating package, class and feature portions of each fully qualified symbol. But, maybe I don't need that level of precision. And, ListSymbols has fine controls to also look at fields and local variables, which are outside the reach of OOMetrics.

April 02, 2025

I tried to generate some representations of Dependency Finder using ChatGPT. I asked it to do them in the style of Studio Ghibli.

Here is one for Dependency Finder:

And here is one for OOMetrics:

March 19, 2025

I removed all references to Cobertura, Clover, and EMMA, now that I use JaCoCo for code coverage.

March 18, 2025

Java 24 is out. I looked at the JVM specification, and I couldn't find any changes to the .class file format. Good.

March 14, 2025

Changed the HTML metrics report to draw EML histograms using Apache ECharts. At first, I hard-coded a log-log plot to highlight how EML might be a power law, as suggested by Kent Beck. But HistogramMeasurement can be used with more than method lengths and for measurements that might not be power laws. So, I added a parameter in its <init> tag to specify the preferred plot for that histogram measurement. For now, it is only applicable to the HTML report.

Here is an example of the distribution of method length in Dependency Finder.

March 13, 2025

Added HistogramMeasurement to all the OOMetrics printers. JSON, text, XML, and YAML printers can print the actual histogram data, but the CSV printer cannot. This is just like it cannot expand collection measurements.

March 12, 2025

Added new OOMetrics measurements for raw method length (RML) and effective method length (EML). RML is based on maxline numbers - minline numbers + 1, and EML is based on |line numbers| (the cardinality of line number entries).

I also fixed SLOC for methods so it includes the method's definition. For most methods, it works just fine. But it also adds 1 "line" to static initializers. SLOC remains an approximation that is close enough.

I made a change to StatisticalMeasurement. When it is empty, it used to return NaN for its sum. This caused problems when I tried to aggregate methods lengths at the class, group, or project levels. I think it is reasonable to have sum return 0 when the measurement is empty. I am beginning to think it wouldn't be a bad thing to have some kind of indeterminate state for measurements that don't have a definitive value, like empty stats or some counters that have not been added to.

I started working on a HistogramMeasurement. It aggregates values from submetrics. One challenge is figuring out what its getValue() should return. For now, I'm having it return the value of the largest bucket in the histogram. I am also having it use integer buckets. It rounds up values to the nearest integer. I will need to modify all the measurement printers so they handle histograms properly.

March 11, 2025

Kent Beck was discussing how long should functions be and I started thinking that I could use OOMetrics to find that out. The default configuration includes SLOC measurements, but they include a value of 1 for abstract methods. Synthetic methods and methods that do not have a Code_attribute get a value of 0, although "not definitive" would be more accurate. The CounterMeasurement type of measurements does not have an indeterminate state, so 0 will have to do.

I could create a new measurement for function length and populate it based on LineNumberTable_attribute. I could either count the entries in the attribute, or do arithmetic with the line numbers it contains. I could even possibly refactor the existing SLOC measurement calculation to make use of it.

For example, take the load(DataInput) method of ClassfileLoader. It is an abstract method in an abstract base class.

    line 62      protected Classfile load(DataInput in) throws IOException;

It is a line of code, so it should be included in the SLOC measurement. And, its length is clearly zero.

Now, consider its implementation in AggregatingClassfileLoader.

    line 75      protected Classfile load(DataInput in) throws IOException {
    line 76          var result = getFactory().create(this, in);
    line 77
    line 78          classfiles.put(result.getClassName(), result);
    line 79
    line 80          return result;
    line 81      }

Its LineNumberTable_attribute only includes lines 76, 78, and 80.

Its SLOC measurement is 3. Looks like a bug is causing it to ignore the method declaration on line 75.

The function length is more ambiguous. Do we count the declaration (line 75)? Do we count the closing brace on line 81? There are 3 active lines in the body, but 5 lines in total. Should the length be 3? 4? 5? 7?

Kent Beck's article shows many 1-line and 2-line functions. So I feel he is not including the method declaration and closing brace in his definition. He's only looking at the body of the method. Most people don't include empty lines and comments either, but the article does not say.

I am tempted to add two new measurements: a raw count that is simply maxline numbers - minline numbers + 1 and an actual count that is the count of line numbers. For the load(DataInput) method implementation, the raw count would be 80 - 76 + 1, so 5. The actual count would be 3. Its SLOC measurement should be 4.

February 02, 2025

Renamed the Micronaut and React projects to more official Dependency Finder names.

• micronaut-depfind-test ⇒ dependency-finder-micronaut
• dependencyfinder-react-app ⇒ dependency-finder-react

January 31, 2025

Figured out how to use environment variables and Docker settings to point the Micronaut service to custom code and dependency graph locations. The defaults in `application.properties` are for running the service locally, outside of Docker. When running it in a container, I can use environment variables to point the service to `/code`, like the Dependency Finder webapp, and then mount volumes at these locations.

I wrote some integration tests for Micronaut. I started with LoadController and ExtractController. I was able to do simple GET requests and verify the load and extract metadata. I could put test-specific properties using special .properties files, but I was not successful is making POST requests to trigger loading or extracting.

I broke up the single micronaut.depfind.test package into:

• com.jeantessier.dependencyfinder.micronaut
• com.jeantessier.dependencyfinder.micronaut.controllers
• com.jeantessier.dependencyfinder.micronaut.services

The whole Micronaut service is still called micronaut-depfind-test, but it's a start.

January 29, 2025

Dependency Finder is now 24 years old.

Looking at code coverage data, I realized there was a missing Fit test for ListDynamicInstructions.

There are no Fit tests for ClassDump either. But that is because it is not even a proper Command-based CLI tool. It was always meant to stand on its own, with no dependencies outside the JDK. When ClassDump encounters a .class file, it reads it one byte at a time, in a while (true) {} loop. The EOFException when it reads past the end of the file will kill it out of the infinite loop. This is not very clean, but the I/O interfaces are not very good at letting me detect the end of the file.

I was curious how big my .class files got to be. The largest one I could find in the Dependency Finder codebase was TestXMLPrinter, in the :integration-tests subproject, at 115KB. It has a lot of tests and most of them use jMock to re-create specific situations. Since most tests didn't interact with the file system, I broke it down between unit tests and the few legitimate integration tests. 196 tests ended up moving to the :lib subproject.

I added a navigation bar to the React app. I couldn't find a standard way of doing this in React or Vite, so I went with a custom component for now.

I pushed out version 1.4.1. While I was following instructions to generate the sample data for the new release, I realized the Linux scripts were not escaping spaces correctly. For example, when I called:

    JarJarDiff -name "Dependency Finder" ...

The differences report had a name of "Dependency" only. I had to fix the launching scripts to handle escaped spaces correctly.

I pushed out version 1.4.1.1 with the fixed scripts. The launching scripts are the only thing that's changed, so I will not update the documentation and everything at this time.

January 25, 2025

I managed to fix the connection between the Micronaut microservice and the React frontend, even through an ngrok tunnel. After digging through the Vite documentation, I moved the .env.local file that was setting the service URL to http://localhost:8080. By renaming it to .env.development, it now only applies in development mode, and it is no longer part of the final build. There, it uses document.location.origin instead. So, whatever URL gets you to the app also gets it to the service.

I used the same URL for actions in the service and pages on the frontend, which can lead to confusion. If you go to / and then navigate to /extract, you get the Extract UI. If you go directly to /extract, you get a JSON with Extract metadata. So, I moved the actions in the service to /api URLs. Now, you can no longer go directly to /extract, it only works when navigating from the home page. You can reach the Extract action by going to /api/extract.

There are a few things I still need to work on:

1. The Micronaut app needs a better name than micronaut-depfind-test.
2. I don't like the package name for the Groovy code, micronaut.depfind.test.
3. The names of code and dependency graph files need to be more user-friendly, given that it is hard to change them in a running container.
4. The Home page for the React app is still the default Vite demo page.
5. I need to make the navigation accessible on all pages in a way congruent with Vite.

But, all the major functionality is there.

January 24, 2025

Created a Docker image to run Dependency Finder Micronaut app and React frontend in a Docker container.

I built the React app and dropped it in the Micronaut app's src/main/resources/static/frontend/ folder. This way, it is included when the Micronaut app gets deployed. If I run it in a Docker container and export port 8080, my browser can download the React app from http://localhost:8080 and it can contact the microservice back at http://localhost:8080.

I created a tunnel to my app running locally using ngrok. I could download the React app from another computer, through the ngrok tunnel, but it still believed it was being hosted at localhost, so it couldn't connect back to the microservice to load any data.

I also tried to deploy the Micronaut Docker image using GitHub actions, but I couldn't get the workflow quite right. In Micronaut, interactions with Docker are mediated through Gradle. I will need to explore how to make it work with the GitHub actions for Docker. Or, maybe I just use a bunch of shell commands instead of a third-party GitHub action.

January 23, 2025

Found a problem with JSON and YAML printers and irregular values. An empty StatisticalMeasurement has all sub-values as NaN. It is possible for a RatioMeasurement to have a value of NaN, infinity, or negative infinity, which Java renders as "NaN", "Infinity", and -"Infinity" respectively.

JSON does not have a representation for these values. I changed the printers to put null instead.

YAML does have representations for these values. I changed the printers to put .NaN, .Inf, and -.Inf as appropriate. YAML is not case-sensitive, but I liked these capitalized versions.

And it was not just the printers, but the MetricsToJSON and MetricsToYAML XSL stylesheets too.

With these fixes in place, I was able to add the /metrics page to my React app and complete it. I also used Apache ECharts to render the dependency metrics chart and all the histograms.

January 22, 2025

I needed to add a JSON output to dependency metrics from DependencyMetrics so they can be fed from the Micronaut service to the React frontend.

I was looking at MetricsReport. It exposes charts, but each of the eight charts are the same lengths, padded with zeroes as needed. This seems wasteful to me. I added a histogram version of each chart that only includes parts where there is actual data.

I also split the old MetricsReport into an abstract base class MetricsReport that has all the flags, and a specific TextMetricsReport for the details of writing textual metrics reports. With this split in place, it was easier to introduce a class that renders the report as a JSON object.

January 21, 2025

Wrote a JSON printer for dependency cycles.

These cycles:

    a
        --> b
            --> a
    a
        --> b
            --> c
                --> a

Become the following JSON document:

    [
        [
            {"type": "package", "name": "a"},
            {"type": "package", "name": "b"}
        ],
        [
            {"type": "package", "name": "a"},
            {"type": "package", "name": "b"},
            {"type": "package", "name": "c"}
        ]
    ]

January 13, 2025

Finished converting all tests to JUnit Jupiter.

I used Perl to do some of the heavy lifting, then reviewed each file manually.

These regex would move the assertion message to the last parameter. They would get confused when expressions were too complex or had method calls with extra commas in them.

    s/assert(Equals|Same|NotSame)\((.*), +(.*), +(.*)\);/assert\1(\3, \4, \2);/;
    s/assert(True|False|Null|NotNull)\((.*), +(.*)\);/assert\1(\3, \2);/;

These regex would change test method signatures and add annotations.

    if (/^(\s*)protected void setUp\(\)/) {
        print($1 . "\@BeforeEach\n");
        s/protected void/void/;
    }

    if (/^(\s*)public void test\w+\(\)/) {
        print($1 . "\@Test\n");
        s/public void/void/;
    }

    if (/super.setUp\(\)/) {
        $_ = "";
    }

January 12, 2025

I think I found a workaround for my problems with DOT files. I added a switch to DependencyReporter to force it to output nodes that match the filter criteria, along with nodes that match the scope criteria. The new switch is called -include-filter-nodes. With it, the XML output will have everything needed by DependencyGraphToDOT to render the graphs I want.

I also changed the URL prefix for DTDs from https://depfind.sourceforge.io/ to https://jeantessier.github.io/dependency-finder/. This way, these resources will come from the official site on GitHub instead of the old site back on SourceForge.

January 11, 2025

There are some issues with the way I generate the DOT files. I currently do it in two passes:

1. Generate nodes based on packages, classes, and features in the source XML.
2. Generate dependency edges based on <outbound> elements.

The initial samples I was working from when I handcrafted the XSL stylesheet was essentially the raw output of DependencyExtractor. It included all the relevant nodes and all edges were represented by both <outbound> and <inbound> elements. It made sense, under these circumstances, to only look at <outbound> elements to prevent duplication. And every dependency was between nodes present as either <package>, <class>, or <feature> elements.

But most graphs come from DependencyReporter instead, so we can focus on specific dependencies. These graphs may include only <inbound> elements, for example. The source parameters will result in <package>, <class>, or <feature> elements, but filter parameters will not.

For example, if I want to find out which com.jeantessier.dependencyfinder.* classes are using GraphCopier or GraphSummarizer, I can do the following:

    DependencyExtractor -xml lib/DependencyFinder.jar |
    c2c -xml
        -scope-includes '/Graph(Copier|Summarizer)/'
        -filter-includes /com.jeantessier.dependencyfinder/

The XML output is as follows. There is one <package> element and two <class> elements. The references to com.jeantessier.dependencyfinder.* classes are only in <inbound> elements.

    <dependencies>
        <package confirmed="yes">
            <name simple-name="com.jeantessier.dependency">com.jeantessier.dependency</name>
            <class confirmed="yes">
                <name simple-name="GraphCopier">com.jeantessier.dependency.GraphCopier</name>
                <inbound type="class" confirmed="yes">com.jeantessier.dependencyfinder.ant.DependencyReporter</inbound>
                <inbound type="class" confirmed="yes">com.jeantessier.dependencyfinder.cli.DependencyReporter</inbound>
                <inbound type="class" confirmed="yes">com.jeantessier.dependencyfinder.gui.DependencyFinder</inbound>
            </class>
            <class confirmed="yes">
                <name simple-name="GraphSummarizer">com.jeantessier.dependency.GraphSummarizer</name>
                <inbound type="class" confirmed="yes">com.jeantessier.dependencyfinder.ant.DependencyReporter</inbound>
                <inbound type="class" confirmed="yes">com.jeantessier.dependencyfinder.cli.DependencyReporter</inbound>
                <inbound type="class" confirmed="yes">com.jeantessier.dependencyfinder.gui.DependencyFinder</inbound>
            </class>
        </package>
    </dependencies>

And here is the graph that I generate from this XML output.

The first thing I can do is look at both <outbound> and <inbound> elements. There might be some duplicate edges as a result, but since I use a strict digraph in the DOT file, Graphviz should de-dupe them for me.

There will still be a problem that the nodes from filter parameters are not expressed correctly, though.

Where what I really want is closer to:

January 10, 2025

I saw someone's Advent of Code solution where they were generating a DOT file for a graph and rendering it using Graphviz. I thought I should be able to render a dependency graph the same way. I wrote an XSL transform to convert the XML graph file to the DOT language. I had to add each node's simple name to the XML output.

January 09, 2025

Converting more unit tests to JUnit Jupiter. I migrated all tests to jMock with JUnit Jupiter. I migrated all JUnit 4 parameterized tests to JUnit Jupiter parameterized tests. I migrated all JUnit 4 tests to JUnit Jupiter. I used a Perl script to do some of the more tedious conversions, but I still had to manually inspect hundreds of files. All I have left are a bunch of JUnit 3 tests in com.jeantessier.dependency, and then I'll be all on the most up-to-date JUnit.

January 08, 2025

Converting more unit tests to JUnit Jupiter.

January 07, 2025

Started converting some of the unit tests to JUnit Jupiter.

I really like assertLinesMatch() for comparing text that spans multiple lines. I got creative with Stream.of() and flatMap() to build the expected output for some printers.

January 06, 2025

Finished converting all the integration tests to JUnit Jupiter.

I started converting some unit tests, too. I created the class com.jeantessier.MockObjectTestCase as an almost drop-in replacement for the one from jMock for Junit 3. It saves me from having to edit all the calls to mock() and checking(), but the name is ugly. I still have to edit all the assertions, mostly to put the error message as the last parameter. It's tedious work.

January 05, 2025

Started converting some JUnit 3 tests to JUnit Jupiter. I started with integration tests, since there are not as many of those. I'm hoping I can eventually remove dependencies on JUnit 3 and 4 completely, but that might take a while.

I'm still using jMock, but I don't have access to MockObjectTestCase anymore. I managed to add mock() and checking() methods to one base class so I wouldn't have to edit too many tests. I guess I could write my own MockObjectTestCase class, maybe as a temporary stopgap.

I am keeping setUp() methods as setUp(), but with the @BeforeEach annotation from JUnit Jupiter. I need to be careful when a test class hierarchy needs setup at multiple levels. I got burned because the subclasses were overriding their superclass's setUp() and it wasn't being called by JUnit as a result. I will be using more descriptive names in superclasses going forward.

November 21, 2024

Ran this journal through a spellchecker.

November 19, 2024

Finished reformating the /query page of my React app using Grid Layout. I was able to get rid of <blockquote> tags and replace them by <div> tags with CSS classes. I used the ch unit to re-create the indentation from <blockquote>.

November 18, 2024

I'm reformating the /query page of my React app using Grid Layout.

November 11, 2024

Dependency Finder received PR #13 from GenZmeY that fixes the Windows scripts. I haven't had access to a Windows machine for years, so I didn't have a way to test them. Thank you, GenZmeY!

October 20, 2024

Added a fully functional /query page to my React app. You can specify scope and filter criteria, run the query, and view the results. It doesn't do graph surfing or set query string search parameters, yet. And, I don't know if I will get to the Advanced Query page. I might do closures and metrics first.

I found a "bug" in com.jeantessier.dependency.JSONPrinter. It rendered boolean values for confirmed nodes as the strings "true" and "false", instead of the raw boolean values true and false that are allowed in JSON. I had to fix quite a few test data templates. The templates didn't end with a newline character because I compared them to the printer's output, which doesn't have one either. But IntelliJ IDEA decided to add a newline on each file that I touched, out of the goodness of its heart. I would have been tedious to open each file in another editor, just to remove a newline. So, I adjusted the test to add a newline to the printer's output instead.

October 19, 2024

Copied the compiled React app to a static assets folder There is an example in the Micronaut guides. It sets up a rather convoluted sample app with a messaging service, but you can tease out the crux of how to do static assets if you squint. under Micronaut. With a single server running the Micronaut app, I can get to the React app's home page and its extract and load pages to trigger the matching extract and load endpoints in the Micronaut app.

October 18, 2024

Fixed the update checkbox on the /extract page. Also, added title and logo in the browser tabs.

And now, when I build the React app to static assets, it all works just fine. I don't know what I did that made it work, but it is nice.

October 17, 2024

More tinkering with my React-based experiment. I added the /load page to load a pre-extracted dependency graph instead of creating one from compiled Java code. This got me to create a shared <Stats> component for both pages.

I wired the forms to actually trigger the corresponding actions on the Micronaut service. It even forces a refresh of the <Stats> component with updated data from the service.

I tried adding dotenv so I could put the service URL in a .env file. After getting it all setup, I found out Vite already includes dotenv, complete with a .env.local file that is already being ignored by Git. So, I reverted my dotenv work in favor of the "standard" one.

October 16, 2024

More tinkering with my React-based experiment. I fleshed out the /extract page and created sub-components for parts of the page. I found react-hook-form to try to handle the form to launch the extraction, but I am struggling to make it work properly.

October 11, 2024

Figured out how to log requests in Micronaut for my React-based experiment.

October 03, 2024

I tried to do static exports of my Next.js project, but it seems to be losing the CSS styling.

I looked for more alternatives to create-react-app in 2024. One Reddit user recommended using Vite instead. The Vite documentation describes how to add React Router, which is what I have been using on other React projects. It also described using SWR for fetching data. I managed to create a /extract route that fetches source and filter information from the /extract endpoint on the Dependency Finder service. But when I build the project to generate static assets, the page does not render for some reason.

October 02, 2024

My experimentation with Next.js is not going well. It looks like Next.js wants both a server-side component and a client-side component. It requires something like a Node server running to serve the app. I'm already toying with a Micronaut app to interact with the graph, I don't need yet another one just to mediate with the frontend. I have built I used create-react-app, which is no longer the popular, hip way to setup a React project. It is while searching for a replacement that I found Next.js. React apps that deploy as static assets and call to a backend to get their data. I was hoping Next.js could give me something just like that. There is a way to compile the app to a static version, but routing gets very limited. I couldn't do what they call dynamic routing in a static version, but I might be able to make it work with extract and query pages, and the like.

September 30, 2024

More experimenting with Micronaut and Next.js.

I also read an article on the new Class-File API that started previewing in Java 22. It includes features to read .class files, similar to my com.jeantessier.classreader package. The Class-File API also includes features to generate .class files and manipulate code, which I don't do.

September 28, 2024

More experimenting with Micronaut. I managed to extract and load graphs, and issue simple queries.

I google'd how to do React in 2024, and Next.js came on top. So, I'm making my way through the tutorial.

September 27, 2024

Finished work on printers in com.jeantessier.dependency. I can now generate HTML (partial), JSON, text, XML, and YAML directly from inside the application. For JSON and YAML, I tried to make the direct output match the output from the corresponding XSLT stylesheets as best I could.

September 26, 2024

Refactored tests for printers in com.jeantessier.dependency. Instead of using complicated line-matching logic in ad hoc tests, I switched to external files that contain the expected outputs. The files have proper extensions so tools can help with the editing. And, I can use parameterized tests to match test setups to expected output files. I got through text and XML outputs, and did JSON output, too. I also want to do YAML and HTML, for to have the coverage.

September 25, 2024

Experimented with Micronaut to see how I could add the JAR files for Dependency Finder to a Micronaut app and call its classes from the application. I managed to wire together a simple /version endpoint that returns data from the Version bean inside Dependency Finder. I could create controllers that rough match the JSP pages in :webapp. I'll have to figure out how to move a Micronaut app from a top-level Gradle project to a subproject in Dependency Finder.

I have an XSLT stylesheet to convert dependency graphs from XML to JSON. But, I don't have a way to generate JSON directly from within the application.

September 21, 2024

I was reading up on Declarative Gradle, and I found a deprecation warning in build scans. So, I converted all the build configurations to the new Test Suite plugin.

September 20, 2024

Crazy thought: could I replace :webapp with a microservice with a React frontend? Should I? I could write the microservice with Micronaut. I would need a JSONPrinter for dependency graphs.

September 19, 2024

I want to rename getSignatureDeclaration() to locateMethodDeclaration(). This will draw a parallel with the existing locateMethod(). It also links the caveats that applies to both methods: they can only locate methods and declarations in classes that were loaded by a Dependency Finder ClassfileLoader.

Actually, by the same loader as the one that loaded the current classfile.

Actually, only if it is an AggregatingClassfileLoader.

The following example shows a bridge method when implementing Enumeration<E>.

The declaration for the bridge method would be java.util.Enumeration.nextElement(): Object. The declaration for the method with the actual implementation would be the method itself. ⊕ We could try to match the signature of the bridge method to that of the implementation method, but such a match would only work when the methods differ only in their return type.

September 18, 2024

Finished getSignatureDeclaration() by adding a method on Classfile to walk the class hierarchy.

One issue that comes up is what to do when the same method is declared in multiple interfaces.

In the following example, the method m() is declared in both SuperInterface and OtherInterface.

Which one should I pick when trying to locate the declaration of ConcreteClass.m()? Should I always return a list of all candidates? Even if most of the time, it will have only one element?

A second issue comes up when a method comes from an ancestor that Dependency Finder has not parsed. Take the following example that extends java.io classes.

If Dependency Finder only sees the com.acme.foo package and not the java.io package, then it will think the close() method was declared in FooReader, and it won't even see the read() methods. ⊕ Maybe as unconfirmed nodes in a dependency graph.

September 17, 2024

Java 23 is out. I looked at the JVM specification, and I couldn't find any changes to the .class file format. Good.

I tried to add a getSignatureDeclaration() method to Method_info. The goal is to walk up the class hierarchy to locate the declaration of a given method. I don't really like the name, but I can't think of a better one for now.

September 16, 2024

Following up on the Mastodon question about identically named methods, but with different return types. The original poster gave an example that implemented Enumeration<E>, and its nextElement() that should return an E.

I updated my illustrative example.

    public interface Factory<T> {
        T create();
    }
     
    public class Circle {}
     
    public class CircleFactory implements Factory<Circle> {
        public Circle create() {
            return new Circle();
        }
    }

CircleFactory still has two create() methods: one is a bridge method that returns an Object from the definition of Factory with type erasure, and the other remains the method from the source that returns a Circle.

During the discussion, I had to double-check my definitions for descriptors and signatures. A method's signature JLS §8.4.2 is its name + parameter types. A method's descriptor JVMS §4.3.3 is its parameter types + return type. It's interesting to note that strictly speaking, a method's name is not part of its descriptor.

From this, we can state that the compiler will reject source code for a class with duplicate signatures JLS §8.4.2 "It is a compile-time error to declare two methods with override-equivalent signatures in a class.". But, the JVM will accept the bytecode of a class that has methods with matching signatures, as long as their descriptors are different JVMS §4.6 "No two methods in one class file may have the same name and descriptor (§4.3.3).".

September 14, 2024

Someone on Mastodon asked a question about identically named methods, but with different return types. This is a good example of bridge methods. It is very timely that I recently learned about these.

I came up with a nice little example to illustrate.

    public interface Shape {}
     
    public interface Factory {
        Shape create();
    }
     
    public class Circle implements Shape {}
     
    public class CircleFactory implements Factory {
        public Circle create() {
            return new Circle();
        }
    }

Now, CircleFactory has two create() methods: one is a bridge method that returns a Shape, and the other is the method from the source that returns a Circle.

September 05, 2024

Released version 1.4.0, with:

• better handling of bridge methods
• unique name of method includes its return type
• ListSymbols can output CSV, YAML, JSON, and XML

September 04, 2024

Looping back on the discussion on bridge methods and code differences I brought up on 2024-07-04, imagine the following interfaces:

    package org.ietf.foo;
     
    public interface Foo {}
     
    public interface FooBuilder {
        Foo buildFrom(String name);
    }

And the initial implementation below. They chose to name their classes exactly like the interface they implement, because reasons Maybe they like to keep names simple, like Foo. The alternative is to distinguish them artificially, either as FooInterface and Foo, or else as Foo and FooImpl. Nobody in their right mind would use something as silly as IFoo and FooImpl, though. Right?!? Regardless of the names, the compiler will create a bridge method..

    package com.acme.impl;
     
    public class Foo implements org.ietf.foo.Foo {}
     
    public class FooBuilder implements org.ietf.foo.FooBuilder {
        public Foo buildFrom(String name) {
            return new Foo();
        }
    }

The Foo in the method's return type is really a com.acme.impl.Foo. The type is congruent with the interface definition, so there is no problem. The compiler creates a bridge method to "bridge" the difference, and everyone is happy.

Let's say the developers decide to remove the ambiguity and write a new version.

    package com.acme.impl;
     
    public class Foo implements org.ietf.foo.Foo {}
     
    public class FooBuilder implements org.ietf.foo.FooBuilder {
        public org.ietf.foo.Foo buildFrom(String name) {
            return new Foo();
        }
    }

It is a little bit more verbose, but removes the need for the bridge method. What should JarJarDiff say in its reports about the changes from the initial version to the new version, if anything?

The initial version has three method symbols.

• com.acme.impl.FooBuilder.FooBuilder()
• com.acme.impl.FooBuilder.buildFrom(java.lang.String): org.ietf.foo.Foo (the bridge method, calls the other symbol)
• com.acme.impl.FooBuilder.buildFrom(java.lang.String): com.acme.impl.Foo (the implementation from source code)

The call graph for some hypothetical client code that calls buildFrom() looks like this.

    Client.main(java.lang.String[]): void
        --> com.acme.impl.FooBuilder.buildFrom(java.lang.String): org.ietf.foo.Foo
            --> com.acme.impl.FooBuilder.buildFrom(java.lang.String): com.acme.impl.Foo

The new version has only two method symbols.

• com.acme.impl.FooBuilder.FooBuilder()
• com.acme.impl.FooBuilder.buildFrom(java.lang.String): org.ietf.foo.Foo (the implementation from source code)

And a more compact call graph.

    Client.main(java.lang.String[]): void
        --> com.acme.impl.FooBuilder.buildFrom(java.lang.String): org.ietf.foo.Foo

From the client's perspective, nothing has changed. It calls an interface method, and what happens after that is not the client's concern. Whether it is a bridge method that then calls an implementation, or that implementation itself, is completely hidden from the client. Therefore, maybe JarJarDiff should not report any changes?

But there is a public method that has disappeared. Does it not matter?

I tried to modify DifferencesFactory to compare methods by their unique name instead of their signature. It would report the mismatched signature as having been removed, and the matching signature as having a code change. But distinguishing methods based on their unique name makes it impossible to track changes to a method's return type. Instead, it looks like one method was removed and another method was added.

So, I will keep distinguishing methods based on their signature. In our example, the default report will show no changes, just like the client code perceives. If JarJarDiff has the -code switch, it will report a change of the return type, which is not untrue.

    Method Changes:
        old:public com.acme.impl.Foo buildFrom(java.lang.String)
        new:public org.ietf.foo.Foo buildFrom(java.lang.String)

Two months ago, I wasn't sure how to interpret and report on this code change. By looking at it from the perspective of the calling client code, it was instantly clear what I needed to do.

August 27, 2024

I fixed ListSymbols to include the return type of methods, so I can use its output in -includes-list type of switches. The method nodes in dependency graphs include the return type, so list elements need it too so they can match nodes.

I added JaCoCo coverage report to the website. I just copied it into Git. This is simpler, for now, than figuring out how to do partial deployments in GitHub Pages.

I'm not thrilled that there is 0% code coverage for symbol printers.

August 23, 2024

Added steps to the GitHub Actions workflow that generates releases so it will also build the Docker image and upload it to Docker Hub.

August 22, 2024

Created a GitHub Actions workflow to generate releases. I found a way to pass in the VERSION value to And, and through it, to Gradle. This way, Gradle can include it in the JAR file's manifest, and Ant can use it when it generates the Manual and Tools pages, as well as naming distribution files.

I found a trick where pushing a tag can trigger a workflow. The workflow has access to the tag name, so it can pass it to Ant and Gradle. I also found a GitHub Action that creates a release in GitHub and uploads the distribution files to it.

All I have left is to make the workflow create the Docker image and upload it to Docker Hub.

August 21, 2024

The text output from ListSymbols can be used with -includes-list and -excludes-list command-line switches. Methods will have their full signature matched against these lists, so I needed to change TextSymbolPrinter to show full signatures instead of unique names.

I took the opportunity to rename the switches to ListSymbols. Since it reports more than just names, now, I removed the name part of the switches.

• -class-names ⇒ -classes
• -field-names ⇒ -fields
• -method-names ⇒ -methods
• -local-names ⇒ -local-variables
• -inner-class-names ⇒ -inner-classes

and

• -non-private-field-names ⇒ -non-private-fields
• -final-method-or-class-names ⇒ -final-methods-or-classes

August 13, 2024

I played with reproducing the method name analysis from that article using a simple script in Ruby I picked Ruby for this example because it has built-in support for reading CSV files..

For example, I can get the top ten method prefixes with the following. First, I extract the method names to a CSV file.

    ListSymbols
        -method-names
        -csv
        -out symbols
        lib/DependencyFinder.jar

Then, I can use this Ruby code to look at the names only, extract their prefix, and list the ten most common ones.

    require 'csv'
     
    prefix_histo = Hash.new {|hash, key| hash[key] = []}
     
    CSV.foreach('symbols_method_names.csv', headers: true, strip: true) do |row|
        name = row['name']
         
        prefix_regex = /^([a-z]+)[A-Z].*/
        match = prefix_regex.match(name)
        prefix_histo[match[1]] << name if match
    end
     
    puts "Top Prefixes"
    puts "------------"
    prefix_histo.sort_by {|k, v| -v.size}[..10].each {|k, v| puts "#{v.size}\t#{k}"}

As of this writing, the top ten prefixes are:

    Top Prefixes
    ------------
    1524    get
    651     visit
    356     is
    329     set
    72      print
    60      to
    53      add
    51      create
    49      build
    42      fire
    36      has

August 12, 2024

Extracted the logic to break down symbols to SymbolPrinter and wrote text and CSV variations.

Also added JSON, XML, and YAML formats after further refactoring.

August 11, 2024

Worked on CSV output for ListSymbols and breaking down symbols into constituents. I'm trying it out in the CLI first. I'll have to extract it later to some common place (com.jeantessier.classreader?) that can be shared between the CLI and Ant tasks.

August 09, 2024

Refactored SymbolGatherer to track Visitable objects instead of only String. The next step will be to separate output generation between text and CSV, and I might also add YAML, JSON, and XML for good measure.

August 08, 2024

I read an article where they categorize methods by their first letter, their prefix, their suffix, or their functional interface parameter. In the article, they apply this categorization to java.util.stream.Stream, and look for any insights. It got me wondering if I could do the same in Dependency Finder or arbitrary classes. Would it help reveal any useful insights?

ListSymbols is an obvious candidate to host this functionality. It uses SymbolGatherer, which only works in terms of strings. I need to refactor it to track `Method_info` so I can more easily isolate the method name to tease out prefixes and suffixes and so forth. I could have a tabular output where I break down a method into its fully qualified class name, the method name, the method signature, and its return type.

July 25, 2024

I found a library in Java to generate test objects with random data. It is named Instancio. I read about it in this article. It looks somewhat akin to the factory_bot gem in Ruby, and the Faker library in Ruby and JavaScript.

July 17, 2024

I want to replace the cgi-bin/Journal.cgi script with a static page. The script takes too long to generate the contents from all the Journal entries. I would like to automate the construction of a Journal.html static page as part of the continuous build, so I can still edit individual files for each Journal entry, but be able to load the Journal quickly when visiting the site.

I had a GitHub workflow that would generate Journal.html and saves it as a build artifact. I needed to figure out how to deploy it to GitHub Pages alongside the rest of the website.

I found a "standard" GitHub Pages deployment workflow by playing around with Pages settings on the GitHub repo. It packages all files it finds into a single archive and deploys it to GitHub Pages. I added steps to generate the static Journal page and Atom feed (and fixed some URLs), that now I have a CI that generates the static assets that I need.

July 08, 2024

Added code coverage using JaCoCo, a descendent of EMMA. There is a Gradle plugin for it. I used a variation with a standalone utility project to get transitive dependencies between the various test subprojects and the :lib subproject. I set up :integration-tests and :fit-tests as their own subprojects, so they could encapsulate the test data and fixture classes that each one needs. This makes it more complicated for JaCoCo to track coverage of classes in :lib that are exercised by the tests in these subprojects.

It seems the other code coverage tools I had been using in the past, like Cobertura, Clover, and EMMA, have fallen behind.

I also refreshed the Samples page to bring it up to date with the latest versions of Dependency Finder. It had been over 10 years since the last update. For some reason, the layout of the dependency graph page does not work properly with the latest graphs. I can't see any difference, and it works just fine when I point my browser to the local files, or serve them from my laptop using a local webserver. But when I load them from SourceForge, everything is against the left margin.

July 07, 2024

Used GitHub Actions to create the distribution files and store them in GitHub Packages. I will figure out how to create a release from them later. The examples I have, they use a tool to generate semantic versions. But, I am not quite ready for the level of involvement in supporting semver.

I also added a workflow to compile the Journal entries into a single HTML file. I could use it on the github.io to have the Journal page there too. I cannot run CGI scripts on GitHub, so I have to find a way to render static contents instead.

July 04, 2024

Is changing a signature to remove the need for a bridge method a change in signature or the removal of a method?

For example, imagine we have an API package defining interfaces for a Foo abstraction and its factory FooBuilder.

    package org.ietf.foo;
     
    public interface Foo {}
     
    public interface FooBuilder {
        Foo buildFrom(String name);
    }

If we can implement these interfaces in a separate implementation package, we can match the names of the classes to the interfaces they implement.

    package com.acme.impl;
     
    public class Foo implements org.ietf.foo.Foo {}
     
    public class FooBuilder implements org.ietf.foo.FooBuilder {
        public Foo buildFrom(String name) {
            return new Foo();
        }
    }

Since there are no import statements, the implementation of buildFrom() has a return type of com.acme.impl.Foo. Since it is a subtype of the Foo interface, it satisfies the contract of buildFrom() from the FooBuilder interface. But the compiler will create a bridge method with the correct return type and have it call our implementation.

We can alleviate the need for a bridge method by giving our implementation the correct exact return type. It makes the implementation more verbose and potentially harder to read, though. Which is why we were using the form with the bridge method in the first place.

    package com.acme.impl;
     
    public class Foo implements org.ietf.foo.Foo {}
     
    public class FooBuilder implements org.ietf.foo.FooBuilder {
        public org.ietf.foo.Foo buildFrom(String name) {
            return new Foo();
        }
    }

Now, the code looks as follows.

If I run both versions through JarJarDiff while it is using method signatures to analyze differences, it reports:

    Method Changes:
        old:public com.acme.impl.Foo buildFrom(java.lang.String)
        new:public org.ietf.foo.Foo buildFrom(java.lang.String)

That is, it completely ignores the bridge method and compares the change in the implementations of the interface.

But when I modify JarJarDiff to use methods' unique name instead, it reports:

    Removed Methods:
        public com.acme.impl.Foo buildFrom(java.lang.String)

The bridge method is matched to the new implementation, and the old implementation has nothing to compare it against.

So, which is the correct interpretation? Is it a method change or did we remove a method?

July 03, 2024

Did a bit of refactoring. DifferencesFactory looks up fields by their name and methods by their signature. I need it to do it by their unique name instead. The original method to look up a Method_info is:

    public Method_info getMethod(String signature) {
        return methods.parallelStream()
                .filter(method -> method.getSignature().equals(signature))
                .findAny()
                .orElse(null);
    }

Instead of adding a separate variant just for uniqueName, I figured I could use a functional interface to let me pass in what to match against. I can also re-write the existing method in terms of the new functional interface, so I don't break existing callers.

    public Method_info getMethod(String signature) {
        return getMethod(signature, Method_info::getSignature);
    }
     
    public Method_info getMethod(String name, Function<Method_info, String> getter) {
        return methods.parallelStream()
                .filter(method -> getter.apply(method).equals(name))
                .findAny()
                .orElse(null);
    }

With this new getMethod() in place, the DifferencesFactory can call it with Method_info::getUniqueName to find methods by their unique name.

I could even take it one step further and pass in an arbitrary criteria using the Predicate functional interface.

    public Method_info getMethod(String signature) {
        return getMethod(signature, Method_info::getSignature);
    }
     
    public Method_info getMethod(String name, Function<Method_info, String> getter) {
        return getMethod(method -> getter.apply(method).equals(name));
    }
     
    public Method_info getMethod(Predicate<Method_info> filter) {
        return methods.parallelStream()
                .filter(filter)
                .findAny()
                .orElse(null);
    }

Now, it turns out there are just a couple of places where I use getMethod(). I converted these places to use Predicate instead.

    oldClass.getMethod(signature);

became

    oldClass.getMethod(method -> method.getSignature().equals(signature))

And with that, I only needed to keep the one version of getMethod() that takes a Predicate. I did the same to locateMethod(signature), getField(name), and locateField(name) to convert them all to using Predicate as the criteria used in their search.

With this refactoring in place, I could run differences on unique methods by changing DifferencesFactory to call:

    oldClass.getMethod(method -> method.getUniqueName().equals(name))

With one caveat. The current logic uses signatures, like foo(int), and not full signatures like some.package.Foo.foo(int). The current incarnation of uniqueName is fully qualified, and I'm afraid it will break API differences, especially when ClassClassDiff tries to compare two classes with different names.

So, I split the existing getUniqueName() into a method called getUniqueName() that returns an unqualified foo(int): void and another method called getFullUniqueName() that returns the fully qualified version with package and class names. I changed all the places where I had been using getUniqueName() to call getFullUniqueName() instead. Actually, it was the other way around: I did the renaming first, and then I split them.

July 02, 2024

Created the GitHub Action jeantessier/test-summary-action as a separate project. I can use it instead of the custom Ruby code in Dependency Finder, and remove all needs for Ruby to keep it simple.

July 01, 2024

Updated the JUnit dependency to JUnit Jupiter (JUnit 5). The tests are still written for a mix of JUnit 3 and JUnit 4, but at least the package dependency should now be up-to-date. I received a security warning from GitHub, and I'm hoping this upgrade addresses it.

I also added a step in the GitHub Actions workflow to report on tests that pass and/or fail as part of the job summary. The information I need is in XML files produced by JUnit. I used Ruby and the nokogiri gem to parse the XML and generate a Markdown table. I am not too keen to introduce yet another language, Ruby, in Dependency Finder. But, it is the quickest and simplest way to parse XML files and extract the information I need. It is only in the GitHub Actions workflow. For now.

I could extract the test summary logic into my own GitHub Action, separate from Dependency Finder. This would keep the Ruby code out. This new action could either be a Docker container or a composite action that calls Ruby from the runner's shell.

Someone wrote a test-summary GitHub Action that does a similar job. This could be a way to not have to introduce Ruby into Dependency Finder.

June 30, 2024

The next release will be 1.4.0. The change to method unique names will break some setups out there, just like I've been updating regular expressions in Dependency Finder itself.

I renamed the master branch to main.

I've been experimenting with GitHub Actions. I set up a workflow that runs ./gradlew build, which builds everything and runs all tests. I still have to figure out how to publish distributions and Docker images from GitHub Actions.

June 28, 2024

Use methods' unique name in the dependency graph. And also for fields, for good measure. As it turns out, all that was required was to update a regex in NodeFactory and another regex in TextPrinter.

OOMetrics relied on Deprecated_attribute to find deprecated classes. But it used Feature_info.isDeprecated() for fields and methods. I removed the inconsistency, and now OOMetrics no longer looks at Deprecated_attribute directly. It's all hidden behind isDeprecated() methods.

June 27, 2024

Moved the logic to build unique names for method metrics to Feature_info and FeatureRef_info, so they can be consistent across client packages. Fields just use their name, and methods append their return type to their signature. Constructors and class initializers are exempt.

I also updated the copyright notice to 2024.

June 25, 2024

Updated the OOMetrics documentation to use the new names of method metrics. And fixed a number of typos across the documentation in the process.

June 24, 2024

Continued work on issue #8.

It turns out everywhere in OOMetrics where I used a method's signature ended up using the new key. So, I faced reality and make this key the actual name of method-level metrics.

Completed the tests for all metrics printers. They now cover all outputs and a few additional variations.

June 20, 2024

Worked on issue #8.

I added an indexing key to Metrics, separate from their name, for use when tracking metrics. For project-, group-, and class-level metrics, it is the same as their name. But for methods, it is the combination of the method's signature and its return type. Some synthetic methods have the same signature as an existing method, but will have a different return type (e.g., erasures, bridge methods).

I had to change references to method names to use the key instead, so each variation gets reported correctly.

I added some tests for the various metrics printers. Keeping track of all the hidden vs. empty options was getting too complicated to do in my head. I put the expected output in files under integration-tests/src/test/resources and the tests generate an actual output and compare it to the expected output from the files. These tests can be somewhat fragile if the output changes a lot, but I don't feel this will be the case here. I could use data-driven tests to generate most variations. I use the MartinConfig.xml configuration because it has hidden measurements, and it surfaces all variations of empty metrics, hidden measurements, etc.

In the process of testing the printers, I realized the JSON printer was missing a newline character at the end of its output. I had noticed a weird character when I called OOMetrics -json, but never understood that it was because of a missing newline. Now, this mystery is solved.

June 19, 2024

Removed the synchronization around the firing of events. Synchronizing the list of listeners makes a lot of sense if listeners keep registering and de-registering while we are firing events. But in Dependency Finder, we set up the listeners are the beginning and don't change them once the main processing starts. There is no need to synchronize this list when we are firing events. I was copying the list of listeners in a synchronized block to keep the critical section short. By removing the critical section, I don't need to copy lists all the time anymore.

I wrote CodeSpliterator, as a simile to CodeIterator, so the Code_attribute can expose a Stream<Instruction> via its stream() method.

June 18, 2024

There were some flaky tests in the metrics printers. Looking into it, it turned out that when generating random percentiles, they sometimes used the value 0. Percentiles have to be 0 < p ≤ 100, so a value of 0 was causing problems.

OOMetrics and ClassMetrics were only using Synthetic_attribute to identify synthetic elements and they were ignoring elements that have the ACC_SYNTHETIC in their access_flags. I refactored them to use the helper method isSynthetic(), which adequately checks both the attribute and the access flags.

I also updated both OOMetrics and ClassMetrics to include reports of all variations of classes, fields, methods, and inner classes. They had fallen behind during the update to Java 21. I took this opportunity to refactor ClassMetrics to remove a lot of duplication in how it prints reports. When I switched to the Streams API, I mistakenly added quite a few empty lines in reports with the -list switch. I was able to restore the look of the reports back to how it was before, while preserving the new additions.

June 13, 2024

Every once in a while, an item in the Classfile structure will reference an entry in the constant pool. The constant pool index starts at 1 and the Classfile structure uses a special value of 0 to indicate an empty reference. For example, Classfile has a item called super_class that indicates the superclass of the current class being described by the Classfile. The item super_class can have the value 0 if the current class does not have a superclass, e.g., Object or interfaces.

For each of these items, the interfaces in com.jeantessier.classreader expose a number of methods to get the index value, the raw constant pool entry, or a more helpful value. For example, for super_class, we have:

• int getSuperclassIndex()
• Class_info getRawSuperclass()
• String getSuperclassName()

In the code, when we need to verify if there is a superclass or not, we would check the value of getSuperclassIndex(). This introduces a magic constant 0 that means "there is no superclass."

Magic constants tend to make the code harder to read. So, I added a new method for such items:

• boolean hasSuperclass()

The new method handles the check for 0 so the magic constant does not spread throughout the code. It makes the client code easier to read.

Since I had a pervasive pattern for checking for 0 everytime I looked up these items from the constant pool, I found a few places where I was checking for 0 when the specification clearly states that it must not be zero. In these cases, I removed the check and I expect the Classfile structure to be compliant with the specification.

June 07, 2024

Looked at adding percentiles to OOMetricsGUI. I haven't looked at the Swing code that makes up the GUI applications since I first wrote it over 20 years ago. It uses sets of constants to build column names, so the set of subvalues is essentially hard-coded. I will have to do some major refactoring of the models that feed the GUI to make column names more dynamic and make them based on what's in the metrics configuration.

June 06, 2024

Added percentiles to the CLI and Ant versions of OOMetrics. In the process, I wrote unit tests for all the subclasses of com.jeantessier.metrics.Printer.

June 05, 2024

Found another problem, and this one is a big one.

I've always assumed field names and method signatures were unique within a given class. After all, the Java language specification states that:

It is a compile-time error for the body of a class declaration to declare two fields with the same name.

It is a compile-time error to declare two methods with override-equivalent signatures in a class.

While this is true for Java code that you feed to the compiler, it is not strictly true inside the classfile structure. Bridge methods are allowed to share a signature with the method they are a bridge for. Here is an example.

Suppose an interface package a.

    package a;
     
    public interface Foo {}
     
    public interface FooCreator {
        Foo buildFrom(String name);
    }

And its implementation package a.impl:

    package a.impl;
     
    public class Foo implements a.Foo {}
     
    public class FooBuilder implements a.FooBuilder {
        public Foo buildFrom(String name) {
            return new Foo();
        }
    }

The interface method a.FooBuilder.buildFrom(String) returns an instance of interface a.Foo. The implementation method a.impl.FooBuilder.buildFrom(String) is returning an instance of class a.impl.Foo. This is legal, since a.impl.Foo is a a.Foo. While the methods' signatures match, their descriptors do not, because the return types are different.

To resolve this, the compiler generates a bridge method in a.impl.FooBuilder that reconciles the descriptors. It has the same signature as the implementation method, but the descriptor matches the one from the interface. It is as if the implementation This code will not compile because there are two methods with the signature buildFor(String). had been:

    package a.impl;
     
    public class Foo implements a.Foo {}
     
    public class FooBuilder implements a.FooBuilder {
        public Foo buildFrom(String name) {
            return new Foo();
        }
         
        public a.Foo buildFrom(String name) {
            return buildFrom(name); // meaning, the other method defined above
        }
    }

In this case, you could fix it by explicitly returning the interface type in the declaration of the implementation method.

    package a.impl;
     
    public class FooBuilder implements a.FooBuilder {
        public a.Foo buildFrom(String name) {
            return new Foo();
        }
    }

But this may be impractical if package names are long, or if the result is one method with a very long declaration when all the other methods in the class are much more concise.

Sometimes, it is inevitable. Consider the following interface and its implementation. ⊕ Funnily, when I tried to run ClassReader on this Wrapper class, I found another bug. A name deep inside an attribute structure turns out to be optional and, my code assumed it was always present.

    public class Wrapper<T> implements Comparable<Wrapper<T>> {
        private final T wrappedObject;
         
        public Wrapper(T object) {
            wrappedObject = object;
        }
         
        public int compareTo(Wrapper<T> other) {
            // Implementation compares
            // "this" to "other" and returns
            // -1, 0, or 1 as appropriate.
        }
    }

After type erasure, Comparable demands a method with the signature compareTo(java.lang.Object). But the compiler demands that we write a method with the signature compareTo(Wrapper<T>). We will not be able to avoid this bridge method.

Classfile stores a collection of methods, so the bridge methods and the methods they are bridging are present and accounted for.

Now, for the bug. There are times in Dependency Finder when I use maps to store information about methods. These maps are keyed on the methods' signature. For example:

• FeatureNode in a dependency graph
• method-related differences
• method-related metrics
• Classfile.getMethod(String) will return the first method with a matching signature, but it searches the methods in an indeterminate order.

Depending on the case, two methods with the same signatures may end up merged together (this is the case in dependency graphs), or one method may completely mask the other (this is the case in difference reports).

I will need to identify all the places where I use a maps keyed on method signatures and figure out an alternative implementation, including changing assumptions downstream from them that methods signatures are unique within a class.

June 04, 2024

Wrote a bunch of unit tests for CSVPrinter to specify its exact behavior.

Added skinparam backgroundColor transparent to PlantUML diagrams so they look better on the Journal page, which gets an off-white background from Tufte CSS.

June 03, 2024

Fourteen years later after the fact, I find out that ⊕ "Users are encouraged to use similar features in newer Java versions, where available." Jakarta ORO has been retired. I use it all over the place in DependencyFinder, so I will not be ripping it out any time soon. But I might start using java.util.regex in select places, especially when the Streams API makes it even more powerful.

I came up with a way to request various percentiles as part of the init text for StatisticalMeasurement. It was nice and contained in the measurements themselves, until I got to CSVPrinter. It prints a header that would need column names for each requested percentile, but it hasn't seen the measurements at this point. It operates solely in terms of descriptors, and the descriptors are measurement type agnostic. The printer already has a special case for StatisticalMeasurement, so I could call static methods of this class to help me figure out the percentile columns.

I found a troubling bug in com.jeantessier.metrics.TextPrinter. It renders all metrics according to the project-level measurement descriptors. It renders group-level, class-level, and method-level metrics as if they had been populated with the project-level descriptors. This means that it does not render any measurement whose name does not appear among the top, project-level configuration. ⊕ I first noticed it when my class-level report in CSV had a "Character Count" CNCC measurement that did not appear in text reports, even though all the other switches were the same. Most of the other printers, JSONPrinter, XMLPrinter, and YAMLPrinter, explicitly navigate down to each level and pass on the appropriate collection of descriptors. For CSVPrinter, it is done externally by the calling tools. But TextPrinter uses a single collection of descriptors for all metrics and this needs to be fixed.

June 02, 2024

Refactored StatisticalMeasurement to add percentiles. The statistics that it exposes are minimum, median, and maximum. But this does not tell us how far outliers are from the other results. For that, it would be helpful to have values line the 90^th percentile or the 99^th percentile. And maybe even the 1^st and 10^th percentile for the lower end of the range. Maybe, once I have a percentile function, I can replace getMedian() with the 50^th percentile!

I keep thinking of the median as a value from the set. But looking at the code, I noticed it was averaging the two central values when the dataset had an even number of elements. That is how they recommend to do it on Wikipedia, and it is likely where I picked it up. So I left the calculation of the median as it was, but with a little bit of clean up to use the Streams API and inline some trivial methods.

I did add getPercentile(n) and I will look for the best way to expose percentiles in OOMetrics reports. I could use canned percentiles, like 1, 10, 50, 90, and 99, but I'd like to let users decide which percentiles make sense for them. Maybe there is a way to specify them in the configuration, as part of the initialization text for StatisticalMeasurement.

May 31, 2024

Switched the Introduction page and the Journal page to Tufte CSS.

I have been using Tufte CSS for my personal blog for a few years now, and I like the look. It uses white space intelligently to help the eye read the text with minimal straining. I write my blog using Markdown and a JavaScript library converts it to HTML in the browser. This conversion step sometimes interfere with some of the features of Tufte CSS, so I cannot use it to its full potential. But the Journal page is rendered HTML on the server, and I have complete control on it. I can use features like sidenotes and figures to their fullest.

May 23, 2024

Drew up a map of all the tools that create classfile loaders and will need an extra parameter to target JDK versions when reading multi-release JAR files.

The green ones use TransientClassfileLoader. The red ones use AggregatingClassfileLoader. Two-colored ones, like OOMetrics, use both. The yellow boxes use neither.

May 22, 2024

Worked on JarClassfileLoader to handle multi-release JAR files and only load a single version of a class, potentially based on a target JDK. The changes removed any commonality there was between ZipClassfileLoader and JarClassfileLoader, so I made them sibling classes instead of one inheriting from the other. When loading a JAR file from an InputStream, I copy it to a temporary file so I can better use the Streams API to sort and filter entries.

Next, I will have to update all the tools and their documentation to add a switch to target specific JDK versions.

Created a multi-release JAR file from the command line. I'm using the same .class file multiple times. I just need to have the same class in multiple META-INF/versions/N locations.

    jar cf multi.jar -C integration-tests/build/classes/java/main test.class
    jar uf multi.jar --release 17 -C integration-tests/build/classes/java/main test.class
    jar uf multi.jar --release 9 -C integration-tests/build/classes/java/main test.class
    jar uf multi.jar --release 22 -C integration-tests/build/classes/java/main test.class
    jar uf multi.jar --release 21 -C integration-tests/build/classes/java/main test.class

And here is the resulting JAR file.

    jar tf multi.jar
    META-INF/
    META-INF/MANIFEST.MF
    test.class
    META-INF/versions/17/test.class
    META-INF/versions/9/test.class
    META-INF/versions/22/test.class
    META-INF/versions/21/test.class

May 21, 2024

Wrote a test to create a multi-release JAR file and read it using JarClassfileLoader. I don't quite know how I will approach processing the JAR file to zero-in on a single .class file, but I now have a test in place to let me know how I'm doing.

May 13, 2024

I am looking for a way to pick a single version of a class from a multi-release JAR file. Such JAR files can contain any number of versions of a given class in various META-INF/versions/N directories, plus a default version in the top-level directory. If I ignore the META-INF/versions/N directories, then I might be ignoring more advanced versions. If I only look at the first file I encounter, then I might be picking an arbitrary version.

JAR files tend to list files in alphabetical order, so it might list versions of an example class com.foo.Bar as:

1. META-INF/versions/11/com/foo/Bar.class
2. META-INF/versions/17/com/foo/Bar.class
3. META-INF/versions/21/com/foo/Bar.class
4. META-INF/versions/22/com/foo/Bar.class
5. META-INF/versions/9/com/foo/Bar.class
6. com/foo/Bar.class

This JAR file supports up to JDK 22. But maybe the user is limited to JDK 17 for business reasons. They would want to analyze the JDK 17 version and ignore all others.

JAR files tend to list files in alphabetical order, but they don't really have to.

I could add some filtering function in JarClassfileLoader. But its usage is buried pretty deep in the framework. It would be difficult to pass a filtering criteria for which JDK version to target through constructors. I could add a filter between the loader and the visitor.

    loader.addLoadListener(new MultiReleaseFilter(targetJDK, new LoadListenerVisitorAdapter(gatherer)));

The filter could zero in to the most adequate version that meets the targetJDK criteria. It would need receive versions in decreasing order of JDK version, so I might have to tweak JarClassfileLoader anyway. It would traverse JAR file entries in order of version instead of alphabetically.

1. META-INF/versions/22/com/foo/Bar.class
2. META-INF/versions/21/com/foo/Bar.class
3. META-INF/versions/17/com/foo/Bar.class
4. META-INF/versions/11/com/foo/Bar.class
5. META-INF/versions/9/com/foo/Bar.class
6. com/foo/Bar.class

If the target is JDK 19, it would forward events about the third entry and skip all others.

May 10, 2024

Added SingleValueMeasurement to fix the problem with tracking major and minor version numbers in OOMetrics.

It doesn't fix the problem with processing multiple copies of the same class in multi-release JAR files, though.

For example, I extracted log4j-api.jar to /tmp so I could analyze the various versions of the Base64Util class separately.

For the JDK 8 version of the class in the top-level declaration:

    OOMetrics \
        -txt \
        -classes \
        -methods \
        /tmp/log4j-api/org/apache/logging/log4j/util/Base64Util.class

    Class metrics
    -------------
    org.apache.logging.log4j.util.Base64Util
        Major Version (MaV): 52
        Methods (M): 3
        Attributes (A): 2
        Inner Classes (IC): 0
        Single Lines of Code (SLOC): 27
        Subclasses (SUB): 0
        Depth of Inheritance (DOI): 0
     
     
    Method metrics
    --------------
    org.apache.logging.log4j.util.Base64Util.Base64Util()
        Single Lines of Code (SLOC): 1
     
    org.apache.logging.log4j.util.Base64Util.encode(java.lang.String)
        Single Lines of Code (SLOC): 8
     
    org.apache.logging.log4j.util.Base64Util.static {}
        Single Lines of Code (SLOC): 15

For the JDK 9 version of the class in the META-INF/versions/9 directory:

    OOMetrics \
        -txt \
        -classes
        -methods \
        /tmp/log4j-api/META-INF/versions/9/org/apache/logging/log4j/util/Base64Util.class

    Class metrics
    -------------
    org.apache.logging.log4j.util.Base64Util
        Major Version (MaV): 53
        Methods (M): 3
        Attributes (A): 1
        Inner Classes (IC): 0
        Single Lines of Code (SLOC): 5
        Subclasses (SUB): 0
        Depth of Inheritance (DOI): 0
     
     
    Method metrics
    --------------
    org.apache.logging.log4j.util.Base64Util.Base64Util()
        Single Lines of Code (SLOC): 1
     
    org.apache.logging.log4j.util.Base64Util.encode(java.lang.String)
        Single Lines of Code (SLOC): 1
     
    org.apache.logging.log4j.util.Base64Util.static {}
        Single Lines of Code (SLOC): 1

When I run OOMetrics on both files at the same time, notice how the SLOC and A measurements adds up across both versions of the class.

    OOMetrics \
        -txt \
        -classes \
        -methods \
        /tmp/log4j-api/META-INF/versions/9/org/apache/logging/log4j/util/Base64Util.class \
        /tmp/log4j-api/org/apache/logging/log4j/util/Base64Util.class

    Class metrics
    -------------
    org.apache.logging.log4j.util.Base64Util
        Major Version (MaV): 52
        Methods (M): 3
        Attributes (A): 3                   // 2 + 1
        Inner Classes (IC): 0
        Single Lines of Code (SLOC): 32     // 27 + 5
        Subclasses (SUB): 0
        Depth of Inheritance (DOI): 0
     
     
    Method metrics
    --------------
    org.apache.logging.log4j.util.Base64Util.Base64Util()
        Single Lines of Code (SLOC): 2     // 1 + 1
     
    org.apache.logging.log4j.util.Base64Util.encode(java.lang.String)
        Single Lines of Code (SLOC): 9     // 8 + 1
     
    org.apache.logging.log4j.util.Base64Util.static {}
        Single Lines of Code (SLOC): 16    // 15 + 1

The Major Version measurement now uses the new SingleValueMeasurement, so it takes the value of the last version that was visited. When analyzing a JAR file, OOMetrics visits the versions in META-INF/versions/N before the top-level version.

May 09, 2024

Found a problem with how OOMetrics collects major version information. It assumes it visits each class only once, which may not always be the case. I found at least two situations where it visits a class more than once. Since MetricsGatherer calls addToMeasurement(), it ends up accumulating the value of major version, leading to confusing results.

The first case is module-info. Module information is stored in a top-level .class file named module-info.class. It contains a classfile structure with the ACC_MODULE flag turned on, the name `module-info`, is part of the default package, and it doesn't have a superclass. If you run OOMetrics on multiple JAR files together, it is possible that more than one of these files will have a module-info.class entry and OOMetrics will treat them as multiple presentations of the single module-info "class". When it processes the "Major Version" measurement, it adds the value of the major version to the measurement instance. If two module-info.class files have the major version 61, the resulting metrics measurement has a value of 122.

Unfortunately, OOMetrics only has method, class, group, and project levels of metrics. Either it can treat module-info as a class, when it is not one, or it can ignore it altogether. It is probably unlikely that it would have a different major version from the actual classes in the JAR file, so we would not lose any information by ignoring it.

I came across the second case while running OOMetrics against the LOG4J archives. It has to do with "Multi-release JAR files", as described in the JAR File Specification. The META-INF/versions/N directory inside each JAR file can have additional .class files that replace the top-level declaration of their respective class. For example, log4j-api.jar has both:

• META-INF/versions/9/org/apache/logging/log4j/util/Base64Util.class with major version 53
• org/apache/logging/log4j/util/Base64Util.class with major version 52

Again, OOMetrics traverses both classfile structures but uses the same Metrics instance for class Base64Util to aggregate measurements across both structures. The result is a "Major Version" measurement of 105.

Instead of using a CounterMeasurement with an add() method that accumulates values, I could create a new type of measurement that only records the first or last value passed to add(). The only visitors are printers, so the impact would be contained.

February 22, 2024

I discovered Apache ECharts as a JavaScript library for building charts in the browser.

Here is an example, with charts about the Robert C. Martin metrics for Dependency Finder.

January 29, 2024

Dependency Finder is now 23 years old.

January 26, 2024

Refactored Log4J calls to use lazy string interpolation.

Lines like:

    debug("msg " + val);

became:

    debug("msg {}", val):

So Log4J only does the string interpolation if the log is enabled.

Some lines where a value was expensive to compute became:

    debug("msg {}", () -> val);

So that even the computation is deferred until the log actually needs it.

January 24, 2024

Replaced Log4J with Log4J2. This meant changing log4j.properties to log4j2.properties and adapting it to the new format. I could have used the XML format, which all Log4J2 docs use, or the YAML format, but the properties file seemed most straightforward way to go for now.

I also figured out how to copy JAR files from Gradle dependencies, instead of fetching them by hand and hard-checking them in Git. As a result, I had to rework the launching script to dynamically build a CLASSPATH from the list of JAR files in lib/.

I don't know how to do that under Windows, so all the Windows launching scripts are broken at the moment.

I removed the scripts to run JUnit since I run JUnit through Gradle, now.

I removed the scripts to run Chainsaw since it no longer ships with Log4J, but as a separate download.

January 23, 2024

Removed some unnecessary dependencies from Gradle subprojects. Also removed some unnecessary uses of Log4J in tests.

January 19, 2024

I'm looking into what it would take to include Java modules as an additional layer of the dependency graph, above packages. The Module attribute has requires structures to get the modules that are dependencies of the current module, and uses structures for services, possibly in other modules, that it references. I could use the ModulePackages attribute lists the packages in a module, but I would need some mechanism to unclaimed packages to the default unnamed module.

The Node hierarchy is not an articulation point in the design of Dependency Finder. Adding a new kind of dependency would have a sizeable impact (i.e., a large blast radius).

As I was looking at Fit tests, my IDE kept complaining that <table border="1"> is no longer valid HTML. I tried replacing it with CSS, only to realize that the FitNesse CSS was overwriting it anyway. I took the opportunity to remove all these obsolete border attributes in Fit tests.

December 15, 2023

I moved the Ant logic that was in integration-tests/jarjardiff/build.xml to the :integration-tests:jarjardiff:old Gradle subproject. I managed to create the archives in the :integration-tests:jarjardiff:old subproject, but I'm not quite sure on the best way to move them back up to either the :integration-tests:jarjardiff subproject or the :integration-tests subproject. For now, the tests that use these archives use explicit paths into the integration-tests/jarjardiff/old/build/archives folder.

Now, I can run:

    ./gradlew check

from a clean folder, and it takes care of everything.

December 14, 2023

I merged the migration to Gradle back into the master branch. I verified that everything works: exec environment, distribution files, WAR file, and the Docker image. I still need Ant at the top level to coordinate some last cross-project dependencies, until I can figure out how to do it from within Gradle.

Adjusted the Docker instructions to use Gradle instead.

Refreshed the Tutorial on the website. The old HTML was geared towards Internet Explorer and no longer works in modern browsers. I don't have access to PowerPoint anymore, so I loaded it in Apple's Keynote and saved as HTML again. I also added a PDF version.

December 13, 2023

Fixed Ant targets for Tomcat 7+. I have an experimental branch from 2011-05-19 that was trying to do that too. I had noted back then that HttpUnit tests were not able to launch the webapp.

Fixed webapp descriptor location in the :webapp subproject. Gradle expects all webapp-related files to be under src/main/webapp.

Fixed location of javadocs and GUI resources in :lib subproject. Gradle expects all non-Java files to be under src/main/resources.

Cleaned up distribution files to remove duplicate JAR files.

December 10, 2023

Moved the remaining build.test.xml logic to build.xml. Moved the last bits of tests/JarJarDiff to the :integration-tests:jarjardiff subproject. Cleaned up tests and their imports.

Removed source and *.bz2 distribution archives. The sources are automatically available via GitHub; it even puts them as release files without having to ask for it.

Skipped all :webapp tests. I had to convert them to JUnit 4 so I could use @Ignore to skip them. Now, I can just type ./gradle check and it runs all tests across all subprojects (and skips the HttpUnit tests).

December 08, 2023

Migrated the Fit tests to a Gradle subproject called :fit-tests. I was able to "upgrade" fitlibrary from 20060116 that I had been using until now to 20070619. The next version up introduces some changes that break my tests. I think something changed in DoFixture, but I cannot find release notes that would explain how to update my tests. It's been 16 years, so I'll have to do some digging.

December 07, 2023

I successfully separated the pure unit tests from "integration tests" that need test data: compiled Java code. The main :lib subproject can run all the unit tests that don't require external data. The :integration-tests subproject has the sample code that the other tests are running against. I don't know how to effectively share test data between projects, yet. So the tests dig directly into the subprojects to find what they need.

I moved the webapp code to its own subproject. It seems to compile fine, but the HttpUnit tests are broken. They were broken to begin with, so they will need more work later. But the status quo is preserved.

December 06, 2023

Created a branch to try and move at least of the build to Gradle. I can still have Ant at the top level and use it to call some Gradle builds to generate some of the artifacts. Until I can find a way to migrate all Ant tasks over time.

November 30, 2023

Experimented with replacing Ant with Gradle. New versions of Gradle use sub-projects to organize the build. My Ant-based build has a lot of custom pieces, like compiling sample code for the tests to parse and operate against. I can compile them in sub-projects, but it's not clear how to share build products between sub-projects. The documentation shows how to share JAR files on CLASSPATH, but what I want is read them from some of the tests. I could have all the unit tests in one sub-project and have "integration" tests that also depend on sample code in another sub-project. I could also build the WAR file in a sub-project separate from the library JAR file. Lots of possibilities, but wiring it all together will be a challenge.

November 22, 2023

Cleaned up the com.jeantessier.commandline package.

November 20, 2023

Finished removing the explicit public qualifiers from interfaces in com.jeantessier.classreader. I also moved some behavior from abstract base classes to default methods in their related interfaces.

More for() loops converted to Stream API. More dead code removed, more hand-written mocks replaced with jMock.

November 19, 2023

I'm finally getting rid of explicit public qualifiers in interfaces. The Java 21 language specification says it is discouraged.

I moved some default implementations to default methods in interfaces instead of abstract base classes in the dependency, diff, and metrics packages. I also converted more for() loops to the Stream API, and even wrote a custom Spliterator for use in DescriptorHelper.

I realized that after I started processing StackMapTable_attribute for dependencies, I inadvertently introduced dependencies on arrays of primitive types. A stack frame might reference a local byte[] with the descriptor "[B" and reference it via a Class_info in the constant pool. The CodeDependencyCollector would mistakenly parse it as a type B[] and treat that as the name of a class. I fixed it so it ignores them from now on, just like it ignores dependencies on primitives in general.

November 14, 2023

Cut release 1.3.1 which includes dependencies on lambdas, method references, and other dynamic callsites.

November 13, 2023

Renamed InvokeDynamicFinder to ListDynamicInstructions. Wrote documentation for this new tool.

I found some documentation on callsites in the javadocs for the java.lang.runtime.SwitchBootstraps class and the java.lang.invoke package. It is very technical, and I'm not sure how exhaustive it really is.

I'm tempted to move up to Java 21 so I can use type pattern matching in switch expressions. They make the code much more compact and streamlined.

It turns this somewhat awkward code:

    Object owner = attribute.getOwner();
    if (owner instanceof Classfile classfile) {
        syntheticClasses.add(classfile);
    } else if (owner instanceof Field_info field) {
        syntheticFields.add(field);
    } else if (owner instanceof Method_info method) {
        syntheticMethods.add(method);
    } else {
        Logger.getLogger(getClass()).warn("Synthetic attribute on unknown Visitable: " + owner.getClass().getName());
    }

Into this:

    switch (attribute.getOwner()) {
        case Classfile classfile -> syntheticClasses.add(classfile);
        case Field_info field -> syntheticFields.add(field);
        case Method_info method -> syntheticMethods.add(method);
        default -> Logger.getLogger(getClass()).warn("Synthetic attribute on unknown Visitable: " + attribute.getOwner().getClass().getName());
    }

But if I do this, I will need to compile the library to Java 21 and users will need to be on Java 21 to use Dependency Finder.

November 12, 2023

Wrote InvokeDynamicFinder, a tool to show the object graph rooted at invokedynamic instructions. It helps me see better the variations in callsites, so I can better determine if CodeDependencyCollector is doing a good enough job or not.

November 11, 2023

I realized ClassMetrics uses lists to accumulate elements, so it needs some major deduping. I switched to TreeSet to get rid of duplicates and sort the elements alphabetically to boot. I had to make Classfile and Field_info and Method_info implement Comparable so I could store them in TreeSet.

I took to opportunity to refactor the compareTo() and equals() methods to replace if cascades with guard clauses. They are more fashionable.

November 10, 2023

All these dependencies to java.lang.invoke were bugging me, so I refactored the default traversal in VisitorBase to do something special in the case of invokedynamic instructions. I added Instruction.getDynamicConstantPoolEntries() to look for MethodHandle_info in arguments to bootstrap methods and collect what these handles are referencing. This way, I can skip java.lang.invoke artifacts in dynamic calls and focus on what the code is trying to express.

Going back to the sample code from issue #6:

    import java.util.function.*;
     
    public class Test {
        public void someMethod(){
            Consumer<String> function = (x -> System.out.println(x));
        }
    }

The dependency graph now looks like this:

        Test
            --> java.lang.Object *
            Test()
                --> java.lang.Object.Object() *
            lambda$someMethod$0(java.lang.String)
                --> java.io.PrintStream *
                --> java.io.PrintStream.println(java.lang.String) *
                --> java.lang.String *
                --> java.lang.System.out *
                <-- Test.someMethod()
            someMethod()
                --> java.lang.String *
                --> Test.lambda$someMethod$0(java.lang.String)

And even ClassReader now shows the intended reference:

        public void someMethod();
            CODE
            0:	invokedynamic void test.Test.lambda$someMethod$0(java.lang.String)
            5:	astore_1
            6:	return

November 09, 2023

Now that I have code to parse BootstrapMethods_attribute, I can do a better job at processing invokedynamic instructions for dependencies. Someone opened issue #6 about tracking dependencies on lambdas. Lambdas are called using the invokedynamic instruction, but go through a callsite to figure out what actual method to call. The callsite resolves the target using data from the BootstrapMethods_attribute.

Issue #6 includes this sample code:

    import java.util.function.*;
     
    public class Test {
        public void someMethod(){
            Consumer<String> function = (x -> System.out.println(x));
        }
    }

Running it through javap -c shows the following:

    public class Test {
      public Test();
        Code:
           0: aload_0
           1: invokespecial #1                  // Method java/lang/Object."<init>":()V
           4: return

      public void someMethod();
        Code:
           0: invokedynamic #7,  0              // InvokeDynamic #0:accept:()Ljava/util/function/Consumer;
           5: astore_1
           6: return
    }

Here is a diagram of the information structure for where the method calls the lambda (click for larger version).

Attributes are shown in green and constant pool entries are shown in pink, with their index in the constant pool.

We see that javap stopped at the InvokeDynamic_info that the instruction points to, at index 7 in the constant pool. It only showed the information from the NameAndType_info at index 8 in the constant pool. With Dependency Finder, I can follow the references all the way to the MethodRef_info for the lambda (index 35 in the constant pool), down in the BootstrapMethods_attribute.

The default Classfile traversal in VisitorBase will navigate from the Instruction through the BootstrapMethod and visit both MethodRef_info with IDs 39 and 35. #39 is the internal JVM machinery to create the callsite and execute through it. #35 is the lambda. Because VisitorBase goes through both, we end up with an extra dependency on java.lang.invoke.LambdaMetafactory.metafactory() and a few other items in the java.lang.invoke package.

        Test
            --> java.lang.Object *
            --> java.lang.String *
            --> java.lang.invoke.CallSite *
            --> java.lang.invoke.LambdaMetafactory.metafactory(java.lang.invoke.MethodHandles$Lookup, java.lang.String, java.lang.invoke.MethodType, java.lang.invoke.MethodType, java.lang.invoke.MethodHandle, java.lang.invoke.MethodType) *
            --> java.lang.invoke.MethodHandle *
            --> java.lang.invoke.MethodHandles$Lookup *
            --> java.lang.invoke.MethodType *
            Test()
                --> java.lang.Object.Object() *
            lambda$someMethod$0(java.lang.String)
                --> java.io.PrintStream *
                --> java.io.PrintStream.println(java.lang.String) *
                --> java.lang.String *
                --> java.lang.System.out *
                <-- Test.someMethod()
            someMethod()
                --> java.lang.String *
                --> java.lang.invoke.CallSite *
                --> java.lang.invoke.LambdaMetafactory.metafactory(java.lang.invoke.MethodHandles$Lookup, java.lang.String, java.lang.invoke.MethodType, java.lang.invoke.MethodType, java.lang.invoke.MethodHandle, java.lang.invoke.MethodType) *
                --> java.lang.invoke.MethodHandle *
                --> java.lang.invoke.MethodHandles$Lookup *
                --> java.lang.invoke.MethodType *
                --> Test.lambda$someMethod$0(java.lang.String)

I find it annoying that it pollutes the dependency graph with these artifacts, but we can skip them with a judicious -filter-excludes /^java.lang.invoke/, if you need to. Though, this will also exclude some legitimate dependencies to that package if your code happens to call into it.

I also refactored TestRegularExpressionParser to be data-driven tests.

November 08, 2023

Updated the official DTDs on the website to 1.3.0.

Refactored some code, mostly in com.jeantessier.dependency, to use more modern Java. A lot of Stream API and using var where possible.

November 06, 2023

Now that I'm done with the work to bring the .class file parser up to the latest specification, I'm ready to do a release. There were API changes that broke backwards compatibility, but there were no major new features. If I were to adhere strictly to semantic versioning, I should go to 2.0. But, I'm hesitant to do this since there are no major additions to Dependency Finder. I'm also uncomfortable just going to 1.2.1, because it's more than just a bugfix and there are some large-ish changes in the Classfile structure, after all. So, I've decided to go with 1.3.0.

I'm doing my development using Java 21, but I'm reluctant to release bytecode with major version 65. It all compiles and passes the tests with JDK 17. I will not force users to upgrade to the latest Java just yet, but I will not stay in the Java Dark Ages either. I'm using some recent language features, so I cannot really go back much further than Java 17.

November 02, 2023

Refactored some code to make better use of switch expressions. A long time ago, I was a big fan of having a single exit point in methods. I would often set up a result variable at the top of a method and have a single return statement at the bottom. It helped with debugging because I could put a print statement just before returning and see what was what. Since then, as I've been working with other languages, I've become more prone to short methods that simply return an expression. So, I'm making a few changes here and there to make the code "look better".

November 01, 2023

Completed support for Java 17 by adding PermittedSubclasses_attribute, and its helper classes processing in VisitorBase and XMLPrinter.

I also tackled rendering tableswitch and lookupswitch instructions. I wanted to explore the "new" switch expressions in Java and I needed a way to visualize the bytecode. I wanted to see if the various forms of switch statements resulted in different bytecode. It appears that they do not, so they are essentially syntactic sugar.

I added some helper methods to Instruction to help with combining bytes to build values that are then used in the execution of the instruction.

October 31, 2023

Completed support for Java 16 by adding Record_attribute, and its helper classes processing in VisitorBase and XMLPrinter.

October 30, 2023

Completed support for Java 11 by adding NestHost_attribute, NestMembers_attribute, and their helper classes processing in VisitorBase and XMLPrinter.

Also restyled the Release Plan page to improve readability. And also to make it easier for me to edit.

October 28, 2023

Completed support for Java 9 by adding ModulePackages_attribute, ModuleMainClass_attribute, and their helper classes processing in VisitorBase and XMLPrinter.

October 27, 2023

Added Module_attribute and its helper classes processing in VisitorBase and XMLPrinter.

I made one change that is not backwards compatible: in =classfile elements, the module element is now called is-module. This lets me repurpose module to represent module references that are part of Module_attribute.

October 26, 2023

Started support for Java 9. Implemented Module_attribute and its helper classes. It was straightforward, but tedious work.

October 25, 2023

Completed support for Java 8 by implementing MethodParameters_attribute and its helper classes. I first drafted the changes that were required to support Java 8 back in 2016. Now, I can say that part is done.

Java 9 defined modules. The Module attribute is a big attribute, but it is very straightforward. Nests from Java 11 are trivial. Records were a big addition in Java 16, but the Record attribute is a name with a descriptor and an attribute lists. Easy stuff. The latest change to the .class file format was in Java 17 with the addition of PermittedSubclasses attribute. It aggregates together classes from the constant pool and will also be easy to implement.

I noticed that the JVM specification uses the plural "access flags" but my code is using the singular "access flag". I changed it everywhere to use the plural. Strictly speaking, it breaks backwards compatibility, but I feel it's minor enough to slip it by.

October 24, 2023

Updated the classfile DTD to include elements for RuntimeVisibleTypeAnnotations_attribute and RuntimeInvisibleTypeAnnotations_attribute, along with their substructures.

October 23, 2023

Started support for Java 8. Implemented RuntimeVisibleTypeAnnotations_attribute and RuntimeInvisibleTypeAnnotations_attribute, along with their helper classes. It turned out to be trickier than I expected. The TypeAnnotation is especially gnarly, with lists of unions of bespoke micro-structures. I have looked at many JAR files on my laptop in various Java-based tools and libraries, but I still haven't found a class with either one of these attributes.

I'm updating the DTD for classfile. It was missing many attributes definitions.

October 22, 2023

Implemented the StackMapFrame parts, including a StackMapFrameFactory. I did all the wiring for Visitable and Visitor support. I implemented rendering as XML using XMLPrinter.

I adjusted the Release Plan to have separate items for each Java version that introduced changes to the .class file format. I filled in dates for the work I've just been doing. The remainder work for Java 8 through 21 should be straightforward. (The last change was for Java 17.)

October 21, 2023

I used ClassMetrics to see which attributes I'm currently skipping in a given codebase. I looked at Dependency Finder itself, a JAR file that was in my JDK installation, and the JAR files for Groovy. StackMap is by far the most common attribute being skipped at the moment.

I took another look at what I would need to support it. There are many ancillary classes that will require interfaces in com.jeantessier.classreader and implementations in com.jeantessier.classreader.impl. I looked at how I had done it for ElementValue and I could see a way through if I skimped on the testing for the more obvious wiring bits.

I implemented the VerificationTypeInfo parts. In doing this, I came upon a way to test even the wiring with some judicious data-driven tests. I haven't made them Visitable, yet. The total support for StackMap will add many, many methods to Visitor and I will do them all at once.

Working on VerificationTypeInfo gave me a path through implementing StackMapFrame next. The tagging for frames is different from what I've encountered before. Previous tags were each discreet values. Specific characters for ElementValue and specific integers for VerificationTypeInfo. But StackMapFrame uses ranges. This is going to be fun!

October 19, 2023

I'm going to skip the remainder of Java 5 support for now. Generic signatures don't add enough information to what's already there to make it worth the complexity of parsing their signatures. And the StackMap is just too crazy. I will look for some lower-hanging fruits, first.

I implemented the BootstrapMethods_attribute structure and attendant helpers. It took all day. I had a couple false starts and fell in a couple of traps, but it worked out in the end.

October 18, 2023

Looked at what it would take to bring Dependency Finder support up to Java 21.

I have had an open feature to handle Java 5.0 additions to the .class file format since 2007. The code to parse these additions is there, but I'm missing the piece to interpret generic signatures. The specification for these signatures is very complex and is mostly text processing.

Back on 2016-08-21, I had drafted the classes I would need to bring support up to Java 8. There are a lot of new structures and those for the StackMap element are particularly gnarly.

The additions for Java 9 through 21 seem fairly straightforward.

I took the opportunity to refactor many classes in com.jeantessier.classreader.impl to deal make private fields final and use the Stream API to traverse the Classfile structure. I managed to find a few tricks to mock streams in visitor tests. For instance, I can mock stream() to return a canned stream with Stream.of(). I can mock forEach() by creating a side effect with CustomAction.

I removed some dead code, mostly "testing only" constructors that were not being used. I noticed that I've been meaning to remove ClassDependencyCollector and FeatureDependencyCollector since 2003-08-29!

The code coverage on the website dates back to 2010-11-29. It is woefully out of date.

October 16, 2023

Refactored the ClassCohesion CLI tool and wrote documentation for it on the Tools page. At the moment, it is only accessible via the CLI. There is no Ant task for it. Refactoring the various print...() helper functions was a nice exercise in combining streams. Merging two streams was easy enough to do with Stream.concat(). But merging a stream of XML text with opening and closing tags was more tricky.

    return Stream.of(
        Stream.of("<tag>"),
        getStreamForContents(contents),
        Stream.of("</tag>")
    )
    .flatMap(Function.identity())
    .collect(joining(System.getProperty("line.separator")));

I have to wrap the opening and closing tags in streams of a single value and then use flatMap() to convert Stream<Stream<String>> to a Stream<String>. The nice thing was building up streams of streams of streams with a single joining() at the very top to collapse them back together.

October 12, 2023

Refactored com.jeantessier.classreader.impl.Classreader to use IntStream.range() instead of a for loop when reading parts of a .class file.

The code used to look like this:

    for (var i=0; i<attributeCount; i++) {
        Logger.getLogger(getClass()).debug("Attribute " + i + ":");
        attributes.add(attributeFactory.create(constantPool, this, in));
    }

And now it looks like this:

    IntStream.range(0, methodCount).forEach(i -> {
        try {
            Logger.getLogger(getClass()).debug("Method " + i + ":");
            methods.add(new Method_info(this, in));
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    });

I think it looks "clever" and more modern, but I have to wrap exceptions because the functional interface does not allow for checked exceptions such as IOException. The try/catch block makes it a little ugly, and I am not sure that the resulting code is really that much better.

October 11, 2023

Refactored com.jeantessier.text.Hex to use the Stream API and java.util.Formatter to render bytes in text and XML outputs. All the print() methods were used for single bytes, so I replaced them with calls to Formatter instead. That left the toString() static method, which I refactored to use streams to traverse the byte array and write to a String. I used this opportunity to format the bytes with extra spaces that make the output easier to read.

October 10, 2023

More tweaking of com.jeantessier.classreader.impl.Classfile to allow for parallel streams.

I researched how to process byte arrays using streams. I drew inspiration from this article where they use java.nio.ByteBuffer.

October 09, 2023

Refactored com.jeantessier.classreader.impl.Classfile to use more modern Java constructs, such as var local variables and using the Stream API.

September 20, 2023

Committed some changes that introduce some modern Java constructs, such as default methods in interfaces and some more use of the Stream API. I had been trying them out a while back but never got around to committing the changes. I have not applied the changes across the codebase, which is why I hadn't committed them.

I noticed the number of methods (M) measurement didn't have a constant in BasicMeasurements. This is because it is a computed measurement instead of being fundamentally derived from the .class file. MetricsGatherer creates a submetrics for each method, and only for methods. So, we can compute the number of methods in a class by counting these submetrics. If we ever decide to have submetrics for things other than methods, we'd have to derive M from the .class file, just like we do for the number attributes (A) and the number of inner classes (IC).

March 05, 2023

Used XSLT to convert XML metrics to YAML format. It is also somewhat easier than doing it in Java. But I'm toying with the idea of keeping both implementations. There were some interesting uses of the Stream API in those Printer subclasses. But I still need to fix how JSONPrinter deals with commas in lists.

I re-wrote the JSONPrinter to capitalize on Collections.joining(). It clones itself to render sub-parts. It will end up using more memory as it writes sub-parts using java.io.StringWriter and only writes them out at the very end.

February 28, 2023

Used XSLT to convert XML metrics to JSON format. It turns out to be much easier than doing it in Java.

February 27, 2023

Created YAML and JSON formats for OOMetrics. The JSON format is particularly tricky because it uses commas as separators in lists, and the Visitor pattern I am using doesn't know where it is in the list. How is it supposed to know not to put a comma after the last element? The Stream API has a joining() collector, but I will have to skip the Visitor pattern in the JSON implementation if I want use it.

February 26, 2023

Found a bug in OOMetrics. Group-level basic measurements were not being computed for defined groups, that is, groups that don't match a Java package name, but rather a regular expression in a group-definition. The metrics gatherer was only computing them for the group metrics of the current package. It was never looking for matching defined groups.

I also moved the P measurement for packages from the top-level project to each group, and used a SubMetricsAccumulatorMeasurement to aggregate them at the project-level. Groups that represent individual packages will have a value of 1, but defined groups using regular expressions can cover multiple packages.

February 24, 2023

Using Stream API to simplify (and modernize) some of the logic in OOMetrics.

February 22, 2023

Came upon Jarviz after seeing a toot about it on Mastodon and an article on InfoQ. One interesting use case for Jarviz is finding the version of Java classfiles, so we can make sure dependencies are compatible.

I looked at adding something similar to OOMetrics. But the current metrics are assumed to be numbers. And the set of metrics is fixed by the config, instead of creating metrics based on the data, like versions "60" and "63".

I added class-level metrics for "major version" and "minor version". But I will need to do more work to aggregate them at the group- or project-level. I would like to build a histogram of different major versions in JAR files. And maybe go so far as listing out the classes by version number.

February 09, 2023

Removed the dependency on Guava. I wasn't really using it and some of the functionalities have equivalents in recent JDK versions. This makes one fewer external dependency to manage.

I could go one step further and replace Jakarta-ORO with java.util.regex. It would allow me to get rid of MaximumCapacityPatternCache. But the Jakarta-ORO API is so much simpler.

And if I were to replace Log4J with java.util.logging, I would have no external dependencies at all. I like the Log4J API much more, though.

January 27, 2023

Added DependencyGraphToJSON and DependencyGraphToYAML. They use XSLT to translate dependency graphs in XML to JSON and YAML formats respectively.

January 26, 2023

I'm playing with the idea adding JSON and YAML output formats, in addition to text and XML. At first, I thought of simply adding new Printer subclasses.

I started drafting what the outputs might look like for dependency graphs.

Then, I realized I might be able to pull it off using XSLT. It wouldn't be much different from DependencyGraphToText.xsl. It already has logic to treat inbound and outbound dependencies as separate collections, even though they are a thrown together in a single collection in XML.

January 22, 2023

Added some support for Java 9 modules. I added the support for accessibility and constant pool entries. I'm still missing the Module, ModulePackages, and ModuleMainClass attributes.

January 21, 2023

Added tests for inner classes treatment in ListSymbols to improve code coverage. And added switches to ListSymbols tools.

Released version 1.2.1-beta6.

January 18, 2023

Fixed instrumenting code for code coverage. When going from Java 1.7 to Java 1.8, the JVM became more strict when validating bytecode. The instrumented bytecode is failing this stricter validation in Java 1.8. Passing the -noverify JVM arg allows it to skip this verification and run the instrumented code just the same.

• Emma: code coverage now works with Emma 2.0.5312.
• Clover: Clover 1.3.13 is still failing to create its coverage database.
• Cobertura: Cobertura 1.9.3 cannot handle try-with-resource code.

January 17, 2023

I tried to get some code coverage numbers for the changes I've been making to ListSymbols, but all three ways that I know of, Emma, Cobertura, and Clover, are broken at the moment. I don't know if it's the Java version, or the tool versions, or what the latest recommended way to get code coverage in Java these days.

January 16, 2023

Fixed how SymbolGatherer gets accessibility of inner classes. When visiting classfiles, SymbolGatherer waits to see if there is an InnerClass structure for it. If so, it passes this InnerClass structure to the Strategy to determine whether to gather it or not. If there is no InnerClass structure, then it uses the Classfile structure to make that determination.

January 15, 2023

Java specs talk of accessibility. So, I renamed VisibilitySymbolGathererStrategy to AccessibilitySymbolGathererStrategy.

January 13, 2023

Wrote VisibilitySymbolGathererStrategy to filter symbols based on their visibility, as determined by access flags. I realized that top-level classes can only be public or else package-visible, but inner classes use the full range of visibility. ListSymbols does not look at inner class metadata, but I will need to make it do so to properly filter inner classes.

January 12, 2023

A user asked for a way to limit ListSymbols to public and protected features. There are a number of decorators for the SymbolGathererStrategy that ListSymbols uses to filter symbols, but none of them match this specific requirement.

I would like to add a new decorator subclass that would filter based on visibility, but the test matrix is daunting. I looked a doing parameterized tests in JUnit, and found a way in JUnit5. I'm still on JUnit4. I will need to figure out the impact of an upgrade. I now wish I had a Gradle project setup so I could use Spock and write my tests in Groovy.

Maybe I can use Fit tests instead, to minimize the immediate impact.

August 02, 2022

I was reading an article comparing Fernflower and Quiltflower where they showed something in Java called try-with-resources. It's a way to write try blocks where you close a resource in the finally clause. It's been in Java since Java 7. I switched all the places where I would close a resource in a finally clause, usually reading files, and replaced them with the try-with-resource pattern.

June 17, 2021

Fixed the calculation of median values for StatisticalMeasurement. I'd always believed that the median value of a list had to be a member of that list. When the list's size is odd, that's easy enough. But when the list size is even, I would use the rightmost of the two middle elements. It turns out that for lists of numbers, we're supposed to use the average of the two middle numbers instead.

January 29, 2021

I started Dependency Finder twenty years ago, today.

May 04, 2020

Created a Docker image to run Dependency Finder in a Tomcat container.

April 16, 2020

Somewhere between Java 5 in 2006(?) and Java 7 in 2013, the invokedynamic instructions grew from 3 bytes to 5 bytes. The two extra bytes are nulled out all the way through to Java 14.

I should do a pass through all instructions to see if any others have changed.

April 13, 2020

I noticed a new issue (#6). It shows a limitation of Dependency Extractor: it fails to capture dependencies on lambda expressions. Take the following example:

    import java.util.function.*;
     
    public class Test {
        public void someMethod(){
            Consumer<String> function = (x->System.out.println(x));
        }
    }

You would expect a dependency graph that includes the following:

        Test
            lambda$someMethod$0(java.lang.String)
                <-- Test.someMethod()
            someMethod()
                --> Test.lambda$someMethod$0(java.lang.String)

The problem is that the reference to the lambda in the bytecode is through the invokedynamic 0xBA instruction. This instruction allows the JVM to resolve a method call at runtime. It relies on very intricate data structures and its description in the Java Virtual Machine Specification, Java SE 8 Edition, took five entire pages. I don't know when I will find the time to work on this very tricky piece of code. The latest Java Virtual Machine Specification is a little more concise, so maybe there is hope, yet. I'll have to take a look at what else is new or changed.

April 09, 2020

Turned on HTTPS on SourceForge. This changed the URL from http://depfind.sourceforge.net/ to https://depfind.sourceforge.io/. SourceForge put some redirects in place, but I tried my best to change all references nonetheless.

April 03, 2020

Fixed a cut'n'paste error in the dependencies DTD that had been going unchecked since 2005-03-22.

April 01, 2020

Fixed the broken tests.

In one case, the compiler was optimizing away the unused variable c in the sample test class. I figured a way to tweak the code so the compiler would keep the variable by declaring it in an enclosing scope.

Tests for CodeDependencyCollector were testing filtering by excluding /java.lang/ and making sure java.lang entities were absent from the dependency graph. But after I added inheritance data to the graph, it became impossible to avoid java.lang.Object since test derives from it. So I switched the package to exclude to java.io instead. These tests were broken since 2008-08-17.

I realized that when I ran tests with Ant instead of textjunit, some tests were failing. I had to add JUnit classes to the CLASSPATH used by Ant's task.

Finally, all the tests for com.jeantessier.dependencyfinder.web are failing because they use HttpUnit, which is broken. I've excluded these tests for the time being, until I can find another way to test JSPs.

March 25, 2020

In Java 9, the com.sun.javadoc package was rewritten into the jdk.javadoc.doclet package. Some functionality was moved around and some of the classes changed names. In Java 13, they finally removed the old, deprecated com.sun.javadoc package. As a result, I will need to rewrite ListDocumentedElements.

March 24, 2020

Explored using Saxon for XSLT processing. I can run a CLI tool straight out of the JAR file. I could add its JAR file to Dependency Finder's lib/ folder and only change the XSL script templates to refer to that JAR file only. One problem, though, is the command-line switches to specify input and output files are different. (And not to my taste.)

    $ DependencyGraphToHTML -in df.xml -out df.html

has become:

    $ DependencyGraphToHTML -s:df.xml -o:df.html

I could have parsed the parameters inside the scripts and re-write in and out switches on the fly. I could have done it for the UNIX shell scripts, but I don't have access to a DOS machine to do the same for the Windows batch files.

Saxon's HTML output specifies a DOCTYPE. With any DOCTYPE, browsers are more strict with the CSS stylesheet. Without a DOCTYPE, I can get away with bare numbers when specifying margins. With a DOCTYPE, I have to provide a unit. "50" has to become "50px" or else we lose all indentation in the HTML output and it looks terrible. Fixing the CSS stylesheets to be more explicit is easy enough and doesn't break anything.

March 23, 2020

Somewhere between Java 8 and Java 11, they removed com.sun.org.apache.xalan.internal.xslt.Process from the JDK. This means that I have to change XSLTProcess if I want to continue using XSL templates to convert XML documents to HTML. I have over 45 tools that are based on XSL and I am not looking forward to having to rewrite all of them.

March 15, 2020

Upgraded the following development components:

JUnit	:	4.12	->	4.13
Tomcat	:	9.0.19	->	9.0.31

I still have 78 failing tests. :-(

Many of the tools that I was using 10 years ago either don't exist anymore or have widely evolved in incompatible ways. Many of them can only download from Maven. My instructions for setting up a developer environment are pretty much obsolete. People have been asking for the ability to download Dependency Finder from Maven. And it would be nice to write tests in Spock (and write code in Groovy, for that matter). I have some thinking ahead of me.

August 09, 2019

Upgraded the following development components:

JDK

:

1.8.0_172

->

1.8.0_181

April 17, 2019

I found a large number of files that were in DOS format, with \r\n as line terminators. I fixed them all to be in UNIX \n format, instead.

March 28, 2019

A user pointed out a problem analyzing some JAXB files. Once again, the files contain new constant pool entries for Java 9. They proposed a patch to ignore a specific filename, but I think it's better to catch the Exception, log it, and continue extracting the class information. Now, there is a small risk that .class files will be skipped "silently". Users could find out by scrutinizing the output with -verbose, or with clever Log4J settings, but it's more likely they won't realize they only have partial data.

I will need to update the com.jeantessier.classreader package to the latest bytecode.

May 14, 2018

Upgraded the following development components:

JDK	:	1.5.0_04 (?)	->	1.8.0_172
Ant	:	1.9.7	->	1.10.1

I don't know when it started, but the default Ant target no longer works. These three invocation all do the same thing:

    ant
    ant all
    ant distclean dist

Both distclean and dist depend on the init target, which sets a bunch of variables and creates output folders. But `distclean` deletes these output folders so they no longer exist by the time `dist` runs.

I tried to generate a new release, 1.2.1-beta5. However, I am now using Java 8 and it generates .class files that older versions of Java will not be able to execute. I wonder if that wil be a problem.

The instructions for generating a build need some updating.

April 27, 2017

Aborting efforts to convert to Grails. For the time being.

Grails 3.2.9 just came out and this project is still on Grails 1.3.7! There is no obvious upgrade path for Grails apps out of 1.x. And I never really got around to do anything substantive. So, I am giving up and deleting the grails branch. I'd have to start over anyway, and there isn't much to salvage.

August 21, 2016

I've been looking at what's needed to bring the com.jeantessier.classreader package up to date with the latest JVM spec for Java 8. I already took care of the new constant pool elements, but there are new attributes too. And those are not trivial.

Here is the class structure that supports Java 4. The constant pool is in blue and the various attributes are in yellow.

And here is the class structure needed for Java 8. The new elements are in green.

Bear in mind, these are only the public interfaces in com.jeantessier.classreader that will be visible to other callers. There will also be implementation classes in com.jeantessier.classreader.impl and additional support classes, like factories for the Target, StackMapFrame, and VerificationType_info hierarchies. It will also impact implementation of the Visitor interface.

August 10, 2016

A user pointed out to me that they were not able to analyse some Lucene files. I determined that the files contained new constant pool entries for Java 7/8 and quickly added support for them in the com.jeantessier.classreader package. This way, we can at the very least parse the files. The support is very barebone for now, only printing them out in logs and as part of ClassReader. After reading about it, I think there are also new attributes that I'll have to look into.

I realized that the installation instructions still reference Sun Microsystems. I should update them to point to Oracle instead.

July 20, 2016

Aborting efforts to convert to Struts.

July 19, 2016

Figured out how to let Gradle know where to find the classes for Javadoc Doclets with the following dependency in build.gradle:

    compile files(org.gradle.internal.jvm.Jvm.current().toolsJar)

It adds tools.jar to the compiler's classpath. It which contains the com.sun.* classes used by the Doclet code.

July 18, 2016

Ripped out WebWork. It was a great framework years ago, but there are better alternatives, now.

July 17, 2016

Upgraded the following development components:

Ant	:	1.9.4	->	1.9.7
Guava	:	r09	->	19.0
JUnit	:	4.8.2	->	4.12
Log4J	:	1.2.15	->	1.2.17

July 15, 2016

Played with replacing the Ant build script with Gradle. It has a number of pre-defined tasks to deal with Java project. It also expects a certain code organization that echoes what I have seen on other projects. Source code is under src/main/java and unit tests are under src/tests/java. It is possible to change these defaults to match legacy projects, but I'm more tempted to move files around to match these growing best practices.

I can declare dependencies in build.gradle and Gradle will download them automatically. Such as:

    compile "log4j:log4j:1.2.17"
    compile "oro:oro:2.0.8"
    compile "com.google.guava:guava:r08"

I can also include dependencies for WebWork and such.

I need to find better examples on how to deal with launch scripts and web applications.

April 01, 2016

Here is the most recent link to the JVM specification and the production rules for signatures.

March 31, 2016

Someone emailed me a question about dependencies on types in declarations for generics.

Here is an example:

    import java.util.*;

    public class SomeClass {
        private Set<OtherClass> property = new LinkedHashSet<OtherClass>();
    }

And its dependency graph:

    Main
        --> java.lang.Object *
        Main()
            --> java.lang.Object.Object() *
            --> java.util.LinkedHashSet.LinkedHashSet() *
            --> java.util.Set *
            --> Main.property
        property
            --> java.util.Set *
            <-- Main.Main()

No mention of OtherClass whatsoever.

This is because the Java compiler applies type erasure to generics. It stores the original signature in Signature_attribute attributes attached to the class, field, and/or method that had a generic declaration. At this time, CodeDependencyCollector ignores the contents of these attributes. Their production rules are very complex, given the wide variety of generic declarations.

For instance, the signature for SomeClass.property above is:

    Ljava/util/Set<LOtherClass;>;

In parsing it, we have to detect the inner L...; sequence before we can process the outer one. And this is a trivial case. Method declarations and wildcards introduce other levels of complexity that only compound each other.

I looked a little into it back on 2009-06-04, but haven't made any progress since, I'm afraid.

January 22, 2016

I found out some of my code samples in the Developer Guide got mangled by the pseudo-wiki generator. Whenever a code sample had a == check, the generator took it to mean an empty <code> tag. I fixed it to leave these alone.

A podcast mentioned GitBook for publishing books. I could move the guides there, provided I rewrite them using Markdown.

October 03, 2015

Moved the project hosting to GitHub.

I can use GitHub Pages to host the project's website, so I moved the older website folder to the gh-pages branch and moved everything in the older DependencyFinder folder up one level. GitHub Pages cannot run CGI scripts, so I will have to find an alternative for the Journal.

I will also need to change references to SourceForge from the pages and fix deployment instructions.

January 30, 2014

Fixed the problem that prevented me from compiling Dependency Finder. A test class was embedding an irregular character "¥" saved directly in the source Java code. The compiler decided that it was going to choke on it from now on. So I changed it for the escape sequence \u00A5 instead and it worked just fine.

There are still 77 broken tests. I vaguely remember that some were broken when I last ran them (years ago). But 77! Many of them relate to a missing Tomcat class, so it's probably because have shifted in Tomcat 7.0.14. If only there was a tool to help me locate a class in folders with a bunch a JAR files in them. :-)

November 25, 2013

Fixed the problem with Perl. I was using an older syntax to load the "getopt" module to parse command-line switches. I updated to a newer syntax and now it's loading just fine.

November 24, 2013

I tried to upgrade from Grails 1.3.7 to the latest, 2.3.3. It proved to be more than I could handle straight out of the box. There have been many many profound changes in Grails since version 1.3.7 and I might have to start from scratch. Lucky for me I haven't had time to get very far in my Grails integration.

On a sadder note, I haven't really kept up with my environment and I realized that I can no longer compile Dependency Finder. Somewhere between computer upgrades and OS upgrades, Perl and Java have changed and my code no longer works. I will have to find the time somewhere to figure out what's changed and fix it.

March 07, 2013

Someone asked for an example of using OOMetrics, such as a constructor that scans an entire folder and a simple getter that returns all metrics for a given class. OOMetrics is highly configurable and slices and dices the data many different ways. This usecase is very simple and setting things up for it is a little involved.

Here's the code:

    import java.util.*;

    import com.jeantessier.classreader.*;
    import com.jeantessier.metrics.*;

    class OOMetrics {
        /**
         * names the file that describes which metrics to compute.
         */
        public static String METRICS_CONFIG_FILE = "config.xml";

        /**
         * holds the metrics definitions and collected data.
         */
        private MetricsFactory metricsFactory;

        /**
         * reads the metrics descriptions from the config file and
         * collects metrics on all the .class files on paths.
         */
        public OOMetrics(Collection<String> paths) throws Exception {
            // Setup the MetricsFactory which will hold all the metrics.
            MetricsConfiguration metricsConfiguration = new MetricsConfigurationLoader().load(METRICS_CONFIG_FILE);
            metricsFactory = new MetricsFactory("OOMetrics", metricsConfiguration);
            com.jeantessier.metrics.MetricsGatherer gatherer = new com.jeantessier.metrics.MetricsGatherer(metricsFactory);

            // Read in all the classes to analyze
            ClassfileLoader classfileLoader = new AggregatingClassfileLoader();
            classfileLoader.load(paths);

            // Collect metrics on all the classes.
            gatherer.visitClassfiles(classfileLoader.getAllClassfiles());
        }

        /**
         * converts the internal metrics data to name-value pairs.
         */
        public Map<String, Number> getMetrics(String className) {
            Map<String, Number> results = new TreeMap<String, Number>();

            Metrics metrics = metricsFactory.createClassMetrics(className);
            for (String measurementName : metrics.getMeasurementNames()) {
                results.put(measurementName, metrics.getMeasurement(measurementName).getValue());
            }

            return results;
        }
    }

The constructor reads the configuration from an external file because hard-coding a configuration would have been very complex. It is using an AggregatingClassfileLoader because it is more versatile even though it uses a lot more memory to hold all the classes at the same time.

The getter reduces the measurements to simple numbers. It could return the whole Measurement object and let the caller sort it out. The traversal is a little awkward because Metrics is meant to be traversed using the visitor pattern instead of through the programmatic API. It ends up with the measurements' short names, which can be a little cryptic. An alternative that shows the longer names would look like this:

    Measurement measurement = metrics.getMeasurement(measurementName);
    results.put(measurement.getLongName() + " (" + measurement.getShortName() + ")", measurement.getValue());

I don't know the full context in which they want to use these metrics, but it may be simpler to use OOMetrics as is, either from the CLI or through Ant, and then parse the output file.

    OOMetrics -classes -xml -out metrics.xml dist/classes/

There is good support for parsing XML in most languages and application frameworks. You'd use it to read the output from the tool and convert it to a lookup table and use XPath expressions to narrow in on the classes you want.

January 29, 2013

Dependency Finder is now 12 years old.

August 07, 2012

Found a Grails plugin, Standalone App Runner, that packages the webapp in a JAR file with a choice of Tomcat or Jetty, so you can run the app simply by executing the JAR file. I was considering a distribution with Jetty a while back and this would be a really easy way to do it. All I have to do is rewrite the webapp as a Grails app.

January 29, 2012

Dependency Finder is now 11 years old.

January 12, 2012

Fixed calls to <javac> in the various build.xml files to include the includeantruntime attribute needed by Ant 1.8.

This is stupid. For the sake of preserving some old build.xml files, they polluted all future build.xml files. They could have made it a command-line switch on Ant itself so old projects could fix it without corrupting all of our projects.

January 08, 2012

Added -excludes-list to ListSymbols, complete with tests. After two years in Groovy, Java code feels so different.

January 06, 2012

Someone asked about listing methods in a code JAR file that are not called from a separate test JAR file. They put their code in one JAR file and their tests in another one, and they want to use Dependency Finder to find what methods are not called directly from the tests.

I figured out a way by listing symbols and using them in an exclusion list, but ListSymbols does not have that switch. Adding it is not too complicated, but there are quite a few tests to adapt. I have to immerse myself in that code again. It's been a while.

November 17, 2011

Someone opened a bug about reporting line numbers as part of dependency graphs, bug 3438741. Their arguments are that 1) why not use it when it's there and 2) other tools can pinpoint statements, so why can't Dependency Finder?

So I tried to clarify my reasons in the user's manual. I'm fairly limited by relying only on compiled code and the information that's there. Line number information, when available, is only for methods and not for fields or classes. Implicit return statements and constructors can throw off line number information, making it harder to really trust them. Other tools, aside from exception stacktraces, typically work off of the source code and build their own parse tree, giving them access to the entire code.

I have been tempted for some time to gather information per dependency, such as the number of times a given dependency is repeated, or possibly what line number it occurs on. But for that, I would need an new object to model the dependency, and that would augment the memory footprint. Right now, dependencies are just Node references in Map instances, keeping things small. But my merging together all copies of a dependency, it becomes impossible to report specifics.

November 11, 2011

Upgraded the following development components:

Ant

:

1.7.0

->

1.8.2

May 19, 2011

Tried to upgrade to Tomcat 7, but HttpUnit is having problem creating the ServletContext needed to run the tests. I cannot find information on this problem online, so I'm going to wait a little.

May 18, 2011

Upgraded the following development components:

Guava

:

r08

->

r09

March 09, 2011

Tried to convert extract.jsp to a Grails GSP. All the scriptlets are really getting in the way. And the same problem I had with WebWork is surfacing again: extract.jsp streams its progress directly to the browser and I haven't quite found how to do this in these frameworks. Maybe a taglib can help.

March 07, 2011

Upgraded the following development components:

Grails

:

1.3.6

->

1.3.7

February 06, 2011

Copied changes to ClassMetrics CLI to the corresponding Ant task. Also updated the documentation for both of them.

February 03, 2011

Finished including annotations being used when compiling class metrics. I used some features of the Guava library to convert a set of Annotation instances to their type string and remove duplicates. It let me write the transformation in a more functional way, but Java is so very verbose compared to Groovy.

Here is the code in Java:

    Iterable<String> annotationTypes = Sets.newTreeSet(Collections2.transform(metrics.getUsedAnnotations(), new Function<Annotation, String>() {
        public String apply(Annotation annotation) {
            return annotation.getType();
        }
    }));

And the equivalent in Groovy:

    def annotationTypes = metrics.usedAnnotations.collect*.type as SortedSet

The nice thing with Groovy is that since it's a JVM language, I can just put DependencyFinder.jar on my CLASSPATH and run that line in some Groovy code.

Upgraded the following development components:

Guava

:

r06

->

r08

February 02, 2011

Working on a unit tests to check that MetricsGatherer also accumulates annotations that are being used by the code under examination.

January 29, 2011

Dependency Finder turns 10 years old.

January 13, 2011

Wrote unit tests for MetricsGatherer's handling of runtime annotations when compiling class metrics.

January 01, 2011

Trying to write unit tests for Field_info so I can tackle bug 1766486. Because I decided, a long time ago, to create everything from data streams in constructors and strictly mirroring the data structure from the classfile specification, I end up having to mock a lot of things to setup a test for a field with a string value. There are times when you question some of the earliest design decisions.

December 31, 2010

Minor tweaks to text docs.

Started working on bug 1766486 about escaping special characters when rendering classes as XML.

December 30, 2010

Upgraded the following development components:

Grails

:

1.3.3

->

1.3.6

December 29, 2010

Fixed the bug with EnclosingMethod_attribute when the enclosing method is a constructor. I took this opportunity to write some tests for generating XML for EnclosingMethod_attribute.

December 28, 2010

Researched the bug with EnclosingMethod_attribute when the enclosing method is a constructor. It turns out that when the enclosing method is a static initializer, the EnclosingMethod structure does not have a method reference.

December 27, 2010

A user reported a bug with EnclosingMethod_attribute when the enclosing method is a constructor. When producing XML output from ClassReader, they sometimes get the string "<init>" in some of the text, which XML parsers think is a start tag and that the overall XML document is not well-formed. I suspect this is because an anonymous inner class is declared inside a constructor, since the bytecode calls all constructors "<init>". I also suspect that anonymous inner classes declared inside a static initializer might produce an EnclosingMethod structure with a method name of "<clinit>".

December 09, 2010

Upgraded the following development components:

IntelliJ IDEA

:

9.0.4 CE

->

10.0 CE

November 29, 2010

Updated the code coverage reports for release 1.2.1 beta4.

November 28, 2010

Figured out the problem with the ArrayIndexOutOfBoundsException on wide instructions. There are helper methods inside the Instruction class to fetch individual bytes from the bytecode. They all take offsets from the instruction's start position in the bytecode. One branch of a case statement was adding the instruction's start position to the offset before calling the helper, resulting in a fetch at twice the start position. If the instruction was past the half-way park, it resulted in ArrayIndexOutOfBoundsException.

Found another problem with escaping text in XML documents from ClassReader. It is possible for some string constants in the bytecode to contains characters that are not strictly UTF-8. I had to brush up on my XML to fix this one. I found a promising library in Commons Lang from the Apache Project, but it did not handle low value control characters that are below the range of normal visible characters. So I had to rework com.jeantessier.classreader.XMLPrinter.escapeXMLCharacters() myself. I don't think I'll have the same problem with the other XML generators because they don't deal with the raw constant strings in Java the way ClassReader does.

Released version 1.2.1 beta4.

November 26, 2010

Received the class that was breaking Dependency Finder back on 2010-10-23. ClassReader chokes on it but javap is fine. Today's a holiday and I'm busy with visitors from out of town, so it'll have to wait a little.

November 18, 2010

Found a back link for apache.com that should have been apache.org instead.

November 12, 2010

Upgraded the following development components:

IntelliJ IDEA

:

9.0.3 CE

->

9.0.4 CE

November 01, 2010

Back on 2010-08-29, I switched from google-collect to Guava. Then, on 2010-09-08, I removed the now obsolete google-collect.jar file but I forgot to update the scripts that lists JAR files explicitly. That's now fixed.

October 23, 2010

Someone is getting ArrayIndexOutOfBoundsException when Dependency Finder encounters a wide instruction (opcode 0xCA). If they can send me the class that causes the problem, I might be able to figure where the problem lies.

October 18, 2010

Someone was asking for help with the dependencyextractor And task, and I noticed that the documentation contained a typo. It mentioned a task attribute named filter-excludes but the task has a setter for setFilterexcludes() instead. So where would the dash in the XML attribute come from? I read the documentation and confirmed that Ant is case-insensitive when it comes to task attributes, but it does not add or remove dashes. So the sample code in the documentation does not work. Oops. All fixed now.

September 08, 2010

Moved the Google Analytics tracking code from the old Urchin-based to their new Asynchronous Snippet. Quite frankly, nobody should notice the difference.

I had to cut and paste the snippet to many files. That was annoying, so I extracted it to a separate .js so I could copy a single line script inclusion instead. Then, I realized that defeated the purpose of Google's Asynchronous Snippet, whose purpose is to not delay the loading of the page by including other scripts synchronously. So I'm back to cut and pasting "large" snippets.

August 29, 2010

Upgraded the following development components:

Tomcat	:	6.0.10	->	6.0.29
google-collect	:	1.0-rc2	->	none
Guava	:	none	->	r06

August 28, 2010

It turns out Guava, strictly speaking, also depends on JSR 305. But I am still able to compile Dependency Finder without it and run the tests.

August 27, 2010

While looking for the latest version of Google Collections, I found out it was now something called Guava. I might consider switching.

August 25, 2010

Upgraded the following development components:

JUnit

:

4.7

->

4.8.2

I also ran JarJarDiff on various versions of JUnit from 3.8.2 to 4.8.2. I added the results to the Samples page for JUnit.

August 19, 2010

Looked at AddThis as a possible mechanism for coworkers to share interesting Dependency Finder results on the webapp. The default AddThis email button puts the URL in the email message. The URL is enough to recreate the query and see the results. I need to explore further if I can add the query's results to the message.

August 12, 2010

A user asked for clarifications on the behavior of the advanced view in the GUI. From the layout of the interface, in appears the checkboxes govern the use of regular expressions for packages, classes, and features. But they don't. You can think of it as a two-step process:

1. Select nodes based on all the regular expressions.
2. Summarize the selected nodes based on the checkboxes.

GraphSummarizer does both steps at the same time, but conceptually, this is what is happening.

I never claimed the UI was very intuitive. There is definitely room for improvement.

July 24, 2010

At ShopWell, we do a lot of design using post-it stickies on foam boards. We have many, many very large foam boards, to the order of 100"x60". We can stand at the foam board and have a hallway chat about things. We can more stickies around and group them and shift things around. We can easily more boards from one room to the next as needed.

On Dependency Finder, I'm a little overwhelmed by all the tasks I've identified and written up over time, in so many places: the Release Plan, Pivotal Tracker, 5"x7" cards, notes, etc. So I figured that if I could get them all on one foam board, where I could see them all and easily manipulate them, it might get me going again. I went out, bought myself a board (a more modest 36"x24"), some stickies, and a bunch of sharpies, and I'll get started running my own board for Dependency Finder.

July 22, 2010

Upgraded the following development components:

Grails

:

1.2.2

->

1.3.3

July 20, 2010

Upgraded the following development components:

IntelliJ IDEA

:

9.0.2 CE

->

9.0.3 CE

May 11, 2010

Pushed the grails branch back to the Git origin repository on SourceForge. I had to dig out the commands, since it had been a while. Roughly, it looked like this:

1. git checkout -b grails
2. ... a bunch of commits ...
3. git push origin grails
4. git checkout master
5. git branch -D grails
6. git branch grails origin/grails
7. git checkout grails

I created the branch locally at first and made some commits to it. I then pushed it to the remote repository. I wanted to change my setup so my local branch would track against the remote branch, so I had to delete the local branch before I could create the mapping I wanted. I have to admit I found this part was a little scary. But it worked just fine.

May 06, 2010

I got an email today from a user who found a bug in Dependency Finder. It turns out that I implemented the null check in Classfile.isInnerClass() in the wrong direction. I looked for usages of this method but found none. Not even a test. The shame.

I also discovered a 2 year old bug report about the access flags for inner classes. Given these classes:

    public class Outer {
        private class Inner {}
    }

The bytecode for Outer has an access flag of 00000000 00100001: ACC_PUBLIC and ACC_SUPER. The bytecode for Inner has an access flag of 00000000 00100000: only ACC_SUPER because it is not a public class. The .class file format does not provide for other class-level visibility levels. The modifiers of the inner class are covered in the InnerClass part of InnerClasses_attribute. The bytecode for both Outer and Inner include identical copies of InnerClass with an access flag of 00000000 00000010: ACC_PRIVATE.

The bug report was trying to copy the InnerClass access flag to the Classfile instance. This will not work because the meaning of each bit is different between the two classes. The interfaces in com.jeantessier.classreader are meant to provide an accurate representation of the information in the bytecode. It is meant for people who think in terms of the bytecode, not people who think in terms of the Java language. There is a subtle difference. If you're dealing with an inner class, you have to get its access level from the enclosed InnerClasses attribute.

To work on this problem, I fired up a grails shell in a Grails application where I had copied DependencyFinder.jar. It was really cool to simply use scripting to manipulate data. Java has so much boilerplate code when compared to Groovy.

I fixed the link to JSR 202 on the Resources page. The latest .class file specification includes full description of the StackMap element. At last.

April 28, 2010

Grails already includes JAR files for Log4J 1.2.15 and Jakarta ORO 2.0.8, the same versions that Dependency Finder uses. As an experiment, I created an empty Grails project and copied only DependencyFinder.jar to its lib/ directory. I then started grails shell and I was able to run the same operations as DependencyExtractor by hand. No big surprise there, but it was cool.

I want to redo the webapp as a Grails application. I'm struggling with file system issues, though. Should I create a directory under the current tree where I would put the new app, next to web/ and webwork/? That would be consistent, but the new app would get a name from that directory, which would most likely be called grails. An alternative is to start a brand new tree somewhere else, completely dissociated from the current Dependency Finder development tree. There, the application can be called DependencyFinder or some such. But I would have to start a new Git repository for it. In either case, I'll have to be creative to manage DependencyFinder.jar as it evolves.

April 23, 2010

Upgraded the following development components:

IntelliJ IDEA

:

9.0.1 CE

->

9.0.2 CE

April 17, 2010

My license for IntelliJ IDEA is about to expire. I haven't been able to work on Dependency Finder since I started working at ShopWell, so I don't know if they'll consider the project active enough to deserve a free license still. But JetBrains has a community edition that is free and does everything I need, so it is not a complete loss. I installed it tonight to try it out.

Upgraded the following development components:

IntelliJ IDEA

:

9.0

->

9.0.1 CE

January 26, 2010

A user was having difficulty accessing the DTDs. After investigating, I realized SourceForge disabled automatic directory listings ... in 2008. They left instructions on how to re-enable it, so I added .htaccess in the dtd/ folder to take care of it.

December 17, 2009

Tried to use Dependency Finder to analyze some Groovy code. But the way that Groovy mediates method calls so it can treat getters like properties makes it difficult (impossible?) for Dependency Finder to see the real dependency. Instead, it sees dependencies to Groovy's internal method resolution code.

I installed the latest version of yEd only to find out its XML parser cannot read the GraphML files I generate with DependencyGraphToyEd. Something to do with forward references. I guess I'll have to rewrite it to process write out all nodes before it writes edges.

November 19, 2009

I've been setting up another development on a new laptop. I couldn't find the version of FitLibrary that I'm using with Dependency Finder. I'm using FitLibraryRunner20060116, but the oldest version available is FitLibraryRunner20070217. There were some API changes, starting with this version, that are incompatible with my code and I have not found a workaround. I am going to have to include my copy of the 2006-01-16 version with the source for Dependency Finder if people want to run the Fit tests.

November 07, 2009

I'm thinking of replacing the web app with a Grails app instead.

October 09, 2009

Found some useful instructions on tying Git and SVN at the online documentation for Git for the git-svn command.

October 08, 2009

Fixed the WebWork-based web app, somewhat. It turns out that if I want to use Spring IoC, I need to add a few JAR files to the application and an empty Spring config. But maybe I just don't need Spring IoC.

Now, the web app starts, but nothing happens when I try to run extract.action. I know the right method gets called because start and stop get assigned, but ExtractAction does not look at the resources listed in source.

I tried to update the SVN repo first, in the hopes of copying the change to Git afterwards. This is probably a bad idea if I don't have access to the original Git client where I first fetched the SVN repo.

October 06, 2009

Converted the Subversion repository to Git, hosted on SourceForge. I'll try to keep them in sync for a while, but I might eventually switch only to Git and cancel the SVN repo, just like I did for CVS a while back.

October 01, 2009

Added sample data files for 1.2.1 beta releases.

Added build.cobertura.xml for computing code coverage using Cobertura.

September 30, 2009

Worked on backlog of data files from 1.2.1 beta releases.

September 27, 2009

Released version 1.2.1 beta3.

Updated the release instructions.

September 23, 2009

Added missing flag values on Classfile parts when rendered in XML. I also added comprehensive tests for all flag values.

• Classfile
• synthetic
• annotation
• enum
• Field
• enum
• Method
• bridge
• varargs
• InnerClass
• annotation
• enum

Updated the DTD too.

September 21, 2009

Found a bug with the latest release. When I added Google Collections, I forgot to update some of the Unix launch scripts. I'll probably cut a new release with the fixed scripts.

September 17, 2009

Upgraded the following development components:

HttpUnit

:

1.6.2

->

1.7

Added the output of JarJarDiff on HttpUnit to the Samples page.

September 16, 2009

Going through the Groovy tutorial. It is a nice scripting language, a cool extension of Java. You can name closure parameters, just like Ruby, but it also has a default it if the closure needs only one parameter. I was able to compile to bytecode, using groovyc, and run Dependency Finder on it. But the code it generates is very hard to read, so don't try it. Generated code is meant to stay hidden.

Speaking of generated code, I saw a presentation on IxEdit, a tool to generate JavaScript to tie controls on a page. While it is very slick for generating interactions, it generates code that you then cut and paste into the HTML page. I'm pretty sure this is not maintainable. The generated JavaScript is fairly opaque. The demo didn't cover how or if you could re-insert the code in the editor to refine interactions.

September 14, 2009

A friend mentioned the Groovy language. It is a scripting language built on top of Java. It compiles to bytecode and runs on the JVM, so I might be able to run Dependency Finder on it and see what comes out. :-)

Videos on Groovy:

• Guillaume Laforge at Google
• Guillaume Laforge on AppEngine

The Groovy site mentions Cobertura to do code coverage. Its sample output looks nicer than what I'm using right now: Clover and Emma. I'll try to run it over Dependency Finder when I have a moment.

September 11, 2009

Released version 1.2.1 beta2.

The distribution file management has changed a lot since the last release. I need to update the release instructions. See release instructions on SourceForge in the meantime.

September 09, 2009

Pushed the logic for separating fully qualified class names into package name and simple class name to the Class_info class, with tests. Added tests for Classfile.locateField() and Classfile.locateMethod(), including proper handling of inherited package-visible methods.

Added LoadAdapter, a no-op implementation of LoadListener, so implementations don't have to duplicate no-op methods just to fulfill the interface.

Started Google Collections in spite of my reservations regarding JSR 305. Next, I want to start using Guice too.

September 04, 2009

Added JarJarDiff output for EasyMock and its class extension.

August 31, 2009

I'm still debating whether interactions between visitors should use double dispatch or not. Double dispatch is useful when you don't know the exact type of the element being visited, so you cannot know which visit() method to use. But when the decorating visitor is ready to dispatch to the decorated visitor, it knows the concrete type of the visited element, so there is no need for double dispatch. I'm still tempted to keep using double dispatch nonetheless, since it is the normal mechanism for passing control to a visitor and it makes the interactions with the decorated visitor seem more natural.

I have produced an alternate sequence diagram that shows the flow of control between three visitors composed visitor to render a dependency graph as text.

I also produced a matching version of the dependency graph for rendering a dependency closure as a call graph.

I really like sdedit for creating sequence diagrams.

August 28, 2009

Tried to used Google Collections in Classfile. I was hoping that Iterables.find() would simplify methods like getField() and getMethod(), even with the added boilerplate for the anonymous inner class for the Predicate. But find() throws an exception if it doesn't find a match whereas my code has been returning null. Once I throw in the exception handling, the methods is much harder to read than the simple for() loop I had before.

Google Collections uses the @Nullable annotation from JSR 305. But the project doesn't have any binaries, so I had to steal the JAR file from the FindBugs project. It doesn't feel as clean as I would want it. Maybe I need to let Google Collections to bake for a little while longer.

August 27, 2009

Refactored Classfile to rework locateField() and locateMethod(). I now have a special constructor to allow testing. I marked it with a comment while I decide whether or not I want to pull Google Collections into the project so I can use its @VisibleForTesting annotation.

August 22, 2009

Finished a possible test for Classfile, using a test-only constructor. I marked it @Deprecated until I figure out how I want to introduce the Google Collections API.

I should rewrite Classfile.locateField() so it is similar to Classfile.locateMethod(). For consistency.

August 19, 2009

Looked at how I'm using MaximumCapacityPatternCache and how its capacity() method returns a hard-coded 20. This capacity serves to seed the expression cache in ORO's Perl5Util. I'm thinking that most of the time, 20 is overkill. And when I need more than 20, I need a lot more than 20. So it's never quite the right number, but I cannot refine it without lots of experimentation. At least, ORO's GenericCache.DEFAULT_CAPACITY is also 20, so somebody thinks it's a sensible value. Maybe I should reference it instead of hard-coding a mysterious 20.

August 18, 2009

Changed dependencies on java.io.DataInputStream to java.io.DataInput instead.

Looked at Classfile.locateMethod() and I found something that seemed strange at first glance: it will return a method defined on the class regardless of its access privilege, but will only return one that is inherited if it is public or protected. This makes sense, since otherwise it would not be inherited. But it is missing the case of a package-level method from a superclass or interface in the same package. The expected behavior is not documented and there are no tests to tell me what I had in mind when I first wrote it. So I tried to write a test for it, but the constructor for Classfile does way too much work. It reads a lot of data from the DataInput, including the constant pool and interfaces and fields and methods and attributes. Specifying all of these in a test is too brittle, I'd much rather supply mocks for all these collaborators. If I used the Google Collections, I could have a special constructor and mark it @VisibleForTesting.

August 17, 2009

Tweaked tests to use is() matcher with assertThat() consistently.

August 14, 2009

Refactored com.jeantessier.metrics.TestMetricsGatherer so it uses mostly the new assertThat() syntax from JUnit 4.5 instead of the various assertTrue() and assertEquals() and so forth. I was able to simplify the tests drastically with judicious uses of helper methods and Hamcrest's hasItems() matcher.

It is still more an integration test than a pure unit test. It is somewhat brittle since it depends on an actual compiled class and lots of internal details of com.jeantessier.metrics, but there is only so much I can do in one pass. At least, it is very readable now that I have removed all the duplication.

August 13, 2009

Upgraded the following development components:

JUnit

:

4.6

->

4.7

The new JUnit introduces a new concept called rules. They are essentially interceptors for tests and a new extension mechanism for JUnit, now that tests don't have to derive from TestCase anymore. This looks promising. They re-implemented expected exceptions and you can now supply matchers for more precise matching of exceptions thrown by the test methods. Rules appear much more flexible, but they are still poorly documented. It is unfortunate that the javadocs bundled with JUnit 4.7 do not include the documentation of already provided rules.

Extracted styles from the Resources page.

August 07, 2009

Another example sequence diagram for the call flow across composed visitors to, this time to render a dependency closure as a call graph in text. Click the image for a better view. See also the source file.

I wanted to completely separate the textual rendering from the traversal of the graph, but I may not be able to do that. The current TextPrinter makes assumptions on the nature of the graph so it can remove "empty nodes". I am not sure in this logic belongs in the rendering or the traversal or somewhere else altogether. If it belongs somewhere else, this complicates the setup to do a simple printout to text.

August 05, 2009

Found sdedit for creating sequence diagrams. Here is an example for the call flow across composed visitors to render a dependency graph as text. Click the image for a better view. See also the source file.

Adjusted the yEd transformation so it renders unconfirmed nodes a different color from the confirmed nodes.

August 03, 2009

Found some references to StackMap in the release notes for JDK 1.6. They point to JSR 202, which itself points to JSR 030 and JSR 139. It appears StackMap is used to simplify class verification by splitting the process between the compiler and the runtime. JSR 139 has an early specification where the attribute is named "StackMap" instead of "StackMapTable". JSR 202 has the full specification, so I will be able to use it to parse "StackMapTable" attributes fully in com.jeantessier.classreader.

July 30, 2009

Had a discussion with Russ Rufer, Tracy Bialik, and Gene Volovich on my recent experiment combining decorator and visitor patterns. Russ recommended that I look at the code for ASM where he saw a way to combine visitors and graph traversal that he liked. This might be inspirational when I try to apply the same model to com.jeantessier.classreader.Visitor.

They also suggested that I could migrate Dependency Finder to ASM. This way, I wouldn't have to keep up with language developments. I vaguely remember taking a glance at ASM some time ago and finding it rather ugly. Maybe it's worth another look, if only to properly document why I don't like it. :-)

Found instructions for switching my MacBook Pro to JDK 1.6.

I was looking at CodeDependencyCollector and I noticed that some of the visit...() methods were not calling their superclass counterparts like the others. They were:

• visitClass_info()
• visitFieldRef_info()
• visitMethodRef_info()
• visitInterfaceMethodRef_info()
• visitExceptionHandler()

I couldn't think of a reason they should not call to super, so I fixed them. They all end up calling to VisitorBase which has all no-ops anyway. Strange, nonetheless. This brings me back to a paper I wrote years ago for my M.Sc. on what I called Inherited Behavior at the time.

Upgraded the following development components:

JUnit

:

4.5

->

4.6

I also noticed that the new SourceForge layout has a large download button on the project's home page and that it points to version 1.1.0-beta2, which is about 5 years old. It took a while to track down how to change it to the latest version, 1.2.1-beta1, and I don't know how long it will take for the change to propagate all the way to the project's home page. I don't know how long it's been this way. The download statistics at SourceForge show nothing for the last week. I hope it's just that their stats are down and not that nobody is downloading Dependency Finder!

July 29, 2009

Refactored SelectiveTraversalStrategy to create an actual ComprehensiveTraversalStrategy to use in defaults instead of an unconfigured SelectiveTraversalStrategy. I figure the indent is more explicit this way.

Changed SelectiveVisitor to reuse the superclass implementations when it is appropriate. This meant changing the tests because the superclass was using double dispatch to pass control to the delegate visitor and the original implementation of SelectiveVisitor was calling the delegate directly.

July 28, 2009

I don't know what I was thinking when I first wrote VisitorDecorator. It was calling the counterpart method on the delegate directly instead of using double dispatch like I originally wanted to. I fixed the tests, mocking more things along the way, and then fixed the code.

Cleaning up the interface, I added traverseInbound() and traverseOutbound() to the Visitor interface. Almost all implementations already had them already and the others ended up being no-ops.

I wrote a BasicTraversal visitor that uses a TraversalStrategy to do some filtering, but I was not happy with it. It was not clear where scope versus filter selections should happen. I felt like I needed a separate class to do the scoping and the filtering, but the composition of decorators would become non-trivial. Finally, I found a way to make it work and I got BasicTraversal and SelectiveVisitor to be just the way I wanted.

I changed my IntelliJ IDEA configuration to use JDK 1.6 on my MacBook Pro. My shell is still stuck on JDK 1.5. I wonder how they will interact, when I happen to compile in IDEA and try to run some tool from the command line.

July 27, 2009

Started down the path of composite visitors. Wrote com.jeantessier.dependency.VisitorDecorator to start composing visitors for dependency graphs.

July 24, 2009

Fixed some tests. Some of the Fit tests needed better explanations. There was one broken test in com.jeantessier.classreader.TestXMLPrinter. This last one was checked in broken, after I made a last minute change to the XML format. I need to be better disciplined at running all the tests before I check in code.

July 23, 2009

Fixed a typo in the web app's style.css.

Finished adding the label to the web app pages, complete with HttpUnit tests.

July 22, 2009

Adding a secondary title to the web app pages, called "label", and which can be set from load.jsp or extract.jsp. The goal is to let automated graph updates tack a label to the current data set, such as a revision number or a timestamp. This lets users know which data their queries are running against.

July 21, 2009

Fixed CodeDependencyCollector to correctly track dependencies on classes and enum values referenced in annotation values.

Found a bug: when CodeDependencyCollector visits the attributes of a class, it attaches dependencies to the last method it visited instead of to the class itself. Luckily, there is a handy hook called VisitorBase.visitClassAnnotations() that lets me intercept control and restore the current node right after visiting fields and methods and before visiting the annotations.

July 15, 2009

Spiked on printing the call graph for closures as a chain of calls instead of a flat dependency graph.

Back on 2008-11-05, I toyed with the idea of using composition to deal with sharing behavior in the com.jeantessier.classreader.Visitor hierarchy. At the time, I was thinking of wrapping a TextPrinter inside a TraversalVisitor, where the outer visitor would be responsible for traversing the structure and would call on the inner visitor for specific processing at each node. This created problems for XMLPrinter which needs to do both pre- and post-processing on each node.

But what if the Printer was the outer visitor and the traversal was done by the inner one? Then, the printer would have free reign in doing all the "around" work it wants. It would call in to the inner visitor when it is ready to traverse a nodes subsequent structure. The inner visitor would still have to call out to the outer one whenever it moves to a new node, though. Reminds me of cross recursion, somehow.

The code for visitMethod_info() would look something like:

    public abstract class VisitorDecorator implements Visitor {
        private Visitor delegate;

        public Visitor getDelegate() {
            return delegate;
        }

        public void setDelegate(Visitor delegate) {
            this.delegate = delegate;
        }
    }

    public class TraversalVisitor extends VisitorDecorator {
        public void visitMethod_info(Method_info method) {
            for(Attribute_info attribute : method.getAttributes()) {
                attribute.accept(getDelegate());
            }
        }
    }

    public abstract class Printer extends VisitorDecorator {
        ...
    }

    public class TextPrinter extends Printer {
        public void visitMethod_info(Method_info method) {
            append("    ").append(method.getDeclaration()).append(";").eol();
            method.accept(getDelegate());
        }
    }

    public class XMLPrinter extends Printer {
        public void visitMethod_info(Method_info method) {
            indent().append("<method>").eol();
            raiseIndent();

            indent().append("<declaration>").append(method.getDeclaration()).append("</declaration>").eol();

            method.accept(getDelegate());

            lowerIndent();
            indent().append("</method>").eol();
        }
    }

Notice how the printers call method.accept(getDelegate()) to get further processing of the current node. The TraversalVisitor instead calls attribute.accept(getDelegate()), moving the processing further along the structure.

The flow of control would constantly pass back and forth between the two visitors. It all pretty much depends on which one starts the ball rolling.

The reason I'm revisiting this issue is because I find myself in need of a com.jeantessier.dependency.Visitor that would traverse the graph depth-first along inbound and outbound dimensions, instead of a the current shallow traversal.

The code to use this would look like:

    Printer printer = new TextPrinter(...);
    Visitor traversal = new TraversalVisitor(...);

    printer.setDelegate(traversal);
    traversal.setDelegate(printer);

    printer.traverseNodes(factory.getPackages().values());

And I could use it to print transitive closures, based on the start criteria.

July 14, 2009

I have been running an experiment to launch Dependency Finder as an instance of Jetty with the web app UI. But Jetty seems flaky. It runs fine at first, but after a while, it starts throwing messages that can cannot find classes that are obviously in the JAR. I don't get it. I've reverted to using Tomcat for now. I removed all the applications in its webapps folder and deployed Dependency Finder as ROOT so I don't need a path element to access it.

July 09, 2009

Found another example of an annotation that takes array and class values even closer to home in org.junit.Test and org.junit.runners.Suite.SuiteClasses. And even better, TestAll has both of these, so all I need to do to find an example is analyze TestAll.

I did a quick spike to try to track annotations as part of com.jeantessier.classreader.MetricsGatherer. Early results are promising, but this was just a spike so I will have to throw away that code and redo it properly, test-first.

July 08, 2009

Found a nice example of an annotation that takes array and class values in org.testng.annotations.Test. Though, looking closer to home, java.lang.annotations.Retention takes an enum value and java.lang.annotations.Target takes an array of enum values. I can play with these manually to validate the XML format for annotation information.

Fixed XML rendering of AnnotationDefault_attribute to remove the extraneous <element-value> node around the actual element value.

July 07, 2009

Played with filtering out files directly from DependencyExtractor. I had forgotten about FileFilteringLoadListener and ClassfileFilteringLoadListener. At first, I thought of using FileFilteringLoadListener to short-circuit even the parsing of unwanted .class files, but that didn't work. It only filters the beginFile() and endFile() events. It does not remove any of the logic that happens between them, nor any beginClassfile() and endClassfile() event.

At the very least, I need to augment the tests for GroupFilteringLoadListener and FileFilteringLoadListener so they at least suppress all events with matching group or file names respectively. Not just their own begin ... end events, but also those that happen in between. After that, maybe I'll consider turning the begin event into something closer to a VetoableChangeListener, where I can suppress processing between match begin and end events based on the handling of the begin event.

The javadocs for LoadListener and LoadEvent are dreadfully out of date.

July 02, 2009

Fixed CodeDependencyCollector to correctly track dependencies on annotations. For now, it only captures dependencies on the annotation itself, not on any element values that it might contain. This will come next.

I implemented support for rendering elements of runtime annotations in XMLPrinter.

I also modified EnumElementValue.getTypeName() and ClassElementValue.getClassInfo() so they convert class names in path notation to a proper class name. Internally, these class names are stored as UTF-8 strings, not ClassInfo_attribute structures, so these classes have to take care of converting strings containing encoded class names such as "Lfoo/Bar;" to proper class names such as "foo.Bar".

Jens Dietrich uses Dependency Finder in his gpl4jung project.

Removed dependencies on toString() when getting the names of elements of Classfile. It is much better to be explicit about which piece of data is needed rather than rely on what an object's string representation is. Also, it is impossible to mock toString() using jMock.

July 01, 2009

Happy Canada Day!

Someone asked why Dependency Finder was not picking up dependencies on runtime annotations. The answer is simple: I haven't implemented that part of Java 1.5 support yet. The code is there to read in annotations from the .class file and include them in the Classfile structure, but I don't do anything with it just yet.

I implemented support for rendering runtime annotations in XMLPrinter. It's a start. I didn't get around to handling all the different types of element values, though.

June 29, 2009

Tweaked the CSS used when converting dependency graphs to HTML (with XSLT).

June 25, 2009

Verified site ownership with Google Webmaster Tools by adding a meta tag on the site's home page.

June 23, 2009

Found out about the Google Visualization API.

June 15, 2009

Changed Classfile to track methods with a simple Collection instead of a Map. This way, bridge methods don't trample each other. But I use a LinkedList internally, so methods are no longer sorted by signature. All the tests are passing for now, but suspect it will impact the logic in OOMetrics and/or JarJarDiff.

I also tried to resurrect the WebWork web app, with limited success. I want to move it squarely into Struts2 instead of clinging to WebWork 2.2.6.

June 11, 2009

One user asked:

    i have a project P that contains in some package Z a class C.
    I want to know all of C's dependencies....

If all you want is C's direct dependencies, you can get them from analyzing the compiled code in C.class. If you want the transitive closure, you sill need to analyze all the classes along the way too. You can point Dependency Finder to a folder and it will scan it and all its subfolders for possible Java code, be it in JAR files, zip files, or loose .class files.

Here is how you would do it using the CLI. Start by extracting all dependencies for project P and saving them in df.xml. You can add third party JARs to the command if you want to drill into their dependencies too.

    % cd [to P's output directory]
    % DependencyExtractor -xml -out df.xml . [add third party JARs too]

Then, reduce all the dependencies in df.xml to class-to-class dependencies and save them to c2c.df.xml.

    % c2c -xml -out c2c.df.xml df.xml

Now, use DependencyClosure to list the first and second degree dependencies from Z.C in the graph in c2c.df.xml. "Inbounds" are classes that depend on Z.C whereas "outbounds" are classes that Z.C depends upon.

    % DependencyClosure c2c.df.xml -class-start-includes /Z.C/ -maximum-inbound-depth 0 -maximum-outbound-depth 1
    % DependencyClosure c2c.df.xml -class-start-includes /Z.C/ -maximum-inbound-depth 0 -maximum-outbound-depth 2

If you want to use the GUI instead, use the "Closure" tab and the File >> Closure menu command. First, use File >> Extract to extract dependencies for project P. Then select the "Closure" tab. In the "Select programming elements" group of controls, check "classes" and put /Z.C/ in the "including" box. In the "Show dependencies (stop for closure), select "classes" but leave the textboxes empty. Set "Follow inbounds" to 0 and set "Follow outbounds" to 1 to first degree or 2 for second degree dependencies. Click File >> Closure and see the results in the large results area.

June 05, 2009

Looking at JavaCC as a potential help in parsing Signature_attribute values. While looking for possible books on JavaCC, I stumbled upon Decompiling Java quite by accident. It came out in 2008, so it might even be up-to-date, but the reviews give me pause.

June 04, 2009

Reading up on Guice 2.0.

Wrapping my head around signatures. I based SignatureHelper on DescriptorHelper, but that may not be enough. The structure of signatures is much more complex and has a recursive nature. I may need a number of classes to handle all the production rules in the JVM specification section 4.4.4.

June 03, 2009

A colleague pointed me to DepAn, another dependency analysis tool for Java. It seems to be built on the Eclipse platform and uses ASM for decompiling Java bytecode. It is distributed as a binary for the Linux platform. It won't work on my Mac and I couldn't get it to start on my Linux box either.

Looking into running DependencyFinder on Google App Engine or Amazon EC2. Users would have to upload their JAR files to the server, though.

May 27, 2009

Found a misnamed method. VisitorBase.visitClassfileAnnotations() should have been called VisitorBase.visitClassfileAttributes().

Worked on Method_info.getSignature() to use the generic signature of a Signature_attribute when it is available. In the process, I realized that I never finished SignatureHelper.

May 26, 2009

Found out about Crap4J, but I'm having a hard time fine tuning the sample build.xml file so I can run it on Dependency Finder. I should add it to the Resources page.

I want to redo ClassList to simply use a LoadListener to collect class names as they are being read in. Coupled with a lightweight Classfile, it would reduce its memory footprint. Right now, it uses an AggregatingClassfileLoader and parses an entire JAR completely before simply writing out class names. This is a waste of time and memory and it hurts when dealing with really large JAR files.

May 18, 2009

Someone found a typo in the documentation: -csv was written -cvs. It's all fixed, now.

I fixed com.jeantessier.metrics.MetricsGatherer to use Classfile.isSynthetic() to determine if a class is synthetic or not. It used to look onl for the presence of a Synthetic_attribute, but in Java 1.5 the synthetic property can be set in the ACCESS_LEVEL field of the .class file. I will need to fix it for methods and fields too.

The deprecation level is computed the same way, by looking for the Deprecated_attribute. But in Java 1.5, it can also be set via an annotation. This will require more work.

May 13, 2009

Refactored CodeDependencyCollector.visitClassfile() so it calls super.visitClassfile(classfile) to traverse the parts of the Classfile instead of doing it explicitly. This means that it will now visit the attributes of the Classfile in addition to its fields and methods. For now, CodeDependencyFinder ignores the attributes that can be attached to a file, but I'm tempted to collect dependencies from attributes such as:

• EnclosingMethod_attribute
• InnerClasses_attribute
• Signature_attribute

EnclosingMethod_attribute is particularly interesting because it would be the first time we have an explicit dependency going from a class to a feature.

I also noticed that com.jeantessier.metrics.MetricsGatherer only looks for a class' Synthetic_attribute and does not look at the class' ACCESS_FLAG. I need to fix this so it calls Classfile.isSynthetic() instead. The same probably goes for other visitors and for other properties of things with attributes.

I have been doing a lot of thinking about large projects that run out of memory to hold the dependency graph. In one case, I have a very large JAR file with all the project classes in it and even if I give DependencyExtractor over 1.5GB of memory, it still runs out. I could add scoping to the extractor, but this feels like piling too many responsibilities on one tool, specially when it is already the purview of DependencyReporter. I could partition the graph across many machines. I can already merge graphs, so the client could ask all partitions (in parallel) and merge the results together. But now I'm facing the problem of how to partition the graph and how to prevent parsing the .class files multiple times. I am exploring the possibility of having lightweight implementations of Classfile and its constituents to help with deciding how to partition the graph, and how this plays with the notions of group and file in a loading session.

May 11, 2009

I've been thinking about how to deal with very large projects that Dependency Finder cannot fit into memory. I could partition the dependency graph between multiple servers, query them in parallel, and merge the results together. I already have the merge logic. Next, I need a way to partition the graph efficiently.

I could do preprocessing of the codebase to determine the partitions automatically. I could use a variation on LCOM4 for this. Well, not really LCOM4, but more of a general graph partitioning algorithm. I could generate a directed package-to-package graph with a weight on each edge that tells how many classes are involved in that edge. Then, I do a normal graph partitioning but I only consider edges with weight greater than N. By varying N, I can find the optimal partitions, including which packages are in each one.

To do the preprocessing, I could simply have a DependencyListener listen in and tally the dependencies. But CodeDependencyCollector, which triggers the listeners, is way too tightly coupled to NodeFactory. I would like to run CodeDependencyCollector without a NodeFactory so it would only generate DependencyEvent messages, and those would have enough information to distinguish between different kinds of dependencies. I could use some kind of non-caching NodeFactory, or rather have some other class create the dependencies so I can have a very lightweight graph in the factory while I just tally information about the dependencies but not the dependencies themselves for this specific analysis.

See also graph partition algorithms at The Stony Brook Algorithm Repository.

May 05, 2009

Fixed com.jeantessier.classreader.TextPrinter so it does not put a ";" after a class' static initialization block.

May 04, 2009

Fixed bug 2663753 which deals with the signature for a class' static initializer.

I wrote a test (using JUnit 4) which exposed that Feature_info.getName() and Feature_info.getSignature() were calling UTF8_info.toString() instead of UTF8_info.getValue(), which tripped the mocks created by jMock. It also forced me to change the signature of the Feature_info constructor to depend on DataInput, an interface, instead of DataInputStream, a concrete class.

Because the new test is a JUnit 4 test, I switched all TestAll suites to use the JUnit 4 mechanism for constructing suites. It lets me compose suites from JUnit 3 and JUnit 4 suites and test, all mixed and matched. I was bring back com.jeantessier.metrics.TestMetricsGatherer into a single test target for all tests. The Fit tests are still in JUnit 3 suites. I will need to take a good look at them to upgrade them too, but I am too tired for this right now.

April 17, 2009

Finally got around to fixing bug 2663688 where com.jeantessier.classreader.TextPrinter might encounter a null Code_attribute on abstract or native methods.

March 31, 2009

Tried to get a simple example of Guice working with Struts2 in Tomcat, but no matter what I tried, I couldn't get it to work. It's like it cannot find my bindings, for some reason. I would really love to use Guice and WebWork (now Struts2) in the webapp.

March 19, 2009

Got a presentation on FindBugs by Bill Pugh, the author himself. It is a really cool tool that uses bytecode analysis, just like Dependency Finder, but in a much more targeted way to find patterns in the code that are likely to be bugs.

March 11, 2009

Refactored the way APIDifferenceStrategy collects field names and method signatures. I extracted a method that takes a function object, a poor cousin to blocks in Ruby. At first, I used an inner interface inside APIDifferenceStrategy and anonymous inner classes at the call sites. But then I thought I might reuse it elsewhere, so I pulled everything out as stand-alone classes in com.jeantessier.classreader: interface FeatureMapper and implementations NameMapper and SignatureMapper. Don't confuse them with PackageMapper which is completely different.

March 10, 2009

Considering moving the behavior from CodeDifferenceStrategy.isCodeDifferent() to Code_attribute.equals() instead. This way, if I introduce a Null Object pattern for Code_attribute, I can make sure that nothing equals the null object. And the code in CodeDifferenceStrategy would become a simple:

    public boolean isCodeDifferent(Code_attribute oldCode, Code_attribute newCode) {
        return oldCode.equals(newCode);
    }

This code assumes that oldCode is not null, or you'd get a NullPointerException. I checked all the call sites for isCodeDifferent() and they all verify that neither oldCode nor newCode is null, so this new version should be perfectly safe.

March 09, 2009

Tried to replace MockDifferenceStrategy with jMock, but some of the tests are too involved with real Classfile instances to make it practical. I will need to rewrite those tests to use mock Classfile and associated structures instead, which will take some time.

March 06, 2009

Minor changes to phrasing in the Tools page.

Added a link to Pivotal Tracker for Dependency Finder as one of the menu items on the site's left-hand panel. I also tried to make the "Download" link stand out more by using a <strong> tag (I wanted to use <b> but IntelliJ IDEA complained that it was deprecated and suggested <strong> instead), but the effect may be too light still.

March 04, 2009

I made the Pivotal Tracker project public so everyone can see how it's doing.

Also, today, I found a bug in ClassReader. In text mode, when it encounters an abstract method, it would throw a NullPointerException. The code in =TextPrinter] blindly calls entry.getCode().accept(this) without checking that getCode() returned something. I could use the Null Object pattern, but I'll have to check everywhere I call getCode() first.

March 02, 2009

Playing with Jetty, I managed to make it serve a complete web app straight from the WAR file. This simple program takes a port number as the first parameter and a WAR file as the second.

    public static void main(String[] args) throws Exception {
        Server server = new Server(Integer.getValue(args[0]));

        WebAppContext webapp = new WebAppContext();
        webapp.setContextPath("/");
        webapp.setWar(args[1]);

        server.setHandler(webapp);

        server.start();
        server.join();
    }

For now, the web app takes its configuration from the web.xml file that is buried in the WAR file. I'll have to figure out a way to pass in the values for the source and file context parameters.

February 24, 2009

Created a project on Pivotal Tracker to try and manage my effort on Dependency Finder. I know a few people at Pivotal Labs and they introduced me to Tracker a few years ago. It's a pretty cool tool, but I'm not sure how useful it will be on a single-person project with a very irregular schedule.

February 19, 2009

I am toying with the idea of using Jetty. With it, I could embed a webserver in the application instead of using Swing. This way, I can provide the user experience entirely from the web application instead of maintaining a separate Swing version.

Documentation on configuring Jetty with an existing web application is sketchy at best. I was hoping for a simple, fool-proof recipe, but it looks like I might have to do actual work to get this to work.

Reading a little more about this, it looks like Jetty already ships as part of WebWork. WebWork's QuickStart uses Jetty to provide a quick starting point for development. This QuickStart may not be the best approach for deploying an application, though.

February 13, 2009

Upgraded the following development components:

IntelliJ IDEA

:

8.0.1

->

8.1

January 26, 2009

Upgraded the following development components:

JUnit

:

4.4

->

4.5

I still haven't started converting all the test cases to JUnit 4.

January 23, 2009

Fixed targets in build.tomcat.xml that were not getting the right paths because they were not depending on the init target.

January 14, 2009

Found a minor bug with ClassReader where it does not reset the internal counter it uses to print a constant pool in text mode. If I ask it to print two classes, the indices of the second class pick up where the first one left off.

I tried to write an integration test first, but there is so much indirection involved between TextVisitor and Classfile that it would have required quite a bit of setup and would have ended up fairy brittle. So I settled for a small unit test of VisitorBase.visitConstantPool() instead.

Fixed a constant duplication in data2atom.cgi where it was referencing Journal.cgi directly instead of the more generic $DOCUMENT.cgi.

January 08, 2009

Updated the copyright notice to 2009.

December 02, 2008

Better escaping of & in Atom feed. Fixed some broken entity references in the Journal.

December 01, 2008

Looked at generating an Atom feed based on the Journal. In the process of turning data2html.cgi into an Atom generator, I found a bug in data2html.cgi when generating links to dated entries.

November 27, 2008

Upgraded the following development components:

IntelliJ IDEA

:

8.0

->

8.0.1

Added CodeFinder to replace the explicit iteration in Method_info.getCode().

November 26, 2008

Cleaned up how XMPrinter deals with instructions that have a value embedded in them, such as bipush and iconst_1. Also made it show the destination address of goto and jump instructions.

Augmented TextPrinter so it also shows the bytecode, similar to what javap -c does, including the exception handling.

As I was looking at coverage reports to make sure I covered all the cases in printing instructions, I was reminded that some ExceptionHandler structures do not have a catch type, meaning they trap everything and apparently that is the bytecode presentation of finally blocks.

I also realized that I was reading offset values for goto and branch instructions as unsigned values, when they are actually signed. The same goes for embedded values in bipush and sipush instructions.

November 25, 2008

Fixed the use of id attributes in classfile.dtd to use index instead. The attribute id has special meaning in XML.

I realized that local variables can be active for only a portion of a method's execution and that the VM can reuse positions in the local variable table for different variables whose scope does not overlap. That significantly complicated LocalVariableFinder. It had to take into account the position of the instructions trying to locate a local variable and what local variables are active at that position in the code. To complicate matters further, instructions that set local variables, such as istore, are such that the variable does not really exist at that instruction position but starts immediately after. Couple that with the wide instruction and things get pretty nasty pretty quickly.

Minor refactoring.

November 24, 2008

Added Instruction.getIndexedLocalVariable(), Instruction.getOffset(), and Instruction.getValue() to help with text rendering of instructions. Used them to make traces in Code_attribute and instruction data from XMLPrinter more useful.

Simplified Instruction tests by using jMock and putting many of the tests methods in one test class.

Bad news for Dependency Finder: I just got Tomb Raider: Underworld! >:-)

November 20, 2008

Wrote LocalVariableFinder to help find the LocalVariable of a Code_attribute from its index in an Instruction.

Removed getCode() from the Instruction interface. The implementation can still hold on to the Code_attribute, but that's an implementation detail, and does not need to be part of the exposed interface.

November 19, 2008

Modified journal entries to turn dates into hyperlinks.

Extracted the implicit index from instructions xload_<i> and xstore_<i> to be the value of <i>. Those values point to entries in the local variable array of the current frame. I could use them to provide better text rendering of instructions by naming the variable (when there is a LocalVariableTable present in the Code_attribute).

November 18, 2008

Modified data2html.cgi to render dates as URLs to the corresponding journal entry.

November 14, 2008

Simplified build.test.xml by grouping all JUnit tests under one target. With junit-4.4.jar in $HOME/.ant/lib, it can run JUnit 4 tests too. com.jeantessier.metrics.TestMetricsGatherer is a JUnit 4 test and it cannot run as part of a JUnit 3 test suite.

November 13, 2008

Back on 2008-11-03, I simplified some constructors at the cost of less useful debug traces. I was looking at traces from ClassMetrics to see how Dependency Finder was dealing with annotations in the wild and I realized that echoing only constant pool indices is not very helpful. I really need to look up those values in the constant pool and include them in the trace message.

Found a bug! It turns out that the constructor for AnnotationElementValue was not passing the ConstantPool to its Annotation. Now that the constructor for Annotation calls getType() for its debug trace, it forces a constant pool look up, which exposed the problem. Hurray for tests!

Since Java 1.2, you can deprecate a programming element with the javadoc tag @deprecated. It adds a Deprecated attribute to the programming element. Since Java 1.5, you can deprecate a programming element with the runtime annotation @Deprecated. I wrote a special Visitor, named DeprecationDetector, that traverses the Classfile structure looking for either one. I will use it to rewrite Deprecatable.isDeprecated() in Classfile and Feature_info. This is part of a larger effort to reduce dependencies by parts of the Classfile structure upon their containing structures.

As I was working on DeprecationDetector, I realized that Annotation did not expose its type property through its interface. It is odd that I did not notice it earlier. And, I must admit, a bit scary. What else might I have missed?

As I was trying ClassReader -xml, I realized that the elements of the constant pool have an attribute id that gives their position in the pool. This is unfortunate since id has a very specific meaning in XML. I should definitely think of changing it to something more like index.

November 11, 2008

Upgraded the following development components:

IntelliJ IDEA

:

8.0 Milestone 1

->

8.0

November 05, 2008

Added visitor hooks for the substructures in *Annotations_attribute.

I've been thinking to move the com.jeantessier.classreader.Visitor hierarchy to use more composition. I could have one class handle traversal of the Classfile structure and delegate processing to another class that would just focus on what needs to be done.

For instance, the visitMethod_info() methods would look like:

    public class VisitorDecorator implements Visitor {
        public void visitMethod_info(Method_info method) {
            delegate.visitMethod_info(method);
        }
    }

    public class TraversalVisitor extends VisitorDecorator {
        public void visitMethod_info(Method_info method) {
            super.visitMethod_info(method);
            for(Attribute_info attribute : method.getAttributes()) {
                attribute.accept(this);
            }
        }
    }

    public class TextPrinter extends Printer {
        public void visitMethod_info(Method_info method) {
            append("    ").append(method.getDeclaration()).append(";").eol();
        }
    }

But it does not work for XmlPrinter, which must generate output both before and after processing the method's attributes.

    public class XmlPrinter extends Printer {
        public void visitMethod_info(Method_info method) {
            append("    <method>").eol();
            append("        <declaration>").append(method.getDeclaration()).append("</declaration>").eol();
            // Attributes go here, somehow
            append("    </method>").eol();
        }
    }

I could have hooks for pre- and post-traversal, but that's what led to all the complexity in com.jeantessier.dependency.VisitorBase. So, I'm going to stick with the inheritance-based design:

Now, the code looks like this:

    public class VisitorBase implements Visitor {
        public void visitMethod_info(Method_info method) {
            for(Attribute_info attribute : method.getAttributes()) {
                attribute.accept(this);
            }
        }
    }

    public class TextPrinter extends Printer {
        public void visitMethod_info(Method_info method) {
            append("    ").append(method.getDeclaration()).append(";").eol();
            super.visitMethod_info(method);
        }
    }

and for XmlPrinter:

    public class XmlPrinter extends Printer {
        public void visitMethod_info(Method_info method) {
            append("    <method>").eol();
            append("        <declaration>").append(method.getDeclaration()).append("</declaration>").eol();
            super.visitMethod_info(method);
            append("    </method>").eol();
        }
    }

Maybe I should revisit com.jeantessier.dependency.VisitorBase, now.

November 04, 2008

Fixed a problem with IntelliJ IDEA. I am using version 8.0M1 on a MacBook Pro. My laptop wouldn't wake up after going to sleep yesterday, so I had to do a hard reboot. After that, IntelliJ wouldn't recompile my classes. It kept clearing the classes directory and then couldn't find classes to run the tests. I completely erased ~/Library/Caches/IntelliJIDEA80 and restarted IntelliJ and that seemed to do the trick.

Added tests for Custom_attribute.

Added visitor hooks for *Annotations_attribute and AnnotationDefault_attribute, but I did not get to their substructures, yet.

November 03, 2008

Removed ConstantPool from the public interface of *Annotations_attribute components. The fact that some of them hold on to the constant pool and do lookup at runtime is an implementation detail.

Fixed Parameter so it actually reads its annotations from the byte stream. I also simplified its interface to remove the back reference to RuntimeParameterAnnotations_attribute.

Added getAttributeName() to all attributes. I had to simplify some of the constructors that were echoing the data as they read it in order to keep the tests simple.

• ConstantValue_attribute
• EnclosingMethod_attribute
• Signature_attribute
• SourceFile_attribute

But after trying it out on some sample code, I realized that it hampers usability, so I will have to put them back in. I also removed dependence on the behavior of toString() in a few places.

Added AnnotationDefault_attribute.

November 02, 2008

Exclude abstract base classes for test classes from Ant target to run all tests.

October 31, 2008

More support for annotations.

Removed the back-reference from Annotation back to its containing RuntimeAnnotations_attribute because it did not make sense in the case of an annotation as an element value.

Rigged ElementValueFactory the same way I did for AttributeFactory.

I realized that most attributes hold on to a reference to the Classfile only so that they can get the ConstantPool from it. So I changed the code to inject the ConstantPool directly instead of the Classfile. This refactoring touched 84 files.

October 30, 2008

More support for annotations.

I've built support for some of the annotation attributes and pretty much all types of element value. Now, I need to figure out how to instantiate element values based on the tag in the byte stream. Once I have that, I will be able to finish ArrayElementValue and write ElementValuePair so I can then finish Annotation. Then, I will have to rig it all up into com.jeantessier.classreader.Visitor.

Added some tests for some structures that were not covered before:

• InterfaceMethodRef_info
• Deprecated_attribute
• Synthetic_attribute
• SourceDebugExtension_attribute

jMock is really helping create a stage where I can exercise the classes very easily.

October 29, 2008

More support for annotations. The way the class hierarchy in com.jeantessier.classreader.impl shadows the interface hierarchy in com.jeantessier.classreader is very tricky.

October 28, 2008

Started writing support for annotations, TDD-style. I found a cool way to use jMock to mock the input stream from which data structures are read. This way, I don't have to create an arbitrary bytestream, all I have to do is mock the calls to readInt() and so forth on the data stream.

While I was trying to mock a DataInputStream, in which all methods are final, I found out about the DataInput interface, which is much better (and I can mock it, no problems).

October 24, 2008

Paired with Nicolas Wettstein to add filtering capability for LoadEvent listeners. It was nice to pair program with someone on Dependency Finder for a change.

The hierarchy looks like this:

This will help with filtering certain classes based on regular expressions. The filters extend the decorator so they can be composed with other LoadListener implementations, such as LoadListenerVisitorAdaptor, for example.

October 23, 2008

Removed redundant field com.jeantessier.metrics.MetricsGatherer.projectName and associated getter and constructor parameter. The MetricsFactory inside MetricsGatherer already has the project name. Nobody was using this field on MetricsGatherer.

October 18, 2008

Upgraded the following development components:

Testability-Explorer

:

1.10RC1-r34

->

1.2.0-r85

I have to rewrite the <testability> tasks in build.testability-explorer.xml to bring it in line with their latest Ant task.

October 10, 2008

Worked on logging from the webapp. I instrumented extract.jsp and load.jsp to log progress both to Log4J and to the servlet container logs. But maybe this logs too much data, so I will not submit this change just yet.

October 08, 2008

Cleaned up the command-line validation in Fit tests.

I was hesitating between check and reject and ensure rows, but I came up with a system. I use ensure rows to show good, valid command-lines and reject to show broken command-lines. I ended up using check for switches -echo, -help, and -version because while they are allowed command-lines, they do not lead to running the application but short-circuit it instead.

I've been having trouble with OOMetrics lately. When I use the -csv switch and save the output with the -out switch, the value of the -out switch is taken as the prefix for the multiple output files, one for each of project, group, class, and method metrics. Since the output stream is opened by the Command superclass, it doesn't know that it has a special meaning in OOMetrics and creates a file named after the prefix during application startup. But that file is never used and ends up being empty.

I encapsulated Command.out behind a getter and a setter so I can do lazy initialization on it. This way, since OOMetrics never calls Command.getOut(), the empty file is never created.

Amended the tech docs to add Google Docs to the list of possible consumers for CSV files.

October 03, 2008

Found a way to update the project website on SourceForge. I cannot SSH into SourceForge anymore, but I could upload files with SFTP. This is annoying. I was used to updating the project webpage directly from Subversion. I remember a long time ago when I had it as a cron job that ran automatically at 3am. Now, I need to maintain a client and then upload files to the site. Some of the documentation on SourceForge about the new tools for administrators hint at a new cron-like service, so maybe there is still hope.

September 18, 2008

Added JarJarDiff report for jMock to the Samples area of the website.

September 17, 2008

Released version 1.2.1-beta1. It's been long overdue and I had to update the instructions on making a release.

September 15, 2008

A user contacted me asking how to collect the transitive closure of classes from some starting point so they could build a single JAR file with only what's needed. Here's my recipe.

Suppose you want to package the tools for the CLI together with all the classes they require, but no other. First, get the total dependency graph for all the JAR files in lib/ and save the graph somewhere convenient:

    DependencyExtractor -xml -out dependencies.xml lib

Next, compute the outbound transitive closure from the com.jeantessier.dependencyfinder.cli package:

    DependencyClosure -xml -out closure.xml -start-includes /cli/ dependencies.xml

Next, reduce the closure to classes only, eliminating java.* and pseudo-classes like array types. This step also removes the dependencies, since we don't really need them anymore.

    c2c -scope-excludes /^java/ -scope-excludes /\\[\\]/ -filter-excludes // closure.xml -xml -out closure.c2c.xml

Finally, extract the class names from the output from the previous command.

    DependencyGraphToFullyQualifiedNames -in closure.c2c.xml -out classes.txt

Replace lib with your own JARs, and /cli/ with your own starting point, and you should be well on your way.

Bear in mind the tricky behavior around computing closures. So you might want to reduce the dependency graph to class-to-class dependencies before you compute the closure.

September 10, 2008

Used the new jMock scripting feature for custom actions in TestCodeDependencyCollectorUsingMocks.

Added named anchors to the entries in the Journal so I can create hyperlinks to specific entries.

September 09, 2008

Upgraded the following development components:

jMock

:

2.4.0

->

2.5.1

jMock's a() matcher is now deprecated, so I used any() instead.

September 02, 2008

Spiked to try out SLOC2, based on the minimum and maximum line numbers in a method and in a class. Running it on Dependency Finder, it yields estimates that are about 60% higher than SLOC. This is due in part to blank lines and comments, which it counts. Interestingly, it finds nothing for interfaces and abstract methods.

In the process, I got the idea to split the different concerns in StatisticalMeasurement: collecting a bunch of values and running statistical analysis on those values. To compute SLOC2, I needed a collection measurement where I could collect the different line numbers and get the minimum and maximum values. I used a SumMeasurement to compute max - min + 1. With separate measurements for collecting values and doing statistics, it would make the metrics config even more complex, so it may not be a win for the user.

August 27, 2008

I was using ListDiff and ListDiffToHTML for diff'ing lists of symbols when I noticed ListDiffToHTML.xsl and ListDiffToText.xsl have template text made explicitly for diff'ing published documentation lists. I will have to move that text to another name and make these templates more generic.

I also have problems, under Linux, passing multi-word project names into ListDiff. Somewhere between all the shell script invocations, I lose the fact that more than one word makes a switch value and the tool ends up only using the first word in the name. I have the same problem with the other tools that take an arbitrary project name, such as OOMetrics.

August 26, 2008

Fixed tech docs for OOMetrics.

August 22, 2008

Looked into the possibility of estimating SLOC by looking at the minimum and maximum values in LineNumberTable_attribute instances in a given class. One problem I have, and this is true for the current SLOC calculation too, is that fields that are initialized in initialization blocks (including constructors) are counted twice. Also, looking at overall minimum line number versus maximum line number takes into account vertical space as well and comments.

August 17, 2008

Spiked to adjust CodeDependencyCollector to filter parents in the dependency graph too. If a parent does not match the filter criteria, it is not included in the graph. I'm not sure this is the right thing to do. What triggered the spike is that the tests around the filtering in CodeDependencyCollector were broken when I started extracting inheritance information. The dependency on the parent gets filtered correctly, but there is still an empty node for the parent. I could leave it there to preserve the parent-child relationships or I could leave it out because the filter expresses an intent to ignore it.

August 15, 2008

Fixed ListDiff so the -old-label switch defaults to the value of the -old switch, the way it does for JarJarDiff. And the same for -new-label and -new. Added tests for the Ant task versions of ListDiff and JarJarDiff

Refactored the changes to the CLI version of JarJarDiff by pulling them up to the base class DiffCommand so they can be shared with ClassClassDiff.

Fixed tech docs accordingly.

August 13, 2008

Upgraded the following development components:

IntelliJ IDEA

:

7.0.3

->

8.0 Milestone 1

Added tests to CodeDependencyCollector to capture that exception handling generates a feature-to-class dependency.

For a moment, I had the crazy idea that I could use raw class-to-class dependencies instead of storing inheritance information separately in the dependency graph. By inheritance, I mean both generalization (extends) and realization (implements). But that is not going to work because there could be other reasons besides inheritance to have class-to-class dependencies, such as collapsing the graph with DependencyReporter -c2c.

August 12, 2008

Changed FeatureResolver so it is no longer limited to unconfirmed features but handles all overwritten features.

August 11, 2008

First pass at FeatureResolver. Added functionality to ClassNode and FeatureNode to navigate the inheritance hierarchy to resolve feature names.

Fixed a typo in the Fit tests for LinkMaximizer and LinkMinimizer, but checked them in the same commit as the FeatureResolver work by accident. Oh! and I had the wrong commit message for that, to make matters worse.

August 10, 2008

Added Fit tests for LinkMaximizer and LinkMinimizer. This is in preparation to writing a Visitor that attempts to resolve unconfirmed nodes in the dependency graph by walking the inheritance hierarchy.

August 08, 2008

Store class hierarchy information in the dependency graph as part of CodeDependencyCollector.

August 07, 2008

Upgraded the following development components:

Log4J

:

1.2.14

->

1.2.15

August 06, 2008

Extracted the regular expression parsing logic out of RegularExpressionSelectionCriteria and put it in its own class, RegularExpressionParser, so I can reuse it the Ant version of ListSymbols.

Added FilteringSymbolGatheringStrategy to the Ant task version of ListSymbols.

August 05, 2008

More clamping down on non-private fields, this time in MetricsReport.

Fixed some broken URLs for images in the Journal.

Still can't update the website because SourceForge's Shell service still cannot talk to the Subversion service. Grrr!

August 04, 2008

Fixed TestSymbolGathererStrategyDecorator so it tests the right class, and not one of its descendants.

The fields of MaximumCapacityPatternCache were package-level instead of being private for some reason. I suspect that a manual refactoring was the cause. I made them private and neither the compiler nor the tests complained about it.

Finished FilteringSymbolGathererStrategy, including tests, and used it in the CLI version of ListSymbols.

Normalized the names of tests for SymbolGathererStrategy implementations.

Clamped down on non-private fields, mostly in test classes. I used the new functionality in ListSymbols.

    $ ant -f build.test.xml compile
    ...
    $ bin/ListSymbols -non-private-field-names classes
    ...

There were a few in test code, mostly oversights from promoting local variables to fields manually. There are still a few public fields in the code to support Fit test.

August 01, 2008

Updated documentation for the ListSymbols Ant task.

Added FilteringSymbolGathererStrategy to support inclusion and exclusion regular expressions to ListSymbols.

July 31, 2008

Documented the new switches to ListSymbols on the the Tools page.

Fixed tests broken by new switches to ListSymbols.

Streamlined the building of CLASSPATH in junit and textjunit launcher scripts under Unix.

Switched textjunit to use the normal JUnit 4.4 test runner instead of the old junit.textui.TestRunner.

I noticed that the graphical UI launched by junit is now broken. It has most likely been broken since I switched to the JUnit 4.4 binaries back on 2008-06-09 but I had not noticed. I might get rid of the graphical launcher altogether if I cannot find a suitable replacement to junit.swingui.TestRunner.

Brought the ListSymbols Ant task to feature parity with its CLI version.

Started to create tests for the Ant version of ListSymbols, but the Ant code is not very testable. The Ant tasks have many trivial accessors that I do not want to waste time testing. The interesting business logic is embedded in the execute() method and I have to pull it out before I can write tests against it.

The project's website is in SourceForge's Subversion repository and I update it from SourceForge's Shell service. I noticed today that the Shell service couldn't talk to the Subversion service for some reason. Someone else is having the same problem, so at least it's not just me. Apparently, SourceForge is in the process of moving its infrastructure, so maybe it's related somehow.

July 28, 2008

Worked with the CLI ListSymbols to list final methods and classes. I use this to locate areas in the code that need special attention. I spent some time around InnerClass to categorize types of inner classes: member classes, anonymous inner classes, local classes. I still have to update the Ant task and update the documentation.

Found interesting descriptions of the various types of inner classes on Gamelan.

July 23, 2008

Back from a lengthy vacation break.

Found and fixed a bug in load.jsp where it was setting the load timestamp and duration in the application context using one name and reading it back using another name. A reminder of the dangers of using explicit strings to save and lookup values.

July 02, 2008

Fixed a bunch of URLs in the Journal.

July 01, 2008

Cleaned up straggling -all switches in sample code in the documentation.

Fixed some of the tools so they can read from standard input:

• c2c
• c2p
• ClassCohesion
• DependencyClosure
• DependencyCycles
• DependencyMetrics
• DependencyReporter
• f2f
• p2p

This way, I can chain them on a command-line so the output of one becomes the input of the next one, just like typical Unix tools.

Found a bug with com.jeantessier.metrics.MetricsGatherer where it is not accounting for final classes correctly. Filed it as bug 2008537 Fixed it and added more tests.

While I was testing MetricsGatherer, I noticed some measurements are being gathered both at the project and at the group level separately. I wanted to remove the duplication because I could compute the one at the project level from submetrics at the group level. This would work as long as groups matched packages exactly. But it would breakdown if someone used group specifications to add groups besides packages. This is why there is this duplication.

June 30, 2008

Added a CLI tool for computing class cohesion as per the LCOM4 metric. The current output is still a work in progress. It will need to ignore unconfirmed classes and possibly sort the output by LCOM4 value, both ascending and descending.

June 27, 2008

Fixed LCOM4Gatherer so it ignores constructors, as per LCOM4. I used Perl5Util to match the feature name to a pattern, but I'm debating using java.util.regex instead.

Code with Perl5Util:

    private static final Perl5Util perl = new Perl5Util();
     
    private boolean isConstructor(FeatureNode node) {
        return perl.match("/(\\w+)\\.\\1\\(/", node.getName());
    }

Code with java.util.regex:

    private static final Pattern CONSTRUCTOR_PATTERN = Pattern.compile("(\\w+)\\.\\1\\(");
     
    private boolean isConstructor(FeatureNode node) {
        return CONSTRUCTOR_PATTERN.matcher(node.getName()).find();
    }

For this check, I find the Perl5Util code easier to work with, and I already use Perl5Util everywhere. But I feel java.util.regex might actually be more efficient by using a native implementation in the JVM itself.

Maybe this logic belongs in FeatureNode instead of being replicated in a lot of places. I guess I answered my own question.

Changed the signature of TraversalStrategy.order() so it returns the exact same type that was passed in, using generics.

June 26, 2008

Found a better way to test for components from LCOM4Gatherer that is not order-sensitive. Right now, the components are stored in a HashSet, so the iteration order during tests would change depending on the implementation of HashSet. The previous checks in the tests were assuming a specific order, so the tests were brittle.

June 25, 2008

First attempt at computing LCOM4 from the dependency graph. Right now, the data structure to store components of a class is simply Collection<Collection<FeatureNode>> as I don't really have any metadata that is not represented just as well by Collection.

June 20, 2008

Added support for inheritance data in the dependency graph. ClassNode will point to its parents (classes and interfaces) and children. Other tools will be able to use this information to navigate the inheritance tree when trying to resolve dependencies.

June 19, 2008

Looked at capturing class hierarchy information in the dependency graph as part of CodeDependencyCollector. It needs to keep track of the superclass as well as interfaces the class implements, but maybe it does not need to distinguish between the two.

June 18, 2008

Found a problem with my list of non-final fields and the methods that depend on them. A field may be protected and used by a subclass, but Dependency Finder will not flag it.

For example:

    public class Superclass {
        protected int count;
    }

    public class Derived extends Superclass {
        public void increment() {
            count++;
        }
    }

yields this dependency graph:

     
        Derived
            --> Superclass
            Derived()
                --> Superclass.Superclass()
            count *
                <-- Derived.increment()
            increment()
                --> Derived.count *
        Superclass
            <-- Derived
            --> java.lang.Object *
            Superclass()
                <-- Derived.Derived()
                --> java.lang.Object.Object() *
            count

Here, Superclass.count is clearly a non-private field. But Derived.increment() depends on Derived.count, not Superclass.count. The dependency graph does not contain information about the class hierarchy, so there is no way to discover that Derived.count is really Superclass.count.

June 16, 2008

Refactored SymbolGatherer to use a Strategy pattern when deciding which symbols to gather or not. I want to write a custom strategy for collecting non-private fields, which I couldn't do with the code before I extracted the decision-making process into another object. I'm debating if SymbolGatherer shouldn't use a collection of strategies instead, collecting symbols that match at least one strategy.

Created a custom SymbolGathererStrategy for detecting non-private, non-static, non-synthetic fields.

Added <synthetic> tags to fields, methods, and inner classes in the XML representation of Classfile.

Refactored com.jeantessier.commandline.TestAliasSwitch and com.jeantessier.classreader.TestVisitorBase to use jMock instead of custom mocks. I was able to get 100% code coverage for com.jeantessier.classreader.VisitorBase. I was also able to get rid of com.jeantessier.commandline.MockVisitor.

With this in place, I was able to used Dependency Finder on another project to find dependencies on non-private fields.

    $ DependencyExtractor -xml -out df.xml [JARs ...]
    $ ListSymbols -non-private-field-names -out names.txt [JARs ...]
    $ DependencyReporter -scope-includes-list names.txt -class-filter -show-inbounds -show-empty-nodes df.xml

I could then use these results to track where one class accesses the fields of another class and replace them with accessors.

June 12, 2008

Wrote an Ant build file for running Testability Explorer against Dependency Finder. I know the guys who work on it and it has some interesting ideas about testability behind it. I might start publishing its report along-side the coverage reports with the next release of Dependency Finder.

June 09, 2008

Upgraded the following development components:

JUnit

:

3.8.2

->

4.4

I haven't changed any of the test classes yet. I just upgraded the JAR file that I use in compilation. Everything should still work with older versions of JUnit, except junit.bat and textjunit.bat which refer to the JAR file by name explicitly.

June 06, 2008

Found a way to run my tests with JUnit 4.4 from the command-line. I found [[http://www.devx.com/Java/Article/31983/0/page/3][this article on JUnit4] on the web and modified textjunit accordingly, and it worked just fine. Essentially, it is this command-line:

    java org.junit.runner.JUnitCore $*

With a CLASSPATH that has all the right JARs on it.

June 04, 2008

Tried to compile and run the tests with JDK 1.6. Some of the tests fail because of the new attribute StackMapTable, but other tests in com.jeantessier.diff were also failing. It turns out that APIDifferenceStrategy uses HashMap internally to accumulate symbols when looking for differences. The iteration order is different between JDK 1.5 and JDK 1.6, which results in the strategy using different code paths to decide that two classes are different. This throws off MockDifferenceStrategy and therefore breaks the test. APIDifferenceStrategy could use a TreeMap instead, which would guarantee the iteration order. While I'm at it, I should probably re-write those tests to use jMock instead of a hand-rolled mock and data read from disk.

May 30, 2008

Replaced the explicit name of CLI commands, set in the constructor, with a call to Class.getSimpleName(). This allowed me to get rid of all constructors across the entire Command hierarchy. For some obscure reason, Commnad.Command(name) declared throwing a CommandLineException, but none of the constructors did anything other than set the command's name. It must have been some leftover from ancient times. I removed cruft, yeah!

May 28, 2008

Extracted the default indent text of "    " (four spaces) to constants on the various Printer classes. I can now use these constants to set the default value for all the -indent-text switches.

May 02, 2008

Wrote a better attempt at distinguishing inner classes of the current class from mere references to inner classes of other classes. Wrote more tests for inner classes.

April 29, 2008

In an effort to better understand the InnerClass structure in .class files, I wrote a quick XSL transform that extract just their content from the XML output of ClassReader. It's still a mess, though.

April 28, 2008

More test for MetricsGatherer using jMock in TestMetricsGathererAccumulators, trying to cover InnerClass. But the logic surrounding InnerClasses attributes and their InnerClass structure is pretty convoluted. Each inner class has seems to have an InnerClass structure referencing itself. In enum members, they also seem to reference the first value in the enum. Then, the enclosing class has an InnerClass structure for each inner class in contains. It is proving difficult to understand what I am looking at when processing an arbitrary InnerClass structure.

April 27, 2008

More test for MetricsGatherer using jMock in TestMetricsGathererAccumulators, covering Synthetic_attribute on classes and Deprecated_attribute on classes, fields, and methods.

April 26, 2008

Started rewriting TestMetricsGathererAccumulators using jMock. I got through dealing with fields and most aspects of methods, so far.

April 25, 2008

Started modifying MetricsGatherer in a test-driven way. I will need to use jMock if I want to get through this one. I started with some precompiled classes, but there is just too much noise. Plus, the ones I have so far don't exercise some of the corner cases that interest me.

April 24, 2008

Looked at a subset of metrics to compare how many final elements are in a given codebase. I will need to list elements (fields, methods, etc.) in order to make sense of it all. Right now, MetricsGatherer simply keeps counters for all of these measurements and simply increments them by 1 as it traverses the code. I will need to change MetricsGatherer to send the name along to the measurements so I can start using NameListMeasurement to get a list of element names.

April 18, 2008

I had to change my SourceForge password because I had forgotten it. A little while ago, my password expired on SourceForge and I had to pick a new one. Now, it's been a few weeks since I last used it and I couldn't recall what it was. So I changed it again to something new. Hopefully, I will remember it this time.

April 17, 2008

Figured out how to log from JSPs with Log4J while limiting the log contents to only the JSP code, not all of Dependency Finder. My conversion patterns in log4j.properties have "%c{2}", limiting class names to their last two components. This was keeping me from seeing the fully qualified class names for the generated JSP classes, and therefore I couldn't define loggers with the right filtering level. I took out the "{2}" momentarily, long enough to see the full package name (org.apache.jsp), and put it back along with a new logger definition.

April 16, 2008

Played with better logging in the webapp. Right now, the application does not log anything. I cannot get any usage statistics from Tomcat's logs, since they are essentially empty. In a JSP, I can use applicationContext.log(), but it writes no information about the context of the call. Each page would have to identify itself in the message it's trying to log. I tried to use Log4J instead, but I'm struggling with how to specify a package that only gets the JSPs and not all of Dependency Finder.

April 10, 2008

Updated documentation for OOMetricsGUI to account for new -enable-cross-class-measurements switch.

April 08, 2008

Modified OOMetricsGUI so it can use a TransientClassfileLoader to process large code bases that don't all fit in memory. This is to bring it to feature parity with the CLI equivalent.

April 04, 2008

Modified the Ant task for OOMetrics so it can use a TransientClassfileLoader to process large code bases that don't all fit in memory. This is to bring it to feature parity with the CLI equivalent.

April 03, 2008

Modified the CLI OOMetrics so it can use a TransientClassfileLoader to process large code bases that don't all fit in memory.

March 26, 2008

Fixed the Fit test for ClassFinder. A colleague suggested that rather than getting Fit to parse four spaces as four spaces for the expected value, I try rewriting the actual value in the fixture instead. Now it works fine, even if it is kind of an ugly hack. At first, I tried to have the replacement text as "<em>four spaces</em>" so that it would render in the Fit HTML page as "four spaces". But that didn't work on MacOS for some reason. So I switched to the less pretty but also less controversial "****four spaces****" instead.

March 24, 2008

Fixed some Fit tests that were missing the fit.Summary at the end.

Looked further into why Fit ignores a cell with only spaces in it. This is causing the Fit tests for ClassFinder to fail because the -indent-text switch has " " (four spaces) as its default value and Fit keeps thinking the expected cell value is the empty string. It turns out that somewhere deep inside Fit, it decides to use a ResultValueAdapterByStringCompare for this column (getDefaultValue() returns Object), and it uses fit.Parse.text() to parse the table cell, and text() calls String.trim() before returning its value. Darn it! Now, I'll have to find a way to supply a custom ValueAdapter instead that knows how to keep a cell value as explicit spaces.

Simplified CommandFixture.switches(). It does not need to wrap its return value in an ArrayFixture, FitLibrary will take care of it for you.

March 20, 2008

Added -compact and -indent-text switches to ClassFinder. -indent-text has four spaces as its default value, and it's throwing off the Fit test.

Changed ClassMatcher so that it uses the group information from the endClassfile event instead of relying on the group stack maintained by LoadListenerBase.

Back on 2006-03-13, I had put up "Related links" banners powered by Google to some of the pages. They stopped working a while back and they were only showing an empty box. But today, I noticed that they started saying "Error". So I took them down.

March 12, 2008

Upgraded the following development components:

IntelliJ IDEA

:

7.0.2

->

7.0.3

March 05, 2008

There are only a few dependencies from com.jeantessier.classreader to com.jeantessier.classreader.impl:

    com.jeantessier.classreader
        AggregatingClassfileLoader
            AggregatingClassfileLoader()
                --> com.jeantessier.classreader.impl.DefaultClassfileFactory.DefaultClassfileFactory()
            AggregatingClassfileLoader(com.jeantessier.classreader.ClassfileLoaderDispatcher)
                --> com.jeantessier.classreader.impl.DefaultClassfileFactory.DefaultClassfileFactory()
        TransientClassfileLoader
            TransientClassfileLoader()
                --> com.jeantessier.classreader.impl.DefaultClassfileFactory.DefaultClassfileFactory()
            TransientClassfileLoader(com.jeantessier.classreader.ClassfileLoaderDispatcher)
                --> com.jeantessier.classreader.impl.DefaultClassfileFactory.DefaultClassfileFactory()

These constructors use concrete classes for the default value of parameters. They should let the caller decide what to inject. Then, the caller is free to hardwire specific values or use a dependency injection framework to supply them to the loader.

Looking at production code, I see the following instantiators of these loaders:

    com.jeantessier.classreader
        AggregatingClassfileLoader
            AggregatingClassfileLoader()
                <-- com.jeantessier.dependencyfinder.ant
                <-- com.jeantessier.dependencyfinder.cli
                <-- com.jeantessier.dependencyfinder.gui
            AggregatingClassfileLoader(com.jeantessier.classreader.ClassfileFactory)
            AggregatingClassfileLoader(com.jeantessier.classreader.ClassfileFactory, com.jeantessier.classreader.ClassfileLoaderDispatcher)
            AggregatingClassfileLoader(com.jeantessier.classreader.ClassfileLoaderDispatcher)
        TransientClassfileLoader
            TransientClassfileLoader()
                <-- com.jeantessier.dependencyfinder.ant
                <-- com.jeantessier.dependencyfinder.cli
            TransientClassfileLoader(com.jeantessier.classreader.ClassfileFactory)
            TransientClassfileLoader(com.jeantessier.classreader.ClassfileFactory, com.jeantessier.classreader.ClassfileLoaderDispatcher)
            TransientClassfileLoader(com.jeantessier.classreader.ClassfileLoaderDispatcher)
                <-- com.jeantessier.dependencyfinder.gui
                <-- com.jeantessier.dependencyfinder.webwork

But a lot of the tests call them too. This is not going to be trivial.

March 03, 2008

Tried to generate GraphML where class-in-package and feature-in-class relationships were modeled using subgraphs instead of simple edges, but the resulting graph didn't look too good in yEd.

For example, take this simple dependency graph:

    p1
        C1
            f1
                --> p2.C2.f2
    p2
        C2
            f2
                <-- p1.C1.f1

Using edges to represent containment, I get this picture:

Whereas using subgraphs, I get this picture:

Clearly, this second form will not scale to classes with many features, or to packages with many classes.

A co-worker asked for means to determine what JARs are needed by a given JAR. While Dependency Finder does not store the source of a class, such as its JAR file, there are ways to get close to the answer. You can get the dependency graph for a JAR with DependencyExtractor, roll it up to package-level dependencies with p2p, and then list the unconfirmed packages from p2p's output. Dependency Finder doesn't have a tool for this last step, but I was able to put together an XSL stylesheet to do just that.

I also realized I did not document DependencyGraphToGraphML nor DependencyGraphToyEd on the Tools page. So I took care of it at the same time as I documented the new stylesheet for unconfirmed symbols: DependencyGraphToUnconfirmedFullyQualifiedNames.

March 02, 2008

Added edges for class-in-package and feature-in-class relationships to the stylesheet for yEd. I want to experiment with subgraphs too and see which will autolayout best.

February 29, 2008

Added a variation on DependencyGraphToGraphML.xsl for yEd: DependencyGraphToyEd.xsl. It can put labels on the nodes and uses dashed arrows for dependencies using yWorks-specific extensions to GraphML.

February 28, 2008

Added a very early version of DependencyGraphToGraphML.xsl. It turns out producing XML from XSLT is a non-issue (as you'd expect). Just use <xsl:output method="xml"/>, which is kinda the default if the top result node is not some variation of <html>, according to the XSLT spec.

I cleaned up the XSL templates. I removed explicit <xsl:output method="html"/> tags because these templates make it obvious that they produce HTML. I wanted to remove <xsl:strip-space> too, but that caused memory problems for large documents, so I kept them in. I inlined unnecessary <xsl:variable> tags all over the place.

One issue I have with all of this is that I don't have any kind of automated tests for these XSL templates. I could run a transformation and use XPath to verify its output, but that would be tedious and brittle and hard to maintain. I could record golden outputs and compare the output of transformations to them, but that would also be brittle. This latter solution would nearly be the only way to test XSL transforms that produce text.

February 24, 2008

Tried to write an XSL transformation to convert a dependency graph from the XML format of Dependency Finder to GraphML. So far, XSLT doesn't like for me to define another <?xml?> tag inside the XSL template. So unless I can find a way to circumvent this, I will have to create a full-blown Printer instead.

February 22, 2008

Someone pointed me to Google Singleton Detector, a static analyzer that looks for singleton and produces nice graph output. They use GraphML to render the graphs and yEd to view them.

The same people that offer yEd also have a UML tool called yDoc, but you have to pay for that one.

February 21, 2008

Fixed MartinConfig.xml to remove extraneous value selectors. Both Ca and CaComp aggregated class names from IEPM with these regular expressions:

    IEPM /^(.*)\.[^\.]*\(.*\)$/
    IEPM /^(.*)\.[\^.]*$/

The first one matches the class name portion of method names thanks to the parentheses at the end. The second regular expression (forget the typo for a moment) does not include the parenthesized argument type list, so it can only match field names. But IEPM can only contain method names, never field names, so this second regular expression is totally superfluous. So I removed it.

With just the first regular expression, Ca and CaComp end up collapsing IEPM to a list of distinct class names, which is what they need.

February 20, 2008

Ran testability-metrics against Dependency Finder for fun. With no surprise, the biggest cyclomatic complexity culprits were com.jeantessier.dependencyfinder.gui.DependencyFinder and com.jeantessier.dependencyfinder.gui.OOMetrics because of Swing. The class with the biggest global state cost was Instruction because it stores so much data in the static context.

February 16, 2008

Adjusted vocabulary in Fit tests to reflect Four-Phase Test pattern from the book xUnit Test Patterns.

February 15, 2008

Someone referred me to a Danny Diggs, from UIUC, who has a tool to detect refactorings that are done to a library and can then replay them on client code when new version of the library gets released. This goes one step further than JarJarDiff who can tell you if the API you relied on got changed; it can actually modify your code to bring it in line with the new version of the library. A quick Google search didn't find any obvious matches for this tool, though.

Someone pointed me to testability-metrics, a tool that looks at cyclomatic complexity as a way to estimate the cost of testing any given method. It works off of compiled bytecode, just like Dependency Finder, but uses ASM for decompiling bytecode.

February 14, 2008

Added a Fit test for MartinConfig.xml.

Replaced the names of basic measurements in Metrics with an enum called BasicMeasurements.

Simplified the Measurement interface by using autoboxing for numerical values instead of explicit add(*) and *Value() methods.

Replaced dependencies on Collections.EMPTY_* to their corresponding Collections.empty*() instead for stronger type checking.

February 13, 2008

Found a possible bug with MartinConfig.xml that was the result of a typo in a regular expression. Where it says [\^.] (a caret or anything), it should have said [^\.] (anything but a dot) instead. I will have to write some conclusive tests to verify the behavior of this config before I can go ahead and fix it.

February 04, 2008

Worked on getting access to Signature_attribute information in relevant programming elements (Classfile, Method_info, and Field_info, forgot about possible InnerClass).

Added tests for index attribute of LocalVariable and LocalVariableType and to make sure XMLPrinter renders it, using jMock.

Added information (and tests) about varargs methods.

January 29, 2008

Dependency Finder is now seven years old.

Adjusted the copyright notice for 2008. I found a few files whose copyright notice still said 2004!

January 28, 2008

Minor clean up: removing trailing spaces.

There was a feature added to JProgressBar in JDK 1.4 where you could put it in an indeterminate state when you didn't know how long a task would be. Back in January 2003, I commented out the calls to JProgressBar.setIndeterminate() in OpenFileAction and MetricsExtractAction. I think it is safe to uncomment them now.

January 27, 2008

Played a little bit with JUnit 4.4. It is perfectly backwards compatible, so I could just upgrade to it and keep on writing tests like with the 3.8.2 version. The are moving to a new syntax that uses Hamcrest Matcher classes, like jMock so you can write assertions like assertThat(actualValue, is(expectedValue)). While it makes the code closer to English, it also makes it farther from Java. I'm not totally convinced this is a good thing. It is more expressive, that's for sure. What I would really like to see is an automated way to build test suites, without me having to hard-code which tests are in a given suite. I haven't found the mechanism to do this in JUnit 4.

January 24, 2008

Looking for a specification of the StackMapTable attribute.

Found some interesting parallels to com.jeantessier.classreader in Javassist. There might inspiration on how to deal with annotation-related structures. See their documentation for AttributeInfo.

January 23, 2008

Add jMock libraries to CLASSPATH for JUnit launch scripts (junit and textjunit, both on Windows and Unix).

January 19, 2008

Used jMock to rewrite TestLoadListenerVisitorAdapter.

January 18, 2008

Finished separating the classes that make up Classfile into interfaces and a implementation classes in a sub-package.

Next, I want to break the visitor hierarchy in com.jeantessier.dependency to use composition instead of inheritance for separating graph traversal from the specific functions. A wrapper will take care of traversal and delegate to another visitor for the work to do. This way, I will be able to test traversal logic with jMock instead of relying on a subclass.

January 17, 2008

Started separating the classes that make up Classfile into interfaces and a implementation classes in a sub-package.

January 16, 2008

Clamped down on public constants for access modes in .class files structures.

I am very tempted to extract interfaces from all the classes that make up the Classfile structure and put their implementation into a separate package. I could possibly rewrite part of the tests with jMock and have more control on what gets tested without having to jump through hoops to create test classes that expose specific behavior.

January 15, 2008

Played with Ashcroft to see how well-behaved my unit tests really were. Out of 1403 unit tests, 1224 did not pass the test. This is not totally surprising since most tests get their data from reading .class files. Also, I couldn't run the tests with Ant because Ant tries to write test reports to files and Ashcroft won't let it open the output file.

January 14, 2008

Ran Dependency Finder through FindBugs and found a few quick fixes, including one clear bug in com.jeantessier.dependencyfinder.cli.ListDiff where I was not closing a stream when reading in a file. Twice. And two constants in TransitiveClosure that were not marked final. But there were quite a few false positives too.

January 07, 2008

Finally got back to working on SignatureHelper to process the signature of generics. For starters, I simply copied the code from DescriptorHelper and modified it slightly. I will need to figure out how to extract common code to a shared superclass. I also added Fit tests for DescriptorHelper and SignatureHelper, to eventually replace the JUnit tests. It is easier to test multiple operations on a given string using Fit tests than with JUnit, which would require a certain amount of code duplication.

January 03, 2008

Upgraded the following development components:

WebWork

:

2.2.4

->

2.2.6

I also changed web.xml for both webapps to use JSP 2.1 with EL support. Not that I'm using any of it yet, but just keeping up-to-date.

December 28, 2007

Played with JSTL to see if some of the tags in there could help with i18n for the webapp.

December 27, 2007

IntelliJ has really awesome support for making internationalized resource bundles. Makes me want to build one for the webapp so I can i18n it.

December 13, 2007

Upgraded the following development components:

IntelliJ IDEA

:

7.0.1

->

7.0.2

I looked into WebWork's OGNL stack and expression language. I cleaned some of the WebWork webapp files.

December 12, 2007

Upgraded my development machine to MacOS X 10.5 "Leopard".

November 28, 2007

Took a quick look at XRadar and how it uses Dependency Finder. Interestingly, XRadar is implemented as a series of Ant tasks and XSL stylesheets. One of the Ant tasks uses DependencyExtractor and its output is processed by a specific XSL stylesheet. They are well insulated from any API changes on my part.

November 26, 2007

Found out Dependency Finder is shipped as part of XRadar. Cool! I should check which part of the API they are calling. They are shipping a pretty old version: 1.1.0 dated December 2004.

A user contacted me because they are using Dependency Finder from behind a firewall and the doctype of generated XML documents points to a SYSTEM DTD that they cannot reach. They suggest using a PUBLIC DTD instead. I will have to look into how that works and what the implications would be.

November 19, 2007

Added version information for Clover.

October 30, 2007

Fixed bug 1822770 where a missing switch value on the command-line was throwing CommandLine.parse() into an infinite loop. Thanks to monperrm for reporting the issue.

October 25, 2007

Finished tests in TestClassReport for non-ASCII characters.

October 24, 2007

Unit tests for escaping [ and ] in HTMLPrinter.

October 23, 2007

Unit tests for escaping [ and ] in HTMLPrinter.

October 22, 2007

Unit tests for escaping [ and ] in HTMLPrinter.

October 18, 2007

More work on bug 1800017.

In the process, I made the error reporting for bad regular expressions a little more user-friendly. I tried using a checked exception at first, but the change quickly spread everywhere. The exception class was in com.jeantessier.dependency and as a result, every package suddenly depended on it, including com.jeantessier.classreader, creating a circular dependency. So I went back to an unchecked exception for the time being.

October 09, 2007

Added a test to TestQueryBase to check for properly escaped [ and ] in order to fix bug 1800017.

September 24, 2007

Code cleanup.

September 21, 2007

Found a bug in the web app where elements with [ or ] in them, like main(String[]) for example, are not escaped correctly when creating likes to them to refine queries and generate exceptions.

September 19, 2007

Moved the Tutorial from MS PowerPoint-generated HTML to the new Google Docs presentation.

August 30, 2007

Fixed the documentation for OOMetrics to remove references to the old -all switch in the CLI.

Changed the path in the distribution archives so Dependency Finder extracts to a folder that include the release label in its name. Distributions used to all extract themselves to DependencyFinder and would overwrite each other. Now, they extract to DependencyFinder-<version> instead, like DependencyFinder-1.2.1 for instance. The previous naming scheme worked well when I was developing on a Windows platform and changing the PATH was tedious. But now that I work mostly on Mac and Unix, I can use a simple symbolic link.

I also modified the installation instructions to include the current version number directly in the text. I was already injecting it to label the instructions at the top of the document. It was easy enough to replace the reference throughout the document.

August 29, 2007

Created some test data and a FIT test for tracking extra-package dependencies using OOMetrics. Right now, it can control access to methods, including constructors, but it does not handle fields. That would require a bigger change in OOMetrics.

I'm also considering adding a filtering function similar to the one in OOMetricsGUI. If I can find a way to extract symbols at a given access level, I could then use them to filter the metrics report.

August 28, 2007

Someone was asking for a tool to track public methods that are not used outside their package. They could use such a tool to narrow down the exposed interface of a library. I started thinking how I could do this using Dependency Finder. One problem is that most tools don't record the access level of programming elements. JarJarDiff does it, but it doesn't know about dependencies. OOMetrics keeps a count by access level, but does not keep track of the access level of individual methods. One possibility would be to get a list of public and protected methods, maybe with ListSymbols, and pass them to OOMetrics somehow. The -scope-includes-list switch to OOMetrics controls which dependencies it tracks, not which metrics are reported. With the -show-empty-metrics switch and a <lower-threshold>1</lower-threshold> in the configuration for the IEPM measurement, any method without any extra-package dependency would stand out, but right now that includes private and package methods.

August 27, 2007

Played with DOM and JAXP to try and figure out how non-ASCII characters get encoded by the platform. So far, they don't. I can generate an XML document using DOM that cannot be parsed back in.

August 24, 2007

I've started by renaming Report.toString() to Report.render(), just like I did for ClassReport. Again, I am still not completely satisfied with it, but it will do as a stop-gap measure.

I also took care of escaping special characters in <modified-fields> tags. This introduced an interesting twist. When the value is changed, ClassReport includes it in the content of the tag where it may need to be escaped too. But in tag content, I don't need to escape '"' or '''. In fact, the content is more human-readable if I don't. So I split ClassReport.escapeXMLCharacters() into ClassReport.escapeXMLCharactersInTagContent() for tag content and ClassReport.escapeXMLCharactersInAttributeValue(), a more thorough version for attribute values.

Replaced of dependency on com.sun.org.apache.xpath.internal.XPathAPI, which is internal and was giving me compiler warnings, with javax.xml.xpath. The code is just as clear and this time uses supported interfaces.

August 23, 2007

Redesigning the Classfile hierarchy is proving more complex than I even thought. For now, the easiest thing to do is for me to just add constants with values that need escaping in ModifiedPackage.ModifiedClass in the JarJarDiff test data folder and write tests that read .class files from the filesystem. I'll revisit jMock or EasyMock for something else at some other time.

I modified ClassReport to be a com.jeantessier.classreader.Visitor so I can use double dispatch to escape string values but not numeric values. It makes more sense to have a specific visitor choose the proper rendering based on the output format than to blindly assume one. Still, Report and ClassReport both join two responsibilities: organizing differences and rendering them in XML. These should be split into separate classes, which would open the door to rendering the reports in something other than XML without the need for XSLT.

Refactored the RemovableDifferences hierarchy. The concrete classes used protected setters to save values in the state of the superclass. Instead, I made the getters abstract so each subclass can implement it based on its own state instead of polluting the state of the superclass. This way, I could get rid of the setters altogether. Some of these getters, such as ClassDifferences.get(Old|New)Declaration() and FeatureDifferences.get(Old|New)Declaration(), are now slightly less efficient but I hope it won't impact overall performance too much.

Refactored com.jeantessier.classreader.TestXMLPrinter to use XPath expressions to test the contents of the generated XML. Since it already loaded and processed the test data folder for JarJarDiff, it indirectly tests the proper escaping of special characters. I find that I don't need to encode '"' or ''' since all string values are in the body of tags and never in attributes. The only exception is the name of custom attributes and I doubt these will use quotes or apostrophes.

August 22, 2007

Tried to use EasyMock instead of jMock. Whereas jMock can infer decent default return values for methods, such as false for a boolean or "" for a string, I have to specify everything explicitly with EasyMock. And it seems to have issues with toString() too. I might as well stick with jMock for now.

Lessons learned:

Lesson 1: don't put overly complex logic in toString(). Tools like debuggers rely on it to display feedback, so if there is a bug in toString(), it will cripple your debugger. ClassReport.toString() does all the XML formatting, and that is a bad thing.

Lesson 2: mock object frameworks like jMock and EasyMock are of no help when the path you are mocking uses double dispatch in the middle of it. I haven't found a way to tell the mock to call some other method when it receives a given call. In my case, I am testing part of a Visitor where the graph it is visiting has mocks in it. When the visitor calls accept() on a mock, that mock should call visitX() back on the visitor. There seems to be no trivial way to specify this on the mocks.

Lesson 3: I built the classes in com.jeantessier.classreader with the assumption that they would read well formed .class files. This makes it hard to test small portions of the structure in isolation. All the constructors take a DataInputStream and expect valid bytes on the stream. Setting up unit tests is extremely hard. I cannot create a custom mock by subclassing. There are no interfaces for me to implement. The structure is too complex for jMock or EasyMock. I'm stuck.

Action items:

• Move rendering logic out of ClassReport.toString() and Report.toString().
• Create constructors for Classfile and its dependants.
• Don't rely on toString() for rendering logic.

That should make things easier to work with and speed up some of the tests. I would be able to build a graph of model objects for the tests instead of reading dummy .class files from disk.

I've started by renaming ClassReport.toString() to ClassReport.render(). I still not completely satisfied with it, but it will do as a stop-gap measure for now.

August 21, 2007

Tried to use jMock to mock parts of an API difference structure so I can test that ClassReport.breakdownDeclaration(Field_info) does the proper encoding of XML characters in values of string constants. I am struggling quite a bit. The method I want to test is private, so I have to go through a path inside ClassReport that calls it, and that path makes quite a few calls on other structures around the Field_info. Setting up the mocks is a little ugly and takes over 75% of the test method. Plus, it looks like jMock never mocks calls to toString(). Maybe there is something that I don't quite get.

August 16, 2007

A short while ago, someone filed a bug against JarJarDiff because it doesn't encode special characters in its XML output, like '<' embedded in string values. I got a case of that today as I was comparing two versions of a library and one of the strings contained a non-ASCII character that was not getting escaped.

I started by writing a test against com.jeantessier.classreader.XMLPrinter and the '>' character, only to find out that this one XML generator actually escape certain characters already. But it's the only one to do so and it only handles '<', '>', and '&'.

The person who filed the bug proposed using an XMLUtil class copied from some other open source software. I'm not a big fan of utility classes so we'll see how it all turns out. Because of the way the printer hierarchy is organized, I might have to duplicate the escaping code between them if I don't want to use a utility class.

August 08, 2007

Played with Doxygen to generate UML diagrams. The documentation it generates looks nice and includes where symbols are referenced from. The diagrams are rather busy because they tend to include everything. If I want to make it part of the normal documentation build, it would require people to include Doxygen too. I'm not sure I want to take it that far just yet.

Refactored NbSubMetricsMeasurement to remove some duplication.

August 03, 2007

Added SourceDebugExtension_attribute.

Added counts for the various attribute types in ClassMetrics.

August 02, 2007

Played with JDK 1.6. It screwed up some of the tests because of changes in the language and the .class file format. For instance, there is a new StackMapTable attribute for which I could not find any description on the Internet. Of course, the JDK 1.6 compiler puts one such attribute in the bytecode for test.main(). Right now, Dependency Finder represents it using Custom_attribute, but it threw off TestVisitorBase.testVisitCode_attribute() which wasn't expecting custom attributes. The few bits of text I could find on Sun's website that talk about StackMapTable also refer to a StackMap attribute, which I had never heard off before. Makes me wonder what else is lurking in the shadows.

Since I do most of my Dependency Finder development on a Mac these days, it will still be a while before I can deal with JDK 1.6 issues.

Someone found a bug with the XML generated by JarJarDiff. When it breaks down the declaration of constants, it shows the value for that constant if it can, which, in the case of strings, can contain characters that should be escaped in XML. I suspect it might be an issue for other XML generators too.

July 31, 2007

Fixed an error in metrics.dtd. members was defined as a list of members when it is actually a list of member elements, with no 's' at the end.

July 26, 2007

Finished reading the last Harry Potter book: Harry Potter and the Deathly Hallows. Now I can get back to Dependency Finder. :-)

July 21, 2007

Started reading the last Harry Potter book: Harry Potter and the Deathly Hallows. Won't be working on Dependency Finder for a little while.

July 19, 2007

Back from vacation.

July 12, 2007

Leaving for vacation in Cancun, Mexico. Won't be working on Dependency Finder for a little while.

July 04, 2007

Monitor is too aggressive when multiple versions of a class show up in a codebase. This can happen on large projects that depend on many third-party software that each depend, in turn, on some popular package. For example, both GWT and HttpUnit come with their own version of Xerces; if my project uses both, Monitor is going to see Xerces classes twice when I try to analyze my project. When I designed Monitor, I was assuming that it wouldn't see the same class name twice within a session. Now, I probably want to merge dependencies within a session and still remove everything between sessions, regardless of whether Monitor is setup for open or closed sessions.

Refactored Monitor to use parameterized generics. Removed code duplication in TestMonitor.

June 29, 2007

I noticed that the mnemonic() column in Instruction.html was not turning green or red as tests passed or failed. I was using an HTML class attribute on the cell to give it a monospaced font, but that interfered with Fit's coloring scheme, which also uses a class attribute on the cell. The browser was using only the first class attribute and ignoring the one from Fit. I took mine out; it doesn't look as pretty now, but it is much more useful this way.

In OO, a signature is something that uniquely identifies a method, typically it is its name and the type of each parameter, in order. But Java 1.5 introduced a different meaning to the word signature: the generic information around a class, field, or method declaration. Java in general uses the work descriptor to refer to the combination of return type and parameter types of a method. The object model in com.jeantessier.classreader uses signature in the OO sense and builds it from the method's name and descriptor information. But I feel this may get confusing as I expose the Java 1.5 Signature_info data. I am debating whether I should rename all my *Signature methods to *Descriptor instead; maybe even going as far as leaving the method name out of them and let callers combine them as appropriate. I'll need to look into this more closely.

Feature_info already has a getDescriptor() method that returns the raw string that describes the descriptor (in the Java sense).

June 28, 2007

Refactored classes in com.jeantessier.classreader to encapsulate knowledge of field types and method signatures in the appropriate classes. I want to better encapsulate use of DescriptorHelper. This is a little hairy because this package has rather poor coverage from the tests.

June 26, 2007

Added a test generic method where some parameters are not generic and some are simple types. I think I can use the optional Signature attribute in lieu of the descriptor, instead of trying to merge them somehow.

June 25, 2007

Extracted common code from DependencyCycles, DependencyClosure, and DependencyMetrics and put it into DependencyGraphCommand. They load and merge all the graphs before they can start processing. DependencyReporter benefited also, but to a lesser degree because it optimizes the processing of graphs as they are being loaded instead of waiting for one big graph of everything.

June 24, 2007

Extracted common code from ClassList and cie into DirectoryExplorerCommand.

June 22, 2007

Found a bug with DependencyExtractor. If I don't give it any parameters, it is supposed to start exploring the current directory. Instead of that, I am getting a UnsupportedOperationException in Collection.add() when the tool adds "." to the parameters when the parameter list is empty. I opened a bug against it: Bug #1741810.

This impacts quite a few tools:

• ClassFinder
• ClassList
• ClassMetrics
• DependencyExtractor
• ListDeprecatedElements
• ListSymbols
• OOMetrics

I also found a similar somewhat similar problem with ClassReader, but this tool uses an AtLeastParameterStrategy and requires at least one parameter. So it makes no sense to have a special case for when there are no parameters, since this will never happen. I looked at previous versions of the tool and this duality has been present for a long time, so I kept the AtLeastParameterStrategy. ClassReader is not meant to run against a large set of classes, it is intended for use against a small set so we can inspect their internals.

The Tools page refers to two scripts that do not exist yet: CyclesToHTML and CyclesToText. Either I never wrote them or I somehow forgot them in some Subversion client somewhere. I will have to dig around for them or write them from scratch eventually.

June 18, 2007

Plugged into Google Analytics.

June 14, 2007

Fixed tests so they could run without a network connection. MetricsConfigurationLoader takes an optional boolean as to whether or not validate XML documents against their DTD. When running the tests without a network connection, you can set the DF_VALIDATE environment variable to anything but the string "true" to set the Java property DEPENDENCYFINDERTESTSVALIDATE which in turn controls what gets passed to MetricsConfigurationLoader. Some new tests that verify the behavior of *.XMLPrinter also try to parse the resulting XML and need to turn off the validation feature of the parser when the network connection is not available.

Fixed the command-line test launchers to account for the latest versions of WebWork and Tomcat, which have changed the name and/or location of their JARs. Note that junit and junit.bat require that you pass them -noloading for the HttpUnit-based tests to work. They don't seem to like the classloader that JUnit uses to reload classes between tests.

May 30, 2007

Fixed documentation.

May 25, 2007

Added isSynthetic(), isAnnotation(), and isEnum() to InnerClass. I was even able to test isEnum()!

Rev'ed up the release number to 1.2.1 (-beta1) and adjusted the Release Plan.

May 23, 2007

Renamed SignatureHelper to DescriptorHelper. I used the word signature in its non-java sense of a method's identifier and argument types and order. But Java 1.5 has a specific structure in the .class file named signature to represent information about type parameters of generics. Since my previous SignatureHelper was manipulating descriptor constructs, I renamed it DescriptorHelper to prevent confusion. I will introduce a new SignatureHelper later to deal with the new signature constructs.

I took this opportunity to move some of the functionality that only dealt with class names to the new helper ClassNameHelper. I suspect that SignatureHelper will need it too, so it makes sense to have it in some shared location instead of tying DescriptorHelper and SignatureHelper together somehow.

May 22, 2007

Fixed the copyright notice that was missing from some of the newest files.

Added Signature_attribute, LocalVariableTypeTable_attribute, and LocalVariableType to support generics.

May 15, 2007

Got a new IntelliJ license from JetBrains. Thanks Jetbrains!

May 14, 2007

Removed extra-class dependencies on access level constants in Feature_info and Classfile and used accessors instead.

Used specialized generics in ClassMetrics.

Asked JetBrains to renew my open source license for IntelliJ.

May 13, 2007

Added Field_info.isEnum().

Fixed Feature_info to account for the new "synthetic" access flag for fields and methods.

May 11, 2007

Moved ClassfileLoaderDispatcher actions to a first-class enum instead of an inner class.

May 10, 2007

Implemented EnclosingMethod_attribute to handle "EnclosingMethod" attributes of classes. Changed the classfile DTD to reflect the new attribute.

Fixed some tests that dealt with the ldc instruction.

May 09, 2007

Added a test for Classfile.isAnnotation(), including a sample annotation class that the tests can use.

May 05, 2007

Renamed the test data class TestEnum to testenum to highlight that it is not a regular class but just a construct to test parsing of .class files.

May 03, 2007

Added support for enum and annotation access modifiers for classes. I put together a test for the ACC_ENUM flag, but not for the ACC_ANNOTATION flag yet. A lot of this Java 1.5 support work is not going to be tested: it is just too hard to generate all the possible byte streams for all the structures.

April 30, 2007

Changed AttributeFactory to use a Java enum instead of string constants when deciding which Attribute_info subclass to instantiate. Right now, the enum is a public standalone enum and AttributeFactory is only one static method. I'm thinking of combining them somehow, maybe make the enum a private enum of AttributeFactory.

April 27, 2007

Made Code_attribute implement Iterable so I can use Java 1.5's for-each loop to iterate over instructions.

Moved processing of Instruction in XMLPrinter out of visitCode_attribute() and in to where it belongs: visitInstruction().

April 26, 2007

Modified com.jeantessier.classreader.XMLPrinter so it shows custom attributes and their contents.

April 17, 2007

Fixed the documentation for JarJarDiff (both CLI and Ant task) and ClassClassDiff. I also realized I had not updated the documentation of the Ant tasks that is on the website, so I fixed it too.

April 13, 2007

I have been using jMock at work and it is very interesting. The syntax takes some getting used to, but I might actually consider using it on Dependency Finder.

April 11, 2007

Fixed Ant build files to place the WebWork webapp WAR file in the dist directory, next to the other distribution files.

April 10, 2007

Added a Fit test for the merging behavior of NodeFactory.

April 05, 2007

Figured a way to set properties for Ant targets within IntelliJ. On other platforms I've used, Windows and Linux, IntelliJ IDEA gets the environment variables, but not on Mac OS X for some reason. On the Mac, I have to set them explicitly in the project.

Also on Mac OS X, FitLibrary tests have trouble copying some of the images in the files subfolder, which has a .svn subfolder with Subversion-related files. Some of these are read-only and subsequent runs of the test suite has trouble overwriting them again and again. I had to add a <chmod> task in build.test.xml to circumvent this.

Finally, the Mac OS X has timestamps on files with a resolution to the second. One test was waiting only 50msec and so was failing on this platform. I've raised the sleep time to a full second, but I'm a little concerned at making the tests take even longer.

Released version 1.2.0.

April 04, 2007

Upgraded the following development components:

IntelliJ IDEA

:

6.0.4

->

6.0.5

But it seems IntelliJ IDEA on a Mac doesn't get the environment variables the way it does on Windows or Linux. As a result, Ant targets don't execute correctly.

March 30, 2007

Found a bug with OOMetricsGUI where the names of packages, classes, and features are all prefixed with "com.jeantessier.metrics.Metrics". It turns out the cell renderer from Swing calls toString() on the object being rendered, in my case they are Metrics objects and their toString() method adds a lot of debug information to the string. I modified OOMetricsTableModel to insert only the name of the Metrics in the first column instead of the Metrics itself.

This bug was in Release 1.1.1 and has been around for almost two years before I noticed it.

March 23, 2007

Used an Enum for the actions in ClassfileLoaderDispatcher so test error messages would be more significant. Previously, they were integers.

Downgraded the following development components:

FitLibrary

:

20070217

->

20060116

March 22, 2007

I'm having a hard time getting all the tests to work with the new FitLibrary. I'm thinking of reverting it for now. I'll figure it out after I've launched version 1.2.0 of Dependency Finder.

Upgraded the following development components:

Log4J	:	1.2.13	->	1.2.14
Tomcat	:	5.5.17	->	6.0.10

March 20, 2007

Accessing fields of other classes is bad. Even those of your superclasses. I just got bit by this with some of my DoFixture subclasses. They called setSystemUnderTest() to set it, but were accessing systemUnderTest directly instead of calling getSystemUnderTest() like they should. Of course, the latest version of FitLibrary tightened access to this field and now my code is broken. Luckily, the easy fix is to get it through the accessor and everything becomes alright.

Upgraded the following development components:

FitLibrary

:

20060116

->

20070217

March 18, 2007

I tried to upgrade Tomcat and FitLibrary as part of setting up my environment on MacOS X. Tomcat changed its directory structure between versions 5.5.17 and 6.0.10. FitLibrary changed some of its code so my code no longer compiles against the latest version, FitLibraryRunner20070217.

March 16, 2007

I got a Mac for my next laptop. I'm going to switch development back and forth between my Windows desktop and my Mac laptop.

March 14, 2007

Refactored CycleDetector to have better logging so maybe I can find what's the problem when looking for cycles in large graphs. I also simplified Cycle slightly.

March 09, 2007

Finished documentation for ClassClassDiff and DependencyGraphToRDF in Tools docs.

Refactored common parts of com.jeantessier.diff to use specialized generic structures instead of unqualified ones.

March 08, 2007

Added documentation for ClassClassDiff and DependencyGraphToRDF to the Tools docs.

Refactored common parts of ClassClassDiff and JarJarDiff into a new abstract base class.

March 01, 2007

Fixed DependencyGraphToRDF so at least it runs and generates some output. I still have no way to verify the validity of its output.

February 28, 2007

Tech docs. Still need to document:

• ClassClassDiff
• DependencyGraphToRDF

DependencyGraphToRDF appears broken and I have no way to verify the validity of its output.

February 23, 2007

Use the new (in JDK 1.5) Class.getSimpleName() instead of cutting it out myself as I used to. I also added a similar method on objects that tend to model classes, like Class_info, Classfile, and ClassNode.

February 22, 2007

I noticed that calling CLI commands with the -version switch when the command has mandatory parameters or switches resulted in an error message and no version information. I reorganized the command-line parsing logic so it collects error messages in a manner that does not prevent processing a -version or a -help switch.

February 21, 2007

Fixed the ParameterStrategy for ClassReader, DependencyClosure, DependencyCycles, DependencyMetrics, and DependencyReporter. When I refactored the CLI into Command subclasses, I inadvertently removed the constraint that these tools require at least one file to work on. I looked at the dependencies on ParameterStrategy constructors between version 1.1.1 and the latest code and I could determine which CLI commands needed a different strategy from the others.

Created a launch script for Log4J's Chainsaw. Between times that I actually need to look at logs to debug something, I keep forgetting how to start it. This way, I don't have to remember anything.

February 16, 2007

Tech doc for CLI version of DependencyCycles. Changed scope switches / attributes to start ones instead, similar to DependencyClosure.

Changed CommandLine.getSwitch() to throw an IllegalArgumentException instead of a checked CommandLineException when it cannot resolve a switch name. This simplified the signature of many methods that deal with setting up the CommandLine object in the CLI.

Fixed a bug in DependencyCycles where it was opening its own writer instead of using the one setup by its superclass. Revised all writer usage across CLI.

February 15, 2007

Minor refactoring in the GUI code to remove warnings.

February 08, 2007

Modified extract.jsp so the webapp can filter out dependencies at extraction time too. I also changed the default values of showSource and showFile to true. It makes more sense to show these values while someone is trying to configure Dependency Finder and hide them later, if needed.

February 05, 2007

Someone raised a problem with JarJarDiff and PublishedDiffToHTML.xsl. I was able to reproduce their problem, but when I looked how it works in the latest version, I found some bad text in the Tools documentation as well as the documentation of the matching Ant task. I want to remind everyone that in version 1.2.0, you won't use XSLT to narrow down the diff report but do it right inside JarJarDiff instead with the level switch.

Worked on documenting the new AliasSwitch.

February 02, 2007

Found another competitor, and curiously enough, they're called JarJar. There are even instructions on how to use it to find JAR-to-JAR dependencies.

January 29, 2007

Changed the next version number from 1.1.2 to 1.2.0. The scope of changes warrants a higher version number.

Added package comment to com.jeantessier.dependency to give an overview of the package hierarchy. Maybe that would better belong in an overview.html file, I have fully made up my mind, yet.

January 26, 2007

New -echo command-line switch for all CLI commands.

January 25, 2007

Received praise for Dependency Finder from Peter Lenahan, Technical Director, Information Builder, Inc. I've added it to the website.

January 16, 2007

Upgraded the following development components:

IntelliJ IDEA

:

6.0.2

->

6.0.4

January 06, 2007

Updated the copyright notice to 2007.

January 05, 2007

Working on release instructions with Subversion instead of CVS.

January 03, 2007

Added sample graphs as XML documents for one example in the Developer Guide and for the graph used by the webapp tests.

January 02, 2007

Fixed some of the build files so running Clover analysis doesn't clear Emma reports, or vice versa.

December 30, 2006

Finished refactoring the CLI commands. Made some fixed to the tools documentation in the process.

Adapted the build file for Clover for Clover 1.3 and streamlined the flow to generate a coverage analysis report. Emma copies instrumented files to a separate directory, but I can't seem to get Clover to do the same. Since some of the metrics and API difference tests operate on compiled Dependency Finder classes themselves, the instrumentation throws them off. But the resulting code coverage analysis seems close enough just the same.

December 29, 2006

Validate -maximize and -minimize switches to DependencyExtractor, just like in DependencyReporter, complete with Fit tests.

Clean up Ant targets across build files and clean up documentation. In the process, I found a typo where a '*' was not escaped properly and the HTML generator could not replace the encoded sequence with a '*' in the output.

Listed third-party packages in the Developer Guide, including some tentative version numbers that are known to work with Dependency Finder.

Looked at fixing the Clover code coverage build file. Got rid of the section on jcoverage, since I can't seem to find a version of it that does not require Eclipse.

Added a link to my LinkedIn profile ⊕ Jean Tessier on LinkedIn
at the top of my "Jean's Journal" blog.

December 28, 2006

New AliasSwitch to handle command-line switches that are simply an alias for a collection of other switches. This way, com.jeantessier.commandline can handle aliases and their rendering in usage messages instead of each individual CLI command. I modified DependencyReporter and DependencyMetrics to use the new aliasing mechanism. I had to modify CommandFixture so I could test command-line parsing and alias configurations as part of the Fit tests.

I refactored the buffer handling in com.jeantessier.commandline.Printer to a new class com.jeantessier.text.PrinterBuffer. Other printers will be able to use it too to remove code duplication between them. I already modified com.jeantessier.diff.Printer and com.jeantessier.diff.ListDiffPrinter. Other printers will take longer because they are wrappers around PrintWriter instead of StringBuffer.

Figured out how to exclude sample classes that are only test data from Emma reports. I cleaned up build.emma.xml and the procedure for running the coverage report to make it much simpler. I no longer need to invoke a separate target from build.text.xml to generate the rest of the test data.

December 23, 2006

Unit tests for MultipleValuesSwitch.

December 18, 2006

Refactoring CommandLine to verify satisfaction of command-line switches.

December 16, 2006

Unit tests around command-line parsing to prepare for upcoming support for aliasing.

December 15, 2006

Push isSatisfied() behavior in CommandLineSwitch implementations.

December 14, 2006

Refactored DependencyMetrics as part of the grand CLI refactoring.

Refactored the control flow for parsing and validating the command-line in Command. One problem I still have is that multiple errors trigger multiple printing of the usage text.

December 13, 2006

Refactored DependencyClosure, DependencyCycles, and DependencyReporter as part of the grand CLI refactoring.

Emma recommends that you use an enabled attribute on <emma> tags to control whether instrumentation is done or not. The attribute takes its value from a flag property in the build.xml file under a special target that you put on the command-line, before the instrument or report targets. This way, the flag gets set in the emma target so that it enables the <emma> tags in other other targets.

My Emma-related Ant tasks are dedicated to doing code coverage, so I end up always calling them with emma on the command-line. Since I never turn the flag off, I removed it altogether and got rid of the emma target. Now, I will be able to trigger the code coverage targets from within IntelliJ.

Refactored Command to remove duplication when populating groups of related switches, such as for start and stop criteria and for filtering.

December 11, 2006

Reorganized the build files to get all test reports in one location.

December 10, 2006

Refactored ClassMetrics as part of the grand CLI refactoring.

December 09, 2006

When I reorganized build.xml, I changed the project's name from "DependencyFinder" to "Dependency Finder" with a space. Now, I use the project name to build the name of the JAR file when creating a distribution, so now the JAR file had a space in its name. Yuck! So I changed the name back to "DependencyFinder", without the space.

Refactored ClassFinder and ClassList as part of the grand CLI refactoring.

Cleaned up dependencies between Ant targets for Emma coverage analysis.

December 05, 2006

Reorganized build.xml into a number of build file. This way, someone can compile Dependency Finder with a minimum of third-party libraries, namely the Servlets API and WebWork, instead of needing all the libraries used for testing like Fit and HttpUnit.

I should get rid of the CVS report tasks eventually, now that everything is in Subversion. I don't know if I will create equivalent reports like Subversion, given that SourceForge already has some statistics on source control.

December 04, 2006

Finally got Emma to run. Looking at the coverage data, I found some test classes that were not included in the appropriate TestAll class.

December 03, 2006

Experimented with Emma for code coverage.

December 02, 2006

Tried to run the code coverage analysis with jcoverage, but my version (1.0.5) does not generate valid Java 5 code when it instruments the code. I was not able to find a more recent version; the jcoverage website only mentions Eclipse plugins. I am going to try using Emma instead.

December 01, 2006

Fixed test for ClassClassDiff. Had to move -xml switch out of Command and back into ClassReader and DependencyExtractor to account for tools that will not support multiple output formats, namely ClassClassDiff and JarJarDiff that only output XML.

November 30, 2006

Major refactoring of ClassClassDiff to use common code from Command.

November 29, 2006

Better use of the facilities in FitLibrary to minimize the amount of fixture code and maximize reuse.

Major refactoring of DependencyExtractor and ClassReader to pull common code into the new base class Command.

November 28, 2006

Started refactoring the CLI classes, with the help of FIT tests. It hints that more refactoring is needed in com.jeantessier.commandline.

November 19, 2006

Refactored com.jeantessier.commandline to simplify printers for usage and echo of parsed command-line. The latter still doesn't echo parameters; it only does switches at this time.

Refactored com.jeantessier.commandline to store parameters in the ParameterStrategy instances instead of the CommandLine. Just like switch values are stored in the CommandLineSwitch objects, now the parameters are also store outside the CommandLine object.

Cleaned up classes in the CLI and Ant tasks packages.

Upgraded the following development components:

IntelliJ IDEA

:

6.0.1

->

6.0.2

November 18, 2006

Added com.jeantessier.commandline.TextPrinter to echo the command-line, as perceived by a program. Some recent problems with the CLI stemmed from improper escaping of shell metacharacters. I hope that with some kind of -echo switch users will be able to see what values made it into the program.

I had to rework CommandLineSwitch so each switch contained its own name. This gave me the occasion to add unit tests to that package.

November 16, 2006

Added filtering to DependencyExtractor. This came at the suggestion of a user that it might help deal with very large graphs by removing dependencies that do not really bring new information, like dependencies on classes in the JDK for instance. I added the functionality to both the CLI and the Ant task, but I will also need to do something about the webapp and the GUI.

November 13, 2006

Simplified the command-line switch processing for DependencyFinder, DependencyCycles, and DependencyMetrics. I need to build aliasing support in com.jeantessier.commandline instead of handling aliases in each CLI tool's main() method.

I also adjusted the output form DependencyMetrics so it does not show NaN or infinity when ratios involve a zero.

I am seeing again the case where the report from DependencyMetrics shows strange numbers. The number under "outbound link(s)" is the sum of the breakdown under "inbound link(s)" and vice versa. I remember looking into this a while back and somehow convincing myself that this was right, but I am still confused, which probably means I need to fix these sections so they are more intuitive.

I also removed the -all switch from the CLI OOMetrics.

November 12, 2006

Removed -all and -show-all switches from DependencyReporter, DependencyMetrics, and DependencyCycles. I also removed their equivalent in the matching Ant tasks (DependencyCycles doesn't have an Ant task yet). Now, they act as if -all is on unless the user explicitly specifies at least one of the scope or filter switches (-package-scope, etc.). The same logic applies to -show-all: DependencyReporter will show everything unless the user explicitly requests only certain things. This means that if you give now switches to DependencyReporter, it will print out the input graph as is (but in text instead of XML). I believe these defaults make more sense than the previous ones that would print nothing unless you remembered to actually ask for something.

November 11, 2006

I want to write tests for the improvements I did to XMLPrinter for Instruction and to CodeDependencyCollector, but the classes in com.jeantessier.classreader were among the first I wrote for Dependency Finder and they are not easy to test. Their constructor takes a DataInputStream with the binary data for a complete class. This makes it very hard to get test data without building actual Java classes and compiling them. Especially when trying to test specific instructions. I will need to change all of them so I can construct temporary structures in tests, but that will take a while and I don't want to delay the upcoming release more than I have to.

November 09, 2006

I have half a mind to unify the HTML styles for dependency graphs in the webapp versus the XSLT transform used by the CLI and the Ant tasks. I think the style in the webapp looks better in the end.

November 08, 2006

It turns out the problem with closures comes down to Windows batch files strip '^' characters from parameters. So if you want to filter out java.* with a regular expression /^java/, the Java program only gets /java/ and filters everything with "java" anywhere in it. I filed a bug report (1592817) for it.

For now, the best solution is to enclose regular expressions in double quotes, such as "/^java/", so the shell and script don't strip anything. Similarly under Unix, if you regular expression contains many meta-characters like '\', '(', or '[', you need to escape them properly. Maybe there is no fix for it besides education. I should add a note on this in the documentation.

November 07, 2006

Came upon an interesting consequence of using JDK 1.5+. Take this simple class:

    package test;
     
    public class Test {
        public void test() {
            Class[] array = new Class[1];
            array[0] = String.class;
        }
    }

If I compile it using JDK 1.4 and then run DependencyExtractor on it, I get:

    java.lang *
        Class *
            <-- test.Test.class$(java.lang.String)
            <-- test.Test.class$java$lang$String
            <-- test.Test.test()
            forName(java.lang.String) *
                <-- test.Test.class$(java.lang.String)
        ClassNotFoundException *
            <-- test.Test.class$(java.lang.String)
            getMessage() *
                <-- test.Test.class$(java.lang.String)
        NoClassDefFoundError *
            NoClassDefFoundError(java.lang.String) *
                <-- test.Test.class$(java.lang.String)
        Object *
            <-- test.Test
            Object() *
                <-- test.Test.Test()
        String *
            <-- test.Test.class$(java.lang.String)
            <-- test.Test.test()
    test
        Test
            --> java.lang.Object *
            Test()
                --> java.lang.Object.Object() *
            class$(java.lang.String)
                --> java.lang.Class *
                --> java.lang.Class.forName(java.lang.String) *
                --> java.lang.ClassNotFoundException *
                --> java.lang.ClassNotFoundException.getMessage() *
                --> java.lang.NoClassDefFoundError.NoClassDefFoundError(java.lang.String) *
                --> java.lang.String *
                <-- test.Test.test()
            class$java$lang$String
                --> java.lang.Class *
                <-- test.Test.test()
            test()
                --> java.lang.Class *
                --> java.lang.String *
                --> test.Test.class$(java.lang.String)
                --> test.Test.class$java$lang$String

The compiler created synthetic features to store the reference to String.class. All this machinery adds a lot of complexity to the dependency graph.

In JDK 1.5, they expanded the VM instructions to handle this case and now the compiler can generate much simpler bytecode. If I compile the class with JDK 1.5 and then run DependencyExtractor on it, I now get:

    java.lang *
        Class *
            <-- test.Test.test()
        Object *
            <-- test.Test
            Object() *
                <-- test.Test.Test()
    test
        Test
            --> java.lang.Object *
            Test()
                --> java.lang.Object.Object() *
            test()
                --> java.lang.Class *

Notice the conspicuous absence of java.lang.String from the graph. This is because Dependency Finder does not understand the Java 1.5 features of the bytecode, yet.

I make a quick change to CodeDependencyCollector so it could at least account for the new behavior of two VM instructions (ldc and ldc_w) in Java 1.5. With this change, DependencyExtractor now gives this much simpler, yet accurate graph:

    java.lang *
        Class *
            <-- test.Test.test()
        Object *
            <-- test.Test
            Object() *
                <-- test.Test.Test()
        String *
            <-- test.Test.test()
    test
        Test
            --> java.lang.Object *
            Test()
                --> java.lang.Object.Object() *
            test()
                --> java.lang.Class *
                --> java.lang.String *

While working on ldc and ldc_w handling, I also realized the XML output for instructions wasn't descriptive enough to let me know which entry they are referencing in the constant pool. In the case I was looking at, the instruction could point to a class_info structure or to a Utf8_info structure and I couldn't tell which one it was. So I added an index attribute to the <instruction> tag so whoever reads the file can see exactly what the instruction is referencing.

Someone contacted me with an issue regarding closures. Apparently, filtering is not working correctly. I tried to recreate the problem, but it worked just fine for me. These kinds of bugs are always annoying to work on since I cannot reproduce it to examine it.

November 03, 2006

Tried to convert the tutorial to HTML using OpenOffice instead of PowerPoint. The resulting HTML works in more browsers, but the embedded images look crappy.

October 31, 2006

Did a quick study of the download rates for Dependency Finder, based on the statistics from SourceForge. It's nice how downloads flow from one release to the next, except that every once in a while, some of the old releases still get downloads even when there are multiple new releases after them. I wonder what could motivate someone to fetch an old release. Release 1.1.1 is holding up at over 300 downloads per month; I'm happy.

Cleaned up HTMLCyclePrinter.

October 29, 2006

More closure functional tests to test hyperlinks in the HTML. Also created some functional tests for cycle detection and metrics. I added an HTML implementation of CyclePrinter to provide hyperlinks for the cycle detector portion of the webapp.

October 24, 2006

Someone complained that the Tutorial only works on Microsoft products and I should move it to open source technologies. I'll try OpenOffice when I have some time.

October 23, 2006

More closure functional tests.

October 20, 2006

Upgraded the following development components:

WebWork

:

2.2

->

2.2.4

October 19, 2006

Clean up extraneous logging in TestQueryBase.

October 18, 2006

Clean up warnings in JSPs.

October 11, 2006

Created preliminary tests for closure.jsp.

Upgraded the following development components:

IntelliJ IDEA

:

6.0

->

6.0.1

October 09, 2006

Added hyperlinks on scoped node names in HTMLPrinter for the web app so the user can narrow down the scope to a single package, class, or feature. It is not perfect, and sometimes behaves in surprising ways, but it is as good (or bad) as the hyperlink on dependency names. Note that each link simply forces the scope-includes field to a certain prefix; it does not change the flags for what is in scope, whether it's packages, classes, and/or features.

Suppose a query result has lots of packages, classes, and features. You can click on a class name reduces the scope to elements prefixed with that name. In the query result, if you click on the package's name, it will expand the scope of the query back to all elements in the package.

Modified advancedquery.jsp so it uses HTMLPrinter. Refactored TestQuery to share tests between query.jsp and advancedquery.jsp. I also added tests that actually follow the hyperlinks to verify they do the right thing.

October 06, 2006

Fixed bug where if the scope was reduced to an inferred feature of a confirmed class, the class and its package ended up unconfirmed in the resulting sub-graph. GraphCopier and GraphSummarizer were using the current node's confirmation status when copying a node a propagating it to its parent nodes instead of creating accurate copies of the parent nodes in the first place.

Fixed the webapp's CSS stylesheet to treat visited links to inferred nodes correctly. The rules for a:visited and a:hover trump an explicit style assigned via the class attribute. I had to create explicit descriptions for a:visited.inferred and a:hover.visited.

October 05, 2006

Adjusted dependency printout in HTMLPrinter so the '*' for inferred dependencies is not inside the <a> tag. Also used CSS to highlight scoped node names and subdue inferred nodes (both scoped nodes and dependencies).

October 03, 2006

Upgraded the following development components:

IntelliJ IDEA

:

5.1.2

->

6.0

October 01, 2006

Fixed CLASSPATH in the launch scripts for JUnit to include WebWork libraries.

September 29, 2006

Changed classes in the com.jeantessier.dependency.Printer hierarchy to use separate methods for printing scoped node names from dependency node names. This lets me put scoped nodes in bold in the HTML output and leave dependency names in regular font. I should probably use CSS instead of hard-coding the bold using <b> tags, though.

Fixed CLASSPATH in the launch scripts for JUnit to include all the libraries needed by HttpUnit. For the Windows batch file, I had to list all JARs explicitly because I don't have a good way to detecting *.jar like I have in Bash shell scripts.

There is a weird interaction between the JARs that breaks Commons-Logging when using the Swing UI of JUnit. If I don't use the automatic class reload feature, it works just fine. I tried playing with tests/junit/runner/excluded.properties so JUnit would not reload the obvious candidates (Commons-Logging, Jasper, even ServletUnit), but to no avail.

September 27, 2006

Added an id attribute to <a> tags from HTMLPrinter to help with testing.

September 20, 2006

Made some progress with ServletUnit. It really helped to read through the tutorial. Step 3 had the piece I was missing to correctly submit forms and navigate between pages.

September 19, 2006

I starting to lose patience with ServletUnit. This is a case where I can do something in 10 minutes, but it is taking me weeks to get it under test. This is frustrating. If I don't find a way to get ServletUnit to work soon, I will have to drop it and go without test for the outermost layer of the UI.

I took a look at JUnit 4.1. It uses annotations to mark the test methods and the fixture setup and tear down code. It would be a significant effort to switch to JUnit 4. Plus, I hear that support in IntelliJ and Ant is still behind. I think I'll stick with JUnit 3.8.2 for now.

September 15, 2006

I tried to fix the broken test with ServletUnit, but to no avail. Instead of navigating to the page I want by following links, I tried setting the request parameters directly from the test, but that doesn't seem to work either. I log the HTML that comes back in the response, and it is as if I hadn't set any parameters. Something still escapes me when it comes to using ServletUnit.

September 07, 2006

Minor fix to data2html.cgi to remove target attribute from generated <a> tags.

September 06, 2006

Got my first ServletUnit tests to pass. As long as I just request a single page, it works fine. But when I use the controls on a page (submit button) to navigate to another page, I get a strange NullPointerException from deep within the framework. Sigh.

August 31, 2006

Started writing tests using ServletUnit, but I can't seem to get it right.

August 30, 2006

Searched the web for examples of how to use ServletUnit.

Good news for Dependency Finder: I finally finished Half-Life 2! >:-)

August 29, 2006

Downloaded HttpUnit and checked the documentation for ServletUnit. It doesn't seem as trivial to use as I would have hoped.

August 25, 2006

I want to finish putting web links in the results of the webapp, but how am I going to test them? I searched for "testing servlets" and "testing jsp out of container" and found references to ServletUnit, a part of HttpUnit.

August 05, 2006

Investigated a little more the problems with closure.jsp, but I cannot seem to reproduce the original problem. I tried simple graphs, I tried working from com.jeantessier.dependency.NodeFactory.createClass(), they all seem to work just fine.

August 04, 2006

Played a bit with TeamCity, by the makers of IntelliJ IDEA. While I was trying to set it up for Dependency Finder, I was confronted with the fact that I create the distribution files in the same folder as the build.xml file. I should really put them in their own folder, like dist/, so TeamCity can recognize them as build artifacts. It would also align my build better with the best practices out there.

July 19, 2006

Fixed HTMLPrinter to properly escape the '$' in inner class names. Since '$' has special meaning in Perl regular expressions, it must be escaped otherwise the regular expression does not match anything and you get empty results.

July 10, 2006

Came back from a three-week vacation.

Upgraded the following development components:

IntelliJ IDEA

:

5.1.1

->

5.1.2

June 16, 2006

Looked at fixing closure.jsp so that it always match packages and also match classes when looking for features. It's not quite there yet; I will need to think harder about this problem and why closure.jsp is sometimes unresponsive or plain wrong.

June 15, 2006

Use specialized collections in package com.jeantessier.dependency.

June 14, 2006

Looked a little at closure.jsp and tried to figure out why it returns an empty closure when I give it a perfectly valid starting point. The classes involved with TransitiveClosureEngine are not instrumented with Log4J, which hinders me a little. I will have to take care of that before I can investigate further. I also need to come up with sample data that reproduces the problem. I tried a trivial graph a.A.a() --> b.B.b() --> c.C.c() but that one worked just fine.

I was discussing the com.jeantessier.dependency.Visitor hierarchy with a colleague and complained about the complicated contract for children of VisitorBase, especially for the printers. During the discussion, it dawned on me that I could try composition instead of inheritance. The visitors would delegate traversal to a strategy, the way they do now, but they could also delegate processing of each node to another collaborator. The contract with the processor would still be fairly complex to preserve flexibility, but might be easier to manage.

A user raised an issue with DependencyClosure as they were trying to compute a class-level outbound closure. Some classes at the edges were missing unless they used -maximize during extraction. Without looking, it looks suspiciously related to some tricky behavior I documented in the Manual already.

June 10, 2006

Functional tests for com.jeantessier.dependency.MetricsGatherer.

June 09, 2006

I decided against having CycleDetector summarize the graph to do class-to-class cycle determinations. For the CLI and Ant users, they can use DependencyReporter to reduce the graph explicitly. For the GUI and web app versions, I will hard-code the reduction in the commands themselves. I'll have to see if it takes too long, maybe I can cache the reduced graph instead of recomputing it for every call.

June 08, 2006

For some reason, I cannot compute closures with the web app anymore. I will have to investigate further. query.jsp finds the node I'm interested in and shows dependencies in and out of it. metrics.jsp also sees it and can compute measurements about it. But when I try the same regular expression in closure.jsp, I get an empty result.

There is a problem with DependencyMetrics. The sub-heading under outbound add up to the total for inbound and vice versa. Sure enough, that's what the code is computing. I'll have to study my own code again to figure out what was the original intent. Maybe I can write some tests, this time around, to document what it is supposed to do.

June 06, 2006

Today is 6/6/6!!!

Wrote a functional test to investigate finding dependency cycles at the class or package level with a graph that goes down to the feature level. The fact that CycleDetector extends VisitorBase, with all its baggage, will not make it easy to sort it all out.

June 02, 2006

Opened a ticket (1499643) with SourceForge about my commit problem. Apparently, I am not the only one. I found at least one other ticket (1499575) with the same problem and others are also having trouble commiting but get different error messages.

Fixed my issue with table width in the new webapp pages for dependency cycles.

I tried to load the webapp with the dependency graph for Dependency Finder and find class-level cycles starting at com.jeantessier.classreader.Visitor, specifying it a maximum cycle length of 5. The CPU went to 100% for over two hours and never came back. In the end, I had to kill Tomcat. I'll have to investigate why this happened and if it is a bug or just that CycleDetector is not really useful in real life.

Found out SourceForge added a new permission on user accounts for Subversion access. I granted myself permission but it didn't work right away. I guess there is a propagation delay between the user management system and the permission system. It's all better now.

Upgraded the following development components:

Tomcat

:

5.5.15

->

5.5.17

June 01, 2006

Wrote a page for dependency cycles in the webapp. I still have only a minor issue with the layout of a table.

Cannot commit to SourceForge's Subversion for some reason. I am getting a 403 Forbidden error code.

May 31, 2006

Wrote an Ant task for dependency cycles.

May 27, 2006

Added code to specify the indentation text on CyclePrinter classes and stream data to a Writer.

Wrote a CLI tool for dependency cycles. Next will be an Ant task, a webapp page, and a GUI tab, not necessarily in that order.

May 26, 2006

Wrote a quick XML renderer for Cycle and an initial DTD.

May 25, 2006

Fixed a bug with name length-related metrics. It was assuming method names with parentheses in them and some of the tests were using plain names like "foo". I reworked the name computation so it is more reliable, for features and for classes alike.

May 18, 2006

Re-did the text rendering of Cycle from scratch. This time, I created unit tests around TextCyclePrinter. Getting Fit to process text somewhat properly was more difficult than I expected. At first, I tried using check rows, but it got confused by \r\n versus \n line terminations. Then, I tried using and ArrayFixture to break down the lines of the text, but when Fit reads in the expected cells, it strips the leading whitespace, which again breaks text comparison. For now, I check indentation in unit tests and general order of the lines in Fit.

May 17, 2006

Added the IntelliJ logo on the website, at the request of JetBrains. They asked for it as a small favor and I like IntelliJ so much that it was a no-brainer to put it up. I really think it's the best Java IDE around at this time.

Started working on text representation of Cycle.

May 13, 2006

Added Structure 101 to the Resources page. I think this might be the latest incarnation of reView. Both come from the same company and reView is no longer on their website.

May 12, 2006

Use specialized collections in com.jeantessier.dependency.Cycle and related classes.

Added acceptance tests and code to pass a SelectionCriteria to CycleDetector so I can use scope specifiers when looking for cycles.

May 10, 2006

More Fit tests to show how closures behave in the presence of a stop condition.

May 06, 2006

Created initial acceptance tests for dependency closures.

My license for InteliJ IDEA will expire soon. I must renew it. I hope they still give out free licenses to open source projects.

May 04, 2006

Someone complained that the behavior of the closure engine was not intuitive. I think I will write some acceptance tests to nail down the current behavior and consider some of the improvements they suggested.

April 20, 2006

Added character count and word count for method names. I added all the word and character count measurements to the default OOMetrics configuration.

April 19, 2006

The online documentation of the Ant tasks had fallen behind. I updated it to version 1.1.1.

April 18, 2006

Reduced the scope for version 1.1.2 so I can get it out the door sooner, and maybe even soon.

April 17, 2006

Started working on WordCounter to count the number of words in identifiers, including package names. Adjusted MetricsFactory to use it in computing the number of words in package and class names. I still need to deal with method names.

I might have to start creating metrics for individual fields. That might be quite a bit of work.

Cleaned up MaximumCapacityPatternCache and its tests to remove unused imports and use specialized version of collections.

April 13, 2006

Fixed JUnit run scripts to have correct FitLibrary in their CLASSPATH.

April 11, 2006

Upgraded the following development components:

IntelliJ IDEA

:

5.1

->

5.1.1

April 07, 2006

Finished converting all my development environments to Subversion.

Fixed TestDirectoryExplorer to use dedicated temporary files instead of some arbitrary folder within the project. The tests would break everytime that folder would gain or lose files. With the switch to Subversion, there was an explosion of ancillary files that finally decided me to do something about this test.

April 05, 2006

Got FitLibrary's FolderRunner to run under JUnit.

Tried Subversion for version control. With TortoiseSVN, a Windows Explorer plug-in, I can checkout the project just fine. With the official, command-line client, it kept dropping the connection. I tried the integration in IntelliJ and SmartSVN, but they were having difficulties too. Strange, and annoying. Now, I wonder if I wouldn't be better staying with CVS.

Moved Fit test images to files folder so FolderRunner can copy it to the reports folder.

April 04, 2006

SourceForge's CVS is back online. Time to merge the development lines I've been taking on three different computers.

Worked on the name character count and word count for packages and classes.

Upgraded the following development components:

Fit	:	1.1	->	none
FitLibrary	:	15Feb2005	->	20060116

April 03, 2006

Use specialized collections in com.jeantessier.metrics.

Worked on MetricsGathererFixture, trying to find a good way to represent metrics in a fixture.

I tried to use FitLibraryRunner20060110, which seems to be the latest version of FitLibrary, but it eludes me. DoFixture has changed package. The HTML pages now use CSS to manage colors and state and it seems I have to copy the stylesheet myself between the folder with the input files and the folder with the output files. And apparently, DoFixture now works differently, because I can many complaints about methods not being found, whereas it used to find them just fine using introspection on my classes.

I tried FitLibraryRunner20060110 because there was an incompatibility between FitLibraryForFit15Feb2005 and the latest fit-java-1.1. I was able to do a small change to FitLibrary and recompile it against Fit 1.1, but I'm not sure how the versioning of FitLibrary works. The new FitLibraryRunner20060110 has all the Fit classes inside of it, so I only need one JAR file. But it seems some of the semantics have changed along the way and I haven't seen any documentation so far. I'll have to dig some more.

March 31, 2006

SourceForge's CVS repository was down yesterday and today.

I signed up for a migration to Subversion. I have been heading some good about it and it looks like the switch is easy. Even though SourecForge's CVS is down, they were able to migrate my project and I could browse it using ViewVC. But when I tried to check it out, it failed. IntelliJ did not report any error, but didn't create the project either and the checkout only had website and was missing DependencyFinder. I tried with SmartSVN, but didn't get any further, except that it complained about a bad socket read at some point. Maybe if I have them migrate it again, after CVS is operational again, it will work better. Subversion would be nice: it has atomic checkins and it knows about folders, so you can move or rename parts of the filesystem and retain the history.

Started working on Fit tests for metrics for identifier length. I want to compute the length of package names, class names, and method names, both in number of characters and number of words.

March 30, 2006

I want to add metrics to find long identifiers. I could have length, in characters, of class names, package names, and feature names. With some clever parsing, I could also compute length in number of words.

Modified code using ClassfileLoader and CommandLine to use the generics version of Collection classes so they can use the new for() loop to iterate over collections.

March 27, 2006

Created a Fit test to try to capture the dependency cycle in the Visitor pattern. Adjusted CycleDetector to drill down packages and classes in its search for cycles.

Added CycleComparator to order cycles within a detector to ease testing. Short cycles come before long ones. Cycles of the same length are ordered according to the lexical ordering of the nodes in their path.

March 26, 2006

Augmented CycleDetector to detect cycles at the class level.

Added Fit tests for cycles at the class or feature levels. Next, I will need to look for cycles that span levels, like in the Visitor pattern.

    BaseClass
        accept(Visitor)
            --> Visitor
    ClassA
        --> BaseClass
        accept(Visitor)
            --> Visitor
            --> Visitor.visitClassA(ClassA)
    ClassB
        --> BaseClass
        accept(Visitor)
            --> Visitor
            --> Visitor.visitClassB(ClassB)
    Visitor
        visitClassA(ClassA)
            --> ClassA
        visitClassB(ClassB)
            --> ClassB

These cycles go from some feature to a class and continues through the features of that class. For this to work, CycleDetector would have to look at lower levels when seeking cycles from a node. It would have to drill down from PackageNode to ClassNode and from ClassNode to FeatureNode as appropriate.

I also fixed the image links in the Journal. Since I switched to a CGI script, the URL is no longer a prefix of the images/ folder where the images are. Links to images must now be prefixed with .

March 25, 2006

Added Fit tests to the all tests suite. I don't know why I had not done it before.

March 23, 2006

I figured out how to test each individual path for cycles in Fit tests.

Added more tests.

March 22, 2006

Created a Fit test for detecting dependency cycles. I think I got the basis for a simple fixture to trigger the CycleDetector. But I still need to work on verifying the cycles it detected.

March 14, 2006

Wrote a little more explanation around Perl regular expressions in the Manual.

March 13, 2006

Added new experimental "Related links" banner from Google to selected pages on the website. It does not pay like Google's AdSense, but it might provide useful links to people who visit the site. I added it to the following pages to start with:

• Introduction (between main text and quotes)
• Resources (at the top)
• Journal (at the top)

I also added a copy of the open source license for Dependency Finder on the website. Someone was inquiring and they couldn't find it on the web. It has always been included in the distributions, but it can't hurt to have it posted visibly too.

Upgraded the following development components:

JUnit

:

3.8.1

->

3.8.2

March 11, 2006

Fixed ModifiedOnlyDispatcher to only ignore .class files and delegate everything else, including JAR and ZIP files. This fixes the problem I had been having when pointing the webapp to a collection of JARs instead of unbundled .class files.

March 10, 2006

A user was asking questions about Monitor and how to reload a graph and pick up new classes. I augmented the Developer Guide to describe how you would use a Monitor and a ModifiedOnlyDispatcher with a ClassfileLoader to support reloading the graph and picking up changes.

As I was writing this, I realized that the monitor and the dispatcher are not linked to one another. This means that if the dispatcher decides to ignore a JAR file that hasn't changed, the monitor will not know about the classes in it and will remove them from the graph if it uses closed sessions.

I had experienced this behavior before, when using the webapp with a collection of JAR files instead of the loose .class files I was used to. I'm thinking that if I let the dispatcher travers collections of classes, like JAR or ZIP files, it would let the monitor know about all the .class files and the monitor would not remove them from the graph. That's already what ModifiedOnlyDispatcher does with directories.

March 06, 2006

Augmented DependencyMetrics to report the number of confirmed packages, classes, and features in the graph.

A friend of mine brought Structure 101 to my attention. They have a really nice user interface. Maybe some day I'll have something just as nice.

March 03, 2006

Wrote a long Fit test to check name, length, and index calculation for every instruction in the VM's instruction set.

I fixed a misspelling of dup_x2 (0x5B) and the reserved opcode breakpoint (0xCA).

I also added the new Java 1.5 instruction invokedynamic (0xBA).

March 02, 2006

Updated dependencies.dtd on the website. It appears I had forgotten to do so when I released 1.1.1.

March 01, 2006

Tied Instruction more closely to Code_attribute so that some of the instruction processing could be centralized there. Instruction is now a Visitable and VisitorBase automatically traverses them when processing a Code_attribute. So I was able to pull the code for iterating over bytecode from the various visitors into the abstract base class. Also, now that individual Instruction instances know about their corresponding Code_attribute and its Classfile and its ConstantPool, those that represent opcodes with indices into the constant pool can retrieve the correct ConstantPoolEntry themselves, instead of relying on client code to do it. These two changes together simplified greatly the various operations that walk the bytecode.

Instruction knows whether or not it holds an index into the constant pool. For now, I simply used the list from CodeDependencyCollector, but I will need to take a close look at the entire instruction set to make sure I have a complete solution.

I also take this knowledge into account when comparing instructions in Instruction.equals() and in computing Instruction.hashCode(). If the instruction actually refers to a ConstantPoolEntry, equals() and hashCode() will use the entry's equals() and hashCode() as part of their computation. Otherwise, they just do a byte-to-byte comparison of the instruction's data.

With these new tools, I revisited the case a user had brought to my attention on 2006-02-21. At the time, JarJarDiff only did byte-to-byte comparison when checking for code changes in methods. The user complained that they had changed only one method but JarJarDiff was flagging them all as different after a recompile. It turns out the constant pool was reordered between the two compilations, resulting in different indices and therefore different bytes in the bytecode. Now, instructions are isolated from any shuffling in the constant pool, so I expect that only the modified method will show up in the JarJarDiff report.

There were three. One was the method that had been modified, it had extra instructions. I was able to run ClassReader on the classes to compare the actual instructions for the other two. In both cases, the reordering of the constant pool had pushed the index of a reference beyond 255 and required an ldc_w instruction of 3 bytes instead of a simple ldc instruction of 2 bytes. I could introduce special treatment to equate ldc with ldc_w, but I feel this could lead to fairly convoluted comparison rules.

February 28, 2006

Cleaned up some dead or duplicated code.

February 26, 2006

Upgraded my wireless LAN from 802.11b to 802.11g. Larger bandwidth and somehow a larger range too.

February 24, 2006

Changed CodeDifferenceStrategy to compare instructions instead of the raw bytes. The next step will be to make instruction comparison smarter.

Also added Hex.toString(byte[]) to make it easy to log byte arrays. Previously, Hex could only write to OutputStream or Writer, which forced traces to write bytecode to a StringWriter in order to log it. Now, Hex uses a StringBuffer instead, which should speed things up a bit.

February 23, 2006

Played a little with smarter code difference determination, using the CodeIterator and Instruction classes. Instruction is not tied to its Classfile, so it cannot access the constant pool and lookup indices in the bytecode. I will have to tie it tighter to the object tree if I want it to compare the actual parameter to an instruction when deciding if something is different or not.

I found something interesting. In the compiled code I received from a user yesterday, there is and ldc_w instruction that points to a CONSTANT_class entry in the constant pool. Now, the spec clearly says it has to point to a either an int, a float, or a string. I wonder if maybe they used a non-compliant compiler or something.

Instrumented more debug info in the processing of bytecode from methods, when reading it and when comparing it in JarJarDiff.

Did a massive refactoring to use the new Java 5 for() loop in com.jeantessier.classreader, which meant using specialized versions of the Collections API. I'm not madly in love with generics, but the new for() is really cool.

February 22, 2006

A user complained that JarJarDiff flagged all methods as having code changes when only one methods had changed. They sent me the classes so I could investigate.

I took a quick look. The constant pool is different for the two classes. The compiler didn't put the entries at the same offset in each case. The bytecode makes heavy use of references to the constant pool, so the indices in the bytecode are different because the offsets in the constant pool are different. The way JarJarDiff looks for differences in the bytecode is a simple byte-to-byte comparison; it doesn't interpret the bytecode to see if they are the same instructions with the same parameters. It just compares the byte arrays for each corresponding method together and makes the method as having changed if any single byte is different.

I could probably make it a lot smarter, but that would mean it would have to analyze the bytecode and understand each instruction and its parameters and compare entries in the constant pool to see that they are the same even though they have different offsets. That could take a while, as there are many instructions. I have other tasks that I must finish first before I can think of tackling this one.

If this is critical for you, I might reconsider... Code-level differences is a new feature and I wouldn't want it to start in a useless state. At the same time, I'd like to ship at some point, so if this is not critical, maybe it can wait a future point release. I guess I'll sleep on it for a while...

February 18, 2006

Fixed default value for level in the Ant version of JarJarDiff. Somebody brought to my attention that it wasn't initialized and caused a NullPointerException if you didn't specify it in the <jarjardiff> tag. I also fixed its documentation.

February 15, 2006

Fixed typos in Manual around examples for extracting with the CLI.

February 05, 2006

Cleaned up ExtractAction.

February 04, 2006

Tested actual extraction in new WebWork-based ExtractAction.

January 29, 2006

Dependency Finder is now five years old.

January 27, 2006

I have been wondering how to replicate the current behavior of extract.jsp under WebWork. A WebWork action terminates before the JSP tied to its result is even started. The action has no way to stream data that would end up embedded in the result, the way extract.jsp streams parsing information to keep the user notified of its progress.

But I found that the JSP can use the <ww:action> tag to invoke another action while the JSP is rendered. Maybe I can use this to have a dummy action launch the extract.jsp and from within it, call the real action that extracts the graph and stream progress to the output stream.

January 24, 2006

Added a new XSL stylesheet, DiffToFullyQualifiedNames.xsl, to extract symbols from diff reports generated by JarJarDiff. Someone suggested that they could use these symbols to start an upstream closure to find which tests would possibly exercise the modified code. Apparently, their suites take a very long time to run and this would allow them to quickly run some regression to see that they haven't broken anything. Sounds interesting, I'm curious to see how it turns out.

I added start-includes-list and stop-includes-list switches to DependencyClosure, similar to scope-includes-list on DependencyReporter. This way, DependencyClosure can work from an explicit set of symbols instead of regular expressions. In the process, I had to do quite a bit of code duplication between the two CLI tools, which I don't like very much.

JarJarDiff currently only lists the names of removed/new packages, classes, and interfaces, without any of their internal details. For this trick to work, maybe it would help if it listed all the classes and features in a removed package.

There was already a ListFullyQualifiedNames that operated on dependency graphs. I renamed it DependencyGraphToFullyQualifiedNames to align it with the new DiffToFullyQualifiedNames.

January 19, 2006

Edited the Developer Guide to clarify the source of dependency events and how Dependency Finder removes duplicates.

Incremented the date in the copyright to 2006.

Started writing tests for ExtractAction. The WebWork people did a great job of encapsulating things to make it easy to test. Kudos to them.

January 18, 2006

Started redoing the webapp using WebWork.

Added the following development components:

WebWork

:

none

->

2.2

January 11, 2006

Played a bit with WebWork. I have been toying with the idea of rewriting the webapp using WebWork, but since it is merging with Struts now, I'm not so sure anymore. I don't want the webapp to bloat just because it needs a whole bunch of infrastructure JARs. Already, the default type of WebWork webapp requires 1MB of JARs before you've even started doing anything.

Upgraded the following development components:

Tomcat

:

5.5.12

->

5.5.15

January 09, 2006

Built a CGI script to rebuild the Journal from the individual text entries. I had to do a fair bit of scrubbing of the text files.

January 05, 2006

Studied the impact of logging a little further. If I remove all logging in CodeDependencyCollector, I can save about 15%.

Worked at breaking down the Journal into individual wiki-like files. The large, single file HTML was becoming too unwieldy. IntelliJ IDEA was stalling every time I opened a new tag. With the wiki-like notation, it will make writing entries a little easier since I won't have to type all those HTML tags anymore. I still need to figure out how to rebuild the HTML file from the individual pieces.

January 04, 2006

Took another look at the impact of logging. When analyzing Dependency Finder itself with all logging turned on, some of the lines from VisitorBase are over 300 KB long! But these logging calls are guarded by calls to Logger.isDebugEnabled(), so those 300K lines usually aren't generated.

January 03, 2006

Cleaned up build.xml.

December 30, 2005

Making the switch to Java 5.0. If you're still running it with the JDK 1.4, let me know ASAP.

Trying to compile for Java 1.4 with the 1.5 compiler is just too risky. It catches uses of the new language constructs, but not uses of new API elements. Relying on the compiler to keep me straight is just not reliable. And I'm tired of keeping multiple versions of the JDK just so I can compile Dependency Finder in near-obsolete technology.

Removed JARs for Xalan. I'll rely on the version what ships with the JDK. It means that dependencies shift from org.apache packages to com.sun.org.apache packages, which seems a little fishy, especially since some of those have internal in their name. I'll have to read some more documentation about how we're supposed to use those XSL-related classes. On the plus side, I can now use the default XMLReaderFactory.createXMLReader() method instead of hard-coding the Apache implementation.

Upgraded the following development components:

Xalan

:

2.6.0

->

none

December 18, 2005

Fixed the Manual section on JarJarDiff.

December 17, 2005

Fixed the Manual section on JarJarDiff.

December 16, 2005

Cleaned up build.xml. I had been running the tests from my IDE for too long and some of the targets had gotten broken.

December 15, 2005

Finally settled to a format for my Fit tests. I got rid of my early experimentations and kept only the most comprehensive ones. I also posted a suggestion for a fix to FitLibrary to bring it in line with Fit 1.1. We'll see how that goes and if it gets incorporated.

Upgraded the following development components:

Log4J

:

1.2.8

->

1.2.13

December 14, 2005

Augmenting the Developer Guide to incorporate the extra explanations from yesterday's email. I also want to explain how com.jeantessier.dependency.VisitorBase uses multiple cases of the Template pattern to present a generic graph traversal mechanism with multiple extension points. And I do mean multiple.

December 13, 2005

Sent a long email to a user who was asking how to use the classes in the Dependency Finder library to extract dependencies and how to operate on the graph. I should work that into the Developer Guide.

December 12, 2005

Tech docs around the event-based processing when loading a .class file using a ClassfileLoader.

Enhanced NodeFactoryFixture so I can test both inbound and outbound dependencies in a single Fit table.

December 11, 2005

Tech docs to describe the ClassfileLoader hierarchy.

Came up with a new subclass of DoFixture to help me test dependency graphs as part of Fit tables.

December 07, 2005

Got a bunch of questions from a user on how to use the various classes in the Dependency Finder library to process .class files and generate dependency graphs ready for further processing.

December 06, 2005

I finally figured out why Fit behaved differently between my desktop and my laptop. It turns out they had a different version of Fit. I must have downloaded Fit 1.0 to my laptop by accident. Anyway, I tried diff'ing the fit.jar and noticed they were different. I followed up with a quick run of JarJarDiff, just to be sure, and there it was, lots of signature and code changes. I copied Fit from my desktop back onto my laptop and voilà! it now works like a charm.

Also wrangled DoFixture into doing what I wanted it to do. It works great for dealing with simple values. Now, I need to figure out how to call Node.addDependency() and pass Node values from the Fit tables. I guess I will need one of those fabled TypeAdapter.

December 05, 2005

Wrote Fit tests using fit.DoFixture so I don't need to write custom fixture classes. The book and the documentation are a little sketchy and I need quite a few trial and errors to get it to do what I want. Or, at least, almost what I want.

Fit still behaves differently between my desktop and my laptop. I am still baffled.

December 01, 2005

Wrote FitTestSuite and FitTest to run Fit tests with JUnit. I got it to automatically pickup HTML files in a given folder and to have tests fail when there are failures in one of the tables in an HTML document. Now, I can subclass FitTestSuite and give it a folder name in suite() to have it process a given folder.

Fit automatically compacts phrases like is confirmed into camel-case identifiers like isConfirmed. It works on my desktop but not on my laptop. The Fit version is the same, pretty much everything is the same. Maybe the JVM patch version is different, but that's it. I am baffled.

November 28, 2005

Started writing Fit tests for creating dependency graphs. I hope that using tables to create test setup will be easier. I still have to figure out how to run them automatically using Ant.

November 27, 2005

Started reading about the Fit testing framework.

November 19, 2005

Refactored the code to instantiate the strategy in JarJarDiff into private utility methods. This helps clean up the main code.

Also cleaned up some of the JSPs in the webapp. Now, both extract.jsp and load.jsp highlight missing files.

November 17, 2005

For now, I'll treat typed arrays as independent classes. We'll see how it goes.

Continued work on the circular dependency detector. Dealt with comparing and ordering cycles.

November 16, 2005

Added a maximum cycle length to CycleDetector.

Healed tests for com.jeantessier.dependency.TestHTMLPrinter. It had been a while since all the tests ran green.

In Java 5.0, some code compiles to strange forms. Enums have an implicit values() method that attempts to copy the internal array of the enum's values. It does this copying by calling the array's clone() method, but the array is typed with the enum's type, so it looks like a call to MyEnum[].clone() and I can't seem to decide what kind of dependency that is. I need to investigate further.

November 15, 2005

Started a new com.jeantessier.dependency.Visitor to look for cycles in the dependency graph. It seems that many people out there who are interested in dependencies really want to detect cycles. I still think that not all cycles are necessarily bad, but maybe at least listing them can be of some use. You definitely want to avoid circular dependencies between large-scale components, but one component might be implemented as many, tightly coupled, packages. Since Dependency Finder is not aware of organizing principles beyond the package, it's usefulness at detecting unwanted cycles will be limited.

Further refined the webapp's style.css.

Posted Dependency Finder on Google Base.

November 14, 2005

Experimented with the webapp's style.css so hyperlinks don't stand out too much.

November 09, 2005

Fixed ClassReport so it does not show modified declarations for classes whose declaration has not changed, according to the strategy in use.

November 07, 2005

Merged the published_API branch back into the trunk.

Fixed file count based tests.

November 06, 2005

Fixed interaction between DifferencesStrategy and its DifferenceStrategy so we can overlook class differences when the strategy dictates that we do. This is to support the case when a class has a backwards compatible change to its declaration but some of its features have non-backwards compatible changes. I still want to report the feature changes, but I want to ignore the class declaration difference.

November 03, 2005

Fixed JarJarDiff so the -level switch that takes a possible class name can look for a constructor that takes a DifferenceStrategy before it falls back on the default constructor. This way, users can specify strategies that extend DifferenceStrategyDecorator and have them decorate the same basic strategy that APIDifferenceStrategy and IncompatibleDifferenceStrategy do.

I want to do the same thing for printers, too.

JUnit launch scripts for Unix. I have had the Windows .bat files since forever, but only got around to writing Unix equivalents last year. I finally checked them in.

November 02, 2005

Last tests for IncompatibleDifferenceStrategy.

November 01, 2005

More tests for IncompatibleDifferenceStrategy. I am done with class-level tests and starting field-level tests. Then, I will have to do constructor and method level tests. It's gonna be a while, still.

October 30, 2005

More tests for IncompatibleDifferenceStrategy. I had a little bout of analysis paralysis, but I think I'm out of it now.

October 28, 2005

Refactored of tests for IncompatibleDifferenceStrategy. Not done yet.

October 25, 2005

Reworked the entire DifferenceStrategy hierarchy. For a short time, I considered using a chain of responsibility pattern, but setting it up in a flexible way, from the command-line, was just too complicated. So I will stay with decorators and some limited combinations, based on command-line parameters.

October 19, 2005

I'm going to make IncompatibleDifferenceStrategy a subclass of DifferenceStrategyDecorator.

I read that JDK 1.5 has JMX instrumentation to help with profiling. I should look further into it to help with performance issues with regard to logging.

October 15, 2005

More test data and initial tests for IncompatibleDifferenceStrategy.

October 14, 2005

More test data and new IncompatibleDifferenceStrategy to replace PublishedDiffToHTML.xsl.

October 13, 2005

Created some more test data to test the new DifferenceStrategy to replace PublishedDiffToHTML.xsl.

October 12, 2005

Reworked the way DifferencesFactory determines code differences. It is now all done by the strategy and the factory only sets a flag in the appropriate CodeDifferences instance. I had to adjust DifferenceStrategy in order to expose isCodeDifference(). I eventually would like for constant value and code differences to have their own instance of Differences instead of just being flags. Objects can hold information about the nature of the difference.

I had an idea of using a chain of command pattern instead of my current strategy pattern for DifferencesFactory, but that would make it much more complicated to insert user-defined strategies. So I'll stick with my current design for now.

October 11, 2005

Removed obsolete method Differences.isEmpty().

October 09, 2005

Got rid of the Validator hierarchy. The only one that was used at all was ListBasedValidator which I turned into a decorator for DifferenceStrategy. The resulting design is much cleaner and it makes DifferencesFactory that much simpler. One last strategy for backward-compatibility-breaking changes and I'll be done with this development item.

October 04, 2005

Changed all the JarJarDiff variations to use the new DifferenceStrategy. Each has a new level parameter to tell it which strategy to use. Level can either be a predefined constant, api for APIOnlyDifferenceStrategy or code for ComprehensiveDifferenceStrategy. In addition, users can set it to the name of a class that implements DifferenceStrategy and that has a default constructor; the program will instantiate it and use the instance to analyze API differences. I might so the same thing for the selection of printers in many of the tools.

October 03, 2005

Finished APIOnlyDifferenceStrategy. Now, I need a switch in JarJarDiff to select the strategy at runtime.

October 02, 2005

Introduced APIOnlyDifferenceStrategy to compute API differences without regard for code-only variations. I also want it to not show the value of constants, but that will require more tweaking. At least, I now have tests in place.

September 29, 2005

Tracking value differences for constants via their declaration is too sensitive. I will want cases where I only look at changes that could break backward compatibility and these will show up. I'm back to finding a way to distinguish declaration changes separately from value changes.

I'm seriously considering simply switching to Java 1.5. It will let me use all the features from the library and also some of the new enhancements in the language, notably the new for() loop for iterating over a collection. At a place I used to work, we wrote software that customers would run as part of their operation. Some customers were still using Java 1.3, so our build environment was limited to 1.3 on purpose so we could detect any change that would not be compatible with 1.3. In order to run Dependency Finder reports as part of the build, it had to run under 1.3. I figured there might be other installations with similar constraints, so it would be nice if they could still benefit from Dependency Finder and I have kept it at Java 1.3 ever since. But it is starting to hurt, and I don't care quite as much anymore. Java 1.3 is approaching retirement, so it should be safe to move up to Java 1.4. But Java 1.5 is even better and it has been out for a while now. Surely anybody serious about Java development has moved to it by now!

If you're using Dependency Finder in an environment that requires Java 1.4, let me know. Otherwise, I'm just going to move up to Java 1.5 and ease my own suffering. :-)

September 28, 2005

Implemented equals() and hashCode() in subclasses of ConstantPoolEntry to help in discovering API differences. It started with the need to compare constant values to detect when they change between versions. Since those make up the majority of this hierarchy, I decided to do it all.

One interesting point is when comparing two values of either Float_info or Double_info, where I need to compare float or double values respectively. I cannot use operator ==== because rounding errors could make it fail when the values are otherwise the same. Since Java 1.4, I can use Float.compare(float, float) and Double.compare(double, double), but not in Java 1.3. So far, my policy has been to keep the code runnable on a Java 1.3 virtual machine. I used to do it by compiling the release version with JDK 1.3, but as of 2004-06-18, I use the source and target attributes to the <javac> Ant task in build.xml. This way, I only need JDK 1.5 on my machine. But this control mechanism does not guard against calling features that were not in JDK 1.3, like those compare() methods. The compiler compiles them just fine, and the code runs fine until JarJarDiff hits a float or double constant that didn't change and it tries to call my new equals() method to verify it. It then explodes with a NoSuchMethodError for either Float.compare(float, float) or Double.compare(double, double).

September 26, 2005

Introduced DifferencesStrategy and started using it in DifferencesFactory. In the process, I noticed that I never modified FieldDifferences to take into account the change in value for constants. I fixed it, but I'll have to also fix ClassReport and the XSL stylesheet in DiffToHTML.xsl.

September 23, 2005

Upgraded the following development components:

Tomcat

:

5.5.9

->

5.5.12

September 22, 2005

Started using PackageMapper in DifferencesFactory.

September 20, 2005

Created PackageMapper to have one place where a collection of classfiles get mapped to packages. Reworked its use into ComprehensiveDifferencesStrategy.

September 12, 2005

Renamed JarDifferences to ProjectDifferences, since it deals with the entire project and not any single JAR file.

Finished ComprehensiveDifferenceStrategy. Now I have to use it in DifferencesFactory.

September 11, 2005

Continued work on DifferenceStrategy to encapsulate the behavior behind Differences.isEmpty(). Got through the tests for packages. Now, I need to handle entire projects and create a subclass for backward-compatibility breaking changes.

September 09, 2005

Continued work on DifferenceStrategy to encapsulate the behavior behind Differences.isEmpty(). Got through the tests for classes and interfaces.

It turns out that constants have their value embedded directly into their field entry. I adjusted the new difference engine to look at those values too (the old one was ignoring them).

September 08, 2005

Started work on DifferenceStrategy to encapsulate the behavior behind Differences.isEmpty(). Got through the simple tests for fields, constructors, and methods.

Ran a quick test with a large JAR file: removing all calls to Log4J reduces extraction time by 50%. I should look into more discretionary use of logging to make the application more responsive, specially for large codebases.

September 07, 2005

After staring at the code for some time, I am finally convinced the various subclasses of Differences each expose enough information that I can come up with a strategy object that can look at each one and determine if it is empty or not. At first, I had thought of using a visitor, but that would mean either changing the Visitor interface to have a return value, or make the visitors stateful. Neither is appealing to me, so I'll stick with my idea of using the Strategy pattern instead.

Fixed RemovableDifferences that was directly referencing fields internally instead of going though accessors. Since the places it was doing the accessing will move to a strategy, it made sense to go through an accessor instead so the calling code will be easier to move later on.

September 05, 2005

I am stuck. On two fronts. With JarJarDiff, I can't seem to see a way to insert some kind of strategy in the difference engine to choose between all differences and only those I judge might break backward compatibility. With the HTML-ization of the web app output, I can't seem to find a clean way to go from preformatted text output to hyperlinked HTML. This is frustrating.

September 01, 2005

Fixed a typo in the manual.

August 25, 2005

Tried to create an HTMLPrinter for the webapp so dependencies can be hyperlinks to further queries. Right now, it is just a quick hack. It will need more work to make it pretty and generic enough.

August 24, 2005

Turns out my manual tests for JarJarDiff's filtering were using the wrong filter files. I am glad to report that the filtering function is working just fine. And now, I have tests to prove it.

Upgraded the following development components:

IntelliJ IDEA

:

5.0

->

5.0.1

August 22, 2005

Toyed with the idea of removing deprecation-related entries from JarJarDiff reports, in addition to the documentation-related ones I removed two days ago. In the end, I decided to keep them since they impact the compilation messages you get when building code.

Further modified JarJarDiff to use Validator as a filter instead of the two documentation parameters it had before. Now, it uses a new ComprehensiveValidator by default but if supplied with the -filter switch, it uses a ListBaSedValidator instead to limit the report to selected elements. I have to modify ListBaSedValidator to throw an exception when its input file is missing. I am not finished yet. There are no tests for filtering and when I pass in the switch, the resulting report is empty. This shouldn't take too long to sort out.

August 21, 2005

For some obscure reason, my laptop won't let me start SmartCVS anymore. It complains that I don't have Java 1.4 or later installed. I have JDK 1.5! I tried everything, up to and including uninstalling both SmartCVS and JDK and reinstalling them, to no avail. I figured I'd use the CVS support in IntelliJ IDEA, but the branch only exists on the source code and not on the website module, which confused my IDE for a little while.

Upgraded the following development components:

SmartCVS

:

4.0.5

->

4.0.6

August 20, 2005

Started fixing JarJarDiff to remove all the parts that dealt with the level of documentation as separate entries in the XML report. The next step will be to use the lists from -old-validator and -new-validator to trim the report itself, not with an XSL stylesheet. I will do this work on a separate branch and merge it in when I'm done. The branch is named published_API.

Fixed textjunit.bat and junit.bat to allow for spaces in the JAVA_HOME environment variable.

August 15, 2005

Found more ideas for object-oriented metrics from the works of Kenneth L. Morris. I will have to research it a little further.

August 13, 2005

Thought about my current dilemma regarding what to display in JarJarDiff reports between entire APIs, published ones only, and selective display. I thought I could do it with XSL transformations, but those are becoming too complex. Plus, there is an issue with using function calls in predicates that I still haven't resolved. I'm starting to think I'd better generate different reports from JarJarDiff and have a single stylesheet to render them in HTML. I might do the same for the stylesheets that narrow down dependency graphs instead of using DependencyReporter.

Upgraded the following development components:

JDK	:	1.5.0_01	->	1.5.0_04
IntelliJ IDEA	:	4.5	->	5.0

August 06, 2005

Back from vacation (yeah, it was three weeks).

July 26, 2005

Fixed typos in Manual for UNIX installation instructions.

July 16, 2005

Gone on vacation.

July 13, 2005

Finished support for negative closure.

Checked in a "cleaned up" version of DependencyGraphToRDF.xsl and associated script.

July 11, 2005

Gave some thoughts to computing negative closures using the tools that are already there. All I really need is a new stylesheet to extract the fully qualified node names from the XML graph without any indentation.

July 06, 2005

I started to reorganize the package com.jeantessier.dependency (on paper) to separate the classes that deal exclusively with the graph from those that deal in terms of dependencies (from those about transitive closures). Maybe it's not as bad as I think.

Started looking at how Java 5.0 stores some of its more exotic constructs, like variable argument methods and generics. I found some links to the updated JVM specification. I have some work ahead of me. The good news is that its all new special attributes and new access flags, so everything kinds works already. It's just that they are not really accounted for when extracting dependencies or computing metrics.

Moved Hex from package com.jeantessier.dependencyfinder to package com.jeantessier.text so classes in other packages can use it to dump binary data to log files, like Custom_attribute when logging the content of unknown attributes.

July 05, 2005

Somebody suggested using Dependency Finder to generate data flow graphs. They asked if inbound dependencies on fields couldn't map to setting the value and outbound dependencies to reading the value. The goals are different between dependency graphs and data flow graphs. But this got me thinking that maybe just having a different collector could do the trick. The Node class and its related classes are generic enough to represent any kind of relationships. It is up to the consumer of the graph to infer meaning from things like "inbound" and "outbound" nodes.

June 23, 2005

Upgraded the following development components:

Ant	:	1.6.2	->	1.6.5
JDK	:	1.4.2	->	1.5.0
Tomcat	:	5.0.28	->	5.5.9

June 18, 2005

Took one more look at using JDK 1.5 to generate the code for JDK 1.3 instead of JDK 1.3 itself. The differences are very minor, from what I could see. Mostly, it has to do with how JDK 1.3 was resolving method calls to their latest definition, while from JDK 1.4 forward they are resolved to the most specific class of the reference. This has no ill effect on running the code in the JDK 1.3 virtual machine.

June 17, 2005

Worked a little more on SelectedDiffToHTML.

June 16, 2005

A coworker requested that pages from the web app be bookmarkable. The easy solution is to replace the "method" attribute of <form> tags in the JSPs. The URLs can get pretty large. Just the parameter names for advancedquery make it 600 characters long. With web proxies that can truncate URLs at 1K, that leaves about 400 characters for specifying regular expressions. I haven't checked it in yet.

June 08, 2005

Added code difference information to the XML reports that come out of JarJarDiff. The XSL stylesheets are giving me trouble. They can't seem to pick up the new XML tags. Strange.

It turns out that my CVS configuration in IDEA missed two test data files the other day. I guess I'll stick to SmartCVS for checkins.

June 06, 2005

Started on computing code differences for methods that have not changed otherwise.

Working with IntelliJ IDEA and all its refactorings is really nice. I still have a little difficulty configuring the project just right, especially with support for JSPs without turning the entire project into a web app.

June 01, 2005

SourceForge stats are back and WOW! they're great. Check it out! Release 1.1.0 got over 1000 downloads and Release 1.1.1 is already past 250 after just two weeks. This is awesome. I'm really happy.

May 29, 2005

Still looking at tracking changes in the bytecode as part of JarJarDiff report.

A few additional fixes from IDEA's analysis.

May 28, 2005

Started looking at tracking changes in the bytecode as part of JarJarDiff report.

May 25, 2005

Installed IntelliJ IDEA. Ran the built-in analysis tool and found a few places where signatures included exceptions that were no longer thrown and some unused parameters. I'm used to IDEA on the Mac, so I need to adapt to the key bindings on Windows.

May 24, 2005

Updated the code coverage analyses on the website with runs against Release 1.1.1.

Got my free license for IntelliJ IDEA.

Googled for "Dependency Finder", out of curiosity, and found two interesting hits.

The first one was reView. I had already stumbled on it a while back and I must say the demo on their website was pretty impressive. It has a lot of what I'd want to see in a graphical UI for Dependency Finder. They sell licenses for $1,999 each. Whew! Their brochure has a lot of great arguments for managing dependencies from a financial point of view. And since Dependency Finder is free, it's an even better deal.

The second one was Hackystat. This one is a metrics framework that has Dependency Finder as one of its tools for getting information about a codebase. It is from the University of Hawaii. They use a dependency graph, which they post- process, to extract coupling information about programming elements. I think they might have an easier time collecting their data if they used OOMetrics instead. I'll have to get in touch with them and see if see if that is the case.

May 23, 2005

Updated the website with data regarding Release 1.1.1.

Applied for a free license for IntelliJ IDEA as part of their Open Source License program. I may finally move away from Emacs. It does have a lot more refactorings and a lot of smart features. We'll see.

May 18, 2005

Generated some of the data for the Samples regarding Release 1.1.1.

May 17, 2005

Release 1.1.1.

Also found about a new graphics package for rendering graphs: JUNG.

May 12, 2005

Looked at using JDK 1.5 to compile Dependency Finder for the 1.3 runtime. I can use source and target attributes on the Ant targets, but the .class files are different from the ones compiled with JDK 1.3. I need to investigate further to make sure they will run fine with the 1.3 runtime.

May 02, 2005

Tech docs.

Finished release 1.1.1.

Considered merging ClassFinder and ClassList into a single tool. But in other areas, I have simple tools that each do one thing only, so I might as well let these two follow the same pattern.

April 30, 2005

Tech docs.

April 29, 2005

Someone brought Ivy to my attention. It is a package for analyzing dependencies and producing reports with integration into Ant, Maven, Cruise Control, and even Eclipse. I couldn't understand the sample reports on the website, and I don't think it offers the same kind of dynamic interaction capabilities as Dependency Finder. It is good that there is slowly more and more tools to support object-oriented design.

Tech docs.

April 28, 2005

Tech docs.

April 24, 2005

Realized I had mistakenly swapped the current CVSDiff and CVSChangelog reports when I last updated them.

April 19, 2005

Adjusted the JSPs that manage the graph in memory to keep track of extract, update, and load times separately. This way, extract.jsp limits itself to extract and update times, as appropriate, while load.jsp takes care of load times.

April 18, 2005

Dependency Finder: social networking for your code!

Cleared up the documentation regarding the web application and how to get a dependency graph loaded.

Also changed the obscure delta application variable to the more aptly named duration. It tells how long the last extraction or loading took. In the case of extracting a graph, I should probably have separate numbers for the last full extraction and the last update only.

On my new PC, I can't seem to start Tomcat on port 8080. It keeps complaining that port 8080 is already in use, but I can't figure out by what. I've changed the default config to port 8000 in the meantime and it seems to work just fine.

April 17, 2005

Started working on a new printer for transitive closures. This will be harder than I thought. com.jeantessier.dependency.Printer has way too much machinery geared solely at printing graphs to be useful. I also need to figure out a way to find the closure's starting point after the engine has computed it and possibly after the graph has been reduced possibly as far as package-to-package.

April 14, 2005

Tech docs for closures.

April 12, 2005

Someone is having problems computing closures and finding paths between components. Can't blame them, the output from DependencyClosure is very hard to follow.

They claim that in JDK 1.3.1, there is a path from JTree to Toolkit through MouseEvent. DependencyClosure found one through Component and I looked at the source for MouseEvent from 1.3.1_09 and there is no mention of Toolkit in there. There is in JDK 1.4 but not in 1.3.

At the time, I didn't have rt.jar handy to repeat the tests, so I used DependencyFinder.jar instead. At first, I tried to find a closure from com.jeantessier.dependencyfinder.cli.DependencyExtractor to com.jeantessier.commandline.MultipleValuesSwitch. I thought it called CommandLine.addMultipleValuesSwitch() and would then find MultipleValuesSwitch from there, but it doesn't. Of course, DependencyExtractor does not have multiple value switches, so there is no feature-to-feature path from it to MultipleValuesSwitch. I will need to document this situation, in case someone else comes upon it too.

Gave some more thought to a specialized printer for transitive closures.

Finishing the documentation shouldn't take too long. Definitely not long enough to justify having a beta release while we're waiting.

April 10, 2005

Uploaded screenshots to SourceForge.

April 07, 2005

Adjusting wording on the GUI to make it a little more intuitive. Also played with fonts to get something close to Courier in graph displays.

Had a "teachable moment" with a colleague today. I think he finally saw the benefits of using Dependency Finder when dealing with a large, intricate codebase.

I'm going to make a beta release while I finish updating the documentation.

I also need to start working on a new printer for closures. The normal graph printer makes results too hard to read.

April 06, 2005

Talked to a former coworker who is still using JarJarDiff. He mentioned that having it also report on methods whose implementation changed without impacting their signature. This is the same as Request for Enhancement #900448. So don't despair, it'll come one day.

April 05, 2005

Checked in fix for case where group is a single .class file.

April 04, 2005

Fixed the ClassfileLoaderEventSource to record the first dispatch call and use it when dealing with a group of a single .class file.

April 03, 2005

Wrote a sequence diagram of what happens when you call ClassfileLoader.load(). This will help me figure out how to handle the case where processing a group consisting of a single .class file.

April 02, 2005

Wrote a unit test to replicate the problem with groups of a single .class file.

March 31, 2005

Found a problem when analyzing individual .class files. For some reason, the GUI keeps on skipping them. It turns out the dispatcher gets called twice: once when initiating the group processing and a second time to deal with the file as part of the group. The GUI uses a ModifiedOnlyDispatcher, which instructs the ClassfileLoader to ignore the file after the first time, so the file is never read. The intricate collaboration between the various ClassfileLoader subclasses will make this hard to figure out.

March 30, 2005

There is a problem with DependencyFinder: it ignores .class files passed in explicitly. Somehow, the dispatcher seems to think it should ignore the files. This is because the dispatcher is asked twice about the file: once as a group and once as an individual file. Since the file hasn't changed between being looked at as a group and as a single file, the ModifiedOnlyDispatcher tells the loader to ignore it the second time around. Maybe ClassfileLoaderDispatcher needs both a dispatchGroup() method and a dispatchFile() method instead of the current single dispatch() method.

March 28, 2005

Looking into setting Font for UI elements in the GUI.

March 27, 2005

Finished adjusting all the remaining dependency graph-related stylesheets to use the new information about inferred versus confirmed nodes. I took the opportunity to update all the HTML stylesheets to use the CSS styles instead of hard-coded HTML tags.

March 26, 2005

Fixed NodeHandler to use the new confirmed attribute in the XML description of graphs. In the process, I added new tests for NodeHandler and NodeLoader, which will help a little with the code coverage of the test suite.

March 24, 2005

Copied confirmed status when computing transitive closures.

March 22, 2005

Adjusted dependency graph-related stylesheets to use the new information about inferred versus confirmed nodes. More to come.

March 21, 2005

Added support in the dependency graph's TextPrinter and XMLPrinter to show inferred versus confirmed status of nodes. I had done most of it during a spike back in January, but I didn't have any tests for it. Now, I need to fix the DTD and the XSL transformations. And NodeHandler too.

March 17, 2005

Added windowtitle to <javadoc> target so the Javadocs have a proper window title when viewed with a browser.