Today I found the problem. As I had begun to suspect, there was a simple error in an underlying data-structure.

The lesson? Write unit tests carefully! It turns out that although I had written a test for the faulty method, the particular data-set I used in the test special-cased around the bug. What I should have done was use multiple data-sets (pretty obvious) and made sure they were more realistic (in this case I had used a single contiguous set of bits). If I had done this originally, then I would have found the problem much much earlier.

[Update: Ironically, I originally used java.util.BitSet instead of my hand-rolled data-structure but was running into memory usage problems so I replaced a bunch of Maps and Sets with my own versions optimised for their particular usage in the algorithm. It turns out that for this particular case, the java.util version is entirely adequate. There's another lesson here :-)]

A caveat:

• correctness has been checked on small number of examples only

Benchmarking shows (as expected) quadratic time O(n²) and linear space O(n).

Needless to say, I’m pretty pleased with this result.

With regards the profiler, it was indeed much nicer than the Eclipse TPTP experience, but you need to read the fine print. I can’t say how much time I wasted trying to discover where all my heap was going before I read (my bold):

The total number of objects allocated for each class that Profiler presents (in both “Record Object Creation” and “Record Object Creation and Garbage Collection” modes) is exact, whereas other numbers (such as total object size) and the reverse call graphs are by default obtained statistically.

Ultimately, using a combination of the netbeans profiler, jmap -histo, and the profiler’s ability to record the stack when allocations happen proved to be the most useful approach. The instant illumination of seeing that it’s your Map implementation that’s responsible for all the space, then looking at the allocation stack traces and discovering that it’s actually one particular use of the Map that’s the problem.

It seems that a commercial offshoot of some HP Labs research is actively developing a similar concept.

• highlighting all occurrences of the current element (if the cursor is on a var, method or field, then all occurrences in the file are highlighted, including markers to the right of the scrollbar so I can see where it’s used elsewhere in the file)
• toggle of comment/uncomment; netbeans has these has two separate commands (why would I ever want to comment an already commented line?)
• support for generics in completion (and smart filtering wrt completion in general) is woeful; if I have a variable s declared to be of type java.util.Set<Foo> and I type s = new then I expect completion to a) offer only (non-abstract) classes that implement the Set inteface, and b) throw in the appropriate generics stuff as well, e.g., HashSet<Foo> rather than just HashSet.
• multiple launch profiles
• junit integration

Of course, if sites like these offered decent RESTful interfaces then anybody could work at designing a decent piece of information software.

This is great! I was kicking around this kind of idea with Jim before he left for his PhD in France.
I’m looking forward to a close read of it.

[update: It seems that the problem is not with the serialisation, but that the Resource "contains" the objects twice. How does this happen? First the objects are added to the Resource with getContents().add(objectx), then later something like object1.setParent(object2) is called where parent is a containment relationship. Thus, the resource "directly" contains object1 and object2 but also transitively contains object1 via the containment from object2.

The real question is "why?". In earlier versions of Tefkat this did not happen. When the transitive containment was established the direct containment was removed (I think -- it could be that the serialisation was smart and ignored the duplicates). Either way, this is something that's crept into EMF in one of the more recent releases (2.2.0 or 2.2.1 maybe?). So far, no joy on tracking down the real root of the regression.]

int[] foo = {1, 2, 3, 5, 7, 11};
testPrimes(foo);


or, even worse:

testPrimes(new int[] {1, 2, 3, 5, 7, 11});


rather than being able to just write:

testPrimes({1, 2, 3, 5, 7, 11});