Purposive without Foresight

The puzzle
The participation rules

First of all, many thanks to the participants, all the people who are interested and I apologize for the delay of result delivery. A lot of things has happened in my life during last several months. My daily job has very little to do with Java or Rete-based rule engines now, and almost all weekends during the last several months were occupied for the preparation of my wedding. But they are not good excuse for the lack of discipline and the awareness of responsibility for Drools Puzzle. Sorry everyone, I’m back to Drools Puzzle and will run it regularly from now on.

Here are the links of original submission from participants. Please let me know if I missed any:

• Carlos Bustamante
• Chris Barham (Ellen: Chris, I did get your email but there was no jar attached. Maybe you forgot to attach the jar. Sorry for the late message. I’ll check closely when the email arrives next time!)
• Dan Berindei
• Elmo Nazareno
• Geoffrey DeSmet
• Johan Lindberg
• John Kirchmeier
• Reginaldo Delfino
• Scott Reed
• Dirk Farin

Below is the evaluation of last round from Dr. Gernot Starke:

Drools-Puzzle #2: And the Winner is…Scott Reed (…. applause …) Dunnit! A dozen solutions to the second installment of the (now famous!) Drools puzzle contest arrived at my desk, originating from brave men spread all over the world (all right, at least from Australia, South-America, North-America and Europe). I evaluated during a brief holiday in souther Turkey – without internet access and with only Drools 4.0.1 plus Eclipse 3.2 installed on my MacBook. That initially sounded all right for me – but wasn’t. Read on why I failed to evaluate a few (gorgeously interesting) solutions with this setup!

C++ for highest performance

Like in puzzle #1, all rule-based solutions were outperformed by Dirk Farin’s C++ version. Dirk: You’re great. But we’re using rules to factor business-logic out of java applications… If we ever need to build something ultra-performant, we’ll surely remember your approx-one-nanosecond-solution. Btw: I took your source and could immediately build-and-run it on the Mac Dirk used numbers instead of (symbolic) names, which makes his solution-output a little hard to read… Although admirable in performance, Dirks’ solution still is a C++ program… and I didn’t let those count in the puzzle…

Standalone CLIPS

The next hurdle in evaluation came from Johan Lindberg from Sweden: He send a pure CLIPS solution, which wouldn’t run in Drools either…(Ellen: My bad, I said CLIPS source is welcome in the participation rules. Sorry Johan! We’ll see whether the CLIPS parser is ready in latest Drools distribution)

Johan seemed to be pleased with the puzzle and asked for more. Johan & Dirk – please send us Drools solutions next time

Only Seeing is Believing

Like in any athletic competition I required the winning solution to run on my machine – during my evaluation. I didn’t actually feel the urge to debug or modify submissions so they would/could run in my limited setup (see above). Although this may sound unfair (“but we’re still using 3.x in our project”), I had no other chance.

Therefore, solutions without source-code (Carlos Bustamante), with requirements to “mvn install” additional drools modules (Geoffrey DeSmet) or with Drools 3.1 rule syntax didn’t count either. Sorry guys – life can be tough. More on Geoffreys’ solution later on… Now there were 7 solutions from 6 participants left over to compare. It had already gotten hot in the mediterranean hemisphere – I noticed other people jump into the pool, I smelled roasted garlic and delicious turkish mokka. But I kept evaluating…

Simple and Complex Rules

No participant bothered with any explanation, how or why his solutions works. They all did work correctly, by the way. But with several of them I still don’t know why. The golden ideas of their creators, those shiny bits of knowledge – buried in crusts of unexplained code. I hear people talking about self-explaining code, of the truth that only lies on code, never in documentation. Might hold for guru-readers, but not for humble-ones like myself. I need explanation, abstracted away from source-code. Even for our simple puzzle, several solutions spanned more than 100 lines of code – mostly without any docu. For me, this is one lessons I (once again!) learned from this puzzle: Care for understandability. All solutions gave correct results – but I evaluated only one (from John Kirchmeier) as “clean-and-simple”, without fancy decoration, and gave it top-score in “Drools code understandability”. Well done, John! One case in particular gave me headaches. One participant (he’ll know who I’m talking about), a real Drools-guru, supplied a solution based on the DroolsSolver approach (formerly known as Taseree). In my remote holiday location, without internet access, I couldn’t even build it. He was the only one to submit a solution based on a scoring approach – his program creates just one (arbitrary) family, scores it and, if the score is not perfect, modifies the family… The result surely works – but at what price? The “business logic” of our simple riddle gets mixed up in six business rules, one scoring rule and a pretty complex (java) function to re-order the family relationships – naturally based on Generics, a FamilyMoveFactory and two Switcher-classes… Impressive stuff, I’m sure this approach will solve generations of extremely complex riddles – but simply overwhelmed me. Impressiveness doesn’t count (Ellen: eh, eh, it counts to me! ). Another sting to another participant: 18 rules for this puzzle might be a little over-modularized…

Running for Gold

Now that I bashed a little on overly complex solution approaches (yep rules can be kept simple yet correct!), what solutions remained in the race for the title? (Running on a MacBook Pro, Intel Dual-Core, OS-X 10.4.10, 2GB Ram, Eclipse 3.2 with Java 1.5, running exclusively Eclipse and Finder, using System.currentTimeMillis() for timing). I ran every solution 3 times and took the lowest values for the overall performance (that is: initialization plus rule-evaluation!).

Scott Reed, 1596 overall, 37 for Drools
Christiano Guiffrida, 1682 overall, 33ms for Drools
Dan Berindei (first solution based on fact-retraction), 1720ms overall, 110 for Drools
John Kirchmeier, 1760 overall, 53 for Drools
Dan Berindei (second solution, without retract), 1812 overall, 65 for Drools
Chris Barham, 1875ms overall solution time, 56ms for Drools.
Reginaldo Delfino, 1956 overall, 77 for Drools

Scott – congratulations. Your solution outperformed the other Drools solutions. Please supply the next puzzle to the Drools community! It was close: Christiano’s and Dan’s solutions were pretty fast too. My personal evaluation on understandability: The fastest solutions score 4 out of 5 possible points. With 1760 msec overall runtime, John Kirchmeier’s clean-and-elegant solution provided an excellent compromise between efficiency, maintainability and understandability. John: You’re my moral winner, with a rule-understandablity-score of 5.

Download of submissions

I do not provide downloads for the submissions (Ellen: the links for solution download please see above) – maybe the participants can put their solutions in the JBoss-Wiki and send me the link – I’ll post those here!

Hints for the Future

As code quality remains to be very subjective, it cannot easily be evaluated by a single referee in our little contest. Taking pure performance clearly identifies a winner – but as a software-architect I’d definitely pick the solution with the lowest overall complexity and acceptable performance.

I propose that from now on each “puzzle-creator/judge” publishes her or his set of evaluation criteria… (I failed in this respect – sorry!). Next time I’d even propose a naming convention plus a predefined “main” class structure:

1 2 3 4 5 6 7 8 9

main(...) //... long start =3D System.currentTimeMillis(); init_rule_file(); long begin_fact =3D System.currentTimeMillis(); pre_init_working_memory(); wm.fireAllRules() long end =3D System.currentTimeMillis(); //...

thanx to everybody for participating, especially to Ellen for the idea (Ellen: sorry again for my terrible organization for the 2nd round. I’ll improve it.)!

Dr. Gernot Starke
Doing IT Right

Scott Reed, congratulations again and please contact Mark for your prize – a JBoss T-shirt.

In order to keep the consistency of evaluation rules and evaluation write-ups, I’ll take over this job from next round on. If there are any volunteers from the community for evaluation and the report write-up, you are very welcome! Write to me so we can assign each other’s work.

Puzzle for next round will come up very soon. Participation rules will be modified a bit too. Since I have internet access and do not mind downloading different versions of Drools, code wrote for older versions of Drools are still acceptable to me. The drools CLIPS parser is not finished, so there might be a little bit problem to test CLIPS submissions. But if you really submit CLIPS code, I’ll thrive to figure out a way to run it. I will also accept code written in languages other than Java, as long as the participant find it fun to write solutions for the problem or simply find it fun to challenge me. Ellen should be more diligent and more responsibility-aware. From now on, deadline of the result-report delivery is 2 days after the evaluation. Evaluation deadline is 3 days after submission deadline.

Scott Reed has suggested a problem that is similar to round 1 or round 2′s. I think it might be time for some freshness and difficulty. So, the problem for coming 3rd round will be very different from round 1 and round 2, and is significantly more complex. Stay tuned and please participate if you are interested!

Tags: computer science, drools, drools puzzle, rule engine, software engineering