Case study: Climbing fflib

This site shows an AI agent improving the Apex code of fflib-apex-common, a widely used Salesforce library. The agent tries one small change at a time. Each change is scored in a Salesforce scratch org on tests, code coverage, static analysis and governor limit use, and is kept only if the score goes up. One such run on a group of classes is called an ascent. The tooling is Climber, and each ascent's code is on its own branch of this fork.

The library was climbed twice: once with Salesforce Code Analyzer's default PMD rules, and once with the PMD brace rules turned off. Comparing the two sets shows how far two independent runs find the same improvements.

How to use this site:

  1. Explore the out of the box map and the map adjusted for style to see how the library is grouped and how each group scored before any ascent.
  2. Explore the out of the box ascents and the ascents adjusted for style for each group. Each ascent's report shows every change the agent tried and why it was kept or discarded, and its retrospective sums up the behaviour changes and patterns.
  3. Compare the two sets and read what they taught us about Climber.
  4. Read the summary of what the out of the box ascents taught us, both about fflib itself and about how to climb better.

Caveats:

If you are a coding agent, use the retrospective JSON files: one per ascent, linked in the table, or all of them in retros.json.

Out of the box map

The library grouped by area, sized by lines of code and coloured by score. Scored before any ascent, with Salesforce Code Analyzer's default PMD rules, as its ascents were scored.

Out of the box ascents

Map adjusted for style

The same code, scored from the same test and analyzer run as the map above, but with PMD run without the four brace rules (IfStmtsMustUseBraces, IfElseStmtsMustUseBraces, ForLoopsMustUseBraces, WhileLoopsMustUseBraces), which flag fflib's choice to leave braces out of one-line bodies. This removes 198 of the 229 PMD violations. Six of the ten groups reach a PMD score of 100. SObjects, Unit of work and SObject describe stay lower because of OneDeclarationPerLine and EagerlyLoadedDescribeSObjectResult.

Ascents adjusted for style

The same groups climbed again with the four brace rules left out of PMD, from the lowest scoring group on the map above.

What the two sets taught us about Climber

Climber's goal is to find the best improvements and bug fixes reliably, so two runs on the same code should find the same ideas. The chart counts the distinct ideas kept by each set across all ten groups, leaving out brace fixes.

Changes made to Climber from what these climbs taught us:

Data

ascents.json lists every ascent with its scores and links. retros.json collects every ascent's retrospective (behaviour changes, patterns, test gaps) in one file.