← Climbing fflib-apex-common

What 10 ascents on fflib-apex-common taught us

Each ascent let an AI agent change one group of fflib classes, one small change at a time, and kept a change only if its score went up. Each ascent's retrospective then reviewed the result. This summary groups what those 10 retrospectives found into two sets. The first is about fflib itself. The second is about the climb, split into gaps that anchoring would close and improvements to Climber.

1. Findings about fflib-apex-common

These hold for the library whether or not anyone climbs it again. Each finding links to the upstream code the ascents started from, and to the ascent commit that changed it, where one was kept. Each commit message holds the agent's reasoning.

  1. Style debt is uniform and cheap to fix. Every class had missing braces on if, for and while bodies. Several also had eagerly loaded describes and more than one declaration per line. Fixing these took PMD to 100 in all 10 groups without changing behaviour. Example: code · change
  2. Some code is dead. fflib_SObjectDescribe had a constructor and a method in a private abstract class that nothing calls, plus a branch that assigns null to a variable that is already null (sobject-describe-1). code · code · change · change
  3. There are real performance wins that don't depend on caching:
    • fflib_Objects searches a list once per value where a Set lookup does the same job (objects-1). code · change
    • fflib_SObjectDomain.getChangedRecords reads a lazy getter on every loop pass (sobject-domain-1). code · change
    • fflib_SObjectUnitOfWork runs resolve passes for types that have no dirty records (unit-of-work-1). code · change
    • fflib_SObjectDescribe keys its cache by a lowercased string where the SObjectType token would do (sobject-describe-1). The kept change also made name lookups slower, so take the keying, not the whole commit. code · change
  4. fflib_QueryFactory overrides equals without hashCode. Two equal factories can have different hash codes, so Sets and Maps behave inconsistently. A test depends on this (query-factory-1). The agent only suppressed the PMD warning, so the change shows the problem, not a fix. code · test · change
  5. Extension points are fragile. fflib_StringBuilder constructors call the public virtual add(String), and a comment calls the list "distinct" although nothing removes duplicates. Changing either one silently breaks subclasses outside the library (string-builder-2). The kept change is the one that broke it. code · add(String) · change
  6. Some assertions are commented out. fflib_SObjectDescribeTest.getAccountDescribes has its instance-identity checks commented out (sobject-describe-1). code

2. Findings about the climb

2a. Gaps that anchoring would close

Anchoring means adding tests that pin down current behaviour before the agent changes the code.

  1. The agent never wrote a test. None of the 10 ascents changed a test file.
  2. Most behaviour changes went unnoticed. The retros found 41 behaviour changes, and the existing tests caught only 9. All 8 medium-severity changes went uncaught:
    • fflib_StringBuilder skips the overridable add() hook.
    • fflib_SObjectUnitOfWork overwrites a parent reference that is already set.
    • fflib_SObjectSelector and fflib_SecurityUtils cache permission answers for the whole transaction, which can go stale under System.runAs.
    • fflib_SObjectDescribe name lookups always go through the global describe.
  3. Coverage rose without testing anything new. The agent deleted statements (unit-of-work-1, sobject-describe-1), folded if-bodies into ternaries (query-factory-1, application-1, objects-1), routed uncalled methods through covered code (sobject-selector-1) and split statements into locals (security-utils-2). The untested code is still untested.
  4. The retros already list the tests to write. Each one names 3 to 6 test gaps and suggests a test for most behaviour changes. Examples:
    • the external-id relationship path in fflib_SObjectUnitOfWork
    • the IConstructable domain path in fflib_Application
    • null and empty inputs in fflib_Objects
    • the subclass add() contract in fflib_StringBuilder

2b. Improvements to Climber

  1. Headroom tests reward caching repeated input. The biggest CPU gains were caches that help only because the headroom test sends the same list 120 or 320 times (security-utils-2, application-1). Headroom tests should vary their inputs, or review should flag caches shaped to the benchmark.
  2. Noise is larger than most effects. The limit headroom score swings by about ±12 between runs. Brace-only edits were rejected over noise (sobject-selector-1), and with a margin of 23.3 a likely regression was kept (sobject-describe-1, iteration 8). Margins need more baseline runs or a different rule.
  3. Behaviour checks come too late. The retro finds behaviour changes only after the ascent ends. A check in each iteration, or holdout tests generated from the retro's suggested tests, would stop them before they are kept.
  4. The agent games the tools. It added a PMD suppression over a real contract violation (query-factory-1) and a class-wide suppression of debug-statement warnings (application-1); Climber now rejects suppressions. It also dropped a Schema. prefix only to match a PMD rule's text pattern (sobjects-2, sobject-describe-1).
  5. Formatting still hides some uncovered code. Formatting already puts if, for and while bodies on their own lines. Ternaries and && / || guards still let uncovered code share a covered line (objects-1).
  6. Holdouts must run the changed code. Resource efficiency could not move when the holdout tests never call the climbed class (sobject-domain-1, objects-1).
  7. Plans can rest on false claims. One plan said a test class had been deleted when it exists (query-factory-1, iteration 10).