Workers port report

Proven on this run

6 of 6 Playwright tests ported to Kane. Kane caught every bug the old suite caught (7/7). 1 planted bug slipped past every test in both suites. Kane avoided 2 false alarms the old suite raised.

Generated 2026-11-16T14:52:11.328Z

6/6tests ported
7 vs 7bugs caught, old vs new (of 30 test and bug pairs)
2false alarms avoided
0Kane stronger
0Kane weaker
0needs human
389.8Kane credits used

Matrix Playwright (PW) vs Kane, per test and per break

TestcleanbaselineB1bugB2bugB3bugB4bugB5bugR1refactorR2refactor
should allow me to add todo itemsboth passPW ✓ pass · Kane ✓ pass runneither caughtPW ✓ pass · Kane ✓ passneither caughtPW ✓ pass · Kane ✓ passneither caughtPW ✓ pass · Kane ✓ passneither caughtPW ✓ pass · Kane ✓ passneither caughtPW ✓ pass · Kane ✓ passboth passPW ✓ pass · Kane ✓ passboth passPW ✓ pass · Kane ✓ pass
should append new items to the bottom of the listboth passPW ✓ pass · Kane ✓ pass runboth caughtPW ✗ fail · Kane ✗ failhealedneither caughtPW ✓ pass · Kane ✓ passboth caughtPW ✗ fail · Kane ✗ failhealedneither caughtPW ✓ pass · Kane ✓ passneither caughtPW ✓ pass · Kane ✓ passfalse alarm avoidedPW ✗ fail · Kane ✓ passboth passPW ✓ pass · Kane ✓ pass
should cancel edits on escapeboth passPW ✓ pass · Kane ✓ pass runneither caughtPW ✓ pass · Kane ✓ passneither caughtPW ✓ pass · Kane ✓ passneither caughtPW ✓ pass · Kane ✓ passneither caughtPW ✓ pass · Kane ✓ passboth caughtPW ✗ fail · Kane ✗ failhealedboth passPW ✓ pass · Kane ✓ passboth passPW ✓ pass · Kane ✓ pass
should display the current number of todo itemsboth passPW ✓ pass · Kane ✓ pass runboth caughtPW ✗ fail · Kane ✗ failhealedneither caughtPW ✓ pass · Kane ✓ passneither caughtPW ✓ pass · Kane ✓ passneither caughtPW ✓ pass · Kane ✓ passneither caughtPW ✓ pass · Kane ✓ passfalse alarm avoidedPW ✗ fail · Kane ✓ passboth passPW ✓ pass · Kane ✓ pass
should remove completed items when clickedboth passPW ✓ pass · Kane ✓ pass runneither caughtPW ✓ pass · Kane ✓ passneither caughtPW ✓ pass · Kane ✓ passneither caughtPW ✓ pass · Kane ✓ passboth caughtPW ✗ fail · Kane ✗ failhealedneither caughtPW ✓ pass · Kane ✓ passboth passPW ✓ pass · Kane ✓ passboth passPW ✓ pass · Kane ✓ pass
should be hidden when there are no items that are completedboth passPW ✓ pass · Kane ✓ pass runboth caughtPW ✗ fail · Kane ✗ failhealedneither caughtPW ✓ pass · Kane ✓ passneither caughtPW ✓ pass · Kane ✓ passboth caughtPW ✗ fail · Kane ✗ failhealedneither caughtPW ✓ pass · Kane ✓ passboth passPW ✓ pass · Kane ✓ passboth passPW ✓ pass · Kane ✓ pass

healed = Kane's self-healing re-authored a step during this run. Each cell shows the verdict of both suites and the outcome in words.

Repair loop

should append new items to the bottom of the list (repaired)

Round 1

Weaker on: B3

Agent change to the _test.md:

+ - ## Verify the todo count+ ## Verify the visible todo count+ + Verify visible text "3 items left" is shown on the page.+ 

Verdicts after:

  • clean: Kane fails on clean
  • B1: Kane not run
  • B2: Kane not run
  • B3: Kane not run
  • B4: Kane not run
  • B5: Kane not run
  • R1: Kane not run
  • R2: Kane not run

Round 2

Weaker on: clean

Agent change to the _test.md:

- # Should append new items to the bottom of the list+ # should append new items to the bottom of the list- ## Verify the visible todo count+ ## Verify the todo count and item order- Verify visible text "3 items left" is shown on the page.+ Verify the list shows exactly these items, in this order: "buy some cheese", "feed the cat", "book a doctors appointment".- ## Verify the item order and persistence+ ## Verify the items persist after reloading+ Reload the page.- Reload the page. Verify the list shows exactly these items, in this order: "buy some cheese", "feed the cat", "book a doctors appointment".

Verdicts after:

  • clean: both pass
  • B1: both caught
  • B2: neither caught
  • B3: both caught
  • B4: neither caught
  • B5: neither caught
  • R1: false alarm avoided
  • R2: both pass

Before and after

TestPlaywright lines_test.md linesPorted file
should allow me to add todo items2518tests/01-should-allow-me-to-add-todo-items/01-should-allow-me-to-add-todo-items_test.md
should append new items to the bottom of the list1724tests/02-should-append-new-items-to-the-bottom-of-the-list/02-should-append-new-items-to-the-bottom-of-the-list_test.md
should cancel edits on escape721tests/03-should-cancel-edits-on-escape/03-should-cancel-edits-on-escape_test.md
should display the current number of todo items1817tests/04-should-display-the-current-number-of-todo-items/04-should-display-the-current-number-of-todo-items_test.md
should remove completed items when clicked718tests/05-should-remove-completed-items-when-clicked/05-should-remove-completed-items-when-clicked_test.md
should be hidden when there are no items that are completed518tests/06-should-be-hidden-when-there-are-no-items-that-are-completed/06-should-be-hidden-when-there-are-no-items-that-are-completed_test.md
Total79116