Sapling & A Workflow For Mozilla Work

In my continuing quest to not use git, I have spent the last few weeks coming up with a viable workflow for working with Sapling at Mozilla. While it's still has rough edges, I've been enjoying it enough that I figure it's time to share.

Edit to add: It's worth highlighting that this workflow is 1000% unsupported, YMMV and please don't file bugs about this; let's not make the engineering workflow team's life harder.

What

Sapling is an SCM system built by Meta. Apparently it's been around for a while in different forms, but I only heard about it on it's public debut.

Sapling is designed to handle large repos, and has a workflow that's extremely familiar to anyone who has used Mercurial + Evolve.

Experience

My experience with Sapling has actually been... pretty darn good. I'll detail the workflow below, and highlight a few gotchas, but overall Sapling seems like where I might end up sticking in my continuing quest.

What's Good?

So first and foremost, I'm super happy with the user experience of Sapling. So much of my workflow in mercurial simply moved over with no trouble (naked heads, frequent use of histedit, absorb, hiding commits, etc). Some things are even nicer than they are in mercurial: for example, Mozillians often use hg wip, which is an alias that gets installed by bootstrap, which shows a graphical overview of the local state of the tree. In sapling, that's just the default output of a bare sl if you're in a sapling repo -- which sounds silly, but is a legitimately nice workflow improvement.

Even better than all the familiarity is that in my experience almost everything in Sapling is fast, even with a Mozilla central sized Repo. It feels as fast or faster than git, and definitely faster than Mercurial. Because it is so fast it can make some interesting decisions that surprised me. For example, if you amend a commit in the middle of a stack, it will automatically restack all dependent commits immediately.

Sapling also has some really fantastic tooling:

  • The Interactive Smart Log (ISL) gives you a browser based UI for dealing with your tree. I was totally skeptical, but it is very good. You can rebase with drag-and-drop, clean up you tree all from the UI. It's impressive
  • The Sapling VSCode Plugin is really nice. It builds the ISL directly into VSCode, and also adds some really delightful touches, like showing the history annotation for each line of code as you edit it. Using this for handling merge conflicts is legitimately quite nice.

What's Not as Good

Well, firstly: Mozilla's code base has no idea what to do about sapling, to varying levels of problematic. I've made one fix so far, but organizationally I don't want to make more work for the engineering workflow teams, so some things I sort of expect will at best be clunky in a sapling repo.

Some examples:

  • mach bootstrap doesn't error out or anything, but definitely seems to work incorrectly when run inside a sapling repo.
  • mach clang-format relies on figuring out the outgoing set, so it doesn't work at the moment. It's possible to work around this one however.

Sapling itself for sure isn't perfect:

  • I've run into a crash while rebasing once; nothing seemed to be lost though and sl rebase --continue finished the job.
  • The ISL seems finicky at times; it will throw an exception and be broken until reload occasionally.
  • Some aspects of my workflow haven't been implemented in Sapling. For example, I used to make heavy use of hg rebase --stop which would stop a partially completed rebase and leave some dependent changes as un-evolved; this doesn't seem to have an equivalent in Sapling, which provides only --abort and --continue
  • Getting Sapling setup to work properly took some more effort and a few more gotcha's than I expected.
  • Sapling's histedit doesn't use the lovely TUI that mercurial provides, and thus is just... clunky. Interestingly, the sl amend commit in the middle of the stack workflow is kind of nicer for quick edits.
  • I think Sapling's history editing capabilities seem to be only about 50% as powerful as evolve -- I cannot figure out an equivalent to the hg obslog.

One major pain point for me at the moment that I don't have a good answer for is format-on-commit, which I relied pretty heavily on. Apparently Sapling does have hooks, but I haven't yet figured out if they're usable as pre-commit hooks yet.

The Workflow

Basically, the workflow is the following diagram:

I'll explain some more in detail below

Getting Started

  1. Get yourself Sapling
  2. Get yourself a git-cinnabar clone of central: See the So what's next heading of this blog post
  3. sl clone from the local git repo into a new sapling repo.
  4. Do your work inside your sapling repo! Check out the guide here
  5. To make the smartlogs work properly, and to teach Sapling what is 'public', you need to tell it what remote refs are public: sl config --local 'remotenames.publicheads=remote/central'. If you don't do this expect ISL to fall over, and sl to complain about the number of public commits.

Push to try:

  1. sl push --to tmp-PushToTry
  2. cd ../git-clone
  3. git checkout tmp-PushToTry
  4. ./mach try ...
  5. cd ../sapling-clone

Of course, replace tmp-PushToTry as appropriate. And if you've previously used that branch name, or need to update it --force works wonders.

You'll also likely be interested in this git repo setting: git config --local receive.denyCurrentBranch updateInstead which is a nice quality of life improvement rather than getting yelled at.

moz-phab submit

  1. sl push --to FeatureSubmit
  2. cd ../git-clone
  3. `moz-phab submit --upstream central
  4. cd ../sapling-clone
  5. sl pull
  6. sl (use smart log to find the updated commit with the differential ID added)
  7. sl goto updated commit;
  8. sl hide old stack (technically optional, but recommended)

Future Explorations

  • You can probably intuit that it seems totally feasible to script most of the above interactions with the git clone. Definitely a possible future path for me.
    • Hanging out in the Sapling discord has made me aware that there's experimental work happening on a dotgit mode that will have a .git repo; in that world, I suspect a lot of this workflow would be obviated, but it sounds like this is still experimental and I'm not sure how actively it's being developed.
  • Apparently there used to be a Phabricator extension, since deleted, which might be resuscitable. Ideally this would allow bypassing moz-phab submit

Concerns

I do have some reservations about going whole-hog onto sapling.

  1. Sapling is first and foremost Meta's tool. I worry, despite a fairly clear CONTRIBUTING.md that if I need to fix sapling, it'll be a PITA to actually get fixes landed -- but the repo is already filled with a bunch of merged PRs, so this could be just paranoia!
  2. Add-ons (e.g. plugins) are an important workflow aid... however I'm bad at Python, and from chatting in the Sapling discord, it definitely seems like it's a bit rough -- essentially you write against the internal Sapling python API, which is perhaps more than I would like.

Other Notes for Explorers:

  • Launching ISL on a remote machine manually: sl isl --no-open -p 8081 -- provides token for remote access.
  • You can use sl goto and specify a remote git revision and it will just figure it out, though you have to use the full git hash.

Exciting times ahead.