Remove all options from Homebrew/homebrew-core formulae

[MikeMcQuaid]

Options in formulae don't produce a good user experience because they have to be built from source, we don't test them in CI and each combination of options provides a new chance for new failures to occur. We should seek to (eventually!) remove all options from formulae in Homebrew/homebrew-core in favour of enabling as much non-exclusive functionality as possible in a given formula for widely used options or encouraging the community to maintain their own custom options in a tap (e.g. https://github.com/denji/homebrew-nginx/blob/master/Formula/nginx-full.rb). As an absolute last resort if we need to depend on the same formula multiple times with different options (e.g.
#13133) we can consider vendoring formulae using resource blocks or even duplicating formulae.

This should be done in multiple phases:

1. formulae that have build error issues created with particular options should have the options removed immediately (rather than a fix being merged)
2. less popular formulae should have all their options removed
3. popular formulae (e.g. https://formulae.brew.sh/analytics/install-on-request/90d/) should be prioritised for having their defaults adjusted (depending on analytics usage) to get the best balance of functionality without options.

---

I'm open to thoughts on the above and different approaches to solve the problem as well as additional tooling required in Homebrew/brew to accomplish this.

[jhrmnn]

What about a more general approach akin to Debian's alternatives system? That could handle the issue with options as well as other things.

One thing that I've always missed in Homebrew was being able to pick an implementation of MPI: Open MPI vs Mpich. Currently when one installs Mpich, packages depending on MPI (and so on Open MPI in Homebrew's logic), such as Scalapack, have to be built manually, else Homebrew complains about missing dependencies.

[chdiza]

Options in formulae don't produce a good user experience because they have to be built from source, we don't test them in CI and each combination of options provides a new chance for new failures to occur.

Those things do happen with options, but I don't think it follows that eliminating options provides a better user experience. A formula that lacks needed options means the user makes their own tap, in which case...it will be built from source, not tested by CI, and provides new chances for failure. I.e., those three things that an option-using user currently faces will still be faced just about as often once options are eliminated.

[MikeMcQuaid]

What about a more general approach akin to Debian's alternatives system? That could handle the issue with options as well as other things.

We can't really do this given our limited CI and options aren't a good approach for this. "Picking an implementation" isn't really something Debian supports for compile-time things; as far as I'm away that only applies for things that can be swapped out underneath (i.e. are API and ABI compatible) without an application change. This is theoretically possible as something we could do but mostly people are requesting the ability to swap out things that aren't necessarily compatible.

Those things do happen with options, but I don't think it follows that eliminating options provides a better user experience. A formula that lacks needed options means the user makes their own tap, in which case...it will be built from source, not tested by CI, and provides new chances for failure. I.e., those three things that an option-using user currently faces will still be faced just about as often once options are eliminated.

I think a less flexible Homebrew/homebrew-core that works for every user every time is superior to a more flexible Homebrew/homebrew-core that works for every user 90% of the time. That's a product/design difference of opinion I realise we don't share.

Essentially, though, you touch upon a good point: for things that have to be built from source or build using options there's no real difference between Homebrew/homebrew-core and a tap that doesn't provide bottles. As a result it's much, much easier for the community to step up there and provide solutions to these problems without adding the negative perception when Homebrew/homebrew-core doesn't work and the issues that result from hard to reproduce, option, from source builds.

[zbeekman]

I think a less flexible Homebrew/homebrew-core that works for every user every time is superior to a more flexible Homebrew/homebrew-core that works for every user 90% of the time. That's a product/design difference of opinion I realise we don't share.

I personally tend to agree with @chdiza. (Although I'm willing to concede that I may be in the minority here, and I may not have had the extensive experience that led @MikeMcQuaid to those conclusions.) However, if we go this route, I think we need to greatly improve our documentation for setting up dedicated taps, with all the bells and whistles like bottling etc. I was looking into this a few months ago, and there is no straightforward guide on how to accomplish this. I finally stumbled upon this repository, which seemed like the best example for me to follow. Better documentation for users/contributors of the bottling process would make this type of migration much less of a bitter pill for some (like me!) to swallow.

[MikeMcQuaid]

I think we need to greatly improve our documentation for setting up dedicated taps, with all the bells and whistles like bottling etc. I was looking into this a few months ago, and there is no straightforward guide on how to accomplish this.

The current documentation (e.g. https://docs.brew.sh/How-to-Create-and-Maintain-a-Tap and the brew tap-new command in the manpage) is sufficient for many people to setup taps. If you would like to extend that documentation: please feel free.

with all the bells and whistles like bottling etc.

This doesn't really make sense; options are never bottled anyway so to remove them doesn't imply we need to provide an ability for others to bottle them.

Better documentation for users/contributors of the bottling process would make this type of migration much less of a bitter pill for some (like me!) to swallow.

The issue isn't documentation, really, it's setting up free building of macOS packages (on multiple OS versions) with free uploads to a free CDN. I don't think that problem is ours to solve, really, but again: if you're interested in solving it please feel free.

[zbeekman]

I’ll review the documentation again. It may have been updated since I last looked. I think some insight, not in to how to find a free CDN and CI/CD service, but how we are doing it in our specific case, could go a long way to teach by example. Yes finding the right free/cost effective combo is non trivial, but when I was last looking it, I found the documentation of shipping bottles somewhat opaque. I’ll review it and do some code spelunking to try to improve it myself. I realize the bottling issue is somewhat orthogonal to the options, however an important use case for Homebrew is in CI pipelines. Being able to install precompiled dependencies (built with appropriate options) can mean the difference between a job finishing in time, or a build hitting the CI time limit and failing. I’m not voicing my opinion here to try to get Homebrew to reverse course on this decision; I understand the motivations and trust in the process that led to the decision being made. Anyway, now is the time to “get up or shut up” I suppose, and go review the documentation and see if I can work up a decent PR. 🙂 Cheers!

…

On Wed, Sep 5, 2018 at 4:25 AM Mike McQuaid ***@***.***> wrote: I think we need to greatly improve our documentation for setting up dedicated taps, with all the bells and whistles like bottling etc. I was looking into this a few months ago, and there is no straightforward guide on how to accomplish this. The current documentation (e.g. https://docs.brew.sh/How-to-Create-and-Maintain-a-Tap and the brew tap-new command in the manpage) is sufficient for many people to setup taps. If you would like to extend that documentation: please feel free. with all the bells and whistles like bottling etc. This doesn't really make sense; options are never bottled anyway so to remove them doesn't imply we need to provide an ability for others to bottle them. Better documentation for users/contributors of the bottling process would make this type of migration much less of a bitter pill for some (like me!) to swallow. The issue isn't documentation, really, it's setting up free building of macOS packages (on multiple OS versions) with free uploads to a free CDN. I don't think that problem is ours to solve, really, but again: if you're interested in solving it please feel free. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#31510 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAREPOm8dgPyVzRY-ormPrx99muQhI14ks5uX4qQgaJpZM4WNX12> .