Globs support for defining packages in workspaces

[spydon]

I think this is already planned, but I'm opening an issue so that it can be tracked.

Adding globs support will:

• reduce a lot of user mistakes (like forgetting to add new packages to the workspace)
• make migrations from the current melos version to the future melos version where packages are instead defined in the workspace much easier, since their current definitions could just be moved from melos.yaml to the workspace config file

Design doc discussion: https://docs.google.com/document/d/1UEEAGdWIgVf0X7o8WPQCPmL4LSzaGCfJTM0pURoDLfE/edit?resourcekey=0-c5CMaOoc_pg3ZwJKMAM0og&disco=AAAA9uS5yhA

[jonasfj]

I think add globs in workspace: might be nice, but perhaps it is a follow up enhancement. While it does seem feasible, it definitely will involve some work making sure we can still efficiently validate package_config.json using timestamps.

At-least I don't want it to block launching 🚀

[spydon]

At-least I don't want it to block launching 🚀

Makes sense, I think we'll wait with putting up a migration from Melos to Melos + workspaces until this is done then. Because it will be quite a painful experience for all projects with more than a few packages.

[sigurdm]

Yeah - I don't think this will make it for 3.6 .

[spydon]

I just started migrating over monorepos now when Melos has support for pub workspaces.
Just posting this here to visualize what it looks like today, and how it would look like with globs for Flame (which is not a that big monorepo compared to many other ones out there):

workspace:
  - doc/flame/examples
  - doc/tutorials/klondike/app
  - doc/tutorials/space_shooter/app
  - doc/tutorials/platformer/app
  - packages/flame_behavior_tree
  - packages/flame_behavior_tree/behavior_tree
  - packages/flame_behavior_tree/example
  - packages/flame_test
  - packages/flame_test/example
  - packages/flame_tiled
  - packages/flame_tiled/example
  - packages/flame_oxygen
  - packages/flame_oxygen/example
  - packages/flame_isolate
  - packages/flame_isolate/example
  - packages/flame_texturepacker
  - packages/flame_texturepacker/example
  - packages/flame_lint
  - packages/flame_sprite_fusion
  - packages/flame_sprite_fusion/example
  - packages/flame_devtools
  - packages/flame_fire_atlas
  - packages/flame_fire_atlas/example
  - packages/flame_audio
  - packages/flame_audio/example
  - packages/flame_studio
  - packages/flame_spine
  - packages/flame_spine/example
  - packages/flame_splash_screen
  - packages/flame_splash_screen/example
  - packages/flame_bloc
  - packages/flame_bloc/example
  - packages/flame_kenney_xml
  - packages/flame_kenney_xml/example
  - packages/flame
  - packages/flame/example
  - packages/flame_lottie
  - packages/flame_lottie/example
  - packages/flame_markdown
  - packages/flame_markdown/example
  - packages/flame_console
  - packages/flame_console/example
  - packages/flame_rive
  - packages/flame_rive/example
  - packages/flame_forge2d
  - packages/flame_forge2d/example
  - packages/flame_noise
  - packages/flame_riverpod
  - packages/flame_riverpod/example
  - packages/flame_svg
  - packages/flame_svg/example
  - packages/flame_network_assets
  - packages/flame_network_assets/example
  - packages/flame_3d
  - packages/flame_3d/example
  - packages/flame_jenny
  - packages/flame_jenny/jenny
  - examples/games/trex
  - examples/games/padracing
  - examples/games/rogue_shooter
  - examples

vs. what it would look like with globs:

workspace:
  - doc/**
  - examples/**
  - packages/**

[sigurdm]

Thanks for the example.

Yeah - this is on one hand much more compact - and I think we can make it work.

But I still think having the explicit listing of packages has its benefits too. You can actually see the packages that are expected to be there...

[spydon]

But I still think having the explicit listing of packages has its benefits too. You can actually see the packages that are expected to be there...

In melos we have a command list which shows all the packages that are currently included, maybe we could have something similar here?

I think the pros heavily outweigh the cons of having globs here, if the user wants to have all their packages explicitly listed they are still free to have it like that even after globs are introduced. The main pro we have seen with having globs in Melos is that without it people forget that they have to do the one extra step of manually adding the package to the list, and it then accidentally results in the package being excluded from the ci pipelines, package linking etc, which might not be obvious to see when it happens, until it breaks.

People also tend to have structured monorepos, so if they add a new package somewhere in the file structure where they expect it to be included as a package in the monorepos it would automatically be covered by the glob, and vice versa for packages that shouldn't be added.

[spydon]

I just realized dart pub workspace list already exists! 😃

[pcornelissen]

Thanks for the example.

Yeah - this is on one hand much more compact - and I think we can make it work.

But I still think having the explicit listing of packages has its benefits too. You can actually see the packages that are expected to be there...

Well, I wouldn't force everyone to use globs, but forcing everyone who would benefit from globs to litter the config with tons of lines that you need to write a tool to maintain doesn't sound sensible to me.

[damianololo]

I'm with spydon too. In our project we currently have 100+ internal packages. There are many developers working in this project. Maintaining this without using globs will be difficult. It's easier to add one or two packages to exclude than to remember to add all of them to include