Duplicate maven repositories when importing bazel_deps that use maven.install

[aarontsharp]

When using maven.install via bzlmod in the root module, any bazel_dep which also uses maven.install (with the default repository name) can create duplicate artifacts/conflicts.

For instance, when including a bazel_dep for protobuf 21.7 , the following message is logged:

The maven repository 'maven' is used in two different bazel modules, originally in '' and now in 'protobuf'

This happens because protobuf does a maven.install using maven as the repository name pinning various common dependencies

This is maybe benign in and of itself, but it can create artifact conflicts. For example, pinning different versions of mockito/guava/etc to the root module, the following messages are logged:

Found duplicate artifact versions
    com.google.guava:guava has multiple versions 32.0.0-jre, 31.1-jre
    org.mockito:mockito-core has multiple versions 5.3.1, 4.3.1

It's possible to work around this by using a separate name for maven.install, but it is quite annoying (since you need to set repository_name everywhere) and likewise incredibly difficult to understand where these duplicates were introduced. In my case, I wasn't even directly importing protobuf in my root module, but transitively through another bazel_dep, so this was incredibly confusing.

Is there a better way to understand and resolve these conflicts?

[meteorcloudy]

@shs96c Are maven.install with the same name resolved together or separately? How do you recommend to address the warning message?

/cc @Wyverald

[shs96c]

All the repositories with the same name are resolved together. This allows rulesets to contribute to the user’s jars easily and efficiently. This also implies that the default “maven” name should only be used for this purpose. TBH, relying on the “maven” namespace being as you declared it in your workspace file for anything than the top-level project was always a risky thing to do. Bzlmod has made that more obvious. Inter-ruleset dependencies have to be handled this way because constants cannot be loaded in a MODULE.bazel file. If a rulesets wants to have dependencies only it uses (for example for custom tooling) then it’s recommended that they declare a custom namespace for themselves and depend upon that. If you’d like to see an example, then “contrib_rules_jvm” does this. On 21 Jul 2023, at 15:08, Yun Peng ***@***.***> wrote: @shs96c Are maven.install with the same name resolved together or separately? How do you recommend to address the warning message? /cc @Wyverald —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[Ubehebe]

I just ran into this too, and I think it's specifically a problem with the protobuf module. Other modules take care to instantiate their maven artifacts with different names to avoid polluting the default "maven" name:

• rules_jvm_external itself uses the name rules_jvm_external_deps
• rules_kotlin uses the name kotlin_rules_maven

Is there a good reason why protobuf should make its internal java dependencies visible in this way? If not, we should ask the protobuf module maintainers to change it.

[meteorcloudy]

Not really, it was added as a patch file at https://github.com/bazelbuild/bazel-central-registry/blob/918ea9ff0a8c5d3f8719c84c2109c5a25631fc47/modules/protobuf/21.7/patches/add_module_dot_bazel.patch#L37, we can make a BCR patch version for protobuf to fix this issue.

[luangong]

I also ran into this issue and I agree with @meteorcloudy. Let’s go and make the patch to protobuf in BCR 💪

[andrew-otiv]

I just ran into this too trying to build https://github.com/entur/schema2proto in bazel. Thanks for working on this!

Update: No longer urgent for us since we ended up disabling bzlmod.

[tpudlik]

As someone who maintains a non-root Bazel module, I remain confused about how to correctly use the maven module extension. Based on the discussion above, I believe I should not use the default name in maven.install. Should I just use a unique name instead?

But I'm worried about the comment above by @shs96c :

Inter-ruleset dependencies have to be handled this way because constants cannot be loaded in a MODULE.bazel file. If a rulesets wants to have dependencies only it uses (for example for custom tooling) then it’s recommended that they declare a custom namespace for themselves and depend upon that.

This makes it sound like a unique name for maven.install is only recommended "if a rulesets wants to have dependencies only it uses". To clarify, does this mean that if my module @foo has a java_library //:library with a Maven dep like @foo_maven//:somepackage, and another module depends on my library as @foo//:library, they'll run into some kind of trouble? So the custom namespace is only suitable if //:library is private to my module?

It would be great to update the documentation to explain this kind of nuance. Currently it just recommends use_repo(maven, "maven") and that's that.

[shs96c]

@tpudlik, there are a couple of things here that appear to be causing confusion. Our docs are primarily written for owners of root modules, but I can add extra details to help explain how best to use rules_jvm_external in a non-root module.

The important thing to bear in mind is that each of the tag classes in rules_jvm_external have a name attribute, and this defaults to maven. This name is used when creating the workspace and merging attributes, and it's advised that non-root module always set this to be something unique to the ruleset (as we do in our MODULE.bazel, and as is done in contrib_rules_jvm)

The install tag class has a lock_file attribute. Using this is a requirement if you're using the maven resolver, but is optional when using the default coursier resolver (you can select that using the resolver attribute of the install tag) It is strongly advised that all modules (root or non-root) use a lock file since it makes builds faster and more reliable by skipping the artifact resolution in each build. This file can point to a value in your own ruleset (eg. here and here). Within your own ruleset, you can call that file anything you like, including maven_install.json -- the value is a Label, and Bazel will know how to resolve these properly. If you're using a workspace-based build, maven_install is a macro, so I would lean to using the fully-qualified label.

With that out the way, the idea is that the maven namespace is reserved for a top-level ruleset. The only reason to add to it is if you require users to use your dependencies at runtime for their own code, and they can't use an external resource (eg. an artifact from a regular maven repo) in their build for some reason.