cmd/cue: eval error of conflicting values combining .json with .cue files

[slewiskelly]

What version of CUE are you using (cue version)?

Originally v0.4.1 darwin/amd64, but tested all versions back to a known working version (v0.3.2 darwin/amd64).

Specifically, this seems to be a regression since #9366, which was released in the 0.4.0-alpha.1 release.

Does this issue reproduce with the latest release?

Yes.

What did you do?

Given this CUE file:

import (
        "list"
)

#Check: {
        ok: true
        ...
}

#Role: "EM" | "IC" | "TL"

#Team: {
        members: [Username=string]: #TeamMember & {email: string | *"\(Username)@acme.com"}
}

#TeamMember: {
        email: string & !=""
        role:  [...#Role] | *["IC"]
}

checks: [string]: #Check

team: #Team

checks: {
        enoughMembers: {
                ok: len(team.members) >= 1
        }

        hasManager: {
                ok: len([ for m in team.members if list.Contains(m.role, "EM") {m}]) >= 1
        }
}

and data from this JSON file:

{
  "team": {
    "members": {
      "alice": {
        "email": "alicez@acme.com",
        "role": ["EM"]
      },
      "bob": {
        "role": ["TL"]
      },
      "clyde": {}
    }
  }
}

I ran:

cue eval team.cue team.json -e checks

What did you expect to see?

I expect this to evaluate to:

enoughMembers: {
    ok: true
}
hasManager: {
    ok: true
}

As the number of team members correctly evaluates to three:

$ cue eval team.cue team.json -e 'len(team.members)'
3

as does the length of managers, to one:

$ cue eval team.cue team.json -e 'len([for m in team.members if list.Contains(m.role, "EM") {m}])'
1

Both of which are obviously >= 1.

What did you see instead?

When unifying the result of this expression with #Check.ok (true), a conflict occurs:

checks.enoughMembers.ok: conflicting values true and false:
    ./team.cue:6:6
    ./team.cue:21:19
    ./team.cue:27:7
checks.hasManager.ok: conflicting values true and false:
    ./team.cue:6:6
    ./team.cue:21:19
    ./team.cue:31:7

I found that this works:

checks: {
	enoughMembers: {
		ok: len(team.members) >= 0
	}

	hasManager: {
		ok: len([ for m in team.members if list.Contains(m.role, "EM") {m}]) >= 0
	}
}

It is expected that one of the following evaluates correctly, given that >= 0 works, but neither of these work:

checks: {
	enoughMembers: {
		ok: len(team.members) == 0
	}

	hasManager: {
		ok: len([ for m in team.members if list.Contains(m.role, "EM") {m}]) == 0
	}
}

checks: {
	enoughMembers: {
		ok: len(team.members) > 0
	}

	hasManager: {
		ok: len([ for m in team.members if list.Contains(m.role, "EM") {m}]) > 0
	}
}

The same check, written differently, works as expected:

checks: {
	enoughMembers: {
		ok: or([ for m in team.members {true}])
	}

	hasManager: {
		ok: or([ for m in team.members {list.Contains(m.role, "EM")}])
	}
}

[myitcv]

@slewiskelly thanks for catching this, and for the repro. I'm reduced this down a bit further and confirmed with 0aaf4c6

# Demonstrate checks are ok
exec cue eval x.cue data.json
cmp stdout stdout.golden

# Import JSON and verify that we can assert checks are ok
exec cue import data.json
exec cue eval x.cue data.cue y.cue

# Assert checks ok using CUE + JSON
exec cue eval x.cue data.json y.cue

-- data.json --
{
	"team": {
		"alice": [
			"EM"
		],
		"bob": [
			"TL"
		]
	}
}
-- x.cue --
import (
	"list"
)

#Team: [string]: [...("EM" | "IC" | "TL")]

team: #Team

checks: {
	enoughMembers: {
		ok: len(team) >= 1
	}

	hasManager: {
		ok: len([ for m in team if list.Contains(m, "EM") {m}]) >= 1
	}
}
-- y.cue --
checks: [string]: ok: true
-- stdout.golden --
#Team: {}
team: {
    alice: ["EM"]
    bob: ["TL"]
}
checks: {
    enoughMembers: {
        ok: true
    }
    hasManager: {
        ok: true
    }
}

Gives:

# Demonstrate checks are ok (0.021s)
# Import JSON and verify that we can assert checks are ok (0.031s)
# Assert checks ok using CUE + JSON (0.019s)
> exec cue eval x.cue data.json y.cue
[stderr]
checks.enoughMembers.ok: conflicting values true and false:
    ./x.cue:11:7
    ./y.cue:1:19
    ./y.cue:1:23
checks.hasManager.ok: conflicting values true and false:
    ./x.cue:15:7
    ./y.cue:1:19
    ./y.cue:1:23
[exit status 1]
FAIL: /tmp/testscript3392139583/repro.txt/script.txt:10: unexpected command failure
error running repro.txt in /tmp/testscript3392139583/repro.txt

whereas it should pass.

Note that this appears to be specifically related to combining CUE + JSON.

Using git bisect this appears to have been introduced in https://cue-review.googlesource.com/c/cue/+/9322 (not https://cue-review.googlesource.com/c/cue/+/9366 as originally reported)

[xinau]

@myitcv Not sure if it helps/you are aware of it already. But the example can be further reduced to a single "check" clause I.e.

checks: {
        enoughMembers: {
                ok: len(team) >= 1
        }
}

What's even more confusing is that running cue eval with -i yields the expected results. Hope this helps with debugging.

[myitcv]

Thanks @xinau.

I've just updated my comment above to indicate the bisect shows that https://cue-review.googlesource.com/c/cue/+/9322 is the problem CL.

[mpvl]

Analysis: the breaking change introduced a separate verification of what it considers the "schema" part, in this case the respective CUE files. The bug is that CUE inadvertently marks the (obvious) failures as "fatal", instead of "incomplete" (that is, could be satisfied by making a value more concrete).

To reproduce that specific issue, one can run cue def x.cue y.cue of Pauls reproducer, which should pass.

[mpvl]

More concretely:

cue def of

b: {...}
check: len(b) == 1

results in

b: {...}
check: len(b) == 1

as it should, but

b: {...}
check: len(b) == 1
check: true

results in

check: conflicting values true and false:
    -:2:5
    -:3:5