Absorb prune into ensure

[sdboyer]

dep prune currently exists as a separate command. This is not optimal; we want to incorporate its behavior into dep ensure directly, obviating the need to run the extra command.

Projects relying on pruning will necessarily have to give up on (fast) vendor/ verification - #121, as that verification will need to rely on a hash of each dependency project's entire tree, prior to pruning. That's just going to be a cost of doing business.

This will involve a few things:

1. Create a set of flags (express as a bitset, e.g. type PruneOptions uint8) in gps representing the orthogonal, combinable pruning modes:
   i. Remove *_test.go files
   ii. Remove non-go files (except LICENSE, COPYING)
   iii. Remove unused packages
   iv. Remove nested vendor dirs (note that gps needs to have this as a flag, but dep unconditionally sets it - we do not give the user the option to NOT strip vendor directories)
2. Introduce a new top-level section in Gopkg.toml that allows the user to specify each of these separately. I'm imagining something like this:

   [prune]
     non-go = true
     test-go = true
     unused-packages = true

   (All default to false)
3. Modify gps.WriteDepTree() to take a struct with a PruneOptions bitfield. (Keep an eye on [WIP] Parallelised gps.WriteDepTree #903, this overlaps with that)
4. Hack up the logic currently in cmd/dep/prune.go, moving it over into unexported functions in gps that gps.WriteDepTree() calls into itself to perform the pruning.

#120 is absurdly long and less relevant now, so I'm going to close it in favor of this.

• Remember to update the README when we're done with this!

[VojtechVitek]

Can we keep LICENSE and COPYING files with a non-go option?

[sdboyer]

@VojtechVitek 💯 yes, will update OP, i think it'd be needlessly creating pain not to do that

[ibrasho]

I assume you meant gps.WriteDepTree instead of gps.WriteVendorTree. 😉

I already did 1, 2 & 3, I can send a PR (it can even be merged before 4 is completed since it doesn't impact behavior). I'm still working on 4 though.

[ibrasho]

@sdboyer Just to check, after reading the comments in #903 I decided to update the signature of gps.WriteDepTree and make it a method of gps.SourceManager. The new signature is:

func (sm SourceManager) WriteDepTree(basedir string, l Lock, options PruneOptions) error

I've moved the current implementation to SourceMgr and added a dummy implementation for the other dummy sms.

[sdboyer]

I assume you meant gps.WriteDepTree instead of gps.WriteVendorTree. 😉

derp, yes. updated OP. once, long ago, it was called WriteVendorTree(), and i still have that stuck in my mind

I already did 1, 2 & 3 already, I can send a PR (it can even be merged before 4 is completed since it doesn't impact behavior)

🎉 🎉 🎊 🎊 killin it!

[karrick]

@sdboyer The code I just uploaded to my version of dep creates a prune tree while it's verifying dependency hashes.

It might be advantageous to look at this and #121 together.

[sdboyer]

@karrick there's definitely a lot of overlap...or at least, adjacency...eventually between pruning and verification, but if the prune tree is the one we discussed in #121 (i haven't looked yet), then i think that's a separate kind of pruning from what's going on here.

now, it's possible that we might be able to join these together. however, my current plan (as noted in the OP) is that if you do the kind of pruning this issue is focused on, then you don't get vendor/ verification. that's probably a bridge we COULD cross, but i'd want that to be a separate thing, as there's a long-term goal of having the hash digests we put into Gopkg.lock be the same as the ones that end up being used for content-addressability in the "third space".