add: commands to export datasets and to import/query them

[iwilltry42]

• knowledge export <dataset> --output foo.zip
• knowledge retrieve -d <dataset> --archive foo.zip "some question"
• knowledge list-datasets --archive foo.zip
• knowledge get-dataset <dataset> --archive foo.zip
• knowledge import foo.zip

Depends On (currently using fork): philippgille/chromem-go#88

NOTE: there are quite a few rough edges and missing pieces, e.g. the server-mode version of this is not yet implemented.. we'll follow up on this in the future!

[StrongMonkey]

💯 🚀

So is it expected that when using archive, user shouldn't be ingesting files since those file ingestion won't be made into archive anyway?

[StrongMonkey]

Looks like f and rc shoud be closed with defer so no longer need to close it here?

[iwilltry42]

I'm closing them right there to prevent possible resource leaks while defer'ing within the for-loop (pretty unlikely with only two expected files, but still 😬

[iwilltry42]

So is it expected that when using archive, user shouldn't be ingesting files since those file ingestion won't be made into archive anyway?

Yeah, if you want to "update" an archive, you would do import -> ingest -> export. That's essentially, because the archive doesn't contain information about the embeddings function.
I guess we could somehow serialize it with a very limited parameter set that makes sense for our setup, but I'm not sure it's worth the effort.