proposal: minimise the amount of unmanaged memory

[jpaskhay]

Is your feature request related to a problem? Please describe.
Using guard pages has a tradeoff of taking up more unmanaged memory pages, which could be a potential concern in high traffic scenarios.

Describe the solution you'd like
Would be nice if there was an option to skip use guard pages when creating LockedBuffers. Can obviously keep using them by default but allow the user to pass in an option to avoid them. Whether this is exposed as part of the Enclave related usage should be considered, but could probably be handled separately, if needed.

Describe alternatives you've considered
Considered using Enclave, but in the expected traffic/usage patterns (cached keys, multiple concurrent users), it would likely lead to more unmanaged memory usage.

Additional context
N/A

[awnumar]

I've thought about doing something like this, but for temporal benefits instead of spacial.

For the use-case you've specified perhaps we can add a Region structure:

type Region struct {
    pages [][]byte // array of page-sized blocks
    data  []byte   // reference to entire region
}

And then LockedBuffers can also export this perhaps, although it would trivially leak the canary if this was inadvertently passed somewhere, so maybe not.

Another solution I was thinking about is a Pool structure that is essentially a queue of byte slices acting like an allocator or a buffer pool. For performance reasons this is useful as it means we don't need to allocate three pages and set them up every time we need a 32 byte region, but it's also advantageous from a spacial perspective as a single LockedBuffer region can be split into N slices and added to this pool, retaining the guard pages but minimizing wasted space. Two birds with one stone.

[awnumar]

We could also implement a custom allocator that's backed by LockedBuffers. It's potentially the most versatile solution but also the most complicated.

[awnumar]

:: Adding a new container without guard pages

Proposal: Add a new LockedBuffer-like container which is essentially the same as a LockedBuffer but without the guard pages or canary.

For

• Cuts down on amount of memory that is allocated per container.
• Simple implementation.

Against

• Creates user confusion, i.e. which container do I use?
• Proposed benefits are specific to a single use-case: keeping the amount of unmanaged memory down.
• Lose the security benefits of having guard pages.

Conclusion: If we go with this then I would rather implement it in the core package. If not, it can very trivially be implemented by third-parties with the help of github.com/awnumar/memcall.

:: Adding a buffer pool implementation

Proposal: Add a queue/stack of fixed-size buffers that are backed by one or more LockedBuffer objects.

For

• Performance benefits for allocating lots of fixed-size objects.
• Minimizes wasted space within LockedBuffer objects.

Against:

• Queue implementation may be tricky. (Growing the queue, handling concurrency bottlenecks and mutex contention, etc.)
• Limited to fixed-size objects (would not make sense for the queue to mix byte slices of varying lengths).
• If slices are adjacent, overflows would be very bad.
• If slices are not adjacent, have to put canary values in-between to detect/mitigate overflows.

:: Implementing a custom allocator

Proposal: Implement a custom allocator that can be queried for N bytes and will return a byte slice N bytes in length, backed by some memory within a LockedBuffer.

For

• Handles variable-length arrays.
• Minimizes wasted space within LockedBuffer objects.
• Performance benefits for allocating lots of things.

Against

• Implementing an allocator is non-trivial (although might be fun).
• Same issues with adjacent regions of memory as the buffer pool.

[awnumar]

Another solution: you can use memcall to allocate unmanaged memory regions of specific sizes, as well as mlock & mprotect them, and disable core dumps.