A way to consume mouse events?

[wolfpld]

Hi,
Is there a way to consume certain mouse events (e.g. button click, dragging)? Example pseudocode:

if( condition && IsMouseClicked( 0 ) )
{
    ConsumeMouseClick( 0 );
}
if( IsMouseClicked( 0 ) )
{
    // should not activate if 'condition' is true
}

The use case for this would be handling interaction with overlays, which should prevent clicks going to UI elements underneath the overlay.

So far I had limited success directly manipulating ImGui IO structure, but I don't know how robust it is and what assumptions are made about preservation of the state.

[ocornut]

The last time I looked at this, my conclusion what that instead of a coming up with a concept of "consuming" we should instead have a system to claim "ownership" of an input, which seemed more flexible, as your actual consumer is likely to want to poll/use the mouse-down and mouse-release events coming after the click.

So in pseudo-code it would be like:

if( condition && IsMouseClicked( 0 ) )
    SetMouseButtonInputOwner( 0, GetID("mytool"));

And then functions like IsMouseClicked() would take an optional owner in the form of an id.

#define ImGuiInputOwner_None    ((ImGuiID)-1)  // return input if it has no owner
#define ImGuiInputOwner_Any     (0)            // return input regardless of owner

The use case for this would be handling interaction with overlays, which should prevent clicks going to UI elements underneath the overlay.

There are many variations of this problem and I'm always interested in new examples to add to my checklist.
What you mention is often solved by using windows (if your overlay is in a window it would capture inputs).
Is the reason you can't use an input that you want things to pass-through when there's no item, as per issue #3368 (coincidentally adjacent to this issue) ?

[wolfpld]

The last time I looked at this, my conclusion what that instead of a coming up with a concept of "consuming" we should instead have a system to claim "ownership" of an input, which seemed more flexible, as your actual consumer is likely to want to poll/use the mouse-down and mouse-release events coming after the click.

This is not the case I am asking for. I need a way to stop further code within the current frame from reacting to given mouse events. As for what happens in next frames - it's up to my logic to handle.

What you mention is often solved by using windows (if your overlay is in a window it would capture inputs).

This part of the UI is drawn using ImDrawList.

Is the reason you can't use an input that you want things to pass-through when there's no item, as per issue #3368 (coincidentally adjacent to this issue) ?

No. The problem that I have is that there are two possible reactions to a mouse event in my code and I want to give one priority over the other. Essentially, a way of saying "this mouse event has been handled, don't do anything with it anymore".

To give this some context, clicking on a zone in Tracy will open the zone information window:

Dragging the left mouse button over the timeline will display a time range:

However, if you want to limit some statistics to a specific time during program execution, you will be presented with an overlay (the striped area). This overlay can be then adjusted by dragging its borders to fit the required time range. While such adjustment is performed, other possible reactions to mouse events should not happen.

[wolfpld]

I have decided to implement my own wrapper for mouse handling: https://github.com/wolfpld/tracy/blob/master/server/TracyMouse.cpp

While this issue was a partial motivation for this move, a more pressing problem had surfaced. I needed to hook up two actions to the same mouse button: panning the view and opening a context menu. While ImGui has facilities to set a threshold before dragging action is reported, it is not enough for my needs.

Handling the context menu opening (i.e. mouse click then mouse release, without mouse movement) has to have some leeway for random mouse jitter, so a drag threshold is needed.

Panning the view should be as smooth as possible and having threshold set introduces a jarring jump when dragging action activates for the first time.

These two problems have solutions that mutually exclude each other. However, the context menu is not always available to be opened, in which case the drag threshold can be disabled. Determining if this is the case can be easily done by checking if a click+release function was called during the frame (a rather badly named IsMouseClickReleased in the code above).

[ocornut]

Hello,

This is not the case I am asking for. I need a way to stop further code within the current frame from reacting to given mouse events. As for what happens in next frames - it's up to my logic to handle.

The idea I presented above would work similarly, we'd just have to coin the right api/semantic to capture e.g. the button down state for the current frame. What I ommitted in my pseudo API paste above is also that decide that the capture is honored by default in all getters functions, rather than requesting an explicit parameter to honor capture.

So far I had limited success directly manipulating ImGui IO structure, but I don't know how robust it is and what assumptions are made about preservation of the state.

Your workaround will be OK for now. I presume you can technically clear the io flags directly as you mentioned, from my selfish pov it may be preferable as we may discover more edge cases this way, if any exists.

---

The behaviors you describe two posts above (with the pictures) are similar to what most high-level widgets already do by relying on HoveredID and ActiveID to claim ownership / take precedence. It might be easier to work at a slightly higher-level than polling mouse inputs sometimes. Just calling InvisibleButton() or ButtonBehavior() selectively sometimes may do the job.

These two problems have solutions that mutually exclude each other. However, the context menu is not always available to be opened, in which case the drag threshold can be disabled. Determining if this is the case can be easily done by checking if a click+release function was called during the frame (a rather badly named IsMouseClickReleased in the code above).

I think you can use IsMouseDragPastThreshold(), it is valid to use on a MouseReleased frame.
(It is poorly advertised but GetMouseDragDelta() return the last delta which it is valid on the mouse release frame (see #2419), and this is possible because we store io.MouseDragMaxDistanceSqr[] for each button and make it sure it is only cleared on click and not on release.)
If I am not misunderstanding your code, all the state you used is already there and can be queried using IsMouseReleased() && IsMouseDragPastThreshold(). The later is merely a wrapper to access io.MouseDragMaxDistanceSqr[].

[wolfpld]

The state is affected by whether IsMouseClickReleased() is or isn't called (essentially through mousePotentialClickRelease). This selects if drag threshold should be honored.

[ocornut]

The state is affected by whether IsMouseClickReleased() is or isn't called (essentially through mousePotentialClickRelease). This selects if drag threshold should be honored.

I'm not sure I understand, I feel you could replace IsMouseClickReleased() with IsMouseRelease() && IsMouseDragPastThreshold() == false. If that code is public I'm curious to see how it goes. Are you doing this to avoid having to communication information from one part of UI code to the other?

I'm currently building code to replicate your case (and add in imgui_demo) and finding that using a zero threshold for immediate panning BUT showing the menu on a very small drag (smaller than the typical non-zero threshold) also feels like a good and consistent solution.

[wolfpld]

Are you doing this to avoid having to communication information from one part of UI code to the other?

Yes.

I'm currently building code to replicate your case (and add in imgui_demo) and finding that using a zero threshold for immediate panning BUT showing the menu on a very small drag (smaller than the typical non-zero threshold) also feels like a good and consistent solution.

I don't want any drag action to happen if menu is to be opened, as this feels much more solid to me. I may be skewed by previously designing for mobile, where you need to account for much larger inaccuracies with touch input.

[ocornut]

The behaviors you describe two posts above (with the pictures) are similar to what most high-level widgets already do by relying on HoveredID and ActiveID to claim ownership / take precedence. It might be easier to work at a slightly higher-level than polling mouse inputs sometimes. Just calling InvisibleButton() or ButtonBehavior() selectively sometimes may do the job.

InvisibleButton() has a long standing flaw that it didn't give access to button flags, among which one of the most important is the possibility to activate on multiple mouse buttons. I exposed those flags now, making it possible to do:

ImGui::InvisibleButton("canvas", canvas_sz, ImGuiButtonFlags_MouseButtonLeft | ImGuiButtonFlags_MouseButtonRight);
const bool is_hovered = ImGui::IsItemHovered(); // Hovered
const bool is_active = ImGui::IsItemActive();   // Held

This makes quite a difference as this small defect was a recurrent reason for people from jump from simple public API to lower-level imgui_internal.h API (ItemAdd, etc.).

The Demo in "Custom Rendering"->"Canvas" also now showcase a little bit of dragging:

[wolfpld]

I'm using InvisibleButton() as a glorified ItemSize(), for layout purposes (i.e. the gray-background frames overview and the timeline, which is further split into the frames header and a vertically scrollable portion containing zones, etc.), but that's that.

In my case the context menu is only appearing when there's an appropriate item under the mouse cursor (frame header, zone, etc.). I wouldn't be comfortable having to handle this through ImGui widgets rather than items I draw myself.

The Demo in "Custom Rendering"->"Canvas" also now showcase a little bit of dragging:

Doesn't feel solid to me, sorry.

[ocornut]

I'm using InvisibleButton() as a glorified ItemSize(), for layout purposes (i.e. the gray-background frames overview and the timeline, which is further split into the frames header and a vertically scrollable portion containing zones, etc.), but that's that.

What I meant is that you also may be able to use one or more layered InvisibleButton() to catch mouse inputs and block them from other layers. You may decide to only submit a wide button under the mouse based on the finer and custom hovering tests you are already doing, etc. It's a possible higher-level approach without having to create widgets for every fine items (which I agree wouldn't be wise here).

I'm mostly saying that for completeness (from my pov I want to reduce some cases of people adding and maintaining their wrappers).

The Demo in "Custom Rendering"->"Canvas" also now showcase a little bit of dragging:

Doesn't feel solid to me, sorry.

I thought I would be nice to demo it this way, but instead I reworked the code to be closer to your logic:

ImGui::Checkbox("Enable context menu", &opt_enable_context_menu);
[....]
// Pan (we use a zero mouse threshold when there's no context menu)
// You may decide to make that threshold dynamic based on whether the mouse is hovering something etc.
const float mouse_threshold_for_pan = opt_enable_context_menu ? -1.0f : 0.0f;
if (is_active && ImGui::IsMouseDragging(ImGuiMouseButton_Right, mouse_threshold_for_pan))
[...]
if (opt_enable_context_menu && ImGui::IsMouseReleased(ImGuiMouseButton_Right) && ....

[ocornut]

I have now pushed a set of new features to manage input ownership and shortcut routing.
The new features are currently in imgui_internal.h and expected to be moved into public API by 1.90 (aka next release AFTER 1.89).

This is not the case I am asking for. I need a way to stop further code within the current frame from reacting to given mouse events. As for what happens in next frames - it's up to my logic to handle.

As mentioned in earlier messages of this thread (e.g. #3370 (comment)),
The general design idea is to favor associating ownership to held keys. This has benefits over "eating" keys/buttons.
I'm hoping you can consider this. The TL;DR; is to call IsKeyXXXX functions with an explicit owner ImGuiID or ImGuiInputOwner_None.

IsKeyPressed(xxxx) -> IsKeyPressed(xxxx, id);  // return key if no owner or matching owner. ID is item id or arbitrary id of your own.
IsKeyPressed(xxxx) -> IsKeyPressed(xxxx, ImGuiInputOwner_None); // return key if no owner.

Relevant API:

// Input function taking an 'ImGuiID owner_id' argument defaults to (ImGuiKeyOwner_Any == 0) aka don't test ownership, which matches legacy behavior.
#define ImGuiKeyOwner_Any           ((ImGuiID)0)    // Accept key that have an owner, UNLESS a call to SetKeyOwner() explicitely used ImGuiInputFlags_LockThisFrame or ImGuiInputFlags_LockUntilRelease.
#define ImGuiKeyOwner_None          ((ImGuiID)-1)   // Require key to have no owner.

// [EXPERIMENTAL] High-Level: Input Access functions w/ support for Key/Input Ownership
// - Important: legacy IsKeyPressed(ImGuiKey, bool repeat=true) _DEFAULTS_ to repeat, new IsKeyPressed() requires _EXPLICIT_ ImGuiInputFlags_Repeat flag.
// - Expected to be later promoted to public API, the prototypes are designed to replace existing ones (since owner_id can default to Any == 0)
// - Specifying a value for 'ImGuiID owner' will test that EITHER the key is NOT owned (UNLESS locked), EITHER the key is owned by 'owner'.
//   Legacy functions use ImGuiKeyOwner_Any meaning that they typically ignore ownership, unless a call to SetKeyOwner() explicitely used ImGuiInputFlags_LockThisFrame or ImGuiInputFlags_LockUntilRelease.
// - Binding generators may want to ignore those for now, or suffix them with Ex() until we decide if this gets moved into public API.
bool IsKeyDown(ImGuiKey key, ImGuiID owner_id);
bool IsKeyPressed(ImGuiKey key, ImGuiID owner_id, ImGuiInputFlags flags = 0);    // Important: when transitioning from old to new IsKeyPressed(): old API has "bool repeat = true", so would default to repeat. New API requiress explicit ImGuiInputFlags_Repeat.
bool IsKeyReleased(ImGuiKey key, ImGuiID owner_id);
bool IsMouseDown(ImGuiMouseButton button, ImGuiID owner_id);
bool IsMouseClicked(ImGuiMouseButton button, ImGuiID owner_id, ImGuiInputFlags flags = 0);
bool IsMouseReleased(ImGuiMouseButton button, ImGuiID owner_id);

// [EXPERIMENTAL] Low-Level: Key/Input Ownership
// - The idea is that instead of "eating" a given input, we can link to an owner.
// - Ownership is most often claimed as a result of reacting to a press/down event (but occasionally may be claimed ahead).
// - Input queries can then read input by specifying ImGuiKeyOwner_Any (== 0), ImGuiKeyOwner_None (== -1) or a custom ID.
// - Legacy input queries (without specifying an owner or _Any or _None) are equivalent to using ImGuiKeyOwner_Any (== 0).
// - Input ownership is automatically released on the frame after a key is released. Therefore:
//   - for ownership registration happening a result of a down/press event, the SetKeyOwner() call may be done once (common case).
//   - for ownership registration happening ahead of a down/press event, the SetKeyOwner() call needs to be made every frame (happens if e.g. claiming ownership on hover).
// - SetItemKeyOwner() is a shortcut for common simple case. A custom widget will probably want to call SetKeyOwner() multiple times directly based on its interaction state.
// - This is marked experimental because not all widgets are fully honoring the Set/Test idioms. We will need to move forward step by step.
//   Please open a GitHub Issue to submit your usage scenario or if there's a use case you need solved.
ImGuiID            GetKeyOwner(ImGuiKey key);
void               SetKeyOwner(ImGuiKey key, ImGuiID owner_id, ImGuiInputFlags flags = 0);
void               SetItemKeyOwner(ImGuiKey key, ImGuiInputFlags flags = 0);           // Set key owner to last item if it is hovered or active. Equivalent to 'if (IsItemHovered() || IsItemActive()) { SetKeyOwner(key, GetItemID());'.
bool               TestKeyOwner(ImGuiKey key, ImGuiID owner_id);                       // Test that key is either not owned, either owned by 'owner_id'
ImGuiKeyOwnerData* GetKeyOwnerData(ImGuiKey key)     { if (key & ImGuiMod_Mask_) key = ConvertSingleModFlagToKey(key); IM_ASSERT(IsNamedKey(key)); return &GImGui->KeysOwnerData[key - ImGuiKey_NamedKey_BEGIN]; }

However for the purpose of interacting with legacy code which is not ownership aware, it is indeed possible to do the equivalent of "eating", by locking a key/button away from whoever can't explicitly provide the owner id.
The canonical use case would be:

if (IsKeyPressed(ImGuiKey_F))
   SetKeyOwner(ImGuiKey_F, 0, ImGuiInputFlags_LockThisFrame);

or

if (IsKeyPressed(ImGuiKey_F))
   SetKeyOwner(ImGuiKey_F, 0, ImGuiInputFlags_LockUntilRelease);

That's if you don't expect to query the pressed or down state again yourself, which is what your request suggested.

If you were to want to still access the key, you could do, e.g.

if (IsKeyPressed(ImGuiKey_F))
   SetKeyOwner(ImGuiKey_F, my_id, ImGuiInputFlags_LockUntilRelease);
[...]
IsKeyDown(ImGuiKey_F, my_id); // Can only access down state if provided same id. non-owner-aware code won't see it.

And to clarify, this works with ImGuiKey_MouseLeft, ImGuiKey_MouseRight, etc.
While we can't eliminate existing functions like IsMouseClicked() etc. everything is mirrored into key data.

IsMouseClicked(0) == IsMouseClicked(ImGuiMouseButton_Left) == IsKeyPressed(ImGuiKey_MouseLeft);

And new functions like ownership and routing are tied to key data to the code is simple and uniform.
Internal helper which might be helpful:

inline ImGuiKey MouseButtonToKey(ImGuiMouseButton button) { IM_ASSERT(button >= 0 && button < ImGuiMouseButton_COUNT); return (ImGuiKey)(ImGuiKey_MouseLeft + button); }

Demos
Because the current api are under imgui_internal.h we currently cannot yet ship the Demo code in imgui_demo.cpp, but the features/demo_input_owner_and_routing branch has said demos over 3 commits.

Pasting here some of the relevant stuff from said branch. Note the third section.

if (ImGui::TreeNode("Key Ownership basics"))
{
    // Demonstrate basic key ownership system
    // Standard widgets all claim and test for key ownership
    // (note that the ActiveId and HoveredId systems also generally prevents multiple items from interacting, but at a different level)
    if (ImGui::TreeNode("1. Standard widgets taking input ownership"))
    {
        HelpMarker("Standard widgets claim and test for key ownership.\n\n\"Keys\" include mouse buttons, gamepad axises etc.");

        const ImGuiKey key = ImGuiKey_MouseLeft; // Note how mouse and gamepad are also included in ImGuiKey: same data type for all.
        const char* key_name = ImGui::GetKeyName(key);
        ImGui::Text("Press '%s'", key_name);

        ImGui::Text("1st read: (button)");
        ImGui::Button("Click and Hold Me Tight!");

        // Assume this is another piece of code running later.
        // The *default* value for owner is ImGuiKeyOwner_Any, same as calling the simplified function:
        //     IsKeyDown(key) == IsKeyDown(key, ImGuiKeyOwner_Any)
        //     IsKeyPressed(key) == IsKeyPressed(key, ImGuiKeyOwner_Any)
        // But notice the "bool repeat = true" parameter in old signature 'IsKeyPressed(key, repeat)'
        // with the new signature becomes 'IsKeyPressed(key, owner, ImGuiInputFlags_Repeat)'
        ImGui::Text("2nd read: (NOT owner-aware)");
        ImGui::Text("- IsKeyDown(%s): %s", key_name, ImGui::IsKeyDown(key) ? "DOWN!" : "..");
        ImGui::Text("- IsKeyPressed(%s): %s", key_name, ImGui::IsKeyPressed(key) ? "PRESSED!" : "..");

        ImGui::Text("3rd read: (owner-aware: ImGuiKeyOwner_None)");
        ImGui::Text("- IsKeyDown(%s): %s", key_name, ImGui::IsKeyDown(key, ImGuiKeyOwner_None) ? "DOWN!" : "..");
        ImGui::Text("- IsKeyPressed(%s): %s", key_name, ImGui::IsKeyPressed(key, ImGuiKeyOwner_None) ? "PRESSED!" : "..");

        ImGuiID another_owner = ImGui::GetID("AnotherItem");
        ImGui::Text("4nd read: (owner-aware: different owner)");
        ImGui::Text("- IsKeyDown(%s): %s", key_name, ImGui::IsKeyDown(key, another_owner) ? "DOWN!" : "..");
        ImGui::Text("- IsKeyPressed(%s): %s", key_name, ImGui::IsKeyPressed(key, another_owner) ? "PRESSED!" : "..");

        ImGui::TreePop();
    }

    if (ImGui::TreeNode("2. Calling SetKeyOwner()"))
    {
        const ImGuiKey key = ImGuiKey_A;
        const char* key_name = ImGui::GetKeyName(key);
        ImGui::Text("Press '%s'", key_name);

        ImGui::Text("1st read:");
        ImGui::Text("- IsKeyDown(%s): %s", key_name, ImGui::IsKeyDown(key) ? "DOWN!" : "..");
        ImGui::Text("- IsKeyPressed(%s): %s", key_name, ImGui::IsKeyPressed(key, false) ? "PRESSED!" : "..");
        ImGui::Text("...when pressed, call SetKeyOwner() with an owner ID.");
        ImGuiID owner_1 = ImGui::GetID("MyItemID");
        if (ImGui::IsKeyPressed(key, owner_1))
            ImGui::SetKeyOwner(key, owner_1);

        // Assume this is another piece of code running later.
        // (same comments as in section 1)
        ImGui::Text("2nd read: (NOT owner-aware)");
        ImGui::Text("- IsKeyDown(%s): %s", key_name, ImGui::IsKeyDown(key) ? "DOWN!" : "..");
        ImGui::Text("- IsKeyPressed(%s): %s", key_name, ImGui::IsKeyPressed(key) ? "PRESSED!" : "..");

        ImGui::Text("3rd read: (owner-aware: ImGuiKeyOwner_None)");
        ImGui::Text("- IsKeyDown(%s): %s", key_name, ImGui::IsKeyDown(key, ImGuiKeyOwner_None) ? "DOWN!" : "..");
        ImGui::Text("- IsKeyPressed(%s): %s", key_name, ImGui::IsKeyPressed(key, ImGuiKeyOwner_None) ? "PRESSED!" : "..");

        ImGuiID another_owner = ImGui::GetID("AnotherItem");
        ImGui::Text("4th read: (owner-aware: different owner)");
        ImGui::Text("- IsKeyDown(%s): %s", key_name, ImGui::IsKeyDown(key, another_owner) ? "DOWN!" : "..");
        ImGui::Text("- IsKeyPressed(%s): %s", key_name, ImGui::IsKeyPressed(key, another_owner) ? "PRESSED!" : "..");

        ImGui::TreePop();
    }

    // Demonstrate using SetKeyOwner() with ImGuiInputFlags_LockThisFrame / ImGuiInputFlags_LockUntilRelease flags.
    // - Using an owner id solves all/most cases as long as everyone is "owner-id-aware",
    //   meaning they call the long form of IsKeyXXX function. This is the preferred way to do things.
    // - Using ImGuiInputFlags_LockXXXX flags is a way to prevent code that is NOT owner-id-aware from accessing the key.
    //   Think of it as "eating" a key completely: only same owner ID can access the key/button.
    if (ImGui::TreeNode("3. Calling SetKeyOwner() with ImGuiInputFlags_LockXXX flags for non-owner-aware code"))
    {
        const ImGuiKey key = ImGuiKey_B;
        const char* key_name = ImGui::GetKeyName(key);
        ImGui::Text("Press '%s'", key_name);
        static bool lock_this_frame = false;
        static bool lock_until_release = false;

        ImGui::Text("1st read:");
        ImGui::Text("- IsKeyDown(%s): %s", key_name, ImGui::IsKeyDown(key) ? "DOWN!" : "..");
        ImGui::Text("- IsKeyPressed(%s): %s", key_name, ImGui::IsKeyPressed(key, false) ? "PRESSED!" : "..");
        ImGui::Text("...when pressed, call SetKeyOwner() with:");
        ImGui::Checkbox("ImGuiInputFlags_LockThisFrame", &lock_this_frame);
        ImGui::Checkbox("ImGuiInputFlags_LockUntilRelease", &lock_until_release);
        if (ImGui::IsKeyPressed(key, false) && (lock_this_frame || lock_until_release))
            ImGui::SetKeyOwner(key, 0, (lock_this_frame ? ImGuiInputFlags_LockThisFrame : 0) | (lock_until_release ? ImGuiInputFlags_LockUntilRelease : 0));

        // Assume this is another piece of code running later. The calls are not owner-aware,
        // due to the lock they won't be able to see the key.
        ImGui::Text("2nd read: (NOT owner-aware)");
        ImGui::Text("- IsKeyDown(%s): %s", key_name, ImGui::IsKeyDown(key) ? "DOWN!" : "..");
        ImGui::Text("- IsKeyPressed(%s): %s", key_name, ImGui::IsKeyPressed(key, false) ? "PRESSED!" : "..");
        ImGui::TreePop();
    }
    ImGui::TreePop();
}

There are many other features/subtleties.

Since this topic went in many direction, may I suggest:

• you have a cursory look at this, see if update didn't break anything
• maybe now or later try to replace your existing TracyMouse.cpp wrapper with ImGuiInputFlags_LockThisFrame, ImGuiInputFlags_LockUntilRelease etc. I would appreciate this as validation that the system can be used for your advanced use cases.
• then we can close this and whenever you end up digging into new features more extensively you can open new issues.