Visualizations that use zoom-level precisely are quite off

[ilan-gold]

Describe the bug
The scale bar is jumpy on the Avivator site (but somehow not on Vitessce). I imaging there is some sort of state management issue. In any case we should be rendering the scale bar in its own view anyway as suggested here

To Reproduce
avivator demo with a scale bar shows jumpiness

Expected behavior
The scale bar should be fixed like vitessce

Environment:

• Release or git hash:
• Browser: FF
• Browser version: 127.0.1

[ilan-gold]

Ok still getting my web development feet wet again. This only happens on FF (of course).

[ilan-gold]

This happens also on google chrome by using the scale bar. So I would venture a guess it has something to do with bounding box calculations.

[ilan-gold]

Ok this also happens with the side-by-side viewer where one side will get thrown off. So I think there is something going on with the zoom level.

[ilan-gold]

It looks like this actually happens all the time, but is only noticed by things that use zoom state explicitly. These numbers should be monotonic

[ilan-gold]

@xinaesthete See here for more discussion. I have a suspicion this is related to visgl/deck.gl#8989

[xinaesthete]

I think I might have another look into this in combination with updating React / MUI / zustand etc as I still think these things may be somewhat related. In MDV we don't have the janky scale-bar issue and can also load views with appropriate zoom, as well as linking viewState between charts. The latter gets confused if the images have different physical pixel size, but apart from that it seems to work reasonably well and most of the relevant code is based on refactored version of Avivator...

There are some warnings from Zustand currently which it'd be good to get rid of and I'm not sure it's in-scope for the other PR, but it'd be nice to get a new release with all this somewhat cleaned up relatively soon... so I think I'll crack on with that now.

[xinaesthete]

I've just been looking at an image that was encoded in a not terribly useful way - it has a few hundred channels which should really be z-slices etc, and no physical size metadata for the pixels... that last point seems to be helping viv keep the side-by-side views in sync perfectly where others go badly wrong.

[ilan-gold]

This has to be something with Avivator specifcially. https://portal.hubmapconsortium.org/browse/dataset/8690897fced9931da34d66d669c1d698?redirected=True#section-kaggle-1glomerulussegmentation-published for example works fine on safari.

[ilan-gold]

import React, { useState, useEffect } from 'react';
import ReactDOM from 'react-dom/client';
import {
  getChannelStats,
  loadOmeTiff,
  MultiscaleImageLayer,
  ImageLayer
} from '@hms-dbmi/viv';
import DeckGL from '@deck.gl/react';
import { OrthographicView } from '@deck.gl/core'
const url = 'https://viv-demo.storage.googleapis.com/Vanderbilt-Spraggins-Kidney-MxIF.ome.tif'; // OME-TIFF

const useImageLayer = true

// Hardcoded rendering properties.
const props = {
  selections: [
    { z: 0, t: 0, c: 0 },
    { z: 0, t: 0, c: 1 },
    { z: 0, t: 0, c: 2 },
  ],
  colors: [
    [0, 0, 255],
    [0, 255, 0],
    [255, 0, 0],
  ],
  contrastLimits: [
    [0, 10005],
    [0, 10005],
    [0, 10005],
  ],
  channelsVisible: [true, true, true],
}
const INITIAL_VIEW_STATE = useImageLayer ? { target: [100, 100, 0], zoom: -1 } : { target: [10000, 10000, 0], zoom: -7 }

function App() {
  const [loader, setLoader] = useState(null);
  const [viewState, setViewState] = useState(INITIAL_VIEW_STATE);
  const [autoProps, setAutoProps] = useState(null);
  useEffect(() => {
    loadOmeTiff(url).then(setLoader);
  }, []);

  // Viv exposes the getChannelStats to produce nice initial settings
  // so that users can have an "in focus" image immediately.

  async function computeProps(loader) {
    if (!loader) return null;
    // Use lowest level of the image pyramid for calculating stats.
    const source = loader.data[loader.data.length - 1];
    const stats = await Promise.all(props.selections.map(async selection => {
      const raster = await source.getRaster({ selection });
      return getChannelStats(raster.data);
    }));
    // These are calculated bounds for the contrastLimits
    // that could be used for display purposes.
    // domains = stats.map(stat => stat.domain);

    // These are precalculated settings for the contrastLimits that
    // should render a good, "in focus" image initially.
    const contrastLimits = stats.map(stat => stat.contrastLimits);
    const newProps = { ...props, contrastLimits };
    return newProps
  }

  useEffect(() => {

    computeProps(loader).then(setAutoProps)

  }, [loader])

  if (!loader || !autoProps) return null;
  const layer = loader && new (useImageLayer ? ImageLayer : MultiscaleImageLayer)({
    id: 'layer',
    loader: useImageLayer ? loader.data.slice(-1)[0] : loader.data,
    contrastLimits: props.contrastLimits,
      // Default extension is ColorPaletteExtension so no need to specify it if
      // that is the desired rendering, using the `colors` prop.
    colors: props.colors,
    channelsVisible: props.channelsVisible,
    selections: props.selections
  });

  return (
    <DeckGL
      layers={[layer]}
      views={[new OrthographicView({ id: 'ortho', controller: true })]}
      viewState={viewState}
      onViewStateChange={arg => setViewState(arg.viewState) && arg}
    />
  );
}
ReactDOM.createRoot(document.getElementById('root')).render(<App />);

is a reproducer without VivViewer and the same thing is still happening. Separately, I don't know why setViewState needs to be used like this but ok.

[ilan-gold]

Some things ruled out:

1. VivViewer or related react-based mechanisms (so zustand included)
2. ScaleBarLayer even though it's very janky
3. Removing signal appears to have no effect so it's not the signal aborting mechanism
4. This happens both with MultiscaleImageLayer and ImageLayer so it's probably XRLayer specific
5. Related to the above, this isn't a data loading issue then

[xinaesthete]

I've just been doing some debugging of viewState in MDV and it occurs to me that transitionInterpolator / transitionDuration may be potentially implicated in interfering with explicit precise setting of it.

[ilan-gold]

You mean trying to set them? I think I tried but can try again...I did notice that things got better for my other issue on FF. To confirm, are you seeing this?