Pytorch implementation of "360 Layout Estimation via Orthogonal Planes Disentanglement and Multi-View Geometric Consistency Perception") (TPAMI'24)
The project is reproducted based on PanoFormer, DOPNet and mvl_tookit.

Prepare data
Please prepare the dataset by following the guidelines in mvl_toolkit. If you have correctly prepared the dataset, the data can be found in the following file:

../mvl_challenge/assets/zips/ 

where includes 5 files: challenge_phase__training_set.zip、challenge_phase__testing_set.zip、pilot_set.zip、warm_up_training_set.zip, and warm_up_testing_set.zip

../mvl_challenge/assets/mvl_data/  

where includes 3 files: img、labels, and geometry_info

If you finish all the work, you can go next!

Generate pseudo labels
In this step, we employ DOPNet to generate pseudo labels. You may need to modify the following file to choose the model, dataset splits, etc.

../tutorial/create_mlc_labels.py
../tutorial/create_mlc_labels.yaml

The pre-trained model weights should be placed correctly in the following folder：

../mvl_challenge/assets/ckpt/

Fine-tune MV-DOPNet with pseudo labels
In this step, we employ the generated pseudo labels to fine-tune the MV-DOPNet.

If you find this work useful, please consider citing the following papers:

@article{shen2024360,
  title={360 Layout Estimation via Orthogonal Planes Disentanglement and Multi-view Geometric Consistency Perception},
  author={Shen, Zhijie and Lin, Chunyu and Zhang, Junsong and Nie, Lang and Liao, Kang and Zhao, Yao},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2024},
  publisher={IEEE}
}