Study of the Joint Framework for Real-World Super-Resolution-Semantic Segmentation

LIU Xiao; WANG Zheng-yong; HE Xiao-hai; REN Chao

doi:10.3969/j.issn.2096-6091.2023.24.002

您当前的位置：

首页 >

文章列表页 >

Study of the Joint Framework for Real-World Super-Resolution-Semantic Segmentation

RESEARCH PAPER | 更新时间：2025-03-26

- Study of the Joint Framework for Real-World Super-Resolution-Semantic Segmentation
- New Generation of Information Technology Vol. 6, Issue 24, Pages: 6-11(2023)
- 作者机构：
  
  四川大学电子信息学院，四川成都 610065
- 作者简介：
- 基金信息：
- DOI：10.3969/j.issn.2096-6091.2023.24.002
  CLC： TN911.73
- Published：31 December 2023
- 稿件说明：
移动端阅览
刘晓, 王正勇, 何小海, 等. 真实世界超分辨率—语义分割联合框架研究[J]. 新一代信息技术, 2023, 6(24): 06-11

LIU Xiao, WANG Zheng-yong, HE Xiao-hai, et al. A Study of the Joint Framework for Real-World Super-Resolution-Semantic Segmentation[J]. New Generation of Information Technology, 2023, 6(24): 06-11
刘晓, 王正勇, 何小海, 等. 真实世界超分辨率—语义分割联合框架研究[J]. 新一代信息技术, 2023, 6(24): 06-11 DOI： 10.3969/j.issn.2096-6091.2023.24.002.

LIU Xiao, WANG Zheng-yong, HE Xiao-hai, et al. A Study of the Joint Framework for Real-World Super-Resolution-Semantic Segmentation[J]. New Generation of Information Technology, 2023, 6(24): 06-11 DOI： 10.3969/j.issn.2096-6091.2023.24.002.

摘要

现有的语义分割方法在干净的图像上可以产生较好的结果，但是在干净图像上训练的分割模型应用到真实世界的图像上时则会出现性能下降，这是因为训练域和测试域之间存在域间隙，从而降低了分割的准确性。针对真实世界语义分割的问题，本文提出了一种超分辨率—语义分割联合框架，用于提升语义分割准确性。具体来说，所提出的框架嵌入了一个两分支网络，其中包括超分辨率分支、语义分割分支和一个特征共享模块。超分辨率任务鼓励网络找到对不同分辨率特征鲁棒的表示，从而分割头部可以使用恢复的“干净”特征进行更好的预测。其中超分辨率分支仅配置在训练过程中，在推理阶段可以丢弃。基于构建的伪真实配对数据集CityDeg进行监督训练，提出的框架联合现有先进的语义分割方法能够在不引入额外计算成本的情况下有效提高低分辨率场景语义分割性能。

Abstract

Existing semantic segmentation methods produce better results on clean images

but segmentation models trained on clean images applied to real-world images experience performance degradation because of the domain gap between the training and testing domains

which reduces the segmentation accuracy. To address the problem of real-world semantic segmentation

this paper proposes a joint super-resolution-semantic segmentation framework for improving semantic segmentation accuracy. Specifically

the proposed framework embeds a two-branch network that includes a super-resolution branch

a semantic segmentation branch

and a feature sharing module. The super-resolution task encourages the network to find a robust representation of features with different resolutions

so that the segmentation head can use the recovered “clean" features for better prediction. The super-resolution branch is configured only during training and can be discarded during the inference phase. Based on the constructed pseudo-real pairwise dataset CityDeg for supervised training

the proposed framework

together with the existing state-of-the-art semantic segmentation methods

is able to effectively improve the performance of semantic segmentation for low-resolution scenes without introducing additional computational cost.

关键词

Keywords

references

YU C Q , GAO C X , WANG J B , et al . BiSeNet V2: Bilateral network with guided aggregation for real-time semantic segmentation [J ] . International Journal of Computer Vision , 2021 , 129 ( 11 ): 3051 - 3068 .

FAN M Y , LAI S Q , HUANG J S , et al . Rethinking BiSeNet for real-time semantic segmentation [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2021 : 9716 - 9725 .

XU J C , XIONG Z X , BHATTACHARYYA S P . PIDNet: A real-time semantic segmentation network inspired by PID controllers [C ] // 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2023 : 19529 - 19539 .

LIU X , SHI X Y , CHEN L F , et al . Efficient parallel multi-scale detail and semantic encoding network for lightweight semantic segmentation [C ] // Proceedings of the 31st ACM International Conference on Multimedia . New York : ACM , 2023 : 2544 - 2552 .

胡杰 , 昌敏杰 , 徐博远 , 等 . ConvFormer: 基于Transformer的视觉主干网络 [J ] . 电子学报 , 2024 , 52 ( 1 ): 46 - 57 .

WEI Y Y , ZHANG Z , ZHENG H , et al . SGINet: Toward sufficient interaction between single image deraining and semantic segmentation [C ] // Proceedings of the 30th ACM International Conference on Multimedia . New York : ACM , 2022 : 6202 - 6210 .

CHEN W T , CHEN I H , YEH C Y , et al . SJDL-vehicle: Semi-supervised joint defogging learning for foggy vehicle re-identification [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2022 , 36 ( 1 ): 347 - 355 .

LI Y , CHANG Y , YU C F , et al . Close the loop: A unified bottom-up and top-down paradigm for joint image deraining and segmentation [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2022 , 36 ( 2 ): 1438 - 1446 .

HASHMI K A , KALLEMPUDI G , STRICKER D , et al . FeatEnHancer: Enhancing hierarchical features for object detection and beyond under low-light vision [C ] // 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2023 : 6725 - 6735 .

HONG Y , WEI K , CHEN L , et al . Crafting object detection in very low light [J ] // Proceedings of the British Machine Vision Conference , 2021 , 1 ( 2 ): 3 .

WANG X T , XIE L B , DONG C , et al . Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data [C ] // 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) . Piscataway : IEEE , 2021 : 1905 - 1914 .

LIU X , LIAO X , SHI X , et al . Efficient information modulation network for image super resolution [C ] // 2023 European Conference on Artificial Intelligence (ECAI) . Amsterdam : IOS Press , 2023 : 1544 - 1551 .

Views

453

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Predictive Study of Breast Cancer Pathological Images Based on DenseNet201

Application and Prospect of Deep Learning Technology in Medical Image Analysis

Study on the Recognition of Maize Leaf Disease Based on Deep Learning with Small Sample Size

Classification of Breast Cancer Ultrasound Images Based on GoogLeNet

Related Author

GAO Ming-ze

MIAO Zhuang

SUN Yue

GUO Jin-xing

CHEN Guang-xin

HAN Yang

LIU Li-bing

FU Li-yao

Related Institution

Hongqi Hospital Affiliated with Mudanjiang Medical University

Basic Medical College of Mudanjiang Medical University

Medical Imaging College of Mudanjiang Medical University

Chongqing Communication Design Institute Co., Ltd

Henan Agricultural University ， College of Information and Mangement Science

⁰