基于邻域一致性的极化SAR图像仿射配准

朱庆涛 殷君君 曾亮 杨健

朱庆涛, 殷君君, 曾亮, 等. 基于邻域一致性的极化SAR图像仿射配准[J]. 雷达学报, 2021, 10(1): 49–60. doi: 10.12000/JR20120
引用本文: 朱庆涛, 殷君君, 曾亮, 等. 基于邻域一致性的极化SAR图像仿射配准[J]. 雷达学报, 2021, 10(1): 49–60. doi: 10.12000/JR20120
ZHU Qingtao, YIN Junjun, ZENG Liang, et al. Polarimetric SAR image affine registration based on neighborhood consensus[J]. Journal of Radars, 2021, 10(1): 49–60. doi: 10.12000/JR20120
Citation: ZHU Qingtao, YIN Junjun, ZENG Liang, et al. Polarimetric SAR image affine registration based on neighborhood consensus[J]. Journal of Radars, 2021, 10(1): 49–60. doi: 10.12000/JR20120

基于邻域一致性的极化SAR图像仿射配准

doi: 10.12000/JR20120
基金项目: 国家自然科学基金(61771043),中央高校基本科研业务费专项资金(FRF-IDRY-19-008, FRF-GF-19-017B)
详细信息
    作者简介:

    朱庆涛(1997–),男,清华大学电子工程系在读硕士研究生,研究方向为极化SAR图像处理。E-mail: zqt19@mails.tsinghua.edu.cn

    殷君君(1983–),女,北京科技大学计算机与通信工程学院副教授,研究方向为雷达极化应用的基础理论,极化合成孔径雷达图像理解、图像分割、数据融合,海洋遥感及生态环境变化监测。E-mail: yinjj07@gmail.com

    曾亮:曾 亮(1990–),男,清华大学电子工程系在读博士研究生,主要研究方向为极化雷达信号处理、微波遥感、精确制导。E-mail: zengliang14@mails.tsinghua.edu.cn

    杨健:杨 健(1965–),男,湖北襄阳人,分别在西北工业大学和日本新潟大学获得学士、硕士和博士学位,2000年回国,现在为清华大学教授,博士生导师,研究方向为极化雷达理论及其应用。E-mail: yangjian_ee@tsinghua.edu.cn

    通讯作者:

    曾亮 zengliang14@mails.tsinghua.edu.cn

    杨健 yangjian_ee@tsinghua.edu.cn

  • 责任主编:陈思伟 Corresponding Editor: CHEN Siwei
  • 中图分类号: TP75

Polarimetric SAR Image Affine Registration Based on Neighborhood Consensus

Funds: The National Natural Science Foundation of China (61771043), The Fundamental Research Funds for the Central Universities (FRF-IDRY-19-008, FRF-GF-19-017B)
More Information
  • 摘要: 极化SAR图像的配准是极化SAR图像处理的基础,需要具备较高的精度与速度。基于深度学习的极化SAR图像配准大多数是结合图像块特征的匹配与基于随机抽样一致性的参数迭代估计来实现的。目前尚未实现端到端的基于深度卷积神经网络的一步仿射配准。该文提出了一种基于弱监督学习的端到端极化SAR图像配准框架,无需图像切块处理或迭代参数估计。首先,对输入图像对进行特征提取,得到密集的特征图。在此基础上,针对每个特征点保留k对相关度最高的特征点对。之后,将该4D稀疏特征匹配图输入4D稀疏卷积网络,基于邻域一致性进行特征匹配的过滤。最后,结合输出的匹配点对置信度,利用带权最小二乘法进行仿射参数回归,实现图像对的配准。该文采用RADARSAT-2卫星获取的德国Wallerfing地区农田数据以及PAZ卫星获取的中国舟山港口地区数据作为测试图像对。通过对升降轨、不同成像模式、不同极化方式、不同分辨率的极化SAR图像对的配准测试,并与4种现有方法进行对比,验证了该方法具有较高的配准精度与较快的速度。

     

  • 图  1  基于邻域一致性的SAR图像配准算法框图

    Figure  1.  Flowchart of image registration based on neighborhood consensus

    图  2  DenseBlock的网络结构

    Figure  2.  Structure of DenseBlock

    图  3  Wallerfing农田数据

    Figure  3.  Wallerfing farmland data

    图  4  舟山港口数据

    Figure  4.  Zhou Shan port data

    图  5  DenseNet+SNCNet算法下,Wallerfing和舟山数据配准叠加图

    Figure  5.  Overlay map of Wallerfing and Zhou Shan data by DenseNet+SNCNet

    图  6  SAR-SIFT+RANSAC算法下,Wallerfing和舟山数据配准叠加图

    Figure  6.  Overlay map of Wallerfing and Zhou Shan data by SAR-SIFT+RANSAC

    图  7  不同算法下,Wallerfing和舟山数据配准的PCK曲线

    Figure  7.  PCK of Wallerfing and Zhou Shan image registration with different algorithms

    图  8  Wallerfing农田数据Span图

    Figure  8.  Span image of Wallerfing farmland data

    图  9  600×600分辨率下,DenseNet-SNCNet和ResNet-SNCNet的配准结果

    Figure  9.  Registration result of DenseNet-SNCNet and ResNet-SNCNet at 600×600 resolution

    图  10  稀疏特征匹配过滤模块的有效性

    Figure  10.  Effectiveness of the sparse filter module

    图  11  显著特征点对的分布

    Figure  11.  Distribution of salient feature pairs

    表  1  特征提取模块的结构

    Table  1.   Structure of the feature extraction module

    网络的层该层输出的尺寸特征提取模块
    Convolution
    Pooling
    600×600×64
    300×300×64
    7×7 conv, stride 2
    3×3 Max Pooling, stride 2
    DenseBlock(1)300×300×256$\left[ {\begin{array}{*{20}{c}} {1 \times 1\;\;\,{\rm{conv}}} \\ {3 \times 3\;\;\,{\rm{conv}}} \end{array}} \right] \times 6$
    Transition layer(1)300×300×128
    150×150×128
    1×1 conv
    2×2 Average Pooling, stride 2
    DenseBlock(2)150×150×512$\left[ {\begin{array}{*{20}{c}} {1 \times 1\;\;\,{\rm{conv}}} \\ {3 \times 3\;\;\,{\rm{conv}}} \end{array}} \right] \times 12$
    Transition layer(2)
    150×150×256
    75×75×128
    1×1 conv
    2×2 Average Pooling, stride 2
    DenseBlock(3)75×75×1792$\left[ {\begin{array}{*{20}{c}} {1 \times 1\;\;\,{\rm{conv}}} \\ {3 \times 3\;\;\,{\rm{conv}}} \end{array}} \right] \times 48$
    下载: 导出CSV

    表  2  各种配准算法的APE值

    Table  2.   APE of different registration algorithms

    MethodWallerfing Pauli imagesWallerfing Span imagesZhou Shan images
    SIFT+RANSAC6.5219.0581.924
    SAR-SIFT+RANSAC4.0536.9272.252
    ResNet-SNCNet9.2229.6413.677
    DenseNet-SNCNet6.1757.5452.553
    DenseNet+RANSAC32.71540.0427.988
    下载: 导出CSV

    表  3  k取不同值时,DenseNet+SNCNet配准结果的APE值

    Table  3.   APE of registration result by DenseNet+SNCNet with different k

    kWallerfing Pauli imagesZhou Shan images
    27.4972.443
    58.9292.080
    88.1573.154
    106.1752.553
    1613.4162.383
    下载: 导出CSV

    表  4  k取不同值时,基于GPU的DenseNet+SNCNet运算时间

    Table  4.   Time consuming of registration based on GPU by DenseNet+SNCNet with different k

    kWallerfing Pauli imagesZhou Shan images
    20.62670.6364
    50.73350.7379
    80.85250.9292
    100.99430.9542
    161.17931.2199
    下载: 导出CSV
  • [1] WANG Shuang, QUAN Dou, LIANG Xuefeng, et al. A deep learning framework for remote sensing image registration[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 145: 148–164. doi: 10.1016/j.isprsjprs.2017.12.012
    [2] YANG Zhuoqian, DAN Tingting, and YANG Yang. Multi-temporal remote sensing image registration using deep convolutional features[J]. IEEE Access, 2018, 6: 38544–38555. doi: 10.1109/ACCESS.2018.2853100
    [3] SARVAIYA J N, PATNAIK S, and BOMBAYWALA S. Image registration by template matching using normalized cross-correlation[C]. 2009 IEEE International Conference on Advances in Computing, Control, and Telecommunication Technologies, Trivandrum, India, 2009: 819–822. doi: 10.1109/ACT.2009.207.
    [4] GAN Rui, WU Jue, CHUNG A C S, et al. Multiresolution image registration based on Kullback-Leibler distance[C]. The 7th International Conference on Medical Image Computing and Computer-Assisted Intervention. Saint-Malo, France, 2004: 599–606. doi: 10.1007/978-3-540-30135-6_73.
    [5] 张涛, 王正勇, 张影, 等. 基于和声算法的图像配准技术[J]. 电视技术, 2014, 38(7): 9–12. doi: 10.16280/j.videoe.2014.07.032

    ZHANG Tao, WANG Zhengyong, ZHANG Ying, et al. Image registration techniques based on harmony search algorithm[J]. Video Engineering, 2014, 38(7): 9–12. doi: 10.16280/j.videoe.2014.07.032
    [6] AVERBUCH A and KELLER Y. FFT based image registration[C]. 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing, Orlando, USA, 2002: IV-3608–IV-3611. doi: 10.1109/ICASSP.2002.5745436.
    [7] HARRIS C and STEPHENS M. A combined corner and edge detector[C]. The 4th Alvey Vision Conference, Manchester, UK, 1988. doi: 10.5244/C.2.23.
    [8] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91–110. doi: 10.1023/B:VISI.0000029664.99615.94
    [9] WANG Yufan, YU Qiuze, and YU Wenxian. An improved normalized cross correlation algorithm for SAR image registration[C]. 2012 IEEE International Geoscience and Remote Sensing Symposium, Munich, Germany, 2012: 2086–2089. doi: 10.1109/IGARSS.2012.6350961.
    [10] DELLINGER F, DELON J, GOUSSEAU Y, et al. SAR-SIFT: A SIFT-like algorithm for SAR images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(1): 453–466. doi: 10.1109/TGRS.2014.2323552
    [11] MA Wenping, WEN Zelian, WU Yue, et al. Remote sensing image registration with modified SIFT and enhanced feature matching[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(1): 3–7. doi: 10.1109/LGRS.2016.2600858
    [12] KRIZHEVSKY A, SUTSKEVER I, and HINTON G. ImageNet classification with deep convolutional neural networks[C]. The 25th International Conference on Neural Information Processing Systems, Lake Tahoe, USA, 2012: 1097–1105.
    [13] 徐丰, 王海鹏, 金亚秋. 深度学习在SAR目标识别与地物分类中的应用[J]. 雷达学报, 2017, 6(2): 136–148. doi: 10.12000/JR16130

    XU Feng, WANG Haipeng, and JIN Yaqiu. Deep learning as applied in SAR target recognition and terrain classification[J]. Journal of Radars, 2017, 6(2): 136–148. doi: 10.12000/JR16130
    [14] JIN Kan, CHEN Yilun, XU Bin, et al. A patch-to-pixel convolutional neural network for small ship detection with PolSAR images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(9): 6623–6638. doi: 10.1109/TGRS.2020.2978268
    [15] DETONE D, MALISIEWICZ T, and RABINOVICH A. Deep image homography estimation[J]. arXiv: 1606.03798, 2016.
    [16] ROCCO I, ARANDJELOVIC R, and SIVIC J. Convolutional neural network architecture for geometric matching[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(11): 2553–2567. doi: 10.1109/TPAMI.2018.2865351
    [17] BALAKRISHNAN G, ZHAO A, SABUNCU M R, et al. VoxelMorph: A learning framework for deformable medical image registration[J]. IEEE Transactions on Medical Imaging, 2019, 38(8): 1788–1800. doi: 10.1109/TMI.2019.2897538
    [18] ROCCO I, CIMPOI M, ARANDJELOVIĆ R, et al. Neighbourhood consensus networks[C]. Advances in Neural Information Processing Systems 31, Montréal, Canada, 2018: 1651–1662.
    [19] ROCCO I, ARANDJELOVIĆ R, and SIVIC J. Efficient neighbourhood consensus networks via submanifold sparse convolutions[J]. arXiv: 2004.10566, 2020.
    [20] HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 4700–4708.
    [21] CHOY C, GWAK J Y, and SAVARESE S. 4D Spatio-temporal Convnets: Minkowski convolutional neural networks[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 3075–3084.
    [22] YANG Yi and RAMANAN D. Articulated human detection with flexible mixtures of parts[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 35(12): 2878–2890. doi: 10.1109/TPAMI.2012.261
    [23] 魏天华. 基于多核DSP的加速SAR-SIFT算法并行计算设计[D]. [硕士论文], 西安电子科技大学, 2020.

    WEI Tianhua. Parallel calculation of speed-up SAR-SIFT algorithm based on multi-score DSP[D]. [Master dissertation], Xidian University, 2020.
  • 加载中
图(11) / 表(4)
计量
  • 文章访问数:  593
  • HTML全文浏览量:  260
  • PDF下载量:  142
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-08-29
  • 修回日期:  2020-11-05
  • 网络出版日期:  2020-11-20
  • 刊出日期:  2021-02-25

目录

    /

    返回文章
    返回