Pytorch-based adaptive deformable convolution

This project is to implement “Attention-Adaptive and Deformable Convolutional Modules for Dynamic Scene Deblurring(with ERCNN)” . To run this project you need to setup the environment, download the dataset, and then you can train and test the network models. ## Prerequiste The project is tested on Ubuntu 16.04, GPU Titan XP. Note that one GPU is required to run the code. Otherwise, you have to modify code a little bit for using CPU. If using CPU for training, it may too slow. So I recommend you using GPU strong enough and about 12G RAM. ## Dependencies Python 3.5 or 3.6 are recommended. ``` tqdm==4.19.9 numpy==1.17.3 torch==1.0.0 Pillow==6.1.0 torchvision==0.2.2 ``` ## Environment I recommend using ```virtualenv``` for making an environment. If you using ```virtualenv```, ## Dataset I use GOPRO dataset for training and testing. __Download links__: [GOPRO_Large](https://drive.google.com/file/d/1H0PIXvJH4c40pk7ou6nAwoxuR4Qh_Sa2/view?usp=sharing) | Statistics | Training | Test | Total | | ----------- | -------- | ---- | ----- | | sequences | 22 | 11 | 33 | | image pairs | 2103 | 1111 | 3214 | After downloading dataset successfully, you need to put images in right folders. By default, you should have images on dataset/train and dataset/valid folders. ## Demo ## Training Run the following command ``` python demo_train.py ('data_dir' is needed before running ) ``` For training other models, you should uncommend lines in scripts/train.sh file. I used ADAM optimizer with a mini-batch size 16 for training. The learning rate is 1e-4. Total training takes 600 epochs to converge. To prevent our network from overfitting, several data augmentation techniques are involved. In terms of geometric transformations, patches are randomly rotated by 90, 180, and 270 degrees. To take image degradations into account, saturation in HSV colorspace is multiplied by a random number within [0.8, 1.2].  ## Testing Run the following command ``` python demo_test.py ('data_dir' is needed before running ) ``` ## pretrained models if you need the pretrained models,please contact us by chenleinj@njust.edu.cn ## Acknowledge Our code is based on Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring [MSCNN](http://openaccess.thecvf.com/content_cvpr_2017/papers/Nah_Deep_Multi-Scale_Convolutional_CVPR_2017_paper.pdf), which is a nice work for dynamic scene deblurring .

Some code of the implements of Geological Modeling Using 3D Pixel-Adaptive and Deformable Convolutional Neural Network

Adaptive Region Aggregation for the Multi-View Stereo Matching using Deformable Convolutional Networks

Here we try to implement deformable convolution from scratch without using most of the pytorch built-in functions. Deformable Convolution add 2D offsets to the positional locations of grid sampling. Hence kernels of convolutions adapt themselves to the input.

该方法引入了可变形卷积(Deformable Convolution，DConv)模块、多比例空间金字塔卷积(Multi-proportion Spatial Pyramid Convolution，MSPC)结构和自适应多尺度特征融合(Adaptive Spatial Feature Fusion Network，ASFFN)策略来增强算法适应性。具体而言，在特征提取网络中用可变形卷积模块替代原卷积模块，使网络能动态调整卷积核大小，增强对不同尺寸瑕疵特征的捕捉能力；多比例空间金字塔卷积结构通过拼接多个1×3和3×1卷积核，提取更丰富的瑕疵特征信息，提高模型对不同比例尺度特征的敏感性；应用特征融合策略整合ASFF结构，高效捕捉多尺度的瑕疵信息。

An adaptive dense fusion pyramid enhances cross-scale interaction and feature integrity, deformable convolution improves localization of irregular objects, and a cross-level fused RoI extractor strengthens focus on both targets and context.