DPCTF

DPCTF enhances preservation of fine structural details in stereo matching and optical flow estimation by replacing naive upsampling in coarse-to-fine matching with a differentiable Neighbor-Search Upsampling module that selects disparity and flow values from neighboring pixels based on matching scores.

Key Features:

• Differentiable Neighbor-Search Upsampling (NSU) Module: The NSU module replaces naive upsampling in Coarse-To-Fine (CTF) schemes by estimating matching scores per pixel and selecting the best-matched disparity or flow value from neighboring pixels using finer-level information.
• End-to-End Trainability: Integration of the NSU module enables end-to-end training within neural network architectures and facilitates incorporation into existing CTF frameworks.
• Enhanced Detail Preservation: By selecting optimal disparity and flow values during upsampling, DPCTF reduces edge blurring and preserves thin bars, holes, and other fine structures across multiple scales.

Scientific Applications:

• Stereo Matching: Accurately captures depth from stereo image pairs and achieves state-of-the-art results on benchmarks such as FlyingThings3D, outperforming baselines like Bi3D with notable improvements in End-Point Error (EPE).
• Optical Flow: Provides precise motion vectors between consecutive frames and ranked first on the KITTI Flow 2012 benchmark, demonstrating capability on complex motion patterns and fine details.

Methodology:

Processes image pairs using a multi-scale coarse-to-fine representation and uses the NSU module at each upsampling step to choose disparity or flow values based on neighborhood matching scores.