利用局部二值模型及空間注意力進行街景圖像修復

Street Image Inpainting with Local Binary Pattern and Spatial Attention

基於現有的圖像修復方法，如傳統暗房技術以及 Photoshop 修復技術等，皆較需費時的人工修補，而若使用自動修補功能，亦經常造成預測修補結構不完整，導致修補效果不理想。為了有效解決上述的問題，我們的專題先讓使用者透過筆刷進行簡單的塗抹覆蓋，模擬照片破損之區域(mask 區域)。再利用 Local Binary Pattern(LBP) Learning Network 經由 Unet++架構生成預測區域修補結構，並透過門控卷積(Gated Convolution)學習圖像及空間資訊，搭配 Spatial Attention 機制，最後利用 Coarse-to-Fine 方法進行Image Inpainting Network 修補，產生 mask 區域之修補重繪結果。與本專題的研發成果相比，Photoshop 及原論文的預測結果皆較無法準確修補應有結構，且後者修補結果有明顯的色差。此外，在 SSIM 和 PSNR 兩項指標上，本專題的修補成果與原論文相比，分別提升 2.4%及 14.6%，達到 0.9848 與 38.82。

 

The existing image inpainting methods, such as traditional darkroom techniques and Photoshop inpainting techniques, all require time-consuming manual restoration. The use of automatic restoration functions often result in incomplete predicted restoration structures, leading to unsatisfactory restoration results. To effectively solve this issue, our project first allows users to simulate the damaged area (masked area) of a photo by simply covering it with a brush. Then, we use Local Binary Pattern (LBP) Learning Network to generate the predicted region repair structure through the Unet++ framework and learn the image and spatial information through Gated Convolution with Spatial Attention. We finally use the Coarse-to Fine method to perform Image Inpainting Network to repair the masked region. Compared with the results of our project, the prediction results of Photoshop and the referenced work are less accurate in repairing the existing structure, and the latter also has obvious color difference. In addition, compared with the referenced work, the repair results of this project were improved by 2.4% and 14.6% to 0.9848 and 38.82 in terms of SSIM and PSNR, respectively.