Inpainting with NL-means
Image inpainting corresponds to filling missing pixels in an image. This can be achieved using NL-means by iteratively denoising missing pixels.
Contents
Installing toolboxes and setting up the path.
You need to download the general purpose toolbox and the signal toolbox.
You need to unzip these toolboxes in your working directory, so that you have toolbox_general/ and toolbox_signal/ in your directory.
For Scilab user: you must replace the Matlab comment '%' by its Scilab counterpart '//'.
Recommandation: You should create a text file named for instance numericaltour.sce (in Scilab) or numericaltour.m (in Matlab) to write all the Scilab/Matlab commands you want to execute. Then, simply run exec('numericaltour.sce'); (in Scilab) or numericaltour; (in Matlab) to run the commands.
Execute this line only if you are using Matlab.
getd = @(p)path(path,p); % scilab users must *not* execute this
Then you can add these toolboxes to the path.
% Add some directories to the path getd('toolbox_signal/'); getd('toolbox_general/');
The Inpainting Problem
First we load an image.
n = 128;
M = load_image('lena', 256);
M = rescale( crop(M,n) );
Then we construct a mask by removing random pixels.
% amount of removed pixels. rho = .7; % random mask, mask==1 for removed pixels mask = zeros(n,n); sel = randperm(n^2); sel = sel(1:round(rho*n^2)); mask(sel) = 1;
Damaged observation (no noise)
% remove pixels y = M; y(mask==1) = 0; % display clf; imageplot(M, 'Image M', 1,2,1); imageplot(y, 'Observation y', 1,2,2);
Initial guess is obtained by iterative filtering.
s = 3; Mb = y; niter = 30; for i=1:niter % blur the whole image Mb = perform_blurring(Mb,s); % impose the known values Mb(mask==0) = y(mask==0); end
Display initial guess.
clf; imageplot(M, 'Image M', 1,2,1); imageplot(Mb, ['Blurred inpainting, SNR=' num2str(snr(M,Mb)) 'dB'], 1,2,2);
Non-local Mean Inpainting
One can replace the Gaussian blurring by a Non-local Means Filtering to improve the results. Check first the numerical tour on NL-means to see how to compute a non-local filtering.
We set w the (half) size of the patches.
w = 4; w1 = 2*w+1;
Size of the area for the search.
q = 10;
We set up larges (n,n,w1,w1) matrices representing the X and Y position of the pixel to extract.
% location of pixels [Y,X] = meshgrid(1:n,1:n); % offsets [dY,dX] = meshgrid(-w:w,-w:w); % location of pixels to extract dX = reshape(dX, [1 1 w1 w1]); dY = reshape(dY, [1 1 w1 w1]); X = repmat(X, [1 1 w1 w1]) + repmat(dX, [n n 1 1]); Y = repmat(Y, [1 1 w1 w1]) + repmat(dY, [n n 1 1]);
We handle boundary condition by reflexion
X(X<1) = 2-X(X<1); Y(Y<1) = 2-Y(Y<1); X(X>n) = 2*n-X(X>n); Y(Y>n) = 2*n-Y(Y>n);
Target dimensionality
d = 16;
Exercice 1: (the solution is exo1.m) Improve the inpainting result by denoising several time with NL-means only the missing pixel. You can reduce the value of tau during the iteration.
exo1;
