how to calculate the motion vector between two adjacent frames of a video sequence?

I am working on a project to stabilize a video using SIFT feature. I have obtained the feature points. Now i have to match those features using nearest neighbor algorithm. i have used the following code to find the nearest neighbor between two frames:

function [neighborIds neighborDistances] = kNearestNeighbors(dataMatrix, queryMatrix, k)

neighborIds = zeros(size(queryMatrix,1),k);
neighborDistances = neighborIds;

numDataVectors = size(dataMatrix,1);
numQueryVectors = size(queryMatrix,1);
for i=1:numQueryVectors,
    dist = (sum((repmat(queryMatrix(i,:),numDataVectors,1)-dataMatrix).^2,2));
    [sortval, sortpos] = sort(dist,'ascend');
    neighborIds(i,:) = sortpos(1:k);
    neighborDistances(i,:) = sqrt(sortval(1:k));
end

I hav put the 2nd frame's feature points as the data matrix and a single point from the first frame's feature point as the query, so that i can get the nearest neighbor point of all the feature points obtained. but i m not getting the appropriate output.