Panning Past And To Image Boundaires

What is the property that controls whether or not the interactive pan tool can pan beyond the boundary of the image? Also, how does one set the allowable limits of panning?

Once the axis limits have been set manually, Matlab seems to not allow panning beyond the image boundary. For instance, if you display the default image like so

    figure;
    image;

then you can pan beyond the boundaries of the image. However, if then you try setting the axis limits manually:

    axis( [ 20 50 20 50 ] )

you'll see that panning is now limited to original boundaries of the image.

How do I control this behavior?