In this thesis, we present a framework to upsample the low-resolution image from the 3D-TOF camera or IR camera by using the auxiliary high-resolution RGB image from normal RGB camera. We utilize a different registering scheme for the 2D-2D or 2D-3D camera rigs. For the ToF-RGB rig, we use an alternative calibration method with outlier rejection. Our framework is based on least square optimization with novel confidence weighting scheme to maintain sharp boundaries and to prevent bleeding artifact during propagation. We compare the performance of our approach with the previous approaches targeting for ToF depth map upsampling and IR image upsampling. Experiments on synthetic disparity map upsampling show that our results outperform the previous works in terms of both PSNR and visual quality. While error measurements on IR image upsampling is not obvious compared to other methods, the result shows clear edge boundary. In addition to the automatic method, we extend our approach to incorporate user corrections. Our user correction methods are simple and intuitive, and it does not require any additional modifications (except for the weighting term) for solving the objective function defined in Section 4.1. In the future, we shall study how to extend this work for video frames upsampling.