A variety of application areas can be attained in the fields of human-computer interaction for augmented and mixed reality, object tracking and gesture recognition. By combining the areas of 3D computer graphics, computer vision and programming, we have developed a fast, yet robust and accurate image feature detector and matcher to solve common problems that arise in the mentioned research areas. In this thesis, frequent computer vision and augmented reality problems related to camera calibration, object recognition/tracking, image stitching and gesture recognition, are shown to be solved in real-time using our novel feature detection and matching technique. Our method is referred to as FIRST - Fast Invariant to Rotation and Scale Transform. We have also generalized our texture tracking algorithm to a near model base tracking method, using pre-calibrated static planar structures. Our results are compared and discussed with other state of the art works in the areas of invariant feature descriptors and vision based augmented reality, both in accuracy and performance.