Machine learning is the study of algorithms that automatically improve their performance with experience. That can provide significant competitive advantages to many organizations by exploiting the potential of large data volume. Intelligently analyzed data is a valuable resource. At the heart of performance is classification accuracy in this specified task. Mostly a crucial problem in machine learning is identifying a representative set of features from which to construct a classification model for a particular task. The classification of data is based on the set of data feature used. The feature selection can provide optimizing performance by using Genetic Algorithm and strongly effect in classification. In making to get improved classification accuracy, author has taken advantage with using hybrid of machine learning methods rather than use of only machine learning approach. Author proposed Information based distance metric to overwhelm one of the weak points of k nearest neighbor classifier. Moreover it provide the comparison of the result of Information based distance metric and Euclidean distance metric on both majority voting and similarity score summing.