#confusion matrix

[aicorr]

What is a Confusion Matrix in Machine Learning - AICorr.com

Confusion matrix is a table that evaluates the performance of a classification algorithm in machine learning.

[machinelearningsite]

Confusion Matrix 101: Understanding Precision and Recall for Machine Learning Beginners

Evaluating classification models? Dive deep into confusion matrix and unlock the power of precision & recall for peak performance.

When we build a machine learning model, choosing the appropriate metric is the key factor for the feasibility of that model. This factor is only realizable when a model is implemented into a practical use-case. While accuracy remains the suitable metric for linear regression problems, it does not work well with classification problems. Instead, the performance of a classifier model is better…

Confusion Matrix 101: Understanding Precision and Recall for Machine Learning Beginners

[ijtsrd]

Detection of Fake News using Machine Learning

by Pujitha E | Dr. B S Shylaja "Detection of Fake News using Machine Learning"

Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-4 | Issue-6 , October 2020,

URL: https://www.ijtsrd.com/papers/ijtsrd33345.pdf

Paper Url: https://www.ijtsrd.com/computer-science/computer-security/33345/detection-of-fake-news-using-machine-learning/pujitha-e

ugcapprovedjournalswithlowpublicationfees, conferenceissuepublication

The problem of Fake news has evolved much faster in the recent years. Social media has dramatically changed its reach and impact as a whole. On one hand, it’s low cost, and easy accessibility with rapid share of information draws more attention of people to read news from it. On the other hand, it enables wide spread of Fake news, which are nothing but false information to mislead people. As a result, automating Fake news detection has become crucial in order to maintain robust online and social media. Artificial Intelligence and Machine learning are the recent technologies to recognize and eliminate the Fake news with the help of Algorithms. In this work, Machine learning methods are employed to detect the credibility of news based on the text content and responses given by users. A comparison is made to show that the latter is more reliable and effective in terms of determining all kinds of news. The method applied in this work is highest posterior probability of tokens in the response of two classes. It uses frequency based features to train the Algorithms including supervised learning algorithms and classification algorithm technique. The work also highlights a wide range of features established recently in this area that gives a clearer picture for the automation of this problem. An experiment was conducted in the work to match the lists of Fake related words in the text of responses, to find out whether the response based detection is a good measure to determine the credibility or not. 

[nrpin]

Machine Learning Model Performance Evaluation

Machine Learning Model Performance Evaluation - We first develop the Machine Learning (ML) Models in our labs and then we deploy it in the production setup to solve real life problems.

We first develop the Machine Learning (ML) Models in our labs and then we deploy it in the production setup to solve real life problems. The goal is to solve the problem accurately and it should be effective as well. Hence, it is not only the question of whether we have selected the right algorithm but has the algorithm been properly trainedto get the desired result. The important question is how…

View On WordPress

[priancaasharma]

Understanding Confusion Matrix

In this bolg let us demystify confusion matrix and understand it with the help of simple examples.

The name itself creates a kind of confusion and it becomes a little difficult to understand the matrix for the first timers, but with practice and regular use in the models one becomes comfortable with them.

Let us Start then!!

Confusion Matrix

Confusion matrix is a Classification…

View On WordPress

[learnerworld]

Performance measures for classifiers:F1-Score

In continuation of my previous posts on various Performance measures for classifiers, here, I’ve explained the concept of single score measure namely; ‘F - score’.

In my previous posts, I had discussed four fundamental numbers, namely, true positive, true negative, false positive and false negative and eight basic ratios, namely, sensitivity(or recall or true positive rate) & specificity (or true negative rate), false positive rate (or type-I error) & false negative rates (or type-II error), positive predicted value (or precision) & negative predicted value, and false discovery rate (or q-value) & false omission rate.

I had also discussed accuracy paradox, the relationship between various basic ratios and their trade-off to evaluate the performance of a classifier with examples.

I’ll be using the same confusion matrix for reference.

Precision & Recall: First let’s briefly revisit the understanding of ‘Precision (PPV) & Recall (sensitivity)’. 

[You may refer https://learnerworld.tumblr.com/post/153292870245/enjoystatisticswithmeppvnpv & https://learnerworld.tumblr.com/post/152722455780/enjoystatisticswithmesensitivityspecificity for detailed understanding of these ratios]

Precision can be interpreted as ‘proportion of positive identifications was actually correct’.

Precision = TP/((TP+FP) )  

If FP = 0, then Precision = 1

Recall can be interpreted as ‘proportion of actual positives was identified correctly’.

Recall = TP/((TP+FN) )

If FN = 0, then Recall = 1

Trade-Off: For evaluating the model performance, we must observe both precision and recall. It’s quite easy to understand that there is a trade-off between the two.

If we try to maximize recall (i.e., reducing false negatives); the classifiers boundary will minimize precision (i.e., increasing false positives) and vice-versa.

Therefore, we need a measure that relies on both precision and recall. One such measure is ‘F - Score’.

F1-Score: This is a weighted average of precision and recall. F-measure is calculated as a harmonic mean of precision and recall. [Harmonic mean is used in place of arithmetic mean as the arithmetic mean is more sensitive to outliers*.

F1-Score =  (2*Precision*Recall)/((Precision +Recall) )

The F-Measure will always be nearer to the smaller value of Precision or Recall. For problems where both precision and recall are important, one can select a model which maximizes this F1 - score . For other problems, a trade-off is needed, and a decision has to be made whether to maximize precision or recall.

F1 score is a special case of the general Fβ measure (for non-negative real values of β):

Fβ =  ((1+ β^2 )*Precision*Recall)/((β^2*Precision +Recall) )

Two other commonly used F measures are the F2 measure and the F0.5 measure.

As the F-measures do not take the true negatives into account, and that measures such as the Matthews correlation coefficient, Informedness or Cohen's kappa  may be preferable to assess the performance of a binary classifier.

References:

*https://www.quora.com/When-is-it-most-appropriate-to-take-the-arithmetic-mean-vs-geometric-mean-vs-harmonic-mean

Sasaki, Y. (2007). "The truth of the F-measure" (PDF). Van Rijsbergen, C. J. (1979). Information Retrieval (2nd ed.). Butterworth-Heinemann. Powers, David M W (2011). "Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness & Correlation" (PDF). Journal of Machine Learning Technologies. 2 (1): 37–63. Beitzel., Steven M. (2006). On Understanding and Classifying Web Queries (Ph.D. thesis). IIT. CiteSeerX 10.1.1.127.634. X. Li; Y.-Y. Wang; A. Acero (July 2008). Learning query intent from regularized click graphs (PDF). Proceedings of the 31st SIGIR Conference. See, e.g., the evaluation of the [1]. Hand, David. "A note on using the F-measure for evaluating record linkage algorithms - Dimensions". app.dimensions.ai. Retrieved 2018-12-08.

[ankaapmo]

Understanding the Confusion Matrix - #Ankaa

Understanding the Confusion Matrix The confusion matrix is one of the most popular and widely used performance measurement techniques for classification models. While it is super easy to understand, its terminology can be a bit confusing. Therefore, keeping the above premise under consideration, this article aims to clear the... https://ankaa-pmo.com/understanding-the-confusion-matrix/ #AI #Artificial_Intelligence #Confusion_Matrix #Deep_Learning #MachineLearning #Matrix

[learnerworld]

Performance measures for classifiers _ Accuracy & Mis-classification Error

Binary classification

 Binary classification is the task of classifying the elements of a given set into two groups on the basis of a classification rule. Different techniques can be used for binary classification such as logistic regression, naïve Bayes, support vector machine, neural network, decision tress, random forest etc. binary classification is also known as binomial classification or two-class classification or 0/1 classification

 Performance measures for binary classification models

 There are many model evaluation matrices to measure performance of a binary classifier or to compare different models. The very first measure is ‘accuracy’ of the classifier, which measures percentage of correctly classified cases. However, it is not the only metric to evaluate the model. Even sometimes, it is not a reliable measure of model performance because of ‘accuracy paradox’.

There are many other evaluation metrics and plots, which are useful (some of these work only for binary classifiers). We should not consider any of these in an isolated way as there is not a best way to evaluate any system, but different metrics give us different (and valuable) insights about the performance of a classifier. We will discuss different measures and their applicability.

To calculate performance measures, firstly we need to understand the confusion matrix (2 x 2 matrix for binary classification).

 The confusion matrix:

 A confusion matrix (also known as error matrix) is a table of dimension n x n associated with a classifier shows the predicted and actual classification, where n is the number of different classes. [Confusion matrix (Kohavi and Provost, 1998)]

 For binary classifier, we’ve presented confusion matrix as below, where out of total (A+B) observations, number of observed positive classes are A and rest are negative classes.

                                                   Table_1

 In the above matrix olive colored cells shows the number of classes which are correctly classified, while red colored cells presents incorrect classifications. Let’s understand some terminology of this matrix:

 TP (True Positive): As the name implied, these are the cases, where we predicted positive and they do have positive outcome.

TP (True Negative): Similarly, these are the cases, where we predicted negative and they do have negative outcome.

FP (False Positive): These are the cases, where we predicted positive but they do have negative outcome. So, it’s a part of error.

 FN (False Negative): Similarly, these are the cases, where we wrongly predicted negative but they do have positive outcome.

  Prevalence:

The ratio of positive outcomes observed in the data is known as prevalence.

Hence, for truly balanced data, prevalence is 0.5 and for highly unbalanced data it is far from 0.5 [close to 0 or 1 according to class lebels].

 Accuracy & Error:

The fraction of correctly classified cases is known as accuracy of the classifier.

For the above matrix, out of total (A+B) cases, number of correct classifications is (TP+TN).

   Similarly, we can define mis-classification error as number of mis-classified (incorrectly classified) cases divided by total number of cases.

  Accuracy Paradox:

 Intuitively, for any model, higher accuracy (& lower error) is desirable. But there are cases where the model with lower accuracy has greater predictive power. It might seem counter intuitive but for unbalanced data [very less (20 % or less) cases in one class and rest in other class, for example, fraud detection, medical diagnosis, spam filters, titanic survival prediction], accuracy can be misleading and we need other performance measures.

 Especially in case of predicting rare events, a model can make prediction for all cases to be in the majority class and achieve high classification accuracy; the problem is that this model is not useful in the problem domain. For example, in fraud detection, if there are 8% cases of fraudulent transactions (& 92 % genuine transactions) in the training data and our classification model predict all transactions to be genuine (majority class), model accuracy becomes 92 % (very high) but this is not a realistic model as it fails to detect any case of fraudulent transactions. Here, accuracy is not a reliable measure of model performance.

This is called ‘accuracy paradox’.

 For other evaluation matrices, wait for my next post.

Happy Learning !