roc曲线 数据集_实用见解roc曲线和不平衡数据集

The ROC curve is simply a graphical plot of the relationship between the False Positive Rate (FPR) and the True Positive Rate (TPR) when the discrimination threshold of a classifier is varied.

ROC曲线只是当分类器的判别阈值变化时，假阳性率(FPR)和真阳性率(TPR)之间关系的图形图。

FPR is the probability of classifying a data point as positive when it is actually negative. This is essentially the probability that your classifier gives you a false alarm, and it is defined as follows:

FPR是当数据点实际上是负数时将其分类为正数的可能性。 从本质上讲，这是分类器给您带来错误警报的概率，其定义如下：

where N is the total number of negatives, which is equal to the sum of false positives (FP) and true negatives (TN).

其中N是阴性总数，等于假阳性(FP)和真阴性(TN)的总和。

Similarly, TPR is the probability of classifying a point correctly as positive.

同样，TPR是将点正确分类为正的概率。

where P is the total number of positives, which is equal to the sum of true positives (TP) and false negatives (FN).

其中P是阳性总数，等于真阳性(TP)和假阴性(FN)的总和。

The area under the ROC curve (AUC) can be interpreted as the probability that the classification model correctly ranks a random positive example higher than a random negative example. So an AUC which is close to 1 is quite often considered to be a confirmation of the model being good. However that might not be the case as we will see.

ROC曲线下的面积(AUC)可以解释为分类模型正确地将随机正例排名高于随机负例的概率。 因此，接近1的AUC通常被认为是模型良好的确认。 但是，正如我们将要看到的那样，情况可能并非如此。

Looking at the curve below, which has an AUC of 0.93, one may naively conclude that this model is excellent when classifying the underlying dataset.

查看下面的曲线，其AUC为0.93，可以天真的得出结论，在对基础数据集进行分类时，该模型非常出色。

But let’s take a look at an example of a dataset which could give rise to this excellent ROC curve, while the underlying classifier being of poor quality.

但是，让我们看一个数据集示例，该数据集可能会产生这种出色的ROC曲线，而基础分类器的质量很差。

In the image below the red dots represent the positive class and blue dots the negative class. We can assume that we are solving a binary classification problem using logistic regression.

在下面的图像中，红色圆点代表肯定类别，蓝色圆点代表否定类别。 我们可以假设我们正在使用逻辑回归来解决二进制分类问题。

The threshold in our classifier can be set to different values resulting in, for example, the two different boundaries shown in the following figure:

可以将分类器中的阈值设置为不同的值，例如，导致下图所示的两个不同的边界：

The classifier to the right has a TPR of ca. 0.5 but a very low FPR (<0.05). The classifier to the left already has TPR of 1.0 but still a very low FPR (~0.1). When we move the boundary-threshold from right to left we classify more data points as positive, increasing the number of false positives and true positives, giving rise to the ROC curve previously shown above.

右侧的分类器的TPR为。 0.5，但FPR极低(<0.05)。 左侧的分类器已经具有1.0的TPR，但FPR仍然很低(〜0.1)。 当我们将边界阈值从右移到左时，我们将更多数据点归类为正，从而增加了假阳性和真阳性的数量，从而产生了上面显示的ROC曲线。

It’s important to note that for both of these classifiers we can let the FPR approach 0 and keep the TPR constant as we add more negative (blue) points on the left part of the figure. Thus we can let the AUC approach 1 as we like by making the dataset more imbalanced.

重要的是要注意，当我们在图的左侧添加更多的负(蓝色)点时，对于这两个分类器，我们都可以让FPR接近0并使TPR保持恒定。 因此，通过使数据集更加不平衡，我们可以根据需要让AUC接近1。

We see that by changing the threshold we would get great results when measured by the AUC. However this metric completely misses the fact that this classifier is very poor when it comes to precision:

我们看到，通过更改阈值，当使用AUC进行测量时，将获得出色的结果。 但是，该指标完全错过了以下事实：分类器在精度方面非常差：

Indeed both of the thresholds in the picture above result in a precision of less than 0.4 but this shortcoming is of course not indicated by the AUC curve. Thus, one should not blindly trust the AUC metric but to investigate other statistical measures that allow for a better judgement of the outcome of the analysis. In this example the ideal way to capture the performance would be the precision recall curve, which shows the tradeoff between precision and recall (TPR) at different thresholds.

确实，上图中的两个阈值都导致精度低于0.4，但是AUC曲线当然没有指出这一缺点。 因此，不应盲目地相信AUC指标，而应该研究其他统计方法，以便更好地判断分析结果。 在此示例中，捕获性能的理想方法是精度召回曲线，该曲线显示了在不同阈值下精度和召回率(TPR)之间的权衡。

All in all we see that it’s critical to keep in mind the meaning behind the metrics used to evaluate model performance, especially when dealing with imbalanced datasets.

总而言之，我们必须牢记用于评估模型性能的指标背后的含义至关重要，尤其是在处理不平衡的数据集时。