spark-wilcoxon computes the scaled Wilcoxon-Mann-Whitney rank sum statistic (also known by other permutations of names) for data which fall into two classes.

The scaled rank sum statistic is equivalent to the area under the ROC curve, considering the data as scores for the binary classification problem of distinguishing class 1 from class 0. As such, it is a fast, simple way to assess the relevance of the data for the binary classification problem. Scaled values near 1 or 0 indicated greater relevance (with 1 indicating positive correlation and 0 indicating negative correlation), and 1/2 indicating irrelevance.