Future development: multiclass ROC curves

This is a question which occurs more and more often: do pROC support multiclass ROC curves?

And the answer is: only a minimal approach is implemented, according to Hand and Till (2001) method. It gives the mean of all two class AUCs.

Better methods exist, such as Ferri (2003) or He and Frey (2008). They propose a "Volume under surface" (VUS) approach, where you have a ROC surface instead of curve, and a volume instead of an area. It can be generalized to more than 3 classes with hypervolumes and hypersurfaces.

This approach is very interesting but not trivial to implement. As I can't do it myself in the short term, I am currently looking for potential collaborations on this topic. If you are interested or if you know someone who would be interested in implementing the VUS, please let me know. pROC is open source, and open to any contribution!

--

Ferri C., Hernández-orallo J. & Salido M. A., (2003). Volume Under the ROC Surface for Multi-class Problems. Exact Computation and Evaluation of Approximations. PROC. OF 14TH EUROPEAN CONFERENCE ON MACHINE LEARNING, p. 108--120.

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.14.2427David J. Hand and Robert J. Till (2001). A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems. Machine Learning 45(2), p. 171-186. DOI: 10.1023/A:1010920819831.

http://dx.doi.org/10.1023/A:1010920819831He X. & Frey E. C., (2008). The Meaning and Use of the Volume Under a Three-Class ROC Surface (VUS). Medical Imaging, IEEE Transactions on, 27(5), p. 577-588. DOI: 10.1109/TMI.2007.908687

http://dx.doi.org/10.1109/TMI.2007.908687