Profile

Cover photo
4 followers|1,620 views
AboutPostsPhotos

Stream

pROC

Shared publicly  - 
 
pROC 1.6 was released, with the following notable new features:
* Power ROC tests
* Confidence intervals for arbitrary coordinates
* Speed enhancements

For more details, see the link below.
1
1
Xavier Robin's profile photo
Add a comment...

pROC

Shared publicly  - 
 
pROC 1.5.1 is out and fixes a performance issue on load time.
Type install.packages("pROC") to upgrade.
1
Add a comment...

pROC

Shared publicly  - 
 
Future development: multiclass ROC curves

This is a question which occurs more and more often: do pROC support multiclass ROC curves?
And the answer is: only a minimal approach is implemented, according to Hand and Till (2001) method. It gives the mean of all two class AUCs.

Better methods exist, such as Ferri (2003) or He and Frey (2008). They propose a "Volume under surface" (VUS) approach, where you have a ROC surface instead of curve, and a volume instead of an area. It can be generalized to more than 3 classes with hypervolumes and hypersurfaces.

This approach is very interesting but not trivial to implement. As I can't do it myself in the short term, I am currently looking for potential collaborations on this topic. If you are interested or if you know someone who would be interested in implementing the VUS, please let me know. pROC is open source, and open to any contribution!

--
Ferri C., Hernández-orallo J. & Salido M. A., (2003). Volume Under the ROC Surface for Multi-class Problems. Exact Computation and Evaluation of Approximations. PROC. OF 14TH EUROPEAN CONFERENCE ON MACHINE LEARNING, p. 108--120.
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.14.2427

David J. Hand and Robert J. Till (2001). A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems. Machine Learning 45(2), p. 171-186. DOI: 10.1023/A:1010920819831.
http://dx.doi.org/10.1023/A:1010920819831

He X. & Frey E. C., (2008). The Meaning and Use of the Volume Under a Three-Class ROC Surface (VUS). Medical Imaging, IEEE Transactions on, 27(5), p. 577-588. DOI: 10.1109/TMI.2007.908687
http://dx.doi.org/10.1109/TMI.2007.908687
1
Add a comment...

pROC

Shared publicly  - 
 
Freakonometrics parle de courbes ROC dans R et de pROC.
 ·  Translate
1
Add a comment...

pROC

Shared publicly  - 
 
What is pROC? Read more in our BMC Bioinformatics article!
2
Add a comment...
In their circles
1 person
Have them in circles
4 people
Xavier Robin's profile photo

pROC

Shared publicly  - 
 
 
Receiver Operating Characteristis (ROCs)
I like ROCs – they allow assessing (for example) the quality of a test without needing to decide on the threshold between "normal" and "pathologic" (yet). They even boil it down to a single number: AUC (area-under-curve). They are known since World War II, where they were developed for Radar interpretation, and Swets wrote a pertinent seminal paper in 1973. In 1993 I used them for analysis but did not place them into the paper – strange in hindsight –, rather showed a combined sensitivity-specificity figure. It took me until 2001 to publish ROC curves (for Pattern-ERG) to detect glaucoma. I've applied ROCs often since, and always programmed the algorithm myself (initially in Excel, then Igor Pro, and in the last few years in R). Until I found this paper “pROC: an open-source package for R and S+ to analyze and compare ROC curves” by Xavier Robin et al.
http://dx.doi.org/10.1186/1471-2105-12-77
They also offer “pROC” as an R package on the standard R site, great, thanks! One of its many advantages are confidence intervals for the AUCs, significance tests between several discriminators, and confidence areas around the traces – my own bootstrapping took much longer than theirs. If you want to follow the bootstrapping in the console, select something like the following option:
options(pROCProgress=list(name="text", width=NA, char=".", style=3))
The figure below is based on as yet unpublished data…
1
Add a comment...

pROC

Shared publicly  - 
 
Discussion about interpolation of sensitivity and specificity in 'coords'.

When you ask for a specific sensitivity or specificity in the coords function, you don't get a threshold. This is because the se/sp is interpolated and does not correspond to a threshold. We cannot interpolate the threshold because it would not match the se/sp back.

What would you like to see in pROC? Join the discussion on LinkedIn:
1
Add a comment...

pROC

Shared publicly  - 
 
pROC 1.5 is out, with a bunch of improvements:
· Variance and covariance
· Univariate Log-Concave Density Estimation smoothing
· Improvements to the plotting function
· New return values in coords
· And many more!
1
Add a comment...

pROC

Shared publicly  - 
 
You can follow pROC development on GitHub!
1
Add a comment...
People
In their circles
1 person
Have them in circles
4 people
Xavier Robin's profile photo
Story
Tagline
pROC: display and analyze ROC curves in R and S+
Introduction
pROC is a set of tools for R and S+ to visualize, smooth and compare receiver operating characteristic (ROC curves). (Partial) area under the curve (AUC) can be compared with statistical tests based on U-statistics or bootstrap. Confidence intervals can be computed for (p)AUC or ROC curves.