Profile

Cover photo
Alex Holcombe
Works at University of Sydney
Attended Harvard University
Lives in Sydney
989 followers|102,751 views
AboutPostsPhotosYouTube

Stream

Alex Holcombe

Shared publicly  - 
 
 I plot and fit my psychophysical data using R with code I wrote over the last several years. But +Dani Linares has produced an R package, quickpsy  (https://github.com/danilinares/quickpsy), that is much cleaner and more modern than my hodgepodge of code. I'm switching to using it now. An added plus is that in using and refining it, I'm contributing to an open source codebase that other psychophysicists can easily use.
quickpsy - quickly fits and plots psychometric functions for multiple conditions
2
Dani Linares's profile photoAnna Ma-Wyatt's profile photo
2 comments
 
Thanks for sharing! That looks really useful.
Add a comment...

Alex Holcombe

Shared publicly  - 
 
A stumbling block for  #openscience  is the need to anonymise participant identities before posting their data. While many people have written about how they integrate open science into lab practices, I haven't seen much on anonymising data.

My R program anonymises my participants' data using an encryption key that I read in from a second file. This allows me to keep a project's experiment code, analysis code, and raw data all in a single git repository that github keeps track of and makes available online.  

The "loadAnonymiseSaveData.R" https://github.com/alexholcombe/MOTcircular/blob/master/dataRaw/loadAnonymiseSaveData.R file finds all the raw data files and de-identifies them when combining them. The participants are still kept separate but they are assigned new names using an encryption key read from a separate, non-public file (its name is listed in .gitignore so that git won't archive it). The use of an encryption key means the participants can be re-identified later by those who are given the key, so I can track things back all the way to the original untouched files generated by the experiment. A full audit trail like this is important for checking for errors (and fraud).

After this anonymisation step, the R files in my analysis folder https://github.com/alexholcombe/MOTcircular/tree/master/analysis analyse the anonymised data file, and all this can be viewed or downloaded by anyone, even while I am still collecting data.

Obviously, if you aren't ready to use git, this isn't going to help you. Software Carpentry (https://software-carpentry.org/) provides free classes to bring researchers up to speed.
3
4
Open Science's profile photoCrystal Tu's profile photoMark DeLong's profile photoDarin London's profile photo
2 comments
 
Hey Tom, yes and to clarify, in this example I take the subjects' initials and encrypt them into a new, fake initials which appears inset into the graph of each participant's data, psychophysics style. As I look at the data, I start to pick up on who is who, but people I show the data to, and eventual readers of the paper, can't work out who had the embarassingly low thresholds etc.

While I agree it seems innocuous to reveal authors and lab members and consenting undergraduates initials in the plots, as vision researchers have done for a century, unfortunately I can't see that any ethics or IRB panel would be eager to approve that practice!

Rather than simply eliminating the initials, encrypting them seems like a good way to do it because I can quickly work out who is who.
 
Because I only recently started doing this, I'm using a simple, rather breakable rotate-through-the-alphabet code (it's at the end of that R file), I ought to do something fancier.

I wanted to link to a best-practice document but don't know of any. However, I know there is a PLoS ONE paper in press (can't remember the author's name) on the issue of anonymising participant data and posting it, which I am keen to read.  I'd be interested to hear what you end up doing.
Add a comment...

Alex Holcombe

Shared publicly  - 
 
Something new I'm putting under my signature when I review manuscripts for journals:

Signed (I sign all my reviews),

Alex Holcombe

To improve the transparency of peer review and editorial decisions, I avoid entering anything in boxes like the "Confidential Comments to the Associate Editor"  provided here. Also I am concerned that such boxes can easily be abused with "stealth rejections" and/or unsupported insinuations.
2
Mariam Aly's profile photo
 
Agreed -- those boxes shouldn't exist. The only thing more frustrating than rejection after bad reviews is rejection after (seemingly) positive reviews. What's the point of feedback that the authors can't see?
Add a comment...

Alex Holcombe

Shared publicly  - 
 
Nature Scientific Reports adopts fast-tracking fees. We protested this policy at several other journals 4 years ago, our concerns appear to have caused some journals to drop it https://alexholcombe.wordpress.com/tag/fast-track-fees/
3
Thom Baguley's profile photo
 
This interacts particularly badly with things like REF. Now UK researchers can use REF income to pay fast-tracking fees to get outputs to get REF income. I mean that's what research funding is for ...
Add a comment...

Alex Holcombe

Shared publicly  - 
 
Timing & Time Perception Reviews, an #openaccess journal with no fees for authors. See, free  #openaccess  can be done! -they use Open Journal Systems free software as the publishing platform, with support by Brill and from Groningen University. http://rjh.ub.rug.nl/index.php/ttpr/issue/view/1674 …
Review journal on all topics associated with timing and temporal cognition.
2
Add a comment...

Alex Holcombe

Shared publicly  - 
 
congrats to the journal Cognition for instituting a data posting requirement!  Only thing is it's copyrighted like most things in the journal so you can't read it- perhaps on oversight. But I'll (illegally) paste it here:

 I will also be instituting a data transparency policy. To
encourage meta-analysis and to help everyone understand
data better, authors of all published papers will be asked to
make their raw data publically available (unless doing so is
prohibited for some good reason). Here is the specific policy
that I plan to institute:
i. All empirical papers must archive their data upon
acceptance in order to be published unless the
authors provide a compelling reason why they cannot
(e.g., expense, confidentiality). The action editor
will be the final arbiter of whether the reason is suf-
ficiently compelling.
ii. ‘‘Data’’ refers to an electronic file containing nonidentified
responses that are potentially already
coded. Normally, the data would represent an early
stage of electronic processing, before individual
responses have been aggregated. The data must be
in a form that allows all reported statistical analyses
to be reproduced while retaining the confidentiality
of individual participants. This entails that the data
are formatted and documented in a way that makes
the structure of the data set readily apparent.
iii. Archiving consists either of submitting the data to
the journal (to be displayed as supplementary material
at the end of the article), sending it to some
other archive that is accessible to established
researchers and maintained by a substantial established
institution, or authors making the data available
on their own website, assuming that they can
assure us the site will be maintained by a recognized
institution for a reasonable period of time. Again,
action editors will be the final arbiters of the appropriateness
of an archive.
iv. Any publication that reports analyses of or refers to
archived data will be expected to cite the original
publication in which the data were reported.
v. This policy is new and therefore open to modification.
Our aim is to implement a policy that maximizes
transparency while minimizing the burden
on authors.
1
Alexander, name from the Gods's profile photo
2 comments
 
Its alrwady been published hasn't it, in some ways you have stuck up for me and I thank you for that.
Add a comment...

Alex Holcombe

Shared publicly  - 
 
Today's reply to an #Elsevier  review request:

Dear Professor XXXX,

Traditional scientific publishers make higher profits than almost every other legal industry, by exploiting scholars' free labor and draining university library funds. Rather than reviewing for Elsevier, I have pledged to spend my time supporting alternatives, such as open access journals. 

The UK Open Access Implementation Group (http://t.co/rhgKN1bGqa) exists to help scholarly societies transition to alternative publishing models. Vision Research has a venerable history of publishing high-quality perception research, and almost all the credit in my opinion goes to the editorial board and the reviewers, who have added a lot of value over the years by improving the quality of authors' research through the reviewing process. My recommendation is for the editorial boards of Elsevier-published journals to resign and pursue alternatives. 

best
Alex

https://alexholcombe.wordpress.com/2013/01/09/scholarly-publishers-and-their-high-profits/
8
4
Ilene Frank's profile photozeltak brisbane's profile photo
Add a comment...
In his circles
313 people
Have him in circles
989 people
palwinder don's profile photo
quality tunes daily's profile photo
Margareta Frye's profile photo
Brain Canada's profile photo
Alejandro Lleras's profile photo
Luis Zapien's profile photo
Rebecca J. Pelletier's profile photo
Mike Murray's profile photo
Bradley Voytek's profile photo

Alex Holcombe

Shared publicly  - 
 
Why are action potentials in animals and plants all-or-none? 

I haven't found anything that explains this, so I've written the below for use in lecturing undergrads. I'd welcome any comments, corrections, or links to other explanations.

The all-or-none nature of the action potential is how we can achieve long-range transmission without degradation of the signal.
-The signal is binary or digital; no action potential or yes an action potential; like 0 or 1, nothing in between.
-Imagine that it was instead analog, a graded signal instead of digital.
-The neuron might have a signal of, say,60 millivolts, travelling down the axon. You need to preserve the 60-mv msg all the way to the destination.
-So each bit of the neuron has to generate the mV that the previous bit generated. 
-But the problem is you get errors, and the errors accumulate, just like in the Chinese whispers game.
-The second millimetre might be affected by random fluctuations and generate 61 mV. The third mm should then generate 61 mV but erroneous fluctuations might have it go to 63 mV. So, who knows what the message will be by the time it reaches the end of the axon.
-This man Claude Shannon had the insight that if you use digital, you can have perfect transmission, no errors.
-Instead of each bit of the neuron set up to try to reproduce the level in the previous bit of neuron,
it’s set up only to detect whether the membrane potential reaches higher than a threshold, and if the electrical potential is greater than that, it creates a predetermined signal.
-The threshold can have a value related to the size of random fluctuations, greater than the random fluctuations, in which case an action potential is never generated in error, and never erroneously not generated.
-Shannon proved all this in his master’s thesis, the greatest master’s thesis in history. It led to all the digital electronics that we use today.

*If you're wondering about the mention of plants, many plants do indeed have action potentials: http://onlinelibrary.wiley.com/store/10.1111/j.1365-3040.2006.01614.x/asset/j.1365-3040.2006.01614.x.pdf;jsessionid=37878CEAC4D234280FB73E9BF27694CE.f01t04?v=1&t=i8o4o3x7&s=ebf6e66ce48f988202c04bb15a028a07b9c52bcf
2
Ricky Grewal's profile photoJonathan Hunt's profile photoSam Schwarzkopf's profile photoAlex Holcombe's profile photo
5 comments
 
Excellent info, thanks +Jonathan Hunt 
Add a comment...

Alex Holcombe

Shared publicly  - 
 
4th-century St. Ambrose read silently- St. Augustine wrote: “When he read, his eyes scanned the page and his heart sought out the meaning, but his voice was silent and his tongue was still.” Some scholars have taken this passage as evidence that people at that time typically read aloud. That hypothesis matches the fact that punctuation was then used only sporadically; vocalizing would help the reader hear the prosody.

Willingham, Daniel T. Raising Kids Who Read: What Parents and Teachers Can Do (Kindle Locations 2657-2659). 

also see: http://web.stanford.edu/class/history34q/readings/Manguel/Silent_Readers.html
1
Add a comment...

Alex Holcombe

Shared publicly  - 
 
Reviewing instructions for the journal Psychological Science which probably started last year. More Methods reporting, part of effort to improve reproducibility.

(1) Manuscript Evaluation Criteria. Editors and reviewers alike are now asked to evaluate submissions with three questions in mind: (a) What will the reader of this paper learn about psychology that she or he did not know (or could not have known) before? (b) Why is that knowledge important for the field? (c) How are the claims made in the article justified by the methods used? Manuscripts that provide clear and compelling answers to these “What,” “Why,” and “How” questions will have the best prospects of being accepted for publication. For background on the rationale for these questions, please see this editorial: http://pss.sagepub.com/content/early/2013/11/25/0956797613512465.full/

2) Enhanced Reporting of Methods. Another new feature of the manuscript submission process for 2014 is a section containing checkboxes for four Research Disclosure Statement items (very similar to existing items confirming that research meets ethical guidelines, etc.). Submitting authors must check each item and, in doing so, actively declare that:

(a) the total number of excluded observations, and the reasons for making these exclusions, have been reported in the Method section(s),

(b) all independent variables or manipulations, whether successful or failed, have been reported in the Method section(s) [Authors skip this step if there were no independent variables or manipulations, as in the case of correlational research.],

(c) all dependent variables or measures that were analyzed for this article’s target research question have been reported in the Methods section(s), and

(d) the Method section(s) describe how sample size was determined and the rule for stopping data collection

As a reviewer, you don’t need to worry about the checkboxes or anything else to do with the manuscript submission process. The key point here is that the manuscript you’ll be reading will contain four categories of methodological details—Exclusions, Manipulations (if any), Measures, and Sample Size—that have not required disclosure under past reporting standards (of Psychological Science in particular or psychology journals in general) but that are important for interpreting research findings. I hope you will find these methodological details helpful to you in evaluating PSCI-15-0379.

(3) New Word Limits. For manuscripts submitted after January 1, 2014, the Method and Results sections of a manuscript will be excluded from the word limits on Research Articles and Research Reports. The new limits on Research Articles and Research Reports will be 2,000 and 1,000 words, respectively, and will include introductory and Discussion sections, as well as notes, acknowledgements, and appendices. The purpose of eliminating strict limits on the Method and Results sections is to afford authors the opportunity to report what they did, and what they found, in a manner that is clear, concise, and complete (see more details here: http://www.psychologicalscience.org/index.php/publications/observer/2013/november-13/whats-new-at-psychological-science.html). I welcome any ideas you may have for improving the exposition of the paper you’re reviewing.

(4) Embracing the “New Statistics.” Psychological Science now recommends the use of the “new statistics”—effect sizes, confidence intervals, and meta-analysis—to avoid problems associated with null-hypothesis significance testing (NHST). The journal sought to aid researchers in shifting from reliance on NHST to estimation and other preferred techniques by publishing a tutorial by Geoff Cumming, a leader in the new-statistics movement, that includes examples and references to books, articles, software, and online calculators that will aid authors in understanding and implementing estimation techniques in a wide range of research settings. The tutorial is available here: http://pss.sagepub.com/content/early/2013/11/07/0956797613504966.full.
Please note that the journal recommends, but does not require, adoption of the new-statistics approach. In certain areas of research—for instance, function magnetic resonance imaging (fMRI) when a brain region of interest hasn’t been pre-identified—NHST methods of analysis may be more appropriate.

(5) Supplemental Online Material. Since January 2012, Psychological Science has allowed the online publication of two types of supplemental material. One type, referred to as SOM-R, includes material that has undergone both an initial review (by two members of the editorial team) and an extended review (by two or more external referees). The other type, SOM-U, includes unreviewed material, or information that has not been vetted by either the editors or the external referees.
The distinction between the two types of supplemental material is discussed in greater detail here (http://www.psychologicalscience.org/index.php/publications/journals/psychological_science/ps-submissions#SM). Here let me emphasize two key points. First, feel free to skip over any SOM-U material that the author may have uploaded, but please look carefully at any SOM-R material. Second, in the Submission Guidelines, authors are advised that the editors take the adjective “supplemental” seriously: both SOM-R and SOM-U should include the sort of material that enhances the reader’s understanding of an article but is not essential for understanding the article. Thus, should you come across any SOM-R material you consider essential, please mention that in your review, so that the authors will know to move the material into the main article.
1
Add a comment...

Alex Holcombe

Shared publicly  - 
 
End of my review of a manuscript for a journal. I expect to re-use this, some of you might say something similar in your own reviews:

"Especially in cases like this where there are a lot of possible interactions that might occur and Bonferroni can really kill one's statistical power, I recommend (for future studies) preregistering one's hypotheses and analyses so that the post-hoc tests don't have to be post-hoc.

Also, I don't like papers where the (anonymized) data are not made available. But I know that's not the norm (yet).

Signed,
Alex Holcombe (I sign all my reviews)"

...Can't remember the name of the website that invites people to go much further than this and refuse to review manuscripts for which the data and code aren't posted.
1
Michael Kubovy's profile photoAlex Holcombe's profile photo
4 comments
 
Hi Michael,
thanks for the comments!
I had forgotten that Bonferroni is too severe, thanks for the reference.

About the idea that hypothesis and analysis plan preregistration stifles exploration, that's a common belief. But the way it should be implemented (and has been implemented at the several journals, https://osf.io/8mpji/wiki/home/,  that have adopted it as an option) is to not stop post-hoc unregistered analysis. One is free to report that as normal. It simply gives the reader the assurance that the p-values of those analyses that were preregistered can be taken at face value, rather than having to worry about how much to correct for multiple comparisons and the greater worry of whether everything was p-hacked. Here's a summary of registered reports with some refs if you haven't seen it http://www.theguardian.com/science/head-quarters/2014/may/20/psychology-registration-revolution .

Finally, I like the way you put "If you can't think of a plausible alternative to your hypothesis, is it worth testing?" - there are far too many sub-fields of psychology where a statistically significant rejection of the null is the primary, even the only goal. But the null hypothesis is practically never true! Too few appreciate that we should be striving to have our theories make predictions, not binary predictions, but actual sizes of effects.
Add a comment...

Alex Holcombe

Shared publicly  - 
 
Researchers frequently claim that motion-blind patients see moving objects as a series of stills (e.g. Ned Block here: http://www.lscp.net/persons/dupoux/papers/Kouider_Dupoux_2007.Partial_awareness_&_phenomenal_consc.BBS.pdf). However I haven't been able to find a report where the patient is clearly indicating that their experience is a series of stills rather than a blur, as would be seen with a long photographic exposure. And this is an important issue for the debate about discrete vs. continuous processes in perception. The famous Zihl akinetopsia paper is silent on this for patient L.M., but they do say that a patient described only in German by Potzl & Redlich felt a moving object appears "as if the visual stimulus remained stationary but appeared at different successive positions". That sounds like a series of stills but no indication of whether the patient was asked whether the successive positions are discrete, with empty space in between or instead might form a continuum, a blur. If anybody has a library with access to Wiener Klinische Wochenschrift, the 1911 volumes, I could ask someone in German to read the original.
In the case of certain psychoactive drugs, perhaps it is clear that a series of stills is seen rather than a blur-trail http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1001056 
3
Azeeza Rangunwala's profile photoDenis Pelli's profile photoAlex Holcombe's profile photo
5 comments
 
This palinopsia review paper clearly makes the distinction between stroboscopic and blurred multiple images, and claims that multiple patients with palinopsia with or without akinetopsia report stroboscopic images, discrete copies of the image of the moving object. I haven't had time to read the original case reports yet to verify this. http://www.surveyophthalmol.com/article/S0039-6257%2814%2900128-3/abstract Thanks +Patrick Goodbourn 
Add a comment...
People
In his circles
313 people
Have him in circles
989 people
palwinder don's profile photo
quality tunes daily's profile photo
Margareta Frye's profile photo
Brain Canada's profile photo
Alejandro Lleras's profile photo
Luis Zapien's profile photo
Rebecca J. Pelletier's profile photo
Mike Murray's profile photo
Bradley Voytek's profile photo
Work
Employment
  • University of Sydney
    Associate Professor, present
Places
Map of the places this user has livedMap of the places this user has livedMap of the places this user has lived
Currently
Sydney
Story
Tagline
Academic. Doing basic psychology and perception research; facilitating improvements to system of science.
Introduction
Education
  • Harvard University
    Psychology, 1995 - 2000
  • University of Virginia
    Cognitive Science, Psychology, 1991 - 1995
Basic Information
Gender
Male