Profile cover photo
Profile photo
Alex Holcombe
Academic. Doing basic psychology and perception research; facilitating improvements to system of science.
Academic. Doing basic psychology and perception research; facilitating improvements to system of science.

Communities and Collections
View all

Tidy data. Below is a hasty description of data analysis workflow, a bit of the principle of "never touch raw data" ( combined with maintaining all information in a single data file. Has anyone seen a blog post or paper that provides a non-crappy (the below is hasty and crappy) description of such a recommendation?

Some participants will be excluded due to criteria that should be stated, like participant percent correct being lower than 65% or something like that, and individual trials should be excluded from eyetracking data when participants move their eyes. One way to deal with exclusions at both the within-subject and across-subject level is to simply delete those records (individual trials, or whole subjects) from the data. However, for record-keeping purposes it is better to have an "exclusion" column of the "spreadsheet" (ideally, a long-format csv or other txt file) and have it be 1 if that particular trial is to be excluded and a 0 if it is to be included. So in the case of an entire subject being excluded, all their rows will have a 1 if the subject is to be excluded. Alternatively, for a criterion like low performance resulting in exclusion which can be calculated by the R script, that column isn't used for that exclusion criteria because the R script works it out on the fly. And for eyetracking, it would be good if the datafile has a column indicating the amount by which the subject moved their eye from the fixation point during the critical trial period. The overall goal here is to not have, for any versions of the raw data files after the originally-generated one, subjects missing or trials missing, because it is sometimes non-trivial to reconstruct whether the missing data was excluded properly or whether it got lost in some kind of copy-paste error.
Add a comment...

Post has attachment
One of the documented problems with the current system of scientific communication is that when strong evidence is published that overturns or moderates a previous claim, it often does not have the impact it should, with many researchers continuing to cite the original paper in blithe ignorance of the subsequent evidence. Thus, increasing the rate at which researchers attempt to replicate previous research will not be enough, we must also have a new focus on combining evidence from across studies. This has led to the rise of the rate of publication of meta-analysis articles. However, it’s been pointed out that these are quite wasteful and slow, wasteful in that meta-analysis articles tend to involve re-doing the analysis without capitalizing on a previous meta-analysis, and slow in that by the time a meta-analysis article is reviewed and published in a traditional journal, more studies will likely have appeared that aren’t included. is a site that addresses these problems with an infrastructure for cumulative meta-analyses that can be updated as soon as new studies are available. As a member of CurateScience’s advisory board, I have very occasionally provided advice.
Curate Science
Curate Science
Add a comment...

For those of you who teach methods, a way to explain "control" groups:

If you've ever traveled to another country, you've likely encountered Passport Control. Its name is a bit confusing - it is a place simply to check passports, not to control them. The term Passport Control derives from an older meaning of "control", which was to check or confirm.

This meaning of control also survives in the scientific terms "control group" and "control experiment", which help confirm that what is observed in the experiment is due to the manipulation of interest.
Add a comment...

For those of you who teach methods, a way to explain "control" groups:

If you've ever gone overseas, lovely students, you've likely encountered Passport Control. Its name is confusing in modern English, because it is a place simply to check passports, not to control them. The term derives from the original meaning of "control", which was to check or confirm. This meaning survives in the scientific jargon of "control group" and "control experiment", which help confirm that what is observed in the experiment is due to the manipulation of interest.

Thanks Nick Brown and Sam Schwarzkopf
Add a comment...

Post has shared content
To the linked list of zero-fee open-access journals, ranked by article influence, in response to a query from a colleague I add these notes on some open access science journals in fields related to mine, most of which charge a fee:

eLife (free for authors as well as for readers, funded by Max Planck, Wellcome Trust, and HHMI, set up as a competittor to Cell and Nature, run by academics rather than professional editors. I believe they use custom JMS (journal management system) and certainly have a custom publishing platform. Impact factor is around 9.

Glossa is a journal created by an editorial board that defected from a top Elsevier linguistics journal. It is free for both readers and authors thanks to the OLH consortium of libraries worldwide. It is too young to have an impact factor, but is recognized by essentially the entire linguistics community as the continuation of the Elsevier journal (which still exists in zombie form, with some elderly replacement editors). The actual cost, were it not paid for by OLH and a European grant, would be 400 Euros.

Open Mind , a new fully open access MIT PRess journal edited by Richard Aslin. APC is $950

Discrete Analysis, no author fee because they use a cheap Scholastica publishing platform and JMS that charges only $10 /submission, covered by a grant. It is set up as an overlay journal to arXiv, but that isn't necessary for their model, although would be a bit more expensive if it were not an overlay journal.

Journal of Vision- perhaps the flagship journal in the study of visual perception, APC >$1000

PLoS journals, including PLoS Biology, which has an impact factor around 9. Waivers for authors who can't pay the fee, although they demand more and more documentation for that, I've heard - there was certainly some abuse when no questions were asked.

Do remind me of those that I have missed.
A list of reasonable free candidate journals  for publishing in (of course, newer outfits like Discrete Analysis will not be listed).
Add a comment...

Post has shared content
I am thinking of starting an effort to mount an arxiv-based overlay journal in logic, in the style of Discrete Analysis (

My idea would be a completely open-access journal focussing on topics in mathematical and philosophical logic, on the overlay journal concept.

Please let me know below if there is interest in supporting such a venture. Would you submit your research articles to such a journal? Would you serve as referee? As editor? To what extend would the community support such a venture?

Please vote-up or share this post if this is a venture that you would support. If there is strong support for such an effort, I will take it more seriously to make it happen.
Add a comment...

Post has attachment
I used to think diagrams like this were probably accurate in showing dichromats, like dogs, having a gradient between only two qualitatively distinct colors at most (of course, what those particular colors are qualitatively is unknowable; surely they don't feel like our blue and yellow). But from this paper I learned that human dichromats claim to see not a smooth continuum, but multiple qualitatively distinct bands ( Some of the qualitatively different colors we see in the spectrum reflect post-receptoral mechanisms that dogs might also have. If so, you should be able to find a boost in dog learning performance (but not discrimination, which reflects the receptor space) for two colors on either side of a band boundary. Don't know if this has been done.
Add a comment...

Post has attachment
The tobacco industry was clever to create a journal, Indoor Environment, even though only 60% of articles had conclusions favorably disposed towards tobacco. They knew they only needed a few articles in a legitimate-seeming journal for their propaganda war. As the saying goes, the amount of energy necessary to refute bullshit is an order of magnitude bigger than to produce it.
Add a comment...

Post has attachment
Add a comment...

Post has attachment
A discussion about the high author fees charged by some #openaccess journals brought up many other issues, some of which were included in a survey. Nearly 400 vision researchers responded to the survey, of which 93% expressed desire for change. See the detailed response breakdown here: .
When I reported these survey results to the community mailing list (CVnet), I invited journal editors and publisher representatives to respond, and that their responses would be sent out after 3 weeks. Here are their responses:

From: Cathleen Moore (

I am writing on behalf of the Psychonomics Society in regard to the recent journal survey results that have been distributed throughout our community. We would like to offer the following statement as the outcome of discussions within the Executive Committee and the Publications Committee. We would be grateful if you would include in this your communications to the community regarding the recent survey results and surrounding discussion.

The mission of the Psychonomic Society is “…the communication of scientific research in psychology and allied sciences."
( That is, communication of the science is the very purpose for our existence. As such, the Psychonomic
Society is committed to making membership in the society, the annual meeting, and all of our journals affordable for all. Open-access publishing is one aspect of the Society’s commitment, as evident in the establishment of our new open-access journal Cognitive Research: Processes and Implications.

Discussions about open access and other models of publishing are ongoing, and will be part of the formal agenda at future meetings of the Governing
Board later this year.


Cathleen Moore
Chair, Governing Board

In consultation with:
Aaron Benjamin, Chair Elect
Bob Logie, Past Chair
Fernanda Ferreira, Publications Committee Chair
From: Dennis Levi (

The topic of open access will be a major discussion issue for the JOV [Journal of Vision] board meeting at VSS in May.
From: Timothy Meese (

Dear Vision Scientists

We at i-Perception and SAGE are pleased to respond to the issues raised in the recent discussion of open access on CVNet. We circulated a general response over CVNet shortly before Alex Holcombe circulated the results of his survey and the invitation to respond to those. We have appended our earlier circular to this email for completeness and for contact details.


i-Perception (iP) is a fully open access journal with papers published under a CC-BY license. As the survey was about OA, most of our response relates to iP. However, iP's sister print journal Perception (P) includes some material that is also open access. We list that here for completeness: Editorials, the Perception lecture (from ECVP), some conference abstracts, and some of the back archive. The journal Perception can be accessed here:

Costs at iP are clearly competitive (375 GBP [~ 568 USD] for regular articles, see below for further details.) We can confirm that these costs will be fixed through 2016. They will be reviewed in 2017 to ensure ongoing viability for all stakeholders.

Regarding academic control, iP Chief Editors meet with SAGE three times a year, and there is also an annual Editorial/Advisory Board meeting at ECVP, with a representative from SAGE. It is our impression, confirmed during our board meetings at ECVP, that iP is not viewed as overpriced and that reform on this matter is not being sought at this time. However, we add that the Chief Editors will do what they can to keep costs down. We would also like to point out that the Chief Editors are always open to suggestions (by email or in person), which can be taken forward to management board meetings for further discussion. Although subject to certain constraints (e.g. the limitations of generic company software packages such as ScholarOne), we have found SAGE to be very accommodating to our requests and suggestion thus far.

The sum that an author pays for publication has two components: 1. Internal production costs (non profit). 2. Profit. For large organisations, isolating item 1 is quite tricky—for example, should this be averaged over all the publisher’s journals or just the relevant journal? As different journals adopt different approaches, comparisons of components 1 and 2 are likely to be problematic. However, the TOTAL cost that an author pays (page charges, OA/CC-BY charges, any other charges and fees) allows for unambiguous comparisons.

At present SAGE do not do this for any of their journals. There is no immediate plan to do so, but SAGE are keeping their eye on the situation for open review. As for post-peer review, we do have a section in iP called ‘Journal Club’ which is intended for published discussion of other people’s publications. This, we believe, is the best way to implement relevant post-publication peer review.

This allows authors to register the format of their study before data are gathered. This can be valuable in justifying the chosen statistical analysis and also for reporting null results. This is something that several SAGE journals do and will be a subject for discussion between SAGE and the chief editors of iP at their next managerial board meeting in June.

This was first raised at our Editorial/Advisory Board meeting held at ECVP in 2013 and then discussed in detail by that board the following year after circulating a detailed paper on the matter. While the value of these badges was acknowledged for other journals and disciplines, their value for vision/perception research was viewed as questionable and there was little or no enthusiasm at the meeting for adopting the COS badges at that time. However, that decision is open to review, particularly in light of the item above.

At present, SAGE do not report submission and acceptance dates for articles in iP, but this is something that will change in the near future. We are also looking into whether it is possible to make average review times for the journal available on the website.

COPE membership for Perception and i-Perception is currently being processed and we expect to be able to acknowledge this on the website very soon.

SAGE provide copyediting and typesetting. Authors see the copyedited and typeset proofs for any final corrections before publication.

We will be able to accept LaTeX submissions very soon.

To register with iP and/or sign up to receive an email alert for each new issue go here:

Chief editors of Perception and iPerception:
Tim Meese
Peter Thompson
Frans Verstraten
Johan Wagemans
Ellie Craven

APPENDIX A (The email below was first circulated over CVNet on 1st February 2016)

There is a new OA journal already…

…It is i-Perception.

i-Perception (or iPerception) was founded in 2010, and is the OA, peer-reviewed, online sister journal to the long-running print journal, Perception, founded by Richard Gregory in 1972.

For many years both journals were owned by the UK publisher Pion but have recently been taken on by SAGE. As editors, we have enjoyed positive relations with both publishers regarding all aspects of the journal. Although the shift to SAGE has meant the loss of the much beloved submission system, PiMMS (we now use ScholarOne) and ‘paper icons’ on the contents page of iPerception, we are now enjoying the benefits of efficiency and outreach that comes with a larger publisher, and one that we have found to be sensitive to the needs and views of the journals' editors and authors.

Perception and iPerception (we often abbreviate the two journals to PiP) are journals run by vision/perceptual/sensory scientists for vision/perceptual/sensory scientists. For example, PiP have a long standing history in supporting major vision conferences (APCV, ECVP, VSS), but particularly ECVP, where they are the chosen outlet for published conference abstracts and sponsors of the keynote ‘Perception’ lecture on the opening evening.

The remit of both journals is the same: any aspect of perception (human, animal or machine), with an emphasis on experimental work; but theoretical positions, reviews and comments are also considered for publication (see website below for details of the various paper categories). Although the majority of the papers published are on visual perception, all other aspects of perception are also covered, including multi-modal studies, giving it a broader remit than either VR or JoV. PiP is sometimes thought to focus on phenomenology (owing to the interests of its founding editor, we think), but hardcore psychophysics is also found within its pages, and much of what is published in VR or JoV would not be out of place in PiP.

Although the two journals are independent (e.g. they have their own impact factors; the IF for iP is 1.482), they are overseen by a common international editorial board who can be found by following the third link at the bottom of this page. An editorial board meeting takes place annually at ECVP.
The four chief editors (based in Europe/Australia, see below) and the administrative manager (Gillian Porter, based in Bristol, UK) hold managerial board meetings three times a year with SAGE (based in London, UK) and enjoy a close working relationship with an open door (email) policy.

Papers in iP are published under a CC-BY license ( (this is Gold OA). Papers in PiP are also branded Romeo Green ( (This is a different branding system from the more familiar Gold/Green OA terminology, and the two should not be confused.) Romeo Green status enables authors to archive their accepted version in their institutional repository, their departmental website, and their own personal website immediately upon acceptance. This is the most open publishing policy possible, of which SAGE (and we) are justly proud.
If your library does not stock Perception, you might think to request that they do—the bundles with which it is included are likely to have changed with the change in publisher (to SAGE).

There is no cost to publishing in Perception.
The cost for publishing in iPerception is a single charge (on acceptance of the paper), depending on paper type as follows:
Regular Articles = 375 GBP (~ 568 USD)
Short Reports = 200 GBP (~ 303 USD)
Translation Articles = 200 GBP (~ 303 USD)
Short and Sweet = 150 GBP (~227 USD)
Journal Club = 150 GBP (~227 USD)
iReviews = no charge.

VAT (value added tax) at 20% is added to the costs above if the paying party is in the European Union, to comply with European Law. Non-UK institutions are exempt from VAT if they can provide a VAT registration number.

We have been watching the debate on OA over CVNet with interest; we agree that a low cost OA option is a desirable forum for our community—preferably one based around a dedicated journal so as to provide a sense of ‘home' rather than an unfocussed (rebellious, even) out camp—; we hope you will join us to help bring iP towards the forefront of that endeavour.


To see content of iPerception follow the link below…

To submit to Perception or iPerception follow the link below to ScholarOne…

To see details about iPerception follow the link below…

Signed (Chief editors of Perception and iPerception)
Tim Meese
Peter Thompson
Frans Verstraten
Johan Wagemans

Add a comment...
Wait while more posts are being loaded