Modern academic culture is founded on perpetual evaluation and relentless ranking of researchers and their results. Obtaining competitive research funds and publishing in top-tier venues are prerequisites to a successful career. Unfortunately, as was shown by several studies (most remarkably by NeurIPS 2014 experiment) the selection process is inherently affected by the factor of “pure luck”, despite the immense community efforts to ensure objective evaluation of the work.
No doubt that some randomness is inevitable. But is its impact only local (conference/grant proposal) or it may have longer-term implications on academic careers? This question is the focus of the recent study published in the Proceedings of National Academy of Sciences (PNAS), which I thought might be of interest to the community. Called “The Matthew effect in science funding“, the study investigates how winning an early career grant in the Netherlands (highly prestigious Netherlands Organization of Scientific Research Programme — NWO) influences the long-term funding success throughout the academic career, as well as the career path in general.
The authors leverage the fact that NWO funding decisions are determined by a predefined threshold: proposals with scores below the threshold are rejected regardless of how close their scores are to the threshold. The closer the scores to the threshold, the more similar the scientific merit among the proposals, and the higher the randomness of the final decision. Thus, the paper considers the career paths of the researchers whose early career proposals had the scores close to the acceptance threshold, yet some were funded and others rejected — almost surely due to “pure luck”.
The results are expected, yet overwhelming. The paper provides the statistical evidence that slightly luckier researchers that had their early career proposals funded had higher chances to be more successful in their future careers. The highlight (paper contains many interesting results) is that they usually managed to obtain more additional funds in the mid- and long-term (see below – reprinted with permission), and they were more likely to be promoted earlier than their less lucky peers. What is more important, in my view, is that the applicants with rejected proposals were less likely to apply for competitive funding again, to begin with, exhibiting the self-restraint that fueled the vicious cycle.
For me, there are two key takeaways from this paper:
1. As advisors, we should teach our graduate students to embrace failures, and accept them without losing self-confidence. The risk of being rejected is high when applying for competitive scholarships and submitting to top-tier publication venues. But we should explain how important it is to avoid the self-destructing self-restraint and to keep trying.
2. Bold ideas come with high risk, so academic institutions should reward young faculty for (even failed) attempts to apply for competitive grants, to partially compensate for the “pure luck” factor in the selection process which is so devastating for early career stages. Fortunately, this is the policy at the Technion and several other Israeli universities.
About the Author: Mark Silberstein is an Assistant Professor in the Electrical Engineering Department at the Technion – Israel Institute of Technology, where he heads the Accelerated Computer Systems Lab.
Disclaimer: These posts are written by individual contributors to share their thoughts on the Computer Architecture Today blog for the benefit of the community. Any views or opinions represented in this blog are personal, belong solely to the blog author and do not represent those of ACM SIGARCH or its parent organization, ACM.
