Movatterモバイル変換

Prepublicationpeerreview should be abolished. We consider the effects that such a change will have on the social structure of science, paying particular attention to the changed incentive structure and the likely effects on the behaviour of individual scientists. We evaluate these changes from the perspective of epistemic consequentialism. We find that where the effects of abolishing prepublicationpeerreview can be evaluated with a reasonable level of confidence based on presently available evidence, they are (...) either positive or neutral. We conclude that on present evidence abolishingpeerreview weakly dominates the status quo. (shrink)

Direct download(5 more)   Export citation   Bookmark   33 citations  

A growing interest in and concern about the adequacy and fairness of modernpeer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables.The present investigation was an attempt to study thepeer-review process directly, in the natural setting of actual journal referee evaluations of submitted manuscripts. As test materials we (...) selected 12 already published research articles by investigators from prestigious and highly productive American psychology departments, one article from each of 12 highly regarded and widely read American psychology journals with high rejection rates (80%) and nonblind refereeing practices.With fictitious names and institutions substituted for the original ones (e.g., Tri-Valley Center for Human Potential), the altered manuscripts were formally resubmitted to the journals that had originally refereed and published them 18 to 32 months earlier. Of the sample of 38 editors and reviewers, only three (8%) detected the resubmissions. This result allowed nine of the 12 articles to continue through thereview process to receive an actual evaluation: eight of the nine were rejected. Sixteen of the 18 referees (89%) recommended against publication and the editors concurred. The grounds for rejection were in many cases described as “serious methodological flaws.” A number of possible interpretations of these data are reviewed and evaluated. (shrink)

Direct download(7 more)   Export citation   Bookmark   190 citations  

Peerreview and publication is one of the factors proposed in Daubert v. Merrell Dow Pharmaceuticals, Inc. as indicia of the reliability of scientific testimony. This Article traces the origins of thepeer-review system, the process by which it became standard at scientific and medical journals, and the many roles it now plays. Additionally, the Author articulates the epistemological rationale for pre-publicationpeer-review and the inherent limitations of the system as a scientific quality-control mechanism. (...) The Article explores recent changes in science, in scientific publishing, and in the academy that have put the system under strain. The Author argues that Justice Blackmun's advice to courts - thatpeer-reviewed publication is relevant, but is not dispositive - is of little practical help. Instead, the Author suggests questions that courts should ask in assessing the significance of the fact that testimony is, or is not, based onpeer-reviewed publication and illustrates with reference to another Bendectin case, Blum v. Merrell Dow Pharmaceuticals, Inc., where some of these questions were asked. (shrink)

Direct download   Export citation   Bookmark   6 citations  

At present,peerreview is the most common method used by funding agencies to make decisions about resource allocation. But how reliable, efficient, and fair is it in practice? The ex ante evaluation of scientific novelty is a fundamentally uncertain endeavor; bias and chance are embedded in the final outcome. In the current study, I will examine some of the most central problems ofpeerreview and highlight the possible benefits of using a lottery as an (...) alternative decision-making mechanism. Lotteries are driven by chance, not reason. The argument made in the study is that the epistemic landscape could benefit in several respects by using a lottery, thus avoiding all types of bias, disagreement, and other limitations associated with thepeerreview process. Funding agencies could form a pool of funding applicants who have minimal qualification levels and then select randomly within that pool. The benefits of a lottery would not only be that it saves time and resources, but also that it contributes to a more dynamic selection process and increases the epistemic diversity, fairness, and impartiality within academia. (shrink)

No categories

Direct download   Export citation   Bookmark   8 citations  

Peerreview process helps in evaluating and validating of research that is published in the journals. U.S. Office of Research Integrity reported that data fraudulence was found to be involved in 94% cases of misconduct from 228 identified articles between 1994–2012. If fraud in published article are significantly as high as reported, the question arise in mind, were these articlespeer reviewed? Another report said that the reviewers failed to detect 16 cases of fabricated article of Jan (...) Hendrick Schon. Superficialpeer reviewing process does not reveals suspicion of misconduct. Lack of knowledge of systemicreview process not only demolish the academic integrity in publication but also loss the trust of the people of the institution, the nation, and the world. The aim of thisreview article is to aware stakeholders specially novice reviewers about thepeerreview system. Beginners will understand how toreview an article and they can justify better action choices in dealing with reviewing an article. (shrink)

Direct download(5 more)   Export citation   Bookmark   1 citation  

Peerreview is a practice used in the evaluation of scientific and scholarly papers in order to select papers for publication in scholarly journals. The practice has also been extended to other domains, such as the evaluation of grant proposals, medical practice, book publication, and even to such areas as teaching evaluation. The primary area that has been of interest to sociologists, however, has been publication in scientific journals. The practice is usually understood to have begun in the (...) seventeenth century in the Royal Society in London, but it has also been claimed that there have been precursors to this practice. In the popular mind,peerreview in science is a means of “bullet‐proofing” research, as a Wall Street Journal article once put it, that is to say as a guarantee of quality. But within science it has a very different meaning. (shrink)

No categories

Direct download   Export citation   Bookmark  

Two important aspects of the relationship betweenpeerreview and innovation includes the acceptance of articles for publication in journals and the assessment of applications for grants for the funding of research work. While there are well-known examples of the rejection by journals of first choice of many papers that have radically changed the way we think about the world outside ourselves, such papers do get published eventually, however tortuous the process required. With grant applications the situation differs (...) in that the refusal of a grant necessarily curtails the possible research that may be attempted. Here there are many reasons for conservatism and reservation as to the ability of a grant allocation process based onpeerreview to deliver truly innovative investigations. Other methods are needed; although such methods need not be applied across the board, they should constitute the methods whereby some 10–20% of the grant monies are assigned. The nomination of prizes for specific accomplishments is one way of achieving innovation although this presumes that investigators or institution already have available the money necessary to effect the innovations; otherwise it is a question of the selection and funding of particular individuals or institutions and requiring them to solve particular problems that are set in the broadest of terms. (shrink)

Direct download(4 more)   Export citation   Bookmark   9 citations  

We examine the criticisms and subsequent changes that arise in the course ofpeerreview. Fifty-two scholars who had recently published in Administrative Science Quarterly were surveyed regarding theirpeerreview experience and how their article changed from initial journal submission to eventual publication. Papers that challenged theoretical perspectives faced distinctively high levels of criticism and change, particularly with attention to methodology, while those that offered a new perspective or that extended or combined established perspectives were (...) less criticized and changed. The number of challenge-oriented publications was small as well, suggesting that either few such submissions survive thereview process or few are submitted in the first place. Overall,peerreview appears open to expansion of the variety of theoretical argument but does little to aid in the winnowing out of established perspectives. (shrink)

No categories

Direct download   Export citation   Bookmark   1 citation  

This paper reviews the published empirical evidence concerning journalpeerreview consisting of 68 papers, all but three published since 1975.Peerreview improves quality, but its use to screen papers has met with limited success. Current procedures to assure quality and fairness seem to discourage scientific advancement, especially important innovations, because findings that conflict with current beliefs are often judged to have defects. Editors can use procedures to encourage the publication of papers with innovative findings (...) such as invited papers, early-acceptance procedures, author nominations of reviewers, structured rating sheets, openpeerreview, results-blindreview, and, in particular, electronic publication. Some journals are currently using these procedures. The basic principle behind the proposals is to change the decision from whether to publish a paper to how to publish it. (shrink)

Direct download(8 more)   Export citation   Bookmark   16 citations  

BackgroundThe Patient-Centered Outcomes Research Institute is obligated topeerreview and to post publicly “Final Research Reports” of all funded projects. PCORIpeerreview emphasizes adherence to PCORI’s Methodology Standards and principles of ethical scientific communication. During thepeerreview process, reviewers and editors seek to ensure that results are presented objectively and interpreted appropriately, e.g., free of spin.MethodsTwo independent raters assessed PCORIpeerreview feedback sent to authors. We calculated the proportion (...) of reports in which spin was identified duringpeerreview, and the types of spin identified. We included reports submitted by April 2018 with at least one associated journal article. The same raters then assessed whether authors addressed reviewers’ comments about spin. The raters also assessed whether spin identified during PCORIpeerreview was present in related journal articles.ResultsWe included 64 PCORI-funded projects.Peer reviewers or editors identified spin in 55/64 submitted research reports. Types of spin included reporting bias, inappropriate interpretation, inappropriate extrapolation of results, and inappropriate attribution of causality. Authors addressed comments about spin related to 47/55 of the reports.Of 110 associated journal articles, PCORI comments about spin were potentially applicable to 44/110 articles, of which 27/44 contained the same spin that was identified in the PCORI research report. The proportion of articles with spin was similar for articles accepted before and after PCORIpeerreview.DiscussionJust as spin is common in journal articles and press releases, we found that most reports submitted to PCORI included spin. While most spin was mitigated during the funder’speerreview process, we found no evidence thatreview of PCORI reports influenced spin in journal articles. Funders could explore interventions aimed at reducing spin in published articles of studies they support. (shrink)

No categories

Direct download   Export citation   Bookmark  

Peerreview is a topic of considerable concern to many researchers, and there is a correspondingly large body of research on the topic. This issue of Science and Engineering Ethics presents recent work onpeerreview that is both grounded in empirical science and is applicable to policy decisions. This research raises two basic questions; (a) how does currentpeerreview operate, and (b) how can it be improved? Topics addressed include descriptions of how (...)peerreview is used in Federal agencies. whetherpeerreview leads to better manuscripts, demographic characteristics of authors or reviewers (status or institutional affiliation), blinding of reviewers, authors, or results, reliability and consistency of reviews, accepting a paper before the study is done, simultaneous submission, and use of dispute resolution procedures such as scientific dialectical and pleading protocols. (shrink)

Direct download(3 more)   Export citation   Bookmark   3 citations  

Retractions of COVID-19 literature in both preprints and thepeer-reviewed literature serve as a reminder that there are still challenging issues underlying the integrity of the biomedical literature. The risks to academia become larger when such retractions take place in high-ranking biomedical journals. In some cases, retractions result from unreliable or nonexistent data, an issue that could easily be avoided by having open data policies, but there have also been retractions due to oversight inpeerreview and (...) editorial verification. As COVID-19 continues to affect academics and societies around the world, failures inpeerreview might also constitute a public health risk. The effectiveness by which COVID-19 literature is corrected, including through retractions, depends on the stringency of measures in place to detect errors and to correct erroneous literature. It also relies on the stringent implementation of open data policies. (shrink)

Direct download(2 more)   Export citation   Bookmark   3 citations  

This chapter critically reflects on the critiques, reviews, and many proposals presented in Parts Two, Three, and Four, and provides a summary conclusion for the entire Zadeh Project. Obvious differences between experience and reporting on experience are highlighted, with particular attention to the ways such differences are detailed by the Zadeh Scenario and in our colleagues’ responses to it. In addition, we discuss a key challenge associated with clinical ethics practice and thepeerreview of such practice: identifying (...) what actually matters for those engaged in these kinds of activities. In each case, we argue, the work of discovering what matters to the individuals actually involved cannot be overlooked. Furthermore, that work may even be the primary ingredient to responsible conduct for both consultation andpeerreview. Building on this idea that responding responsibly to that which is actually encountered in clinical practice serves as a primary activity of, and thus grounds responsibility in, clinical ethics practice, we then argue that ethical analysis of Finder’s activities in the Zadeh Scenario, and for clinical ethicspeerreview more generally, has at least three layers: that which addresses the immediate interactions between an ethics consultant and those whom the consultant engages; that which addresses the ethics consultant’s own sense of self-responsibility; that which addressespeer reviewers’ own frames for evaluating another’s clinical ethics practice. We conclude that the Zadeh Project exemplifies what is at stake in ethics consultation andpeerreview. Taken as a whole, then, the Zadeh Project reveals howpeerreview, in acknowledgement of the responsibility to create a context forpeer education, rests upon a commitment to model – and demand critical engagement with – what is held to be most worthwhile for clinical ethics practice and the field of clinical ethics consultation as a whole. (shrink)

No categories

Direct download   Export citation   Bookmark  

Peerreview has long been regarded as the gold standard of scientific publication, essential to the integrity of science itself. But, as any publishing scientist knows,peerreview has its downside, including long delays and reviewer bias. Until the coming of the Internet, there appeared to be no alternative. Now, articles appear online as preprints almost immediately upon submission. But they lackpeerreview and thus their scientific standing can be questioned. Post-publication discussion platforms (...) such as PubPeer have proven useful, but are no substitute for pre-publicationpeerreview. Nevertheless, some may be tempted to believe thatpeerreview can now be done without. This article challenges that view by analyzing a recent, non-peer-reviewed article in Skeptical Enquirer, a magazine published by the Committee for Skeptical Enquiry (CSI). The article, “Sodom Meteor Strike Claims Should Be Taken With a Pillar of Salt,” casts doubt on one of the most widely read scientific articles of the last decade and provides a stern warning of the cost of abandoningpeerreview. (shrink)

Direct download(3 more)   Export citation   Bookmark  

Research on bias inpeerreview examines scholarly communication and funding processes to assess the epistemic and social legitimacy of the mechanisms by which knowledge communities vet and self-regulate their work. Despite vocal concerns, a closer look at the empirical and methodological limitations of research on bias raises questions about the existence and extent of many hypothesized forms of bias. In addition, the notion of bias is predicated on an implicit ideal that, once articulated, raises questions about the (...) normative implications of research on bias inpeerreview. Thisreview provides a brief description of the function, history, and scope ofpeerreview; articulates and critiques the conception of bias unifying research on bias inpeerreview; characterizes and examines the empirical, methodological, and normative claims of bias inpeerreview research; and assesses possible alternatives to the status quo. We close by identifying ways to expand conceptions and studies of bias to countenance the complexity of social interactions among actors involved directly and indirectly inpeerreview. (shrink)

Direct download   Export citation   Bookmark   63 citations  

An important function in all scholarly and academic activities is the participation in thepeerreview system. One aspect of thispeerreview evaluation is service on committees judging candidates for important awards, prizes and fellowships. Some reflective observations on this process are made in which a number of factors determining the final choice are identified. It is pointed out that the decisions of such committees are based not only on relevant and objective criteria but are (...) also influenced by a number of irrelevant criteria; caution must be exercised to minimize the effect of the latter factors in order to maintain the highest ethical standards in the selection process. The ranking practices of awards committees, national academies and learned societies are briefly reviewed by pointing out some ethical pitfalls, anecdotal incidents, and the ways to avoid the tarnishing of the selection process. (shrink)

Direct download(3 more)   Export citation   Bookmark   1 citation  

In a recent article in the British Journal for the Philosophy of Science, Heesen and Bright argue that prepublicationpeerreview should be abolished and replaced with postpublicationpeerreview (provided the matter is judged purely on epistemic grounds). In this article, I show that there are three problems with their argument. First, it fails to consider the epistemic cost of implementing the change to postpublicationpeerreview. Second, it fails to consider some potential (...) epistemic benefits of prepublicationpeerreview, which involve avoiding bias. Third, it fails to consider some potential epistemic disadvantages of postpublicationpeerreview, which stem from the greater number of papers that would be published under that system. (shrink)

Direct download(2 more)   Export citation   Bookmark   5 citations  

Pre-publication certification throughpeerreview stands in need of philosophical examination. In this paper, philosopher-psychologist Steven James Bartlett recalls the arguments marshalled four hundred years ago by English poet John Milton against restraint of publication by the "gatekeepers of publication," AKA today'speer reviewers.

Direct download(3 more)   Export citation   Bookmark   3 citations  

As universities in the United Kingdom gear themselves up for the next Research Excellence Framework, REF2021, withpeerreview at its core, we critically re-visit the idea ofpeerreview as a gold standard proxy for research excellence. We question the premise that anonymouspeerreview is a necessary and enabling condition for impartial, expert judgement. We argue that the intentions and supposed benefits underlyingpeerreview and its associated concepts have become (...) congealed in received discourse about research quality. Hence we explore the key conceptual issues raised by the nested assumptions and concepts that come into play inpeerreview as currently practised: primarily those of secrecy, anonymity, legitimacy, trust, impartiality and openness. After delineating the benefits attributed topeerreview, we contrast its declared virtues with its problematic features. We locatepeerreview in an audit culture in which the reviewer is an academic labourer. Drawing on recent trends in moral and political philosophy, we question the usefulness of the ideal of impartiality when tied to secrecy. Then we raise more deliberative, intersubjective possibilities for a revised understanding ofpeerreview in the context of an academic community. Finally, we suggest ways in which the academic community could pursue quality in research by recastingpeerreview to be less secret and more open. (shrink)

Direct download(2 more)   Export citation   Bookmark  

Peer-review is neither reliable, fair, nor a valid basis for predicting ‘impact’: as quality control,peer-review is not fit for purpose. Endorsing the consensus, I offer a reframing: while a normative social process,peer-review also shapes the writing of a scientific paper. In so far as ‘cognition’ describes enabling conditions for flexible behavior, the practices ofpeer-review thus constrain knowledge-making. To pursue cognitive functions ofpeer-review, however, manuscripts must be seen (...) as ‘symbolizations’, replicable patterns that use technologically enabled activity. On this bio-cognitive view,peer-review constrains knowledge-making by writers, editors, reviewers. Authors are prompted to recursively re-aggregate symbolizations to present what are deemed acceptable knowledge claims. How, then, can recursive re-embodiment be explored? In illustration, I sketch how the paper’s own content came to be re-aggregated: agonisticreview drove reformatting of argument structure, changes in rhetorical ploys and careful choice of wordings. For this reason, the paper’s knowledge-claims can be traced to human activity that occurs in distributed cognitive systems.Peer-review is on the frontline in the knowledge sector in that it delimits what can count as knowing. Its systemic nature is therefore crucial to not only discipline-centered ‘real’ science but also its ‘post-academic’ counterparts. (shrink)

Direct download(5 more)   Export citation   Bookmark   2 citations  

BackgroundThe aim of this survey was to determine the level of awareness and understanding ofpeerreview andpeerreview models amongst junior hospital doctors and whether this influences clinical decision-making.MethodsA 30-question online anonymous survey was developed aimed at determining awareness ofpeerreview models and the purpose ofpeerreview, perceived trustworthiness of differentpeerreview models and the role ofpeerreview in clinical decision-making. It was (...) sent to 800 trainee doctors in medical specialties on the University College London Partners trainee database.ResultsThe response rate was (178/800) 22%. Most respondents were specialist registrars. Checking that research is conducted correctly (152/178, 85%) and the data interpreted correctly (148/178, 83%) were viewed as the most important purposes ofpeerreview. Most respondents were aware of open (133/178, 75%), double-blind (125/178, 70%) and single-blindpeerreview (121/178, 68%). 101/178 (57%) had heard of collaborative, 87/178 (49%) of post publication and 29/178 (16%) of decoupledpeerreview. Of those who were aware of double-blind, single-blind open and collaborativepeerreview, 85 (68%), 82 (68%), 74 (56%) and 24 (24%), respectively, understood how they worked. The NEJM, Lancet and The BMJ were deemed to have most trustworthypeerreview, 137/178 (77%), 129/178 (72%) and 115/178 (65%), respectively. Thatpeerreview had taken place was important for a journal content to be used for clinical decision-making 152/178 (85%), but the ability to seepeerreview reports was not as important 22/178 (12%). Most felt there was a need forpeerreview training and that this should be at the specialist registrar stage of training.ConclusionsJunior hospital doctors viewpeerreview to be important as a means of quality control, but do not value the ability to scrutinizepeerreview themselves. The unquestioning acceptance ofpeerreview as final validation in the field of medicine emphasises not only the responsibility held by medical journals to ensurepeerreview is done well but also the need to raise awareness amongst the medical community of the limitations of the currentpeerreview process. (shrink)

No categories

Direct download   Export citation   Bookmark  

Previous research has found that funding disparities are driven by applications’ final impact scores and that only a portion of the black/white funding gap can be explained by bibliometrics and topic choice. Using National Institutes of Health R01 applications for council years 2014–2016, we examine assigned reviewers’ preliminary overall impact and criterion scores to evaluate whether racial disparities in impact scores can be explained by application and applicant characteristics. We hypothesize that differences in commensuration—the process of combining criterion scores into (...) overall impact scores—disadvantage black applicants. Using multilevel models and matching on key variables including career stage, gender, and area of science, we find little evidence for racial disparities emerging in the process of combining preliminary criterion scores into preliminary overall impact scores. Instead, preliminary criterion scores fully account for racial disparities—yet do not explain all of the variability—in preliminary overall impact scores. (shrink)

Direct download   Export citation   Bookmark   2 citations  

There is a paucity of studies addressing the nature of the social relations of production prevailing in academia prior to the commodification of academic research. By filling that gap, this paper enables us to better understand the historical presuppositions from which the process of knowledge commodification in academia has evolved. Our theoretically informed analysis will focus onpeerreview, given that it is one of the few academic practices where traces of that historical past can still be found. (...) On the basis of Marx’s exposition of the main features of pre-capitalist social relations of production in the Grundrisse, it will be concluded thatpeerreview reveals that social relations of production in academia were of a pre-capitalist, or natural-like, nature:peerreview is labour of a direct nature; is the practice through which the academic community mediates its own reproduction, determining how – and, sometimes, if – its members enter into contact with the objective conditions of production; finally, in this latter capacity,peerreview presupposes that the academic community relates to the conditions of knowledge production as though they were their collective property. These findings are employed to account for the underlying logics of the ongoing ‘functional transformation’ ofpeerreview. (shrink)

Direct download(2 more)   Export citation   Bookmark  

In this paper, we analyze the role of science and technology studies journal editors in organizing and maintaining thepeerreview economy. We specifically conceptualizepeerreview as a gift economy running on perpetually renewed experiences of mutual indebtedness among members of an intellectual community. While thepeerreview system is conventionally presented as self-regulating, we draw attention to its vulnerabilities and to the essential curating function of editors. Aside from inherent complexities, there are (...) various shifts in the broader political–economic and sociotechnical organization of scholarly publishing that have recently made it more difficult for editors to organize robust cycles of gift exchange. This includes the increasing importance of journal metrics and associated changes in authorship practices; the growth and differentiation of the STS journal landscape; and changes in publishing funding models and the structure of the publishing market through which interactions among authors, editors, and reviewers are reconfigured. To maintain a functioningpeerreview economy in the face of numerous pressures, editors must balance contradictory imperatives: the need to triage intellectual production and rely on established cycles of gift exchange for efficiency, and the need to expand cycles of gift exchange to ensure the sustainability and diversity of thepeerreview economy. (shrink)

No categories

Direct download   Export citation   Bookmark   2 citations  

An error in the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, which wrongly predicted the disappearance of Himalayan glaciers by 2035, fueled doubts about the authority, honesty and rigor of the IPCC as a leading institution in climate science and, correspondingly, raised questions about whether global warming is anything more than a hoax put forward by environmentalists. The late and confusing reaction of the IPCC to these allegations only worsened the matter. By comparing assessment reports issued by (...) the IPCC, this paper asks the question: ‘Why, despite extensivepeerreview, did the Himalayan glacier melting rate error get published?’ I chronicle exactly what happened, consider why it happened as it did, and show how these answers require us to rethink the concept of ‘peerreview’ in scientific practice. I also identify several future directions forpeerreview if it wants to stay ahead of the game, bearing in mind the forthcoming IPCC Assessment Reports. (shrink)

Direct download(2 more)   Export citation   Bookmark  

Peerreview is an opportunity to perform an unlawful censorship which ensures that no apostate notion ever get published in mainstream journals. Or suchpeerreview censorship is an opportunity to steal any content and to claim afterward the priority of first publication. And last but not least, thepeerreview is an academic tool to promote the mainstream pseudoscience.

Direct download(2 more)   Export citation   Bookmark   3 citations  

Supporting young researchers topeer-review and edit journal submissions will also accelerate their training, says Quan-Hoang Vuong. -/- *Citation: -/- Quan-Hoang Vuong. (2022, Sept. 11). Early career researchers can help fixpeerreview delays. Times Higher Education.

Direct download   Export citation   Bookmark   2 citations  

Peerreview is a widely accepted instrument for raising the quality of science.Peerreview limits the enormous unstructured influx of information and the sheer amount of dubious data, which in its absence would plunge science into chaos. In particular,peerreview offers the benefit of eliminating papers that suffer from poor craftsmanship or methodological shortcomings, especially in the experimental sciences. However, we believe thatpeerreview is not always appropriate for the (...) evaluation of controversial hypothetical science. We argue that the process ofpeerreview can be prone to bias towards ideas that affirm the prior convictions of reviewers and against innovation and radical new ideas. Innovative hypotheses are thus highly vulnerable to being “filtered out” or made to accord with conventional wisdom by thepeerreview process. Consequently, having introducedpeerreview, the Elsevier journal Medical Hypotheses may be unable to continue its tradition as a radical journal allowing discussion of improbable or unconventional ideas. Hence we conclude by asking the publisher to consider re-introducing the system of editorialreview to Medical Hypotheses. (shrink)

Direct download(4 more)   Export citation   Bookmark   3 citations  

In this chapter we reflect on the relevance ofpeerreview for assessing the quality of clinical ethics consultation. We contend thatpeerreview in the narrative form as presented in this book provides an alternative to the formal clinical ethics consultationreview procedures typically found in the clinical ethics literature. We elaborate onpeerreview as a reflection on clinical ethics consultation practice, the elements which a story should contain in order to (...) provide a basis forpeerreview, and the differences between the assessments of thepeer reviewers. We argue that a narrative approach to assessing the quality of ethics consultation can be further developed by allowing all stakeholders who are involved in the clinical ethics practice to actively take part in the evaluation process, following a “responsive evaluation” approach. An example of this is creating a Community of Practice, the aim of which is to foster a joint learning process of all parties involved. At the end of this chapter, we draw conclusions onpeerreview as a dialogical tool for evaluating quality of clinical ethics consultation. (shrink)

No categories

Direct download   Export citation   Bookmark  

Peerreview is often taken to be the main form of quality control on academic research. Usually journals carry this out. However, parts of maths and physics appear to have a parallel, crowd-sourced model ofpeerreview, where papers are posted on the arXiv to be publicly discussed. In this paper we argue that crowd-sourcedpeerreview is likely to do better than journal-solicitedpeerreview at sorting papers by quality. Our argument (...) rests on two key claims. First, crowd-sourcedpeerreview will lead on average to more reviewers per paper than journal-solicitedpeerreview. Second, due to the wisdom of the crowds, more reviewers will tend to make better judgments than fewer. We make the second claim precise by looking at the Condorcet Jury Theorem as well as two related jury theorems developed specifically to apply topeerreview. (shrink)

Direct download(6 more)   Export citation   Bookmark   4 citations  

A relatively high incidence of unsatisfactoryreview decisions is widely recognised and acknowledged as ‘thepeerreview problem’. Factors contributing to this problem are identified and examined. Specific examples of unreasonable rejection are considered. It is concluded that weaknesses of the ‘peerreview’ system are significant and that they are well known or readily recognisable but that necessary counter-measures are not always enforced. Careful management is necessary to discount hollow opinion or error inreview (...) comment.Review and referee functions should be quite separate. (shrink)

Direct download(3 more)   Export citation   Bookmark   5 citations  

Recent science funding policy scholars and practitioners have advocated for the use of lotteries, or elements of random chance, as supplementations of traditionalpeerreview for evaluating grant applications. One of the primary motivations for lotteries is their purported openness to innovative research. The purpose of this paper is to argue that current proponents of funding science by lottery overestimate the viability ofpeerreview and thus unduly restrict the scope of lotteries in science funding practice. (...) I further show how this analysis suggests a different way of introducing lotteries into science funding policy. (shrink)

Direct download(2 more)   Export citation   Bookmark   1 citation  

Peerreview of journal submissions has become one of the most important pillars of quality management in academic publishing. Because of growing concerns with the quality and effectiveness of the system, a host of enthusiastic innovators has proposed and experimented with new procedures and technologies. However, little is known about whether these innovations manage to convince other journal editors. This paper will address open questions regarding the implementation of newreview procedures, the occurrence rate of various (...) class='Hi'>peerreview procedures and their distribution over scientific disciplines or academic publishers, as well as the motivations for editors or publishers to engage in novelreview procedures. It shows that in spite of enthusiastic innovation, the adoption of newpeerreview procedures is in fact very slow, with the exception of text similarity scanners. For now,peerreview innovations appear to be restricted to specific niches in academic publishing. Analysing these niches, the article concludes with a reflection on the circumstances in which innovations might be more widely implemented. (shrink)

No categories

Direct download(2 more)   Export citation   Bookmark   1 citation  

Blindreview is ubiquitous in contemporary science, but there is no consensus among stakeholders and researchers about when or how much or why blindreview should be done. In this essay, we explain why blinding enhances the impartiality and credibility of science while also defending a norm according to which blindreview is a baseline presumption in scientificpeerreview.

No categories

Direct download(2 more)   Export citation   Bookmark   1 citation  

Direct download(3 more)   Export citation   Bookmark  

This Open Access book about the Zadeh Project demonstrates and explores a core question in clinical ethics: how can ethics consultants be accountable in the face of a robust plurality of ethical standpoints, especially those that underwrite practices and methods for doing ethics consultation as well as those viewpoints and values encountered in daily clinical ethics practice? Underscoring this question is the recognition that the field of clinical ethics consultation has arrived at a crucial point in its maturation. Many efforts (...) are underway to more formally “professionalize” the field, with most aimed toward stabilizing a specific set of institutional considerations. Stretched between these institutional and practical initiatives resides a crucial set of of ethical considerations, chief among them the meaning and scope of responsibility for clinical ethics consultants. Developed around a long-form case scenario, the Zadeh Project provides a multi-layered series of “peer-reviews”: critique of the actions of the case scenario’s ethics consultant; reflection on clinical ethics method; examination of the many ways that commitments to method and practice can, and do, intersect, overlap, and alter one another. The design and format of this book thus models a key element for clinical ethics practice: the need and ability to provide careful and thoughtful explanation of core moral considerations that emerge among diverse standpoints. Specifically designed for those studying to become and those who are ethics consultants, this book, with its innovative and multi-layered approach, allows readers to share apeer-review-like experience that shows accountability to be what it is, an ethical, not merely procedural or administrative, undertaking. (shrink)

No categories

Direct download(2 more)   Export citation   Bookmark   3 citations  

Direct download(3 more)   Export citation   Bookmark  

A relatively high incidence of unsatisfactoryreview decisions is widely recognised and acknowledged as ‘thepeerreview problem’. Factors contributing to this problem are identified and examined. Specific examples of unreasonable rejection are considered. It is concluded that weaknesses of the ‘peerreview’ system are significant and that they are well known or readily recognisable but that necessary counter-measures are not always enforced. Careful management is necessary to discount hollow opinion or error inreview (...) comment.Review and referee functions should be quite separate. (shrink)

Direct download(3 more)   Export citation   Bookmark   5 citations  

While much literature has sprouted onpeerreview, this is the first book-length, wide-ranging study that utilizes methods and resources of contemporary philosophy. It covers the tension betweenpeerreview and the liberal notion that truth emerges when ideas proliferate in the marketplace of ideas; arguments for and against blindreview of submissions; the alleged conservatism ofpeerreview; the anomalous nature of book reviewing; the status of non-peer-reviewed publications; and the future (...) ofpeerreview. (shrink)

Direct download   Export citation   Bookmark   14 citations  

Peerreview is widely recognized as a mechanism for quality control of academic content. This research article aims at comparing thereview reports and decisions of reviewers who are members of the editorial board of the European Scientific Journal with those reviewers suggested by the authors and who are not affiliated with the journal. 457review reports on 378 papers submitted to the ESJ in the period of October–December 2017 were analysed. Statistical methods including OLS and (...) Wilcoxon rank-sum test were applied based on the score approach toward the reviewers’ assessments of the papers and their characteristics related to the country, gender, and time of revisions. Results show the difference between the decisions these two groups of reviewers made. Even though editor-suggested and author-suggested reviewers need equal time toreview a paper, the former are less favourable towards the authors of the papers. It is also concluded that factors such as time and country of the reviewers influence their decisions. In this regard, the editors should avoid relying their decisions solely onreview reports received from reviewers suggested by the authors. However, further research with larger sample sizes should be conducted. (shrink)

Direct download(2 more)   Export citation   Bookmark  

Mission-oriented climate change research is often unverifiable. Therefore, many stakeholders look topeer-reviewed climate change research for trustworthy information about deeply uncertain and impactful phenomena. This is becausepeer-review signals that research has been vetted for scientific standards like reproducibility and replicability. Here we evaluate the transparency of research methodologies in mission-oriented computational climate research. We find that only five percent of our sample meets the minimal standard of fully open data and code required for reproducibility and (...) replicability. The widespread lack of minimal openness standards inpeer-reviewed climate change research jeopardizes scientific credibility and risks poor societal outcomes as we manage growing climate risks. The scientific community can and must do better. We, the authors, outline necessary openness standards that we commit to as researchers, reviewers, and readers. We invite you to join us. (shrink)

Direct download   Export citation   Bookmark  

Direct download(2 more)   Export citation   Bookmark  

Export citation   Bookmark   1 citation  

Direct download(3 more)   Export citation   Bookmark   1 citation  

This article argues that some reviewers contribute more to research than many authors, and suggests that reviewers meet the ICMJE criteria for authorship in many cases.

No categories

Export citation   Bookmark   1 citation  

This article examines the evolution ofpeerreview and the modern editorial processes of scholarly journals by analyzing a novel data set derived from the Royal Society’s archives and covering 1865-1965, that is, the historical period in which refereeing became firmly established. Our analysis reveals how the Royal Society’s editorial processes coped with both an increasing reliance on refereeing and a growth in submissions, while maintaining collective responsibility and minimizing research waste. By engaging more of its fellows in (...) editorial activity, the society was able to establish an equilibrium of number of submissions per reviewer that was relatively stable over time. Nevertheless, our analysis shows that the distribution of editorial work was significantly uneven. Our findings reveal interesting parallels with current concerns about the scale and distribution ofpeerreview work and suggest the strategic importance of the management of the editorial process to achieve a creative mix of community commitment and professional responsibility that is essential in contemporary journals. (shrink)

No categories

Direct download   Export citation   Bookmark   4 citations  

As philosophers we should have as one of our aims to produce as much philosophical knowledge as possible. A lot of potential philosophical knowledge is lost because of the flaws of thepeerreview system, and so a lot of philosophical knowledge would be gained were the system improved. Accordingly, as authors we should write papers about how to fixpeerreview, and as editors we should accept such papers if they are good. This paper presents (...) some familiar problems withpeerreview, elaborates on and motivates the argument just given, and replies to some objections to it, making the case that fixingpeerreview is both a philosophical problem and one that admits of a solution. (shrink)

No categories

Direct download(2 more)   Export citation   Bookmark   3 citations  

The major challenge facing today’s biomedical researchers is the increasing competition for available funds. The competitivereview process, through which the National Institutes of Health (NIH) awards grants, is built uponreview by a committee of expert scientists. The NIH is firmly committed to ensuring that itspeerreview system is fair and objective.

Direct download(3 more)   Export citation   Bookmark   5 citations  

The double‐blindpeer‐review of manuscripts for potential publication is a longstanding tradition in the production of scholarship. Nursing has adopted this tradition to secure a place of legitimacy and authority for its scholarship amongst the other disciplines in the academy. However, despite its ubiquity and avowed utility, thepeer‐review has not generally been the subject of much research let alone intense philosophical scrutiny and debate. This manuscript attempts such an engagement with a view to uncovering specific (...) concerns about the essentially conservative and sometimes restrictive effects the double‐blindpeer‐review produces. Drawing on the deconstructionist writings of Derrida and his acolytes this paper attempts to dig beneath the surface mechanics of the double‐blindpeer‐review and in so doing, expose its rather shaky philosophical foundations. It is written to open debate from others who too, have harboured doubts about its adequacy and supremacy as a technology in the production of (legitimate) knowledge. (shrink)

Direct download(2 more)   Export citation   Bookmark   1 citation  

Direct download   Export citation   Bookmark