Advertisement
| estimating the reproducibility of psychological science: Experimental Philosophy Joshua Knobe, Shaun Nichols, 2008 This volume provides an introduction to the major themes of work in experimental philosophy, bringing together some of the most influential articles in the field along with a collection of papers that explore the theoretical significance of this research. |
| estimating the reproducibility of psychological science: Reproducibility and Replicability in Science National Academies of Sciences, Engineering, and Medicine, Policy and Global Affairs, Committee on Science, Engineering, Medicine, and Public Policy, Board on Research Data and Information, Division on Engineering and Physical Sciences, Committee on Applied and Theoretical Statistics, Board on Mathematical Sciences and Analytics, Division on Earth and Life Studies, Nuclear and Radiation Studies Board, Division of Behavioral and Social Sciences and Education, Committee on National Statistics, Board on Behavioral, Cognitive, and Sensory Sciences, Committee on Reproducibility and Replicability in Science, 2019-10-20 One of the pathways by which the scientific community confirms the validity of a new scientific discovery is by repeating the research that produced it. When a scientific effort fails to independently confirm the computations or results of a previous study, some fear that it may be a symptom of a lack of rigor in science, while others argue that such an observed inconsistency can be an important precursor to new discovery. Concerns about reproducibility and replicability have been expressed in both scientific and popular media. As these concerns came to light, Congress requested that the National Academies of Sciences, Engineering, and Medicine conduct a study to assess the extent of issues related to reproducibility and replicability and to offer recommendations for improving rigor and transparency in scientific research. Reproducibility and Replicability in Science defines reproducibility and replicability and examines the factors that may lead to non-reproducibility and non-replicability in research. Unlike the typical expectation of reproducibility between two computations, expectations about replicability are more nuanced, and in some cases a lack of replicability can aid the process of scientific discovery. This report provides recommendations to researchers, academic institutions, journals, and funders on steps they can take to improve reproducibility and replicability in science. |
| estimating the reproducibility of psychological science: Towards Bayesian Model-Based Demography Jakub Bijak, 2021-12-09 This open access book presents a ground-breaking approach to developing micro-foundations for demography and migration studies. It offers a unique and novel methodology for creating empirically grounded agent-based models of international migration – one of the most uncertain population processes and a top-priority policy area. The book discusses in detail the process of building a simulation model of migration, based on a population of intelligent, cognitive agents, their networks and institutions, all interacting with one another. The proposed model-based approach integrates behavioural and social theory with formal modelling, by embedding the interdisciplinary modelling process within a wider inductive framework based on the Bayesian statistical reasoning. Principles of uncertainty quantification are used to devise innovative computer-based simulations, and to learn about modelling the simulated individuals and the way they make decisions. The identified knowledge gaps are subsequently filled with information from dedicated laboratory experiments on cognitive aspects of human decision-making under uncertainty. In this way, the models are built iteratively, from the bottom up, filling an important epistemological gap in migration studies, and social sciences more broadly. |
| estimating the reproducibility of psychological science: Psychological Science Under Scrutiny Scott O. Lilienfeld, Irwin D. Waldman, 2017-01-03 Psychological Science Under Scrutiny explores a range of contemporary challenges to the assumptions and methodologies of psychology, in order to encourage debate and ground the discipline in solid science. Discusses the pointed challenges posed by critics to the field of psychological research, which have given pause to psychological researchers across a broad spectrum of sub-fields Argues that those conducting psychological research need to fundamentally change the way they think about data and results, in order to ensure that psychology has a firm basis in empirical science Places the recent challenges discussed into a broad historical and conceptual perspective, and considers their implications for the future of psychological methodology and research Challenges discussed include confirmation bias, the effects of grant pressure, false-positive findings, overestimating the efficacy of medications, and high correlations in functional brain imaging Chapters are authored by internationally recognized experts in their fields, and are written with a minimum of specialized terminology to ensure accessibility to students and lay readers |
| estimating the reproducibility of psychological science: Replication Research in the Social Sciences James W. Neuliep, 1991-01-31 This book is essential reading for anyone conducting research in the social and behavioural sciences. No other volume offers researchers such concise and authoritative information on replication and its importance in the furtherance of knowledge. The 36 chapters include discussions on replication in behavioural research, editorial bias against replication research, and the replication process in social research. |
| estimating the reproducibility of psychological science: Introduction to Psychology Jennifer Walinga, Charles Stangor, This book is designed to help students organize their thinking about psychology at a conceptual level. The focus on behaviour and empiricism has produced a text that is better organized, has fewer chapters, and is somewhat shorter than many of the leading books. The beginning of each section includes learning objectives; throughout the body of each section are key terms in bold followed by their definitions in italics; key takeaways, and exercises and critical thinking activities end each section. |
| estimating the reproducibility of psychological science: Wavelets and Statistics Anestis Antoniadis, Georges Oppenheim, 2012-12-06 Despite its short history, wavelet theory has found applications in a remarkable diversity of disciplines: mathematics, physics, numerical analysis, signal processing, probability theory and statistics. The abundance of intriguing and useful features enjoyed by wavelet and wavelet packed transforms has led to their application to a wide range of statistical and signal processing problems. On November 16-18, 1994, a conference on Wavelets and Statistics was held at Villard de Lans, France, organized by the Institute IMAG-LMC, Grenoble, France. The meeting was the 15th in the series of the Rencontres Pranco-Belges des 8tatisticiens and was attended by 74 mathematicians from 12 different countries. Following tradition, both theoretical statistical results and practical contributions of this active field of statistical research were presented. The editors and the local organizers hope that this volume reflects the broad spectrum of the conference. as it includes 21 articles contributed by specialists in various areas in this field. The material compiled is fairly wide in scope and ranges from the development of new tools for non parametric curve estimation to applied problems, such as detection of transients in signal processing and image segmentation. The articles are arranged in alphabetical order by author rather than subject matter. However, to help the reader, a subjective classification of the articles is provided at the end of the book. Several articles of this volume are directly or indirectly concerned with several as pects of wavelet-based function estimation and signal denoising. |
| estimating the reproducibility of psychological science: Implementing Reproducible Research Victoria Stodden, Friedrich Leisch, Roger D. Peng, 2014-04-14 In computational science, reproducibility requires that researchers make code and data available to others so that the data can be analyzed in a similar manner as in the original publication. Code must be available to be distributed, data must be accessible in a readable format, and a platform must be available for widely distributing the data and code. In addition, both data and code need to be licensed permissively enough so that others can reproduce the work without a substantial legal burden. Implementing Reproducible Research covers many of the elements necessary for conducting and distributing reproducible research. It explains how to accurately reproduce a scientific result. Divided into three parts, the book discusses the tools, practices, and dissemination platforms for ensuring reproducibility in computational science. It describes: Computational tools, such as Sweave, knitr, VisTrails, Sumatra, CDE, and the Declaratron system Open source practices, good programming practices, trends in open science, and the role of cloud computing in reproducible research Software and methodological platforms, including open source software packages, RunMyCode platform, and open access journals Each part presents contributions from leaders who have developed software and other products that have advanced the field. Supplementary material is available at www.ImplementingRR.org. |
| estimating the reproducibility of psychological science: Toward a More Perfect Psychology Matthew C. Makel, Jonathan A. Plucker, 2017 This book presents strategies for not just maximizing the quality and impact of one's own research, but also evaluating and responding to the research of others. |
| estimating the reproducibility of psychological science: The Upside of Your Dark Side Todd B. Kashdan, Robert Biswas-Diener, 2015-09-01 Audible Best Seller of 2017 Inc. 11 Great Business Books New York Magazine Best Psychology Books LinkedIn's 12 Books on Leadership to Read Two mavericks in the field of positive psychology deliver a timely message Happiness experts have long told us to tune out our negative emotions and focus instead on mindfulness, positivity, and optimism. Researchers Todd Kashdan, Ph.D., and Robert Biswas-Diener, Dr. Philos., disagree. Positive emotions alone are not enough. Anger makes us creative, selfishness makes us brave, and guilt is a powerful motivator. The real key to success lies in emotional agility. Drawing upon extensive scientific research and a wide array of real-life examples, The Upside of Your Dark Side will be embraced by business leaders, parents, and everyone else who’s ready to put their entire psychological tool kit to work. |
| estimating the reproducibility of psychological science: The Cognitive Science of Religion D. Jason Slone, William W. McCorkle Jr., 2019-01-10 The Cognitive Science of Religion introduces students to key empirical studies conducted over the past 25 years in this new and rapidly expanding field. In these studies, cognitive scientists of religion have applied the theories, findings and research tools of the cognitive sciences to understanding religious thought, behaviour and social dynamics. Each chapter is written by a leading international scholar, and summarizes in non-technical language the original empirical study conducted by the scholar. No prior or statistical knowledge is presumed, and studies included range from the classic to the more recent and innovative cases. Students will learn about the theories that cognitive scientists have employed to explain recurrent features of religiosity across cultures and historical eras, how scholars have tested those theories, and what the results of those tests have revealed and suggest. Written to be accessible to undergraduates, this provides a much-needed survey of empirical studies in the cognitive science of religion. |
| estimating the reproducibility of psychological science: The Oxford Handbook of the Science of Science Communication Kathleen Hall Jamieson, Dan M. Kahan, Dietram Scheufele, 2017 On topics from genetic engineering and mad cow disease to vaccination and climate change, this Handbook draws on the insights of 57 leading science of science communication scholars who explore what social scientists know about how citizens come to understand and act on what is known by science. |
| estimating the reproducibility of psychological science: Reproducibility Harald Atmanspacher, Sabine Maasen, 2016-07-05 2017 PROSE Award Honorable Mention The PROSE Awards draw attention to pioneering works of research and for contributions to the conception, production, and design of landmark works in their fields. Featuring peer-reviewed contributions from noted experts in their fields of research, Reproducibility: Principles, Problems, Practices, and Prospects presents state-of-the-art approaches to reproducibility, the gold standard of sound science, from multi- and interdisciplinary perspectives. Including comprehensive coverage for implementing and reflecting the norm of reproducibility in various pertinent fields of research, the book focuses on how the reproducibility of results is applied, how it may be limited, and how such limitations can be understood or even controlled in the natural sciences, computational sciences, life sciences, social sciences, and studies of science and technology. The book presents many chapters devoted to a variety of methods and techniques, as well as their epistemic and ontological underpinnings, which have been developed to safeguard reproducible research and curtail deficits and failures. The book also investigates the political, historical, and social practices that underlie reproducible research in contemporary science studies, including the difficulties of good scientific practice and the ethos of reproducibility in modern innovation societies. Reproducibility: Principles, Problems, Practices, and Prospects is a guide for researchers who are interested in the general and overarching questions behind the concept of reproducibility; for active scientists who are confronted with practical reproducibility problems in their everyday work; and for economic stakeholders and political decision makers who need to better understand the challenges of reproducibility. In addition, the book is a useful in-depth primer for undergraduate and graduate-level courses in scientific methodology and basic issues in the philosophy and sociology of science from a modern perspective. “A comprehensive, insightful treatment of the reproducibility challenges facing science today and of ways in which the scientific community can address them.” Kathleen Hall Jamieson, Elizabeth Ware Packard Professor of Communication, University of Pennsylvania “How can we make sure that reproducible research remains a key imperative of scientific communication under increasing commercialization, media attention, and publication pressure? This handbook offers the first interdisciplinary and fundamental treatment of this important question.”Torsten Hothorn, Professor of Biostatistics, University of Zurich Harald Atmanspacher, PhD, is Associate Fellow and staff member at Collegium Helveticum, ETH and University Zurich and is also President of the Society for Mind-Matter Research. He has pioneered advances in complex dynamical systems research and in a number of topics concerned with the relation between the mental and physical. Sabine Maasen, PhD, is Professor for Sociology of Science and Director of the Munich Center for Technology in Society (TU Munich) and Associate Fellow at Collegium Helveticum (ETH and University Zurich). Her research focuses on the interface of science, technology, and society, notably with respect to neuroscience and its applications. |
| estimating the reproducibility of psychological science: Handbook of Meta-analysis in Ecology and Evolution Julia Koricheva, Jessica Gurevitch, Kerrie Mengersen, 2013-04-21 Meta-analysis is a powerful statistical methodology for synthesizing research evidence across independent studies. This is the first comprehensive handbook of meta-analysis written specifically for ecologists and evolutionary biologists, and it provides an invaluable introduction for beginners as well as an up-to-date guide for experienced meta-analysts. The chapters, written by renowned experts, walk readers through every step of meta-analysis, from problem formulation to the presentation of the results. The handbook identifies both the advantages of using meta-analysis for research synthesis and the potential pitfalls and limitations of meta-analysis (including when it should not be used). Different approaches to carrying out a meta-analysis are described, and include moment and least-square, maximum likelihood, and Bayesian approaches, all illustrated using worked examples based on real biological datasets. This one-of-a-kind resource is uniquely tailored to the biological sciences, and will provide an invaluable text for practitioners from graduate students and senior scientists to policymakers in conservation and environmental management. Walks you through every step of carrying out a meta-analysis in ecology and evolutionary biology, from problem formulation to result presentation Brings together experts from a broad range of fields Shows how to avoid, minimize, or resolve pitfalls such as missing data, publication bias, varying data quality, nonindependence of observations, and phylogenetic dependencies among species Helps you choose the right software Draws on numerous examples based on real biological datasets |
| estimating the reproducibility of psychological science: The Skeptical Professional’s Guide to Rational Prescribing Charles E. Dean, 2022-04-26 The raging COVID-19 pandemic has shaken our trust in science. This volume reviews the evolution of misconduct and fraud in science, the many steps taken to alleviate the problem, and the likelihood that it will continue, given our profit-driven healthcare system. Contents are set in a clinical context, wherein misconduct and fraud affect rational prescribing, a process that depends on balancing the risk–benefit ratio of treatments, whether pharmacologic or psychotherapeutic. The clinical consequences can be significant, in that the efficacy of treatments can be vastly overplayed, adverse effects minimized, and costs to the healthcare system increased if corrective measures are not taken. Key Features • Discusses the various aspects of cheating in publications: spin, protocol changes; failure to publish negative studies, including current data on the publishing industry and its issues, like the menace of predatory journals, poor peer review, coupled with lack of early education in ethics, and its significant impact on rational prescribing. • Assesses the impact of misconduct and fraud on clinicians and healthcare professionals as they attempt to balance the risk–benefit ratio which is supported by multiple contemporary studies. • Presents shocking data on bribes to physicians, journal editors and other key opinion leaders, exposing the ultimate root of the problem which lies in the economics of the healthcare system, badly in need of repair. |
| estimating the reproducibility of psychological science: Computational Social Psychology Robin R. Vallacher, Stephen J. Read, Andrzej Nowak, 2017-05-25 Computational Social Psychology showcases a new approach to social psychology that enables theorists and researchers to specify social psychological processes in terms of formal rules that can be implemented and tested using the power of high speed computing technology and sophisticated software. This approach allows for previously infeasible investigations of the multi-dimensional nature of human experience as it unfolds in accordance with different temporal patterns on different timescales. In effect, the computational approach represents a rediscovery of the themes and ambitions that launched the field over a century ago. The book brings together social psychologists with varying topical interests who are taking the lead in this redirection of the field. Many present formal models that are implemented in computer simulations to test basic assumptions and investigate the emergence of higher-order properties; others develop models to fit the real-time evolution of people’s inner states, overt behavior, and social interactions. Collectively, the contributions illustrate how the methods and tools of the computational approach can investigate, and transform, the diverse landscape of social psychology. |
| estimating the reproducibility of psychological science: Understanding Priming Effects in Social Psychology Daniel C. Molden, 2014-01-10 How incidentally activated social representations affect subsequent thoughts and behaviors has long interested social psychologists. Recently, such priming effects have provoked debate and skepticism. Originally a special issue ofSocial Cognition, this book examines the theoretical challenges researchers must overcome to further advance priming studies and considers how these challenges can be met. The volume aims to reduce the confusion surrounding current discussions by more thoroughly considering the many phenomena in social psychology that the term ?priming? encompasses, and closely examining the psychological processes that explain when and how different types of priming effects occur. |
| estimating the reproducibility of psychological science: Reproducibility Edo D. Pellizzari, Kathleen N. Lohr, Alan Blatecky, Darryl V. Creel, 2017 Science is allegedly in the midst of a reproducibility crisis, but questions of reproducibility and related principles date back nearly 80 years. Numerous controversies have arisen, especially since 2010, in a wide array of disciplines that stem from the failure to reproduce studies or their findings:biology, biomedical and preclinical research, business and organizational studies, computational sciences, drug discovery, economics, education, epidemiology and statistics, genetics, immunology, policy research, political science, psychology, and sociology. This monograph defines terms and constructs related to reproducible research, weighs key considerations and challenges in reproducing or replicating studies, and discusses transparency in publications that can support reproducible research goals. It attempts to clarify reproducible research, with its attendant (and confusing or even conflicting) lexicon and aims to provide useful background, definitions, and practical guidance for all readers. Among its conclusions: First, researchers must become better educated about these issues, particularly the differences between the concepts and terms. The main benefit is being able to communicate clearly within their own fields and, more importantly, across multiple disciplines. In addition, scientists need to embrace these concepts as part of their responsibilities as good stewards of research funding and as providers of credible information for policy decision making across many areas of public concern. Finally, although focusing on transparency and documentation is essential, ultimately the goal is achieving the most rigorous, high-quality science possible given limitations on time, funding, or other resources. “The authors have written a nuanced and thoughtful primer on scientific reproducibility. By highlighting the social, political, and technical importance of reproducibility, together with a precise description of the related concepts of reproducibility, replicability, and repeatability, this primer provides a significant resource that all practicing researchers should read.” Daniel Reed, Vice President for Research and Economic Development, University of Iowa and former Corporate Vice President, Microsoft “This is a well-written, clearly articulated, and timely primer on the developing and evolving rich terminology of reproducible research. The primer, put together by authors with deep experience and expertise in the topic area, focuses primarily on human-centric research in biomedicine, medicine, and the social sciences as well as reproducibility issues in analytics and computational science. The growing focus on reproducibility will open new vistas in research methodologies, meta analysis, comparative studies of research results, and reuse and adaptation of results from prior research. This primer provides an excellent overview of the subject area, and I would recommend it to anyone interested in coming up to speed on current issues in reproducible research.” Chaitan Baru, Distinguished Scientist and Associate Director for Data Initiatives, San Diego Supercomputing Center; current appointment as Senior Advisor for Data Science, Computer and Information Science and Engineering Directorate, National Science Foundation “Pellizzari et al. have taken on the Herculean task of collecting, synthesizing, and relating the various interpretations of reproducibility used in the research community today, and turned the result into an accessible must-read guide. This important work provides a Rosetta Stone for various stakeholders to discuss and implement solutions that make real progress toward a research enterprise that routinely produces reproducible findings.” Victoria Stodden, Associate Professor at the School of Information Sciences, University of Illinois at Urbana Champaign and co-editor of the books Implementing Reproducible Research and Privacy, Big Data, and the Public Good: Frameworks for Engagement |
| estimating the reproducibility of psychological science: Before You Know It John Bargh, 2017-10-17 The world's leading expert on the unconscious mind reveals the hidden mental processes that secretly govern every aspect of our behavior. For more than three decades, Dr. John Bargh has been conducting revolutionary research into the unconscious mind--not Freud's dark, malevolent unconscious but the new unconscious, a helpful and powerful part of the mind that we can access and understand through experimental science. Now Dr. Bargh presents an engaging and enlightening tour of the influential psychological forces that are at work as we go about our daily lives--checking a dating app, holding a cup of hot coffee, or getting a flu shot. Dr. Bargh takes you into his labs at New York University and Yale where his ingenious experiments have shown how the unconscious guides our actions, goals and motivations in areas like race relations, parenting, business, consumer behavior, and addiction. He reveals the pervasive influence of the unconscious mind on who we choose to date or vote for, what we buy, where we live, how we perform on tests and in job interviews, and much more. Before You Know It is full of surprising and entertaining revelations as well as tricks to help you remember to-do items, shop smarter, and sleep better. Before You Know It will profoundly change the way you understand yourself by introducing you to a fascinating world only recently discovered, the world that exists below the surface of your awareness and yet is the key to unlocking new ways of thinking, feeling and behaving.--Jacket. |
| estimating the reproducibility of psychological science: Blueprint, with a new afterword Robert Plomin, 2019-07-02 A top behavioral geneticist makes the case that DNA inherited from our parents at the moment of conception can predict our psychological strengths and weaknesses. In Blueprint, behavioral geneticist Robert Plomin describes how the DNA revolution has made DNA personal by giving us the power to predict our psychological strengths and weaknesses from birth. A century of genetic research shows that DNA differences inherited from our parents are the consistent lifelong sources of our psychological individuality—the blueprint that makes us who we are. Plomin reports that genetics explains more about the psychological differences among people than all other factors combined. Nature, not nurture, is what makes us who we are. Plomin explores the implications of these findings, drawing some provocative conclusions—among them that parenting styles don't really affect children's outcomes once genetics is taken into effect. This book offers readers a unique insider's view of the exciting synergies that came from combining genetics and psychology. The paperback edition has a new afterword by the author. |
| estimating the reproducibility of psychological science: Critical Thinking Jonathan C. Smith, 2017-07-26 Comprehensive and engaging, this extensively revised edition of a student and instructor favorite introduces the basics of critical thinking using the claims of pseudoscience and the paranormal. Guides readers through the critical thinking process by considering different types of support (sources, logic, and scientific observation) and ruling out alternative explanations Allows students to practice and apply their new critical thinking skills on claims of extraordinary cures including energy treatments, complementary/alternative medicine and faith healing as well as four paranormal claims of consequence: astrology, spiritualism and the afterlife, parapsychology, and creationism. Couples a conversational, nontechnical narrative with student-friendly pedagogical tools, including critical thinking questions and a study guide for each chapter. Provides clear and open-minded discussions of the paranormal spectrum, belief justification surveys, the placebo effect, and the relationship between religion and critical thinking |
| estimating the reproducibility of psychological science: The Problem with Science R. Barker Bausell, 2021-01-26 Recent events have vividly underscored the societal importance of science, yet the majority of the public are unaware that a large proportion of published scientific results are simply wrong. The Problem with Science is an exploration of the manifestations and causes of this scientific crisis, accompanied by a description of the very promising corrective initiatives largely developed over the past decade to stem the spate of irreproducible results that have come to characterize many of our sciences. More importantly, Dr. R. Barker Bausell has designed it to provide guidance to practicing and aspiring scientists regarding how (a) to change the way in which science has come to be both conducted and reported in order to avoid producing false positive, irreproducible results in their own work and (b) to change those institutional practices (primarily but not exclusively involving the traditional journal publishing process and the academic reward system) that have unwittingly contributed to the present crisis. There is a need for change in the scientific culture itself. A culture which prioritizes conducting research correctly in order to get things right rather than simply getting it published. |
| estimating the reproducibility of psychological science: Advances in the Sociology of Trust and Cooperation Vincent Buskens, Rense Corten, Chris Snijders, 2020-10-26 The problem of cooperation is one of the core issues in sociology and social science more in general. The key question is how humans, groups, organizations, institutions, and countries can avoid or overcome the collective good dilemmas that could lead to a Hobbesian war of all against all. The chapters in this book provide state of the art examples of research on this crucial topic. These include theoretical, laboratory, and field studies on trust and cooperation, thereby approaching the issue in three complementary and synergetic ways. The theoretical work covers articles on trust and control, reputation formation, and paradigmatic articles on the benefits and caveats of abstracting reality into models. The laboratory studies test the implications of different models of trust and reputation, such as the effects of social and institutional embeddedness and the potentially emerging inequalities this may cause. The field studies test these implications in applied settings such as business purchasing and supply, informal care, and different kinds of collaboration networks. This book is exemplary for rigorous social science. The focus is on effects of social conditions, in particular different forms of social and institutional embeddedness, on social outcomes at the macro level. Modelling efforts are applied to connect social conditions to social outcomes through micro-level behavior in ways that are easily overlooked when argumentation is intuitive and impressionistic. The book sets forth a mixed-method approach by applying different empirical methods to test hypotheses about similar questions. Several contributions re-evaluate the theoretical strengths and weaknesses following from the laboratory and field studies. Improving the theory in light of these findings facilitates pushing the boundaries of social science . |
| estimating the reproducibility of psychological science: Avoiding Questionable Research Practices in Applied Psychology William O'Donohue, Akihiko Masuda, Scott Lilienfeld, 2022-08-23 This authoritative volume presents a detailed analysis of the replication crisis and the use of questionable research practices (QRPs) in psychology, as well as recommended practices for combatting these problems. Ultimately, the book aims to provide a comprehensive, current, and accessible account of the adverse effects of QRPs. The replication crisis in psychology and allied fields has exposed critical flaws in the standard views of research methods, which allow for extensive flexibility in data analysis by investigators and permit the widespread use of QRPs. Chapters examine the intentional use of QRPs such as data fabrication and falsification, along with subtler, unintentional practices such as p-hacking and HARKING (hypothesizing after results are known). Drawing on the growing awareness of these problems, contributors also highlight potential strategies to detect QRPs and minimize their negative impact through open data practices, preregistration of hypotheses and analyses, and adversarial collaborations, in which investigators holding opposing positions on a scientific issue agree to work together on a study in an effort to counteract their respective biases. Among the topics covered: History of controversies in statistics and replication Embracing intellectual humility while designing research Confirmatory vs. exploratory analyses Publication bias and negative results Promoting honest and transparent report writing Avoiding Questionable Research Practices in Applied Psychology provides a deeper understanding of how QRPs impede the reliability and trustworthiness of findings in psychology and the social sciences. It will be a practical, useful resource for students and instructors in graduate and advanced undergraduate level research methods classes, along with psychological researchers interested in improving their own research. |
| estimating the reproducibility of psychological science: Applied Power Analysis for the Behavioral Sciences Christopher L. Aberson, 2019-01-24 Applied Power Analysis for the Behavioral Sciences is a practical how-to guide to conducting statistical power analyses for psychology and related fields. The book provides a guide to conducting analyses that is appropriate for researchers and students, including those with limited quantitative backgrounds. With practical use in mind, the text provides detailed coverage of topics such as how to estimate expected effect sizes and power analyses for complex designs. The topical coverage of the text, an applied approach, in-depth coverage of popular statistical procedures, and a focus on conducting analyses using R make the text a unique contribution to the power literature. To facilitate application and usability, the text includes ready-to-use R code developed for the text. An accompanying R package called pwr2ppl (available at https://github.com/chrisaberson/pwr2ppl) provides tools for conducting power analyses across each topic covered in the text. |
| estimating the reproducibility of psychological science: Psychological Science Under Scrutiny Scott O. Lilienfeld, Irwin D. Waldman, 2017-01-03 Psychological Science Under Scrutiny explores a range of contemporary challenges to the assumptions and methodologies of psychology, in order to encourage debate and ground the discipline in solid science. Discusses the pointed challenges posed by critics to the field of psychological research, which have given pause to psychological researchers across a broad spectrum of sub-fields Argues that those conducting psychological research need to fundamentally change the way they think about data and results, in order to ensure that psychology has a firm basis in empirical science Places the recent challenges discussed into a broad historical and conceptual perspective, and considers their implications for the future of psychological methodology and research Challenges discussed include confirmation bias, the effects of grant pressure, false-positive findings, overestimating the efficacy of medications, and high correlations in functional brain imaging Chapters are authored by internationally recognized experts in their fields, and are written with a minimum of specialized terminology to ensure accessibility to students and lay readers |
| estimating the reproducibility of psychological science: The Science of Weird Shit Chris French, 2024-03-19 An accessible and gratifying introduction to the world of paranormal beliefs and bizarre experiences. Ghostly encounters, alien abduction, reincarnation, talking to the dead, UFO sightings, inexplicable coincidences, out-of-body and near-death experiences. Are these legitimate phenomena? If not, then how should we go about understanding them? In this fascinating book, Chris French investigates paranormal claims to discover what lurks behind this “weird shit.” French provides authoritative evidence-based explanations for a wide range of superficially mysterious phenomena, and then goes further to draw out lessons with wider applications to many other aspects of modern society where critical thinking is urgently needed. Using academic, comprehensive, logical, and, at times, mathematical approaches, The Science of Weird Shit convincingly debunks ESP, communicating with the dead, and alien abduction claims, among other phenomena. All the while, however, French maintains that our belief in such phenomena is neither ridiculous nor trivial; if anything, such claims can tell us a great deal about the human mind if we pay them the attention they are due. Filled with light-bulb moments and a healthy dose of levity, The Science of Weird Shit is a clever, memorable, and gratifying read you won’t soon forget. |
| estimating the reproducibility of psychological science: Can We Be Wrong? The Problem of Textual Evidence in a Time of Data Andrew Piper, 2020-11-19 This Element tackles the problem of generalization with respect to text-based evidence in the field of literary studies. When working with texts, how can we move, reliably and credibly, from individual observations to more general beliefs about the world? The onset of computational methods has highlighted major shortcomings of traditional approaches to texts when it comes to working with small samples of evidence. This Element combines a machine learning-based approach to detect the prevalence and nature of generalization across tens of thousands of sentences from different disciplines alongside a robust discussion of potential solutions to the problem of the generalizability of textual evidence. It exemplifies the way mixed methods can be used in complementary fashion to develop nuanced, evidence-based arguments about complex disciplinary issues in a data-driven research environment. |
| estimating the reproducibility of psychological science: The Choice Factory Richard Shotton, 2018-02-12 Before you can influence decisions, you need to understand what drives them. In The Choice Factory, Richard Shotton sets out to help you learn. By observing a typical day of decision-making, from trivial food choices to significant work-place moves, he investigates how our behaviour is shaped by psychological shortcuts. With a clear focus on the marketing potential of knowing what makes us tick, Shotton has drawn on evidence from academia, real-life ad campaigns and his own original research. The Choice Factory is written in an entertaining and highly-accessible format, with 25 short chapters, each addressing a cognitive bias and outlining simple ways to apply it to your own marketing challenges. Supporting his discussion, Shotton adds insights from new interviews with some of the smartest thinkers in advertising, including Rory Sutherland, Lucy Jameson and Mark Earls. From priming to the pratfall effect, charm pricing to the curse of knowledge, the science of behavioural economics has never been easier to apply to marketing. The Choice Factory is the new advertising essential. |
| estimating the reproducibility of psychological science: Fostering Integrity in Research National Academies of Sciences, Engineering, and Medicine, Policy and Global Affairs, Committee on Science, Engineering, Medicine, and Public Policy, Committee on Responsible Science, 2018-01-13 The integrity of knowledge that emerges from research is based on individual and collective adherence to core values of objectivity, honesty, openness, fairness, accountability, and stewardship. Integrity in science means that the organizations in which research is conducted encourage those involved to exemplify these values in every step of the research process. Understanding the dynamics that support †or distort †practices that uphold the integrity of research by all participants ensures that the research enterprise advances knowledge. The 1992 report Responsible Science: Ensuring the Integrity of the Research Process evaluated issues related to scientific responsibility and the conduct of research. It provided a valuable service in describing and analyzing a very complicated set of issues, and has served as a crucial basis for thinking about research integrity for more than two decades. However, as experience has accumulated with various forms of research misconduct, detrimental research practices, and other forms of misconduct, as subsequent empirical research has revealed more about the nature of scientific misconduct, and because technological and social changes have altered the environment in which science is conducted, it is clear that the framework established more than two decades ago needs to be updated. Responsible Science served as a valuable benchmark to set the context for this most recent analysis and to help guide the committee's thought process. Fostering Integrity in Research identifies best practices in research and recommends practical options for discouraging and addressing research misconduct and detrimental research practices. |
| estimating the reproducibility of psychological science: Social Philosophy of Science for the Social Sciences Jaan Valsiner, 2019-12-11 This is an international and interdisciplinary volume that provides a new look at the general background of the social sciences from a philosophical perspective and provides directions for methodology. It seeks to overcome the limitations of the traditional treatises of a philosophy of science rooted in the physical sciences, as well as extend the coverage of basic science to intentional and socially normative features of the social sciences. The discussions included in this book are divided into four thematic sections: Social and cognitive roots for reflexivity upon the research process Philosophies of explanation in the social sciences Social normativity in social sciences Social processes in particular sciences Social Philosophy of Science for the Social Sciences will find an interested audience in students of the philosophy of science and social sciences. It is also relevant for researchers and students in the fields of psychology, sociology, economics, anthropology, education, and political science. |
| estimating the reproducibility of psychological science: A Decadal Survey of the Social and Behavioral Sciences National Academies of Sciences, Engineering, and Medicine, Division of Behavioral and Social Sciences and Education, Board on Behavioral, Cognitive, and Sensory Sciences, Committee on a Decadal Survey of Social and Behavioral Sciences for Applications to National Security, 2019-06-26 The primary function of the intelligence analyst is to make sense of information about the world, but the way analysts do that work will look profoundly different a decade from now. Technological changes will bring both new advances in conducting analysis and new risks related to technologically based activities and communications around the world. Because these changes are virtually inevitable, the Intelligence Community will need to make sustained collaboration with researchers in the social and behavioral sciences (SBS) a key priority if it is to adapt to these changes in the most productive ways. A Decadal Survey Of The Social and Behavioral Sciences provides guidance for a 10-year research agenda. This report identifies key opportunities in SBS research for strengthening intelligence analysis and offers ideas for integrating the knowledge and perspectives of researchers from these fields into the planning and design of efforts to support intelligence analysis. |
| estimating the reproducibility of psychological science: Handbook of Computational Social Science, Volume 1 Uwe Engel, Anabel Quan-Haase, Sunny Liu, Lars E Lyberg, 2021-11-10 The Handbook of Computational Social Science is a comprehensive reference source for scholars across multiple disciplines. It outlines key debates in the field, showcasing novel statistical modeling and machine learning methods, and draws from specific case studies to demonstrate the opportunities and challenges in CSS approaches. The Handbook is divided into two volumes written by outstanding, internationally renowned scholars in the field. This first volume focuses on the scope of computational social science, ethics, and case studies. It covers a range of key issues, including open science, formal modeling, and the social and behavioral sciences. This volume explores major debates, introduces digital trace data, reviews the changing survey landscape, and presents novel examples of computational social science research on sensing social interaction, social robots, bots, sentiment, manipulation, and extremism in social media. The volume not only makes major contributions to the consolidation of this growing research field but also encourages growth in new directions. With its broad coverage of perspectives (theoretical, methodological, computational), international scope, and interdisciplinary approach, this important resource is integral reading for advanced undergraduates, postgraduates, and researchers engaging with computational methods across the social sciences, as well as those within the scientifi c and engineering sectors. |
| estimating the reproducibility of psychological science: Principles and Methods of Social Research William D. Crano, Marilynn B. Brewer, Andrew Lac, 2023-07-11 Through a multi-methodology approach, Principles and Methods of Social Research, Fourth Edition covers the latest research techniques and designs and guides readers toward the design and conduct of social research from the ground up. Applauded for its comprehensive coverage, the breadth and depth of content of this new edition is unparalleled. Explained with updated applied examples useful to the social, behavioral, educational, and organizational sciences, the methods described are relevant to contemporary researchers. The underlying logic and mechanics of experimental, quasi-experimental, and non-experimental research strategies are discussed in detail. Introductory chapters cover topics such as validity and reliability furnish readers with a firm understanding of foundational concepts. The book has chapters dedicated to sampling, interviewing, questionnaire design, stimulus scaling, observational methods, content analysis, implicit measures, dyadic and group methods, and meta-analysis to cover these essential methodologies. Notable features include an emphasis on understanding the principles that govern the use of a method to facilitate the researcher’s choice of the best technique for a given situation; use of the laboratory experiment as a touchstone to describe and evaluate field experiments, correlational designs, quasi experiments, evaluation studies, and survey designs; and coverage of the ethics of social research including the power a researcher wields and tips on how to use it responsibly. The new edition features: Increased attention to the distinction between conceptual replication and exact replication and how each contributes to cumulative science. Updated research examples that clarify the operation of various research design operations. More learning tools including more explanation of the basic concepts, more research examples, and more tables and figures, such as additional illustrations to include internet content like social media. Extensive revisions and expansions of all chapters. A fuller discussion of the dangers of unethical treatment to research participants. Principles and Methods of Social Research, Fourth Edition is intended for graduate or advanced undergraduate courses in research methods in psychology, communication, sociology, education, public health, and marketing, and further appeals to researchers in various fields of social research, such as social psychology and communication. |
| estimating the reproducibility of psychological science: A Journey into Open Science and Research Transparency in Psychology Jon Grahe, 2021-08-30 A Journey into Open Science and Research Transparency in Psychology introduces the open science movement from psychology through a narrative that integrates song lyrics, national parks, and concerns about diversity, social justice, and sustainability. Along the way, readers receive practical guidance on how to plan and share their research, matching the ideals of scientific transparency. This book considers all the fundamental topics related to the open science movement, including: (a) causes of and responses to the Replication Crisis, (b) crowdsourcing and meta-science research, (c) preregistration, (d) statistical approaches, (e) questionable research practices, (f) research and publication ethics, (g) connections to career topics, (h) finding open science resources, (i) how open science initiatives promote diverse, just, and sustainable outcomes, and (j) the path moving forward. Each topic is introduced using terminology and language aimed at intermediate-level college students who have completed research methods courses. But the book invites all readers to reconsider their research approach and join the Scientific Revolution 2.0. Each chapter describes the associated content and includes exercises intended to help readers plan, conduct, and share their research. This short book is intended as a supplemental text for research methods courses or just a fun and informative exploration of the fundamental topics associated with the Replication Crisis in psychology and the resulting movement to increase scientific transparency in methods. |
| estimating the reproducibility of psychological science: The Oxford Handbook of Law, Regulation and Technology Roger Brownsword, Eloise Scotford, Karen Yeung, 2017-07-24 The variety, pace, and power of technological innovations that have emerged in the 21st Century have been breathtaking. These technological developments, which include advances in networked information and communications, biotechnology, neurotechnology, nanotechnology, robotics, and environmental engineering technology, have raised a number of vital and complex questions. Although these technologies have the potential to generate positive transformation and help address 'grand societal challenges', the novelty associated with technological innovation has also been accompanied by anxieties about their risks and destabilizing effects. Is there a potential harm to human health or the environment? What are the ethical implications? Do this innovations erode of antagonize values such as human dignity, privacy, democracy, or other norms underpinning existing bodies of law and regulation? These technological developments have therefore spawned a nascent but growing body of 'law and technology' scholarship, broadly concerned with exploring the legal, social and ethical dimensions of technological innovation. This handbook collates the many and varied strands of this scholarship, focusing broadly across a range of new and emerging technology and a vast array of social and policy sectors, through which leading scholars in the field interrogate the interfaces between law, emerging technology, and regulation. Structured in five parts, the handbook (I) establishes the collection of essays within existing scholarship concerned with law and technology as well as regulatory governance; (II) explores the relationship between technology development by focusing on core concepts and values which technological developments implicate; (III) studies the challenges for law in responding to the emergence of new technologies, examining how legal norms, doctrine and institutions have been shaped, challenged and destabilized by technology, and even how technologies have been shaped by legal regimes; (IV) provides a critical exploration of the implications of technological innovation, examining the ways in which technological innovation has generated challenges for regulators in the governance of technological development, and the implications of employing new technologies as an instrument of regulatory governance; (V) explores various interfaces between law, regulatory governance, and new technologies across a range of key social domains. |
| estimating the reproducibility of psychological science: The Psychology Research Handbook Frederick T. L. Leong, James T. Austin, 2023-11-16 In the Third Edition of The Psychology Research Handbook editors Frederick T. L. Leong and James T. Austin have assembled experienced expert researchers to provide graduate students and research assistants with a comprehensive framework for conducting many types of psychology research. The book is organized around the idea of a research script, following the step-by-step process of research planning, design, data collection, analysis, and disseminating research. Many chapters are coauthored by advanced graduate students to give their fellow students a sense of real-world research, adding to the clarity and practicality of many chapters. Students and instructors alike will appreciate chapters on topics typically missing from introductory methods texts, including applying for research grants, dealing with journal editors and reviewers, working within research teams, and conducting cross-cultural research. Structures such as recommended readings and exercises guide students to develop and expand their research skills. New chapters include Power and Evidence, IRB as Critical Collaborators in Research, Alternative Data Collection Strategies, Structural Equation Modeling and Replicability and Reproducibility. A comprehensive, easy-to-understand guide to the entire research process, this book quickly and efficiently equips advanced students and research assistants to conduct a full research project. |
| estimating the reproducibility of psychological science: Distrust , 2023-03-23 There is no doubt science is currently suffering from a credibility crisis. This thought-provoking book argues that, ironically, science's credibility is being undermined by tools created by scientists themselves. Scientific disinformation and damaging conspiracy theories are rife because of the internet that science created, the scientific demand for empirical evidence and statistical significance leads to data torturing and confirmation bias, and data mining is fuelled by the technological advances in Big Data and the development of ever-increasingly powerful computers. Using a wide range of entertaining examples, this fascinating book examines the impacts of society's growing distrust of science, and ultimately provides constructive suggestions for restoring the credibility of the scientific community. |
| estimating the reproducibility of psychological science: Data Literacy Neil Smalheiser, 2017-09-05 Data Literacy: How to Make Your Experiments Robust and Reproducible provides an overview of basic concepts and skills in handling data, which are common to diverse areas of science. Readers will get a good grasp of the steps involved in carrying out a scientific study and will understand some of the factors that make a study robust and reproducible.The book covers several major modules such as experimental design, data cleansing and preparation, statistical analysis, data management, and reporting. No specialized knowledge of statistics or computer programming is needed to fully understand the concepts presented. This book is a valuable source for biomedical and health sciences graduate students andresearchers, in general, who are interested in handling data to make their research reproducibleand more efficient. - Presents the content in an informal tone and with many examples taken from the daily routine at laboratories - Can be used for self-studying or as an optional book for more technical courses - Brings an interdisciplinary approach which may be applied across different areas of sciences |
| estimating the reproducibility of psychological science: Stevens' Handbook of Experimental Psychology and Cognitive Neuroscience, Methodology , 2018-02-12 V. Methodology: E. J. Wagenmakers (Volume Editor) Topics covered include methods and models in categorization; cultural consensus theory; network models for clinical psychology; response time modeling; analyzing neural time series data; models and methods for reinforcement learning; convergent methods of memory research; theories for discriminating signal from noise; bayesian cognitive modeling; mathematical modeling in cognition and cognitive neuroscience; the stop-signal paradigm; hypothesis testing and statistical inference; model comparison in psychology; fmri; neural recordings; open science; neural networks and neurocomputational modeling; serial versus parallel processing; methods in psychophysics. |
PSYCHOLOGY Estimating the reproducibilityof psychological …
Here, we evaluated reproducibility using significance and values, effect sizes, subjective assessments of replica-tion teams, and meta-analysis of effect sizes.
Reproducibility, Replicability, and Open Science - UF Research
Jun 29, 2021 · We conducted a large-scale, collab-Scientific statistically claims should not gain credence significant orative effort to obtain an initial estimate of because of the status or …
Response to Comment on “Estimating the reproducibility of …
We agree with them that both methodological differences between original and replication studies and statistical power affect reproducibility, but their very optimistic assessment is based on …
Estimating the reproducibility of psychological science
inspired to address the gap in direct empirical evidence about reproducibility. In this article, we report a large-scale, collaborative effort to obtain an initial estimate of the reproducibility of …
Estimating the reproducibility of psychological science
Reproducibility Project 1 Estimating the Reproducibility of Psychological Science Abstract Reproducibility is a defining feature of science, but the extent to which it characterizes science …
University of Dundee Estimating the reproducibility of …
dataset provides an initial estimate of the reproducibility of psychology and correlational data to support development of hypotheses about the causes of reproducibility. Sampling frame and …
On the Reproducibility of Psychological Science - Taylor
We exploit this fea-ture of the OSC study to describe a statistical model that pro-vides a quantitative explanation for the low reproducibility rates observed in the OSC article. We also …
Estimating the Reproducibility of Psychological Science …
Last Thursday, our article "Estimating the Reproducibility of Psychological Science" was published in Science. Coordinated by the Center for Open Science, we conducted 100 replications of …
Perspectives on Psychological Science An Open, Large-Scale ...
identify the factors that contribute to the reproducibility and validity of psychological science. Ultimately, such evidence— and steps toward resolution, if the evidence produces a call for …
Supplementary Material for - Science | AAAS
Two articles have been published on the methodology of the Reproducibility Project: Psychology. Open Science Collaboration, An open, large-scale, collaborative effort to estimate the …
Comment on: Estimating the reproducibility of psychological …
Estimating the reproducibility of psychological science. Science 349, 943 (2015), with the response from the Open Science Collaboration. Open Science Collaboration, Science 349, 943 …
Researchers overturn landmark study on the replicability of ...
In an attempt to determine the replicability of psychological science, a consortium of 270 scientists known as The Open Science Collaboration (OSC) tried to replicate the results of 100 published …
Commentary: Reproducibility in Psychological Science: When …
Iso-Ahola further claims that researchers have largely focused on isolated indicators—such as effect size and replicability—to determine whether psychological phenomena exist. Contrasting …
Estimating The Reproducibility Of Psychological Science (book)
Estimating The Reproducibility Of Psychological Science: Experimental Philosophy Joshua Knobe,Shaun Nichols,2008 This volume provides an introduction to the major themes of work …
Estimating the Prevalence of Transparency and …
reproducibility-related research practices were far from routine. These findings establish baseline prevalence estimates against which future progress toward increasing the credibility and utility …
Commenton Estimating the reproducibilityof psychological …
A paper from the Open Science Collaboration (Research Articles, 28 August 2015, aac4716) attempting to replicate 100 published studies suggests that the reproducibility of psychological …
Response to Comment on â Estimating the reproducibility of …
Project: Psychology indicates high reproducibility, given the study methodology.Their very optimistic assessment is limited by statistical misconceptions and by causal inferences from …
OUR RESPONSE TO OSCREPLY - Scholars at Harvard
reproducibility: GKPW focused on a variation of one of OSC2015’s five measures of reproducibility how often the confidence interval (CI) of the original study contains the effect size estimate of …
April 9 - Université de Genève
https://unige.zoom.us/j/97094882077?pwd=QlNHUzdwL3RKeDNwRU5MMVhQVEw1dz09 Theme Paper Week 1 March 5 The problem defined: the general overview Munafoet al. 2017.
Estimating The Reproducibility Of Psychological Science [PDF]
Estimating The Reproducibility Of Psychological Science: Towards Bayesian Model-Based Demography Jakub Bijak,2021-12-09 This open access book presents a ground breaking …
PSYCHOLOGY Estimating the reproducibilityof psycholog…
Here, we evaluated reproducibility using significance and values, effect sizes, subjective assessments of replica-tion teams, and meta-analysis of effect …
Reproducibility, Replicability, and Open Science - UF Res…
Jun 29, 2021 · We conducted a large-scale, collab-Scientific statistically claims should not gain credence significant orative effort to obtain an …
Response to Comment on “Estimating the reproducibi…
We agree with them that both methodological differences between original and replication studies and statistical power affect …
Estimating the reproducibility of psycholo…
inspired to address the gap in direct empirical evidence about reproducibility. In this article, we report a large-scale, collaborative effort to obtain an initial …
Estimating the reproducibility of psycholo…
Reproducibility Project 1 Estimating the Reproducibility of Psychological Science Abstract Reproducibility is a defining feature of science, but the extent to …
