Metascience Is More Important Now Than Ever https://undark.org/2025/07/31/opinion-metascience-essential/ #MetaScience This is most likely true, the more difficult question how the field should move forward in terms of culture, approaches, focus etc.
New paper alert! #statistics #metascience "On the poor statistical properties of the P-curve meta-analytic procedure" in JASA. https://raw.githubusercontent.com/richarddmorey/MoreyDavisStober_pcurveASA/refs/heads/main/text/asa_article/Morey_Davis-Stober_2025_JASA_with_supplement.pdf
We show that the "P curve" meta-analysis tests have terrible statistical properties, in spite of being used for over a decade to tell "bad" science from "good". The initial tests should never have made it through peer review. They suffer from extreme sensitivity, arbirary conclusions, inadmissibility, nonmonotonicity in the evidence, and inconsistency in estimation. We recommend they not be used, and that better vetting is needed for methods in metascience.
Journal link: https://www.tandfonline.com/doi/full/10.1080/01621459.2025.2544397
Can metascience help us understand science — or will it become just another broken mirror?
From open science practice, I see its value in questioning structures, making peripheral knowledge visible, and rethinking evaluation.
We need to read not just the books, but the system that produces them.
How can we use metascience for transformation from Latin America?
http://bit.ly/41pO4Np
Puede la metaciencia ayudarnos a entender la ciencia o será otro espejo roto?
Desde la práctica en ciencia abierta, veo su valor para cuestionar estructuras, visibilizar lo periférico y repensar la evaluación.
Necesitamos leer no solo los libros, sino el sistema que los produce.
¿Cómo usar la metaciencia para transformar desde América Latina? http://bit.ly/41pO4Np
I'm looking for someone who'd be interested in co-hosting a new metascience podcast I'm currently developing.
Also editing/producing roles if people would like to be involved. I'm looking for funding possibilities (if anyone knows any please also get in touch).
Happy to chat and share more for those interested. Ideally looking for someone active in metascience but not a strict requirement
Defending science in public we often talk about 'peer reviewed science'. But could this framing contribute to undermining trust in science and holding us back from improving the scientific process? How about instead we talk about the work that has received the most thorough and transparent scrutiny?
Peer review goes a step towards this in having a couple of people scrutinise the work, but there are limits on how thorough it can be and in most journals it's not transparent. Switching the framing to transparent scrutiny allows us to experiment with other models with a path to improvement.
For example, making review open to all, ongoing, and all reviews published improves this. When authors make their raw data and code open, it improves this.
It also gives us a way to criticise problematic organisations that formally do peer review but add little value (e.g. predatory journals). If their reviews are not open and observably of poor quality, then they are less 'thoroughly transparent'.
So with this framing the existence of 'peer reviewed' but clearly poor quality work doesn't undermine trust in science as a whole because we don't pin our meaning and value on an exploitable binary measure of 'peer reviewed'.
It also offers a hopeful way forward because it shows us how we can improve, and every step towards this becomes meaningful. If all we have is binary 'peer reviewed' or not, why spend more effort doing it better?
In summary, I think this new framing would be better for science, both in terms of the public perception of it, and for us as scientists.
“…the sight of a high-ranking university representative stepping in to save a big tech executive from answering a difficult question was deeply embarrassing (or at least should have been) for all concerned.”
And this is the point. #BigTech shielded by academics (a patchy cover at best, though a telling of the power of wealth and its financial control of scientific research). I this case it was about #Google’s AI products where its representative executive Anna Koivuniemi of Google #DeepMind was shielding that the high-ranking university representative and chair, Geraint Rees, ruled an ‘embarrassing’ question out of order.
It’s a short, but enlightening, read well worth your time if your interests include the use of #GenAI in scientific #research.
#ResearchIntegrity #GenAI&ScientificResearch #MetaScience
Read more: https://warrenpearce.pika.page/posts/what-is-metascience-issues-inclusion-and-future-public-value
In my own WiP report #VAR4LCR, I presented a mini #metascience study on "Register and task effects in the International Journal of Learner Corpus Research". The audience proposed extending the study to #SLA articles, register studies, and/or book (chapter) publications.
Opening the second day of the First Workshop on #Replication (#WoReLa1), we are treated to a keynote about "Research on Bilingualism as Discovery Science" by Anne Beatty-Martínez. Anne has lots of food for thought for us all. #linguistics #metascience
Okay, Ana Rita Sá-Leite's talk #WoReLa1 genuinely took us (the audience) on an emotional journey: the results of her work point to a meta-analysis highly influenced by a handful of studies from the same with very small sample sizes and very large effects, Type S error, interpreting absence of evidence as evidence of absence, and more... Find out more: https://doi.org/10.1016/j.cognition.2022.105060 #MetaScience #linguistics
Bernhard Angele is now presenting "Living meta-analyses in Language Sciences" at #WoReLa1. Having explaining the need for such living meta-analyses in a very houmous way, Bernhard demonstrated this very cool project. The main output is a #Shiny app that can perform Bayesian meta-analyses with lots of opportunities to control various parameters AND allows you to upload additional data to an existing meta-analysis. The app can be used as is or the code adapted to your needs: https://dallbrit.shinyapps.io/Breathing_Life_into_MetaAnalysis/. Also check out the associated paper: https://doi.org/10.5334/joc.389 #MetaScience
JOB : Join RoRI as a Research Fellow/Senior Research Fellow in Metascience
https://researchonresearch.org/join-rori-as-a-research-fellow-senior-research-fellow-in-metascience/
Closing 9th of July, based in London/hybrid. Full time, fixed term until 31/12/2027
(apologies for late notice, full disclosure: I am a @RoRInstitute research fellow)
--> please boost for reach <--
You are all warmly invited to our #metascience2025 virtual symposium on Friday 27 June 15:00–16:30 CEST on doing #metascience and #interdisciplinary research as an early-career researcher.
Convenor: Anna Leung (psycholinguist and language teacher; University Hospital, LMU, Germany)
Discussants:
- Shawn Hemelstrand (psychologist and methodologist; The Chinese University of Hong Kong)
- Daniel Kristanto (engineer turned neuroscientist; Carl von Ossietzky Universität Oldenburg)
- Elen Le Foll (corpus linguist and language teaching; University of Cologne)
Abstract: https://nomadit.co.uk/conference/metascience2025/p/16480.
Registration is free: https://cos-io.zoom.us/webinar/register/WN_B5N34T4_S8qwj7t_jhfwKA#/registration.
We look forward to discussing these important topics with you! #ECR #PhD #PostDoc #Academia #OpenScience
🄱🄾🄾🅂🅃🄸🄽🄶 🅃🄷🄸🅂 🅃🄾🄾🅃 = 1% 🄼🄾🅁🄴 🄴🄲🅁 🄴🄼🄿🄾🅆🄴🅁🄼🄴🄽🅃!
"As we report below, even well-meaning scientists provided with identical data and freed from pressures to distort results may not reliably converge in their findings because of the complexity and ambiguity inherent to the process of scientific analysis."
#metascience
The fruits of following my curiosity yesterday - a little Sunday Study on citations.
https://tlohde.com/blog/2025/06/becoming-irrelevant/
Motivated by @sundogplanets 's post on citing old papers: https://mastodon.social/@sundogplanets/114677909402696290
I haven't checked, because I wanted to do it anyway, but I'm sure someone out there has done this more thoroughly and properly.
Open scholarly metadata is actually such a huge mess, especially for monographs, edited volumes, grey lit, i.e. anything that isn't an english STEM journal article.
Which actually explains so much about why the "metascience" scene is how it is....
I and may amazing advisor @nicole_c_nelson and our brilliant collaborator Kelsey preprinted! 
"Adverserial" reanalyses of regulatory data pose a deep problem for existing methods of adjudicating which science our regulations should be based on
This is, unfortunately, a continually-relevant story about what #openscience and #opendata mean for how we make regulations and come to consensus as a society
I just saw that #Metascience 2025 has John Ioannides as a featured speaker and I have to say I‘m surprised by that choice
Should we allow copying / plagiarism in introductions and methods sections of scientific papers? If the context of the work (intro) or description of some techniques used (methods) are the same as previous work, why write it out again in new words? #science #metascience
3 Jahre #PostDoc-Stelle an der #LMU: "Agile-RDM: Einführung einer agilen und bedarfsorientierten Strategie für das #Forschungsdatenmanagement in der #Psychologie“.
https://job-portal.lmu.de/jobposting/555d1dc31ca53521cfa1afa72b95b42717d818cb0?ref=homepage
Das Projekt ist bei Mario Gollwitzer und mir angesiedelt und in enger Kooperation mit dem ZPID.
Kooperation mit dem META-REP Schwerpunktprogramm sowie dem LMU Open Science Center @lmu_osc
Wer sich für #MetaScience und #OpenScience interessiert, könnte hier eine interessante Perspektive haben!
#AcademicJobs
