“An experiment, like every other event which takes place, is a natural phenomenon; but in a scientific experiment the circumstances are so arranged that the relations between a particular set of phenomena may be studied to the best advantage.”
– James Clerk Maxwell, physicist, 1876
Note that we use the term “journal” because this is a publication for disseminating research findings, but it is unlike a typical scientific journal in several important ways.
Anyone can be a scientist in the sense that anyone can be systematic in the way that they approach a question. Non-institutional science is the practice of scientific inquiry by capable individuals who are not affiliated with formal institutions such as universities, government agencies, or corporations.
Douglass C North, a Nobel laureate in economics, defined institutions as “the rules of the game in a society, or, more formally, … the humanly devised constraints that shape human interaction,” (North 1991). Institutional science operates within such rules, which constrain which and how scientific questions can be pursued.
In contrast, non-institutional science allows for greater freedom, letting curiosity drive the questions and unconventional approaches shape the methods. (Cosimo is an organization, but it is not an institution, because it is our explicit goal to encourage thoughtful rule-breaking of institutional conventions.)
Non-institutional science is important because it allows for freer exploration of ideas without the bureaucratic constraints of formal institutions, which, while essential for maintaining high standards and producing reliable research, can also limit creative inquiry. Non-institutional science fills this gap by encouraging unconventional approaches and topics.
Both institutional and non-institutional science are necessary, as they serve different roles in advancing knowledge.
We publish experiment reports and topic investigations. Experiment reports include updates on ongoing experiments, demonstrations of the research process, and interpretations of final results. Topic investigations review academic literature, while also welcoming insights from thoughtful blog posts, forum discussions, interviews, cultural traditions, etc.
We do not use a typical peer-review process; rather, we directly seek and discuss feedback from commentators throughout the research planning and execution. These individuals can be anyone capable and knowledgeable in the topic, focusing on thoughtfulness of commentary rather than formal credentials. One of the ways that we source feedback is through ResearchHub, a scientific collaboration platform where we can communicate with scientists and incentivize reviews with the platform’s token.
We ensure quality and rigor through an iterative, public approach that encourages ongoing improvement, taking feedback from commenters. We invite input from across disciplines, and we value a willingness to explore interpretations over excessive hesitation.
Research results are often hard to evaluate. Just because something is typed up in LaTeX and published in a journal doesn’t necessarily mean it’s correct. Just because something is typed up in Comic Sans on a blog doesn’t necessarily mean it’s low quality work. So how can we actually evaluate what is good science besides using these proxies?
There are many, many ways to do science wrong. There are numerous methodological flaws that can seriously impact study results, sometimes to the point of rendering them meaningless.
But just because something doesn’t meet every requirement of gold standard science, doesn’t mean that it’s without value (especially when the best alternative is even less reliable).
Consider the following evidence investigating whether a particular herb helps with headaches:
The first option has the highest consistency, lowest bias, and highest reliability. It’s also the least feasible, and likely overkill to detect a large effect. On the other hand, as much as I may like my friend’s uncle, it’s hard to say if the herb works based only on his opinion.
In this case, despite its imperfections, the middle study is appropriate, useful, and better than an anecdote.
We can and should apply scientific truth-seeking methods to more unexplored questions, even in the absence of ideal conditions. But we need to understand what is good enough depending on the context.
So how do we decide what standards are appropriate for the situation? At Cosimo, we apply the Gold-Silver-Bronze framework.
If a question is risky to get wrong (e.g. evaluating cancer treatment), it’s best to apply the Gold standard. On the other hand, if it is a relatively low-risk question and there are few existing good studies, we only require the Bronze standard to improve our scientific understanding of the topic. We also consider the feasibility of detecting an effect with the chosen approach.
Truth matters because it empowers us to act effectively and creatively within the world, enabling us to shape what could be by understanding what is.
Our number one commitment is to integrity. This principle guides the way that we conduct each aspect of our work.
We primarily seek funding from individuals who are driven by curiosity and have no vested interest in specific outcomes. If a study is funded by an organization or company, they understand and acknowledge that results will be interpreted according to our best truth-seeking ability.
Additionally, we always aim to protect study participants and contributors. Everyone who contributes to our research in any way should clearly understand what their participation involves and be excited to be a part of it.
