Welcome back to our very occasional “Better Know a Ronin Scholar” series, where we learn more about the journeys of our Research Scholars and their current projects. It’s also doubling as the first post in a new “Fellow Travelers” series, conversations with the different organizations which share some common goals, approaches or philosophy with the Ronin Institute. In late 2018, I talked to the founder of one such traveler, Dr. Rebecca Willén, the founder of IGDORE, also a Research Scholar with Ronin. Rebecca has a PhD in psychology from the University of Gothenburg in Sweden in which her main focus was on deception detection and development of interrogation techniques. After finishing her PhD in 2016 she has since focused on metascience – science about science.
In this interview, we start by discussing the motivations for founding IGDORE, how education fits into IGDORE’s mission, and discuss what “New Academia” is all about. We next talk about the importance of retroactive disclosure statements for transparency in science. We finish on the topics of co-working spaces, changing academic cultures and the future. [This is a edited version of our conversation. Full disclosure: I am an affiliated researcher with IGDORE].
Tell me a bit about IGDORE?
IGDORE stands for the Institute for Globally Distributed Open Research and Education. It’s an independent institute I founded in 2016, directly after I had finished my PhD. It’s a virtual research institute, just like Ronin. We have one non-virtual facility and that is on Bali in Indonesia.
What motivated you to start IGDORE?
There were mainly two reasons why I started IGDORE.
First of all I finished my PhD remotely. I lived the last year of my PhD, I lived in Canada and in Indonesia. I was just working from co-working spaces, from home, from cafes. And that’s how I finished my PhD. I only went to Sweden for my actual defense. And when I started to look for jobs, just before I finished my PhD, and during the Spring of 2016, I found it difficult to find a place where I felt that, “I want to go there with my family and live there”.
I did a job interview in the UK, and when I and visited the university, I felt that, “is this really where I’m supposed to live with my family, we have no connection with this country or that town?”. It didn’t feel right, because we were quite happy with the life we had, and it is also relevant here, that the father of my daughter – we are not a couple any longer – so it makes it more difficult to expect someone to come to a new location when you are not even a couple. So for that reason, that was one of the reasons why I started IGDORE – because I wanted to do research from wherever I was.
“I didn’t want to end up… [experiencing]…the same pressure to publish and to get the low enough p-values to get something published. I wanted to be free to do good science, and to do it from wherever I wanted to reside at the moment”
And the second reason was that during the years that I was at my university as a student, I had lost my faith in science. Because there were a lot of questionable research practices being employed and I had struggled with that, and I struggled to change that from within, and failed. And I didn’t want to end up in a new such situation where I’m doing a postdoc or something at a new university and again experience same the pressure to publish and to get the low enough p-values to get something published. I wanted to be free to do good science, and to do it from wherever I wanted to reside at the moment.
How are the scientists at IGDORE funded? What’s the current funding model?
We have several of the people are affiliated with traditional universities so they have receive the salaries from there. Several of them are preparing, they want to become location-independent for different reasons and want to move away from traditional academia. But are right now standing with one foot in each. IGDORE is not offering any funding.
How is IGDORE set up legally?
In Indonesia we are non-for-profit. In Sweden, we are a limited company. The plan is to become a non-for-profit too. It’s easy to do, but we haven’t done it yet.
So some of the individual scientists at IGDORE have positions, and some of them are freelancers. How do you specifically get your funding, at the moment?
I get my money from seed money from IGDORE. I’m basically living on a minimum.
So Bali can be helpful for that, I imagine?
Bali came before I finished my PhD. No, but living in Bali, makes it possible for me to live on the money I have for a long-time.
Right, the cost of living is probably much lower than in Sweden?
IGDORE and online education
In the future for IGDORE, are you thinking something more like Ronin where it facilitates but doesn’t necessarily provide salaries. Or are you thinking that IGDORE would provide some of that centrally?
I haven’t given it too much thought yet. I want IGDORE to start offering online education, and if we get that going, then there might be the possibility to hire someone. To be able to pay proper salaries to a bunch of researchers. Because I think education will be the main thing that is needed if you want to be able to pay salaries to researchers.
And that’s the “E” in IGDORE?
Where do you see IGDORE evolving the future?
The long-term plan for IGDORE is to become a proper university. A location-independent university. I hope we will have co-working spaces, and laboratories in different parts of the world. And then if people are working for, or studying at IGDORE, then it doesn’t matter where in the world they are, they can be at home, they can choose to go to their nearest campus, co-working space or laboratory. But the long-term plan is to become a full non-formal university.
Right, but a more distributed one? It reminds me a little of Minerva.
They have the education part.
But not the research part.
Exactly, and we are the opposite. Their students are not location-independent. And that is something I want to do differently.
Ah, I didn’t know that, I assumed they were location independent.
The teachers are location-independent, but the students are not. The students are not, they are actually travelling around the world together as one class. They are 40 people, 20 people. Living together for several years, three years. I find Minerva to be very, very interesting. I’m definitely inspired by them. And their tuition fees are low compared to many tuition fees in the US, but for me as a Swede – in Sweden university is free for everyone. But for me – it’s still expensive [in the US]. For me it’s crazy, I don’t know understand how you manage in the US.
Do you spend most of your time in Bali or Sweden?
So, yeah, I’m spending very little time in Sweden nowadays. Since September I’ve been living in Amsterdam in the Netherlands. The plan now is to spend half the time in Bali and half the time in the Netherlands. This is the first time – it’s autumn here now – it’s almost winter here in Amsterdam now and it has been my first autumn in Europe for several years. I had longed for all the colours on the trees. And I really enjoyed that. But now there are no leaves on the trees, it’s grey and dark and cold and now I start to feel that now – ah – this is too long in Europe.
Reactions to IGDORE, finding Ronin, and “New Academia”
How have the reactions been from your colleagues to forming IGDORE?
I think we can broadly speak about three different categories of people there. One category are those people who have been very supportive from the very beginning. One of those is a colleague of mine, we were PhD students at the same time, same laboratory, and she was the first one to join IGDORE, aside from myself. The second category is people who like the idea, they are interested, but they don’t really dare to be openly supportive, because they are careful with their own careers. And the third category are people who basically they think what I am doing is crazy and consider me not belonging to academia any more, and basically have turned away a bit.
Of those three categories, what’s your sense of which are growing and which are shrinking? I think that will tell us a lot in the long run?
I think that in my experience, the third category is getting smaller. With more people joining and more publications including IGDORE’s name, et cetera. That is my experience so far.
Early days for sure. My sense is that the third category can only really shrink at this point. Academia is highly conservative – it’s a bit like the supertanker on the ocean, it takes days just to slow down, let alone turnaround. So inevitably it seems most likely that it will be smaller organizations that lead change.
“It’s much easier – although it’s difficult – it’s still much easier to start up something completely new from scratch, rather than trying to change traditional academia from within.”
That’s an important reason why I also felt I didn’t want to continue to struggle within traditional academia, because it takes so much to change something. And even if I managed to get my department to change one small thing, there would be so much more change needed. It’s much easier – although it’s difficult – it’s still much easier to start up something completely new from scratch, rather than trying to change traditional academia from within.
How did you come to join Ronin?
I found about Ronin when I read a piece in Nature [in 2017] about independent scholarship [see also Ronin blog post]. It was about Ronin and the British, UK one [CORES Sciences and Engineering]. And I had already started IGDORE when I came across this article. And I reached out to Jon [Wilkins] immediately. If I had known about Ronin from the beginning, I would have definitely joined Ronin first. And I think that ultimately I would have started IGDORE at some point anyway, but I think maybe after some years at Ronin. Because I still want to first of all, have the institutional responsibility for good scientific practices. And because I still see the need for a big university doing these things. So something more similar to traditional academia, but making use of all the new and good things. With a good and nice culture and atmosphere. Making it the right way.
I think that IGDORE and Ronin could work together more than we have so far. I think there is lots of potential there. I think an important challenge is to offering financial stability to researchers who work within the New Academia and I think developing educational programs will be a very important route to take. And that is something we could definitely do together. That would be a way to make it easier for people, just as it is within traditional academia, lecturing is the main way to get financial stability, not as a researcher. And if we could do that, we could both educate a new generation of researchers and we could offer work and income to researchers who want to be not in the traditional academia.
You mentioned the term “New Academia”. What is New Academia and how does it relate to independent scholarship?
“There is a problem with the term… “independent scientist”. Because who doesn’t want to be independent? … the term academia should be expanded to include us, and instead to distinguish between “new academia” and “traditional academia”.”
Yes. There is a problem with the term “independent researcher” or “independent scientist”. Because who doesn’t want to be independent? People don’t understand the term. The term refers to a different type of independence, it’s still a problematic term, a confusing term. But I’m also not agreeing with the view that I am not part of academia if I work at IGDORE or at Ronin. Because I’m doing exactly the same work. I’m still applying for the same funding, I’m still struggling with many of the same issues in academia – except for going away from the bad environment and “publish or perish”. So I thought that the term academia should be expanded to include us, and instead to distinguish between “new academia” and “traditional academia” (universities). Because I think what IGDORE and Ronin are is more modern and new – but still academia.
Retroactive disclosure statements and reproducible science
Switching gears a little to talk about your work in metascience. Can you explain the difference between exploratory versus confirmatory analyses? And how they relate to retroactive disclosure statements?
“Psychology is in the frontline of making changes towards more openness and transparency and towards increased scientific rigour. But medicine is very close. Biology is very close. Economics and political science is coming too. It’s spreading.”
There are exploratory or confirmatory analyses within a [single] paper. And right now it’s very common that you frame exploratory analyses as confirmatory. That’s how I was taught how to do science. We refined or even made up the hypothesis after we had the data and analysed the data. And what we can find that has the small enough p-value and that makes sense. And then we refined or made up the hypothesis to fit that. We presented a story, a narrative. It was supposed to be coherent. That was extremely problematic because that is not about truth-seeking, that is just about playing a game to publish. One important thing to do in moving in the right direction is to distinguish correctly between exploratory and confirmatory analysis. That distinction is crucial. If you do something confirmatory, you have specified it before the data collection started.
You state your hypothesis in advance, in other words?
Everything else is exploratory and should be transparently reported as such. There are lots of things happening in this area. We have a new article format – registered reports. Registered reports [164 journals have adopted the format across disciplines] – that’s the future I think. That will be a very common article format in the future. That means you send your methods, your planned data collection and everything to a journal for review. The reviewers look at it, tell you minor revision, major revision, whatever. Once your paper is accepted for publication in that journal, then you start the data collection. And then that means you have to stick to what you planned. The journal will publish it whatever you find.
It totally inverts the whole process?
Yes. And that also makes the review process meaningful. And now the reviewers look at it – and say that you have got this wrong, you need to fix this, you need a larger sample, or you cannot draw this conclusion from this data et cetera, et cetera. That is where the review process makes more sense, rather than afterwards, when they say, well, you should have done that…
You’re almost reviewing experimental design, rather than the results, because you don’t have the results yet?
This is obviously relevant potentially to biomedical studies too – but it’s particularly relevant to psychology. Do you see this being extended into other areas outside your area. Do you talk to people about that kind of thing?
Yes. The retroactive disclosure thing – right now we have a manuscript that will be published next week as a preprint in PsyArXiv. Then we will have [a manuscript] with researchers from most scientific disciplines where we propose how to do retroactive disclosure for a multidisciplinary audience. Psychology is in the frontline of making changes towards more openness and transparency and towards increased scientific rigour. But medicine is very close. Biology is very close. Economics and political science is coming too. It’s spreading, definitely.
How did your co-authors react to your own retroactive disclosures?
The short answer to that is: silence.
Yes, that’s interesting. I guess they didn’t want to draw attention to it
The first two disclosure statements, I actually published that within my PhD thesis. And not a single person has mentioned it at all. And that was reviewed by many people. I didn’t even get any comment on the language. It was in English, and as I’m a Swede, I need help with English. But no, people didn’t even comment [on the language]
It’s almost like they pretended it wasn’t there?
Yeah. That is pretty much how it has been afterwards too. When it later it became a bigger thing on social media et cetera. It’s still silence.
You’ve got the three kinds of categories of responses like we talked about IGDORE earlier. From what I’ve read, people are responding positively to the retroactive disclosures. That tells you that you’re probably going in the right direction.
About the retroactive disclosure statements, people are extremely supportive. People definitely think this is the right way to go – but that is the people who are not authoring the papers.
Changing the culture
“that is something that we miss in the ongoing open-science movement, and open-science revolution. We miss the culture part. And part of that is pure kindness, and that you want to share your work…with others publicly out of pure kindness. You want to be a kind person.”
What are the steps we can take to move the culture away from these “bad environments”. What could help nudge the culture in these better directions?
I think that the work environment is one thing that is not really included right now in the ongoing open-science movement. Which I think it should be. I’m part of the open-science movement, which for me, as a psychologist, the focus is on reproducible science, questionable research practices, et cetera. And for other people, the focus is more on open-access et. cetera, which I’m totally for too. The open-science, originates from the open-source movement, the open-source movement originates from the free/libre software movement. The free/libre software movement they talked a lot about moral issues, ethical issues, philosophy of life.
Richard Stallman and the GNU Manifesto?
Yes, exactly. Stallman and others talked a lot about it as life philosophy. And I think that is something that we should adopt, that is something that we miss in the ongoing open-science movement, and open-science revolution. We miss the culture part. And part of that is pure kindness, and that you want to share your work, your code et cetera, with others publicly out of pure kindness. You want to be a kind person. If we had the same mentality in academia – of course I share my data, of course I share my research materials with everyone. Because it’s great if you can build upon my work, if you can use my data for anything, if you can use my research materials that I’ve developed, that is just awesome because that will help everyone. If you have such a mindset, then the culture will change a lot. Not everyone has to have that mindset in order for it to work. And those few people who end up not having that mindset, who don’t want to share their materials, or data, or articles, will get a bad reputation over time. That is how it has worked within the free software movement.
One of the things that seems to prevent the development of that open, cooperative mindset is the degree of competition to achieve financial stability in the standard salaried academic hierarchy. My feeling is that we need to universalize our public policy solutions by providing healthcare, childcare and basic income to allow greater freedom and space for all to do their creative work.
Yes, I’ve very interested in that too. Science is a bit difficult because typically as a scientist, you don’t typically produce anything that you can or should sell. What we are doing is that we are doing something good for society. We are in the frontline of accumulation of knowledge on society’s behalf. So from that perspective – we need public funding – we need, basically, charity – to function. Because we will not get any income from the work we produce.
In the United States at least, universities are no longer run that way, the reality is they are pushed to focus on a narrower range of research topics and to commercialize innovations as quickly as possible. You’ve got the Bayh-Dole Act, for example. It’s all very short-sighted.
They should part there – it’s problematic with doing that, of course. I don’t think that’s the road to take.
Co-working and co-living spaces
“In Bali – the co-working spaces have added co-living and more culture into the spaces. It’s very relaxed, people are barefoot.”
It’s exciting to see that IGDORE has a co-working space. We have Ronin meetups [in Boston, NYC and other cities]. But my fantasy is a co-working space for scholarship in Boston. And there are plenty of co-working spaces around Boston. But most of them are now priced for tech and biotech startups. But even so, I think the benefits of having a physical space are huge.
And yeah – in Bali – the co-working spaces have added co-living and more culture into the spaces. It’s very relaxed, people are barefoot.
I love that idea. Boston is very traditional in that way. Co-housing, co-living exists – but it’s in the minority. Real estate here is very expensive. And creating any kind of workspace has got to contend with that, unless you own the physical space.
And you don’t have the climate to put up a tent.
Yeah, exactly. We don’t have the right climate. Certain times of the year, though.[laughs] We have the right climate [in Bali].
What’s the thing that’s exciting about this journey to you? What’s the thing that keeps you going and doing this stuff?
To be frank – this is not a fun process.
Has it been more pain than joy?
No, no. It’s very exciting to be building something from the start – from scratch. And there are so many new questions coming up, and so many new ideas coming up all the time. Because once I realized that I would be leaving traditional academia – I started to think out of the box – because anything is possible when you’re building something from scratch. And I think that’s the most exciting thing – anything is possible.
And the most terrifying?
I think the most terrifying is that I will not have enough funding to continue the work. Because what I visualize is IGDORE growing and becoming a full university, a non-formal university. And I would not settle having it as it currently is. I want it to become bigger. So I would need funding for that. So that is probably the most terrifying – that I will fail to make it that big. But I am also not scared to do – to put myself out there – to give it everything I can. To make it work. So if I fail, then no one will remember that failure, no one will remember me or IGDORE. It’s not a big thing.
Well, let’s work to make sure that doesn’t happen! Thank you so much for your time, Rebecca!