The San Francisco Declaration on Research Assessment continues to gain traction in Canada
‘DORA never says publication or traditional scholarly output doesn't matter. What it does say is that scholarly output is diverse.’
Two years into his faculty appointment at the University of Winnipeg, exercise physiologist Yannick Molgat-Seon applied to renew his funding from the Natural Sciences and Engineering Research Council (NSERC). “They emphasized that they had recently signed on to this San Francisco Declaration on Research Assessment,” he said. Though he hadn’t heard of the concept, known as DORA, he read up on it. “I thought it was great, and I figured since I’m funded by this group, I would sign it myself.”
Created in 2012, DORA is a set of principles aimed at expanding how academic research is judged, by encouraging less weight on traditional metrics like journal impact factor (which scores research by the number and type of journal publications). According to the mission statement, DORA encourages institutions to consider a wide range of impact measures, “including qualitative indicators such as influence on policy and practice.”
The University of Winnipeg hasn’t signed DORA, but individual researchers can; in fact, of the 981 Canadian signatories to DORA, just seven are postsecondary institutions. The University of Calgary was the first to sign in 2021, followed by three Quebec institutions: Université de Sherbrooke, Université de Montréal and École de technologie supérieure (ÉTS), part of the Université du Québec network. The University of Victoria and Mount Allison University signed last spring; Concordia University signed in December.
The Tri-Agencies – the federal funding bodies of NSERC, the Social Sciences and Humanities Research Council (SSHRC), and the Canadian Institutes of Health Research (CIHR) – signed jointly in 2019, writing that DORA fit with existing policies on open access publishing and their equity, diversity and inclusion action plan.
Reasoning was similar at the U of C, said William Ghali, vice president of research. “DORA came along on a backdrop of some considerable institutional traction,” he said, noting that the university’s leadership had been developing a research assessment framework that emphasized stronger engagement with the wider community.
Within that, Dr. Ghali said DORA serves as a guide to help researchers internalize that philosophy and view their work in a new light. “For us, DORA is about how academics judge themselves,” he said. “Our own research excellence framework is about the accountability of researchers to tell the story of what they did with their work.”
U of C leadership also incorporated DORA into hiring, advancement and awards processes, encouraging researchers to implement the principles in their applications. Dr. Ghali said he and other leaders led a process of “socializing” DORA among the university deans, emphasizing its expansive nature. “DORA never says publication or traditional scholarly output doesn’t matter,” Dr. Ghali said. “What it does say is that scholarly output is diverse.”
Christie Hurrell, the university’s associate librarian specializing in research support, has worked closely with DORA’s implementation; part of that has been communicating that it isn’t meant to replace anything. “It’s more about creating room for different types of things, as opposed to making people who are happy in the current system change what they’re doing,” she said. She added that herself and other librarians have long been aware of flaws with the journal impact factor, which was originally developed as a tool for librarians to categorize journal subscriptions.
Of all disciplines, science has traditionally relied on publications as the gold standard, which is part of the reason DORA was created by academics from scientific backgrounds. At the UdeS, Jean-Pierre Perreault, vice president, research, said he expected resistance to DORA from his colleagues in science, but instead found many were ready for the change. The conversations, he said, were to the tune of: “I don’t want to know the number of papers you’re publishing. I want to hear something about the real impact for society of what you’re doing.”
Dr. Perreault gave the example of former UdeS professor Réjean Hébert, a renowned geriatrician who developed the functional autonomy measurement system, a diagnostic tool to assess the needs of elderly patients. “He published that in a journal review with an impact factor of two,” Dr. Perreault explained. “However, there’s about 20 million people around the world every year that are using that questionnaire.”
As with Dr. Ghali, Dr. Perreault said hiring and advancement have been one of the concrete ways DORA has shown up in their institution, and leadership at other DORA-signatory universities agree.
“It helps leaders to make sure that we’re taking into account the full range of scholarly outputs,” said Richard Isnor, provost and vice-president at Mount Allison. As a small institution, Dr. Isnor said DORA serves as a unifying guide for their advancement processes, where it’s common for faculty from various disciplines to adjudicate candidates. Dr. Isnor said DORA also helps amplify their work overall. “Smaller universities like us have had to work hard to make sure the research output of our scholars is being treated equitably by researchers, or by the scientific community in bigger universities with different expectations,” he said.
Dr. Molgat-Seon agreed that DORA can help manage expectations for faculty at smaller universities, where there may be fewer resources for research and teaching responsibilities can be greater. “Instead of the publish-or-perish mentality that people often fall into, I’ve shifted my focus towards publishing a few less articles and focusing on doing those things at a high quality, and disseminating the information,” he said.
With relatively few universities having signed, there’s no guarantee that the wider postsecondary community will assess research with DORA principles in mind. Even for signatories, DORA isn’t a binding document.
But Dr. Isnor, who spent many years working for NSERC, says he thinks it fulfills an important piece of large-scale change. “There’s an educational and awareness-building aspect to it, and that is one type of policy instrument that organizations and governments use,” he said. “It helps us as institutions when we share the information and explain why we signed. It reflects a commitment to certain kinds of principles.”
For Christie Hurrell at the U of C, DORA is hovering at a turning point. “We’re in between that top-down socialization of these new ideas, and the friction with people that are trying to go through the system as it’s changing under their feet.”
Featured Jobs
- Education - (2) Assistant Professors (English Language Arts and TESL / Curriculum, Teaching, and Learning) (10-month term)Bishop's University
- Dean, Faculty of Health StudiesBrandon University
- University LibrarianYukon University
- Research Chair in Systems Transformation and Family Justice (Faculty Position)University of Calgary
- Doyenne ou doyen - Faculté de génieUniversité de Sherbrooke
More from News
-
Responding to growing criticism of equity, diversity and inclusion policies
An interview with David Robinson, executive director of the Canadian Association of University Teachers, which represents faculty, librarians and researchers at 125 postsecondary institutions.
-
An historic undertaking for Indigenous education in Quebec
Quebec’s first university by and for Indigenous Peoples in partnership with the First Nations Education Council and Quebec universities.
-
Emojis … as rhetoric?
A new course and lab at U of T Mississauga delve into these digital images 😀
-
Where do the federal election candidates stand on postsecondary education?
Here’s a rundown of each party’s platform on higher education.
More from Research
-
The rise of disaster studies
Disasters happen more often — these programs explore the whys and hows.
-
Engaging the public in sidewalk astronomy
The #popscope project energizes public spaces through experiential, free-choice learning.
-
McGill researchers help advance genetic research through a video game
Borderlands Science, a mini-game within the popular Borderlands 3 video game is putting a new spin on citizen science.
-
Researchers help fill information gap on abortion options with new decision aid
‘When you look online for anything about abortion, it is very difficult to know what you’re reading is legitimate and accurate,’ says one of the creators.
Post a comment
University Affairs moderates all comments according to the following guidelines. If approved, comments generally appear within one business day. We may republish particularly insightful remarks in our print edition or elsewhere.
1 Comments
The claim to diversify output to utilitarian ends without compromising traditional scholarship would be fine if resources, people and material, were infinite and if the fundamental issues at the forefront of knowledge led to immediate applications. But neither is true. This initiative will inevitably serve to limit and devalue basic research, most of which is performed in universities in Canada and is already underfunded.