Universities Canada study uses bad research to get desired results

The U of A, and other Canadian universities, are using a flawed study to push Ottawa for more funding.


On September 18th, Abacus Data released a survey commissioned by Universities Canada to examine how Canadians believe Universities have a role to play today and in the future of Canadian society. On the Universities Canada website, the organization boasts figures such as “94% [of Canadians] support investing in international university research collaboration to tackle global challenges” and “92% support increasing university research funding to comparable levels with our global competitors”.


At face value, these figures are staggering; the survey is representative of the population as a whole, and this overwhelming positive response seems to indicate that nearly all Canadians think that universities ought to play a much larger role in research. However, a closer look at the survey questions show that these results were dubiously collected. Despite the study’s bold claims, I would allege that the survey presents an incomplete picture of Canadians’ perspectives on Universities, for two reasons. 


First, questions were designed to elicit positive responses.

As you can see from the graph, questions used language such as “breakthrough health innovations” and “new solutions” to imply that the research is inherently positive. If a person is asked whether it’s important that Universities research “breakthrough health innovations”, how could they say no? This violates the #1 rule of research design, which is to not use leading questions that encourage one answer over another. The statements and questions posed in the study play into unconscious social pressures of the participants for the purposes of getting the results Universities Canada wants. A better survey might replace “discovering medicines and breakthrough health innovations” with “health and medicine research”, or change “new solutions to feed a growing population” to “researching agriculture and food production technology”


Second, the study does not evaluate Canadians’ broader priorities in regard to government spending.  


The study does not put research funding into the broader economic context in which we live. It’s easy for a participant to say that they support government funding to “bridge differences and encourage peace” when the question only asks whether they support that one statement. It’s more difficult to answer with a firm “yes” when other government spending on health care, education, economic stimulation, and job creation are also taken into account. A more well-designed survey would have asked participants to rank the importance that university research funding within the Canadian economic context. But the study doesn’t ask that, and for good reason: it’s likely that if it had, Universities Canada would have not received the overwhelmingly positive response that they had intended in the first place.


But why does all this information matter? At a recent Senate meeting, U of A President David Turpin has indicated that he, along with other universities in Canada, sees this survey as a lobbying tool that can be used when they meet with government officials to advocate for more funding. As well, in his most recent report to General Faculties Council, President Turpin stated that one of his priorities is to “create a sense of urgency” that increased funding is crucial to Canada’s future. I’m not here to say that research isn’t important – I truly believe that universities are underfunded and that our future does depend on universities having the capacity to conduct high-quality, innovative research. What I can’t support is a university using shady methodology and poorly-designed questions to produce sub-par research, and choosing to use that research to ask for millions more dollars in government funding. I had expected better from a university that claims to be world-class in teaching and research.

Related Articles