THE QUEST FOR EVIDENCE-BASED PRACTICE:
Comment on HEA’s “Towards a Performance Evaluation Framework: Profiling Irish Higher Education”
In Ireland, government policies have given increasing importance to knowledge transfer activities in the higher education sector, as means of driving the knowledge-based economy. There lacks, however, much empirical evidence showing the width and depth of interactions between Irish universities and businesses.
Efforts such as the Community Innovation Surveys are essential to help us understand how enterprises in Ireland source knowledge and build innovation. Nevertheless, these surveys are mainly focused on the business sector and not designed to reveal the full picture of university-industry interactions. An urgent issue which needs to be addressed now in Ireland is to start building empirical evidence on the university side, i.e. how academics engage with businesses and the wider community.
Furthermore, when focusing on university-industry relationships, there has been a shift of research focus from the knowledge transfer of intellectual property (IP) to multifaceted channels and mechanisms of knowledge exchange. As many studies have asserted, IP-related activity is too specific and narrow, a wider view of interactions between academics and businesses is better capable of capturing the comprehensive roles of universities.
In the U.S. and Canada, the Association of University Technology Managers (AUTM) has been releasing its annual surveys for a couple of decades, with its special focus on licensing, patenting, and establishing spin-out companies. The Higher Education Funding Council for England (HEFCE), on behalf of all UK HEIs and the national funding bodies, has been publishing the Higher Education-Business and Community Interaction Survey (HE-BCI) since the academic year 1999/2000. In comparison to the AUTM surveys, the HE-BCI surveys take a more holistic approach and consider the following types of engagements: collaborative research, contract research, consultancy research, facilities and equipment related services, courses for business and the community, and IP-related activity.
Even so, there are critics that these six types of activities only account for a small share of partnerships through which knowledge is exchanged between organisations. More recently, a large research project conducted by the Centre for Business Research (CBR) at the University of Cambridge in 2009 grouped possible modes of interaction into the following categories: people based, problem solving, community based, and commercialisation. In total, the research team considered 27 types of knowledge exchange activities.
As mentioned earlier, there has been little research compiling these data in Ireland, except for a recently report released by the Higher Education Authority (HEA) in December 2013. Entitled “Towards a Performance Evaluation Framework: Profiling Irish Higher Education”, the report represents a milestone in Irish higher education policy. This study, the first of its kind in the country, has attempted to profile Ireland’s higher education sector in the three dimensions of its core mission: teaching and learning, research, and engagement.
The main strengths of the report lie in two aspects. First, it appreciates the diversity of higher education institutions in Ireland and groups them into universities, institutes of technology, and colleges. To develop a framework in which to evaluate the quality of engagement, this is an important step to take as it takes account of institutions’ historical, geographical, socio-cultural specificity and diversity, which would to a large extent determine the type (width), intensity (depth) and locationality (length) of academic engagements.
Second, it puts much effort in collecting data from a variety of sources, in the aim of building a comprehensive performing evaluation framework. In particular, it presents data on topics such as student enrolment, research funding, staff profile, as well as knowledge transfer activities. More importantly, it indicates, in its framework, which data is currently available, under development, or needs to be articulated in the future, the last type of which is what much of the engagement dimension has been labelled as.
Whilst the report has an intensive literature review, in which it searches for and describes the indicators used in different countries when evaluating research, the framework it proposes in the end still has many limitations. In particular, the indicators proposed in the knowledge exchange section are not quite clearly defined. For example, it calls for data collection in joint research programmes with enterprise, but does not specify the nature of those programmes, such as whether they are collaborative research or contract research. The nature of research programmes could have a big impact on what type of knowledge is exchanged, and how intensively that knowledge is exchanged. Also, the framework suggests HEIs to report their involvement in the community, which, again, is too broad and may be really difficult, if not impossible, for all HEIs to provide effective and comparable answers.
All in all, this work has pointed us in the (right) direction of building evidence-based practice in Ireland’s higher education sector by, at least, showing some evidence.