QRiH evaluation
Many humanities research units were evaluated in 2018. To investigate if and how QRiH has been used in the evaluations, the LAP has decided to evaluate the use of QRiH. One can find the main messages from the evaluation and the complete evaluation in the appendices below.
In the autumn of 2019, the results of the evaluation will be discussed with the field. In this meeting, not only the evaluation will be presented, but proposals for adaptation of the instrument will also be discussed.
Basic principles of QRiH
The basic principle behind QRiH is to do justice to the specific nature of humanities research and its wide-ranging diversity, among other things by:
- getting researchers to work with domain panels on formulating quality and relevance criteria
- creating scope for assessment instruments for diverse types of research outputs, such as books, edited volumes, catalogues, audio-visual products, exhibitions, and so on; such diversity can be made visible by working with the indicators referenced in SEP Table D1,
- focusing on the relationship between research quality and societal relevance by giving research units the opportunity to draft self-assessment reports as a narrative, and by also listing hybrid research outputs.
For more information, see QRiH background and development.
QRiH provides information and resources:
- Manual for SEP assessments in the humanities
- Format for self-assessment reports
- Examples of self-assessments (narrative)
- Profile descriptions of research cultures at the national research schools
- SEP Table D1, with examples of indicators
- Descriptions of indicators in terms of reach, usefulness and status
- Resources for using indicators, including Lists of publishers and journals authorised by the domain panels
- Profiles of institutess of institutes and faculties
- Examples of hybrid products and their use
- An explanation of the usefulness of bibliometrics.
Other initiatives
QRiH is based in part on lessons learned and methods developed in the course of other initiatives addressing quality and relevance in the humanities. They are VABB-SHW (Flanders), CRIStin (Norway), and the EU’s ERIH PLUS.
Some aspects of these initiatives have proved useful for QRiH. Examples include designating journals and publishers as communication channels with their own methods of monitoring the quality of publications, and working with panels.
QRiH differs from these initiatives on a number of points
- There is no direct relationship with research funding.
- The Web of Science does not serve as a criterion for quality (the reasons are given here)
- QRiH differentiates between the target groups of publications.
- Composition of the panels is based on the way the humanities are organised in the Netherlands.
- The QRiH system aims to cover as many typical forms of output in the humanities as possible.
The Flemish and Norwegian systems are tied to research funding (budget allocation). Research groups are allocated some of their funding based on their scores in these systems. The ERiHPLUS system is not tied to research funding.
VABB-SHW
VABB-SHW is the Flemish Academic Bibliographic Database for Social Sciences and Humanities. The database lists titles and publishers assessed by the Flemish Authorisation Panel (GP). The GP bases its assessment on the output produced by all social science and humanities researchers in Flanders over the past ten years. The database is updated annually.
The GP authorises a journal if
- it has a peer review procedure
- it has a valid ISSN number
An article in such a journal qualifies if it is at least four pages long.
The scores depend on the journal’s inclusion in the Web of Science (WoS, formerly ISI). The highest score (ISI-a) is awarded to journals that were included in WoS for the entire period. Lower scores (ISI-b to ISI-d) are awarded to journals with a weaker presence in WoS (not a ‘source’ journal for WoS), and the lowest score (ISSN) is reserved for journals that are not reported in WoS. The VABB database of journals contains more than 11,000 titles.
The GP authorises publishers if they satisfy the Guaranteed Peer Reviewed Content (GPRC) criterion. This means that the relevant book must appear on a publisher’s list that is peer reviewed and has an academic editorial board. Separate book titles can also be submitted to the GP for consideration. The GPRC label applies to books (monographs, proceedings and edited volumes).
For more information, see VABB-SHW
CRIStin
Like the VABB-SHW, the Current Research Information System in Norway (CRIStin) documents the total actual output of researchers in Norway. Unlike its Flemish counterpart, however, it covers all domains and disciplines. The system’s working methods and outputs are also used in Denmark, Sweden, Iceland and Finland, in each case based on the country’s own university publications databases.
The Norwegian system accepts a range of publication channels (such as websites), but the main focus is on journals, book publishers, publication series and similar. The system differentiates between level 1 and level 2.
Level 1 is the lower level; to qualify, the publication channel must be peer reviewed. To qualify for level 2, the upper level, the channel must be an internationally prestigious, broad channel of communication for academics in the relevant subject area (specialist journals are not eligible)[1].
Expert panels nominate publications for selection. Journals that appear in WoS (ISI journals) are virtually guaranteed level-2 nomination and selection. Other nominations depend on the expert panel’s decisions; in questionable cases, the main publication committee decides. Level-2 publications must not exceed 20% of the total number of publications in a particular channel in each subject area. Calculation of this percentage is based on the Norwegian Register for Scientific Journals, Series and Publishers (maintained by the Norwegian Centre for Research Data or NSD). The 20% standard is meant to be indicative, but there have been recent attempts to apply it more strictly.
National journals can also qualify as level-2 publications. The system excludes journals regarded as ‘local’, i.e. if more than two thirds of its authors are affiliated with one and the same institution.
Publishers nominated by expert panels can also be included. Once again, they qualify if they are an internationally prestigious, broad channel of communication for academics in the relevant subject area.
System maintenance has now been entrusted to the Norwegian Centre for Research Data (NSD)
For more information, see:
Norwegian Centre for Research Data
ERiH PLUS
ERiH PLUS is an EU initiative. It originally focused on selecting journals in the humanities, and has now been extended to journals in the social sciences. Selected journals must have an ISSN and adhere to an external peer review editorial policy; in addition, the majority of their authors should not be affiliated with one and the same institution (also see CRIStin). Journals are added to the list when nominated by researchers, rather than publishers. The database is maintained by the Norwegian Centre for Research Data (NSD, see foregoing).
ERiH+ does not cover books (monographs), edited volumes or proceedings.
For more information, see:
[1] According to this standard, a journal that an expert panel regards as excellent but that focuses on only a small segment of the subject area may be designated as a level-1 publication but should not be advanced to level 2. The Norwegian instructions do not offer much clarity in this respect.
Frequently Asked Questions
What are Authorised and Reasoned Indicators?
Authorised indicators are validated by the National Authorisation Panel at the suggestion of the Domain Panels. That is the case for lists of the most important journals and publishers in a certain subject area. Reasoned indicators set quality standards based on theoretical-empirical arguments. They include special publications, publications in foreign languages, exhibitions, and cooperation with societal partners. The research unit will have to provide these arguments itself.
(More FAQs will be added.)
Members of Domain Panels
The following scholars participated in the domainpanels (list per 21 september 2017)
ARCHON
Archeology
Dr. Alexander Geurds (UL)
Dr. Roel Lauwerier (RCE)
Prof. Daan Raemaekers (RuG, chair)
Prof. Harry Fokkens (UL)
Prof. Nico Roymans (VU)
Prof. Vladimir Stissi (UvA)
Huizinga
Cultural History
Floris Cohen (Chair)
Frans van Lunteren
Inger Leemans
Joep Leerssen
Lotte Jensen
Michael Wintle (Director HI)
Paul Koopman (Co-ordinator HI)
Wijnand Mijnhardt
Digital Humanities
Digital Humanities
Anne Beaulieu (RUG)
Elli Bleeker (Huygens ING - KNAW Humanities Cluster)
Joris van Eijnatten (Utrecht University)
Franciska de Jong (Utrecht University, Erasmus)
Ju-Sung Lee (Erasmus)
Patrica Lulof (Universiteit van Amsterdam)
Christian Olesen (Universiteit van Amsterdam)
Melvin Wevers (DHLab - KNAW Humanities Cluster)
Filosofie
Filosofie
Anthonie Meijers (TUe)
Frans de Haas (UL)
Marcus Düwell (UU)
LOT
Linguistics
Dr. Aoju Chen (Utrecht)
Dr. Hans Van de Velde (Fryske Akademy)
Dr. Judith Rispens (Amsterdam-UvA)
Dr. Onno Crasborn (Nijmegen)
Dr. Sander Lestrade (Nijmegen), postdoc rep.
Prof. dr. Fons Maes (Tilburg)
Prof. dr. H.E. de Swart.
Prof. dr. Jack Hoeksema (Groningen)
Prof. dr. Lourens de Vries (Amsterdam-VU)
Prof. dr. Marian Klamer (Leiden)
Prof. dr. Nicoline van der Sijs (Meertens Instituut)
Prof. dr. Sjef Barbiers (Leiden), chair
Mediëvistiek
Medieval Studies
Dekker (RUG)
Catrien Santingh (RUG)
Erik Kwakkel (Leiden)
Koopmans (UvA)
Marco Mostert (UU; Chair)
Martine Meuwese (UU)
Sven Meeder (RU)
NICA
Cultural Studies
Anneke Smelik (RU),
Ernst van Alphen (LU)
Frans-Willem Korsten (EUR)
Ginette Verstraete (VU)
Maaike Bleeker (UU)
Murat Aydemir, UvA director NICA.
Patricia Pisters (UvA)
René Boomkens (UvA)
Rosemarie Buikema (UU)
NISIS
Islam Studies
Gabrielle van den Berg (LU)
Petra de Bruijn (LU)
Christian Lange (UU, Chair)
Ruud Peters (UvA)
Thijl Sunier (VU)
NOG
Genderstudies
Dr. Agnes Andeweg (UD)
Dr. Eva Midden (UD)
Dr. Geertje Mak (UD)
Dr. Grietje Dresen (UD)
Dr. Kathrin Thiele (UD)
Dr. Liedeke Plate (UHD)
Dr. Liesbeth Minnaard (UD)
Dr. Rachel Spronk (UD)
Dr. Stefan Dudink (UD)
Dr. Veronica Vasterling (UHD)
Prof. Lies Wesseling (Prof)
Prof. Rosemarie Buikema (Prof)
Prof. Sandra Ponzanesi (Prof)
NOSTER
Theology and Religious Studie
prof. dr. Ab de Jong, Chair
prof. dr. Bert-Jan Lietaert Peerbolte,
prof. dr. Frans Wijssen, Radboud Universiteit
Prof. dr. Heleen Murre-Van den Berg
prof. dr. Judith Frishman,
NWP
Economic and Social History
Prof.dr. A.A.P.O. Janssens
Prof.dr. C.A. Davids
Prof.dr. J. Kok
Prof.dr. M.G.J. Duijvendak
Prof.dr. R. Oldenziel
OIKOS
Classical Studies
Prof. dr. Caroline Kroon (UvA/VU) -Latijn
Prof. dr. G. Boter (VU) - Oud Grieks
Prof. dr. Josine Blok (UU) -Oude Geschiedenis
Prof. dr. Miguel-John Versluys (UL) - Klassieke Archeologie
Prof. dr. Teun Tieleman (UU) - Antieke Filosofie, Chair
OPG
Political History
Prof. dr. Mieke Aerts (UvA)
Prof. dr. Remieg Aerts (Chair, UvA)
Prof. dr. Ida Nijenhuis (Huygens ING / Radboud Universiteit Nijmegen)
Prof. dr. Wim Klinkert (UvA / Defensieakademie Breda)
Prof. dr. Kiran Patel (UM)
Dr. Remco Raben (UU)
OSK
Art History
Dhr. dr. C. Stolwijk (RKD - Nederlands Instituut voor Kunstgeschiedenis), voorzitter
Dhr. dr. J. Abrahamse (Rijksdienst voor het Cultureel Erfgoed)
Dhr. drs. E. Buijsen (Mauritshuis)
Dhr. prof.dr. F. Grijzenhout (Universiteit van Amsterdam)
Dhr. prof.dr. G. Weber (Rijksmuseum Amsterdam)
Dhr. prof.dr. K.A. Ottenheym (UU/Directeur OSK)
Dhr. prof.dr. M.A. Weststeijn (Universiteit Utrecht)
Dhr. prof.dr. P.B.M. van den Akker (Open Universiteit)
Dhr. prof.dr. V. Manuth (Radboud Universiteit Nijmegen)
Mw. dr. H. Berens (Het Nieuwe Instituut)
Mw. dr. M. Schavemaker (Stedelijk Museum Amsterdam)
Mw. dr. M. van Thoor (Technische Universiteit Delft)
Mw. drs. M. Vellekoop (Van Gogh Museum)
Mw. prof.dr. A.S. Lehmann (Rijksuniversiteit Groningen)
Mw. prof.dr. C.J.M. Zijlmans (Universiteit Leiden)
Mw. prof.dr. K. Kwastek (Vrije Universiteit Amsterdam)
OSL
Literary Studies
Dr. Aagje Swinnen (UM)
Dr. Brigitte Adriaensen (RU)
Dr. Stephan Besser (UvA)
Prof. dr. Ellen Rutten (UvA)
Prof. dr. Geert Buelens (UU)
Prof. dr. Henk van der Liet, (UvA, chair)
Prof. dr. Liesbeth Korthals Altes (RUG)
RMeS
Mediastudies
prof. E. Müller (UU)
prof. F. Kessler (UU)
Prof. G. Verstraete (VU)
Prof. M. Broersma (RUG)
prof. R. Rogers (UvA)
WTMC
Science, Technology and Innovation Studies
Prof. S. Wyatt – KNAW.
Dr. A. Meershoek – UM
Dr. B. Pasveer (UM)
Dr. B. van der Meulen – Rathenau Instituut
Dr. G. Valkenburg (UM)
Dr. J. Broerse – VU
Dr. M. Boenink – UT
de Wit (RUG) – PhD student member
Smit (UL) PhD student member
Prof. H. Zwart – RU
Prof. Ir. G. Verbong – TUE
Prof. P. Wouters – UL, chair
Prof. H. Dijstelbloem – external member
Prof. K. Frenken – UU