intermediation Regulatory and quality assurance in higher education: the case of the auditors
Roger King, Paul Griffiths and Ruth Williams
Centre for Higher Education Research and Information, Open University, 44 Bedford Row, London WC1R 4LL
Introduction This paper takes the external quality assurance of universities and colleges as an example of regulation in higher education. In many countries this function is undertaken by national agencies that undertake the classic regulatory functions of setting standards, monitoring activities, and applying pressure to secure behaviour modification where this is required (Baldwin and Cave, 1999). Although many in the national quality agencies would demur that they set standards arguing that universities and their professional communities set standards which the agencies assure the increasing use of codes, benchmarks, and frameworks within which such assurance is undertaken may be considered as constituting forms of standards-setting. This article examines how Quality Assurance Agency (QAA) auditors in England seek to reconcile these external and formal frameworks with long-standing, often less explicit, disciplinary and professional peer approaches.
The piece draws on material gathered from 55 respondents in England, including 17 in-depth interviews with QAA auditors and a survey of a further
1
17 auditors by questionnaire, undertaken between January and May 2005. Additionally interviews were conducted at central level (with officials from the QAA, the Department for Education and Skills, the Better Regulation Unit of the Cabinet Office, and Universities UK, and other experts), and at three universities (Vice Chancellor Office, Dean and senior manager levels). The focus was on the experience and perceptions by auditors of the new transitional arrangements for institutional audits being undertaken by the QAA for the period 2002-5 (see below).
Regulatory intermediation In higher education external quality assurance, key groups found at the interface of the regulator (quality agency) and the regulated (universities and colleges) are positioned to play critical `intermediary' roles. These include the quality assurance agency assessors drawn by the QAA from the academic community (others are senior leaders and quality managers in institutions). As regulatory intermediaries auditors look both `upwards' to the regulator (in accepting regulatory functions) and `downwards' to institutions (to secure reasonably legitimated delivery of regulatory objectives). We may define `regulatory intermediation' as the process by which external regulatory authority is distributed and modified through local, organisational and professional perspectives by key groups such as auditors that occupy the `regulatory space' between the regulator and the regulated. To date, however, it has not been clear how this intermediary role has been discharged or how the potential conflicts within it are reconciled.
2
The interviews with the QAA auditors explored the factors and instruments that were significant in achieving reasonable levels of reconciliation of potentially competing objectives in the role of these intermediaries. The auditors are required to discharge their regulatory responsibilities with objectivity, but also with sensitivity to those being regulated. Moreover, they have to avoid charges from government and others, such as the media, of being too `soft' on institutions, while also avoiding allegations that they are simply state agents carrying out orders from the government or the regulator without proper consideration of the needs or sentiments of the sector and its institutions. A form of `regulatory balance' may be necessary, achieving support for peer sensitivity from the academic community but also avoiding governmental suspicions of `capture' (Moran, 2002). They have to evaluate objectively and deliver regulatory objectives without incurring charges of overzealousness from those being regulated. Generally the auditors take on the governmental or similar external functions of regulatory distribution and evaluation in return for a relative autonomy for their knowledge, authority and expertise. Potentially at least, this allows scope for regulatory modification and adaptation to professional and peer perspectives, a flexibility that may be an essential ingredient for external regulatory effectiveness. Significant variables in these processes may centre on such factors as levels of relational closeness or distance of the auditors to those being regulated (in comparison with, for example, permanent inspectors in regulatory bodies in other sectors), or on the sense of accountability to the regulator and the requirement to apply strongly directive and explicit external codes, frameworks and standards, rather than more informal peer review.
3
Quality Audits The QAA methodology may be described as a form of `meta-regulation' (Parker, 2002) in that it focuses on the whole of an institution, and its processes and procedures, for ensuring quality and standards in student programmes. Effectively, internal institutional regulation is turned `inside-out' and made available for external evaluation and accountability (King, forthcoming).
The process of audit employed for institutional review by the QAA, and managed by it, commences with a self-evaluation report from the university or college, leading to a preliminary briefing meeting, followed by the key weeklong visit to the institution, and results in a published report. Audit examines the institution's structures and processes for quality assurance, including how, and how regularly, the quality and standards of its awards are reviewed and recommendations acted upon. The recent `transitional' phase for this approach by the QAA took place between 2002-5; the next stage (2006-12), normally involving a six-year cycle, will see closer scrutiny of the reliability of the publicly-available information on its standards and quality that is required to be published about the institution (`teaching quality information'). The conclusions of institutional audits are based around the extent of the confidence that QAA has in institutions' quality management, using the three categories of `broad confidence', `limited confidence' and `no confidence'. The two latter conclusions lead to a requirement that an institution prepares an action plan to remedy deficiencies, and this has to be accepted by the QAA before the audit is signed off. Auditors operate with the guidance of the QAA's
4
`academic infrastructure', a wide-ranging set of `codes of practice', a national qualifications framework, programme specifications and subject `benchmarks'. Of the 99 completed audits in the first cycle by October 2005, only six had `limited confidence' outcomes and none had resulted in `no confidence' (Alderman and Brown, 2005).
Findings The findings are drawn predominantly from the in-depth interviews (17) with auditors, but are generally replicated in the survey material from a further 17 auditors. Interviews (12) of managers within audited institutions have also been drawn upon. The interviewed auditors cover a broad range of disciplines, institutional positions, and QAA experience. Eight were drawn from `pre-1992' or well-established universities, and nine from `post-1992' expolytechnic universities. All the auditors that were interviewed were employed (in one case, recently employed) within institutions that were themselves subject to institutional audit by the QAA. 1) Accountabilities Potential processes of regulatory intermediation undertaken by auditors are likely to be influenced by the extent of their sense of accountability to the quality agency. The interviews were aimed at eliciting whether auditors exhibited a high level of primary accountability to the regulator and the extent to which this may be tempered by a wider sense of accountabilities to other stakeholders, such as the institution being audited, its students, the taxpayer, the sector as a whole, or to the government.
5
The responses were categorised as: seven responses that regarded the QAA as the primary source of accountability; six responses that attributed it to the institution being audited, or its students, or to the sector as a whole; and four responses that located them to the government, the taxpayer or the general public. However, nearly all respondents referred to plural responsibilities, recognising that they had accountabilities to a wide group of stakeholders. These dispositions did not vary by pre- or post-1992 institutional locations. A sense of plural accountabilities by auditors is found across the entire sector. It would appear that QAA auditors do not display a narrow regulatory identity and that their regulatory commitment is moderated by a sense of wider responsibilities undertaken within the role. Undoubtedly, in part at least, this reflects their `part-time' status and relational closeness to those they audit.
2) Regulatory intermediation and institutional locations A key element in the concept of regulatory intermediation in the case of QAA auditors is that they feel competing pressures between, on the one hand, external regulatory requirements and methods, and, on the other, more traditional peer and `on-the-ground' practitioner approaches. The formal and frequently standardising frameworks of regulators are often incorporated within the `corporate knowledge' and procedures of institutions by senior leaders, but may jar with more individualistic practices and disciplinary views as to what constitutes good practice as found in the classroom. The interviews with auditors explored whether any sharpness in this potential duality and its resolution in one direction or the other - might be related to the auditor's institutional location. Particularly, auditors were probed to elicit whether, in
6
their own institutions, they felt closer to managerial and corporate knowledge (including its incorporation of external regulatory knowledge), or to academic practices and perceptions at the department or more local level.
The issue was whether an auditor's sensitivity to `basic' disciplinary quality practices in teaching when participating in institutional audit visits for the QAA, might reflect a perceived sense of relational closeness to departmental academics in an auditor's own institutional role position. Alternatively, an emphasis by the auditor on regulatory accountability in external quality assurance might reflect relational distance from the basic unit and closeness to corporate quality knowledge in their own institution. The latter would include close proximity to those procedures internalised and adapted from external regulatory codes and frameworks by the auditor's institution. Successful `intermediation', however, does suggest a level of sensitivity to both the corporate and the local within an auditor's own institution and to those that they assess.
In pursuit of these distinctions respondents were asked whether they regarded themselves as an academic, or as a manager, in their own institution. Only two respondents described themselves as clearly in the `academic' category (both from pre-1992 or longer-established universities); seven respondents described themselves as `managers' (including five from the post-1992 or ex-polytechnic sector); and eight felt that that they were, in some sense, `both' (five pre- and three post-1992 universities). However, even those who described themselves as `managers' felt strongly that they
7
were `still in touch with those on the ground'. This was either because their institutional function brought them into frequent contact, or that they had been academics before entering management or administration, which made their outlooks different, and perceived by academics to be different, to those that were described by them as `career administrators'. That is, they regarded themselves at least as retaining academic sympathies and to some extent academic identities.
Interviewees were also asked to discuss whether `you see any conflict between disciplinary or practitioner knowledge, on the one hand, and corporate or specialist pedagogic knowledge as to what constitutes good practice and quality, on the other?' This again was aimed at eliciting an auditor's position on the corporate-local continuum in quality orientations, and also whether effectively the continuum was more a sharp divide and a source of intra-institutional conflict. That is, do they see themselves as `walking a regulatory tightrope'? For the most part, auditors did not really see `conflict' between the two sources of quality knowledge, although they clearly acknowledged differences and occasional tensions.
Among the reasons advanced for the lack of sharp hostility between corporate, and traditional or local knowledge, were a) that corporate knowledge and external regulatory structures were a means of systematising much of traditional practice (rather than necessarily being at odds with it); b) that younger academics entering the profession increasingly were especially likely to accept the productive trade-off between the two approaches (between
8
external regulatory formality and more local `on-the-ground' and less codified notions of good practice); and c) that most academic staff in departments only needed to be aware of those elements of corporate and regulatory procedure and knowledge that most affected them (such as the QAA's subject benchmarks, or its postgraduate research training codes), while leaving other elements (such as qualifications' frameworks) to the `policy people at the top'. When necessary, the corporate experts would be wheeled in by departments to provide assistance in meeting external regulatory requirements, such as when an audit visit was imminent. Finally, although a core of (mainly older) staff expressed frustration or opposition at corporate or regulatory knowledge and its procedural requirements, most institutions seemed able to keep such staff hidden away, or contained, by policies of `staff shelving' or `staff reconnoitring'.
The respondents were asked why they had become auditors in the first place. Did original motivations illustrate that they were particularly disposed towards discharging external regulatory roles and thus were distinctive to most of their institutional colleagues in possessing a regulatory orientation? In 13 cases respondents emphasised that the prime reason for their interest in external quality assessment lay in a career or personal aspiration to find out what was going on in other institutions. There appeared to be a genuine desire to learn from elsewhere, and auditors often emphasised how few other mechanisms existed in university systems to learn from other institutions (compared to the research and disciplinary networks of academics). The auditors did not display a particular desire to learn about regulation as such, or motivations to
9
want to run the rule over others, but they did reveal a curiosity in other institutions and their practices.
A group of four auditors, however, responded that they had become auditors for the QAA because, as one described it, `the Vice Chancellor wanted me to find out as an insider what it was all about and so I could help the institution when faced with its own external assessment'. All these four auditors were from traditional or pre-1992 universities. This presumably reflects a stronger feeling in that part of the sector that, without longer-standing external quality regulatory traditions of the kind associated with, for example, the now defunct Council for National Academic Awards (CNAA) as found in the ex-polytechnic institutions, generally more work needed to be undertaken by a pre-1992 university in order to prepare for a QAA institutional audit.
3) Governmental and self-regulation The process we describe as regulatory intermediation as undertaken by QAA auditors may be influenced by whether auditors regard their task as governmental (and the QAA as a governmental agency), or as predominantly more self-regulatory, with the QAA envisioned more as an independent body. If the latter, it may be hypothesised that auditors could feel more scope for the exercise of traditional forms of peer evaluation. Governmental regulation, on the other hand, may be regarded as more constraining and as making more difficult the processes of intermediation and the reconciliation of competing perspectives.
10
The research interviews found 12 auditors who described the QAA as `a government regulator', three auditors who regarded it as `independent', and two who saw it as containing `elements of both'. Yet all 12 responses describing the QAA as a `government regulator' were heavily qualified. One saw it as `not as dogmatic as a statutory regulator and more of a watchdog', and would rather have the QAA than governmental inspectors `because of the value of peer review and the more human quality'. Others regarded the QAA as answerable to government although `not simply as a tool of government, but not completely independent either because Ministers can change its methods.' Another auditor saw it as a governmental regulator, but felt that `inhouse and in its day-to-day operations the QAA is largely free of government and is not really accessible to government and other outsiders.'
One auditor regarded the current QAA regime as `more free-spirited' than previous QAA regimes, although it was still undertaking a governmental function. Others claimed the peer review methodology as highly important, especially its requirement for a relational closeness of the auditors to the regulatees in comparison with inspectorial or regulatory authorities found in other sectors, and felt that this helps to moderate any malign governmental influence. Many auditors see the QAA as a relatively benign regulator and stress how important it is that it is `not seen as a governmental body'. The permanent staffing, in the view of most auditors, acts as a buffer to government and has `a life of its own' in comparison to the auditors. The final report from the institutional audit visit is regarded by most of the auditors as the key outcome for governmental or regulatory purposes
11
(although auditors felt that most value or benefit actually flows from the peer discussions with the institution during the visit itself). The bland `civil service' language that reports adopt is regarded as a sign that the QAA does not wish to get offside with government officials, or with Vice Chancellors, by invoking controversy in the media. Most auditors appear to recognise the purpose of the QAA being non-controversial, as `keeping government from imposing something far less acceptable'. However, the auditors also lament that the overall audit process consequently has become overly `paper-driven'.
Most of those auditors using the `government regulator' description for QAA appear to be referring to the Funding Council (HEFCE) rather than to a central government department, reflecting the increased governmental dimension to HEFCE's role in recent years. It is seen as `always in the background'. Moreover, in this view, QAA is required as a consequence of HEFCE and governmental influence to adopt regulatory methodologies that allow comparability and the introduction of consumer information approaches, such as the recent requirements for teaching quality information, which is publicly available data on the standards and quality of an institution.
Nonetheless, generally the auditors saw the QAA nowadays as more `selfreferential' than in its earlier life, so that outsiders (including government) have difficulty in de-coding its processes and outcomes. However, it was regarded as retaining a level of independence only so long as government `was looking elsewhere for the moment', for it still has to `keep government sweet'. If QAA starts to get controversial, for example by allowing newspapers
12
to construct league tables on the basis of the reports, it was felt that Vice Chancellors would rebel and lobby government to change the arrangements. At any sign of difficulty or political turbulence, what looks like independence and self-regulation in times of stability is shown to be something else entirely.
Moreover, although the QAA was independent or self-regulatory in the eyes of a few auditors, it was more broadly regarded by auditors as beginning to `professionalise' its activities by having well-trained, trusted auditors with increasing and built-up experience that began to replicate that found in more explicitly governmental regulators, while (cleverly) yet retaining the important peer review methodology and the relational closeness of the auditors to those being scrutinised. In this view of the auditors, QAA has taken on the accoutrement of a professional body in its own right. Some respondents, however, distinguished the QAA Board and bureaucracy (as entities lay that `outside the higher education system' and which formed the governmental regulatory characteristics of the organisation) from the panel members (auditors) of the visit teams to institutions (who are regarded as `on the inside').
4) Standards and regulatory formalism The development of the `academic infrastructure' by the QAA would appear to indicate a marked movement in the direction of establishing formal or prescribed standards for universities. It involves detailed codes, benchmarks, specifications and frameworks. An important issue, however, is the extent to which the QAA (and therefore the auditors) require institutions to display
13
compliance with such standards and, indeed, whether they are regarded as standards at all. `Compliance' is a word that the QAA seeks to avoid. Rather, it refers regularly in its newsletters to universities `engaging with' the infrastructure and, rather philosophically, as a means for the sector as a whole to `engage in a conversation or dialogue with itself'.
The interviews with the auditors suggest that, while on the whole they do regard the codes, benchmarks and frameworks of the academic infrastructure as like standards (as `approximate standards' as one described them, or as more a range of `professionally agreed aspirations' in the words of another), they interpret these notions quite widely and flexibly. Generally, during the audit visits they are content when institutions display that they have at least considered the elements of the academic infrastructure for facilitating audit discussions, and for systematising and aiding what usually passes for good practice and `what a lot of people are already doing'. The academic infrastructure is also regarded as helping institutions `to tackle colleagues internally'. Mostly it `helped people to reflect'. And while there needed to be compliance `of a sort', it is not required `in a box-ticking way, or 100 per cent'. If the infrastructure was to become more than `a set of principles' it would become too prescriptive and `who is going to monitor compliance there are not enough resources to do it'.
It is also suggested by auditors that institutions seem happier with this `woollier approach' than with `compliance'. But `flagrant breaches' could not be tolerated. On occasions, however, some auditors felt that institutional leaders
14
deliberately interpreted the infrastructure `in a rigidly rule-based way' for `their own internal reasons'. Some institutions, on the other hand, regard the `odd maverick' auditor as riding particular `hobby horses', particularly with respect to the academic infrastructure, and, according to one auditor, `treating it as a bible then it is a real disaster'.
For the most part, the infrastructure is regarded as a set of guidelines `a guide to how you approach standards'. Nonetheless, some elements are perceived as having much stronger elements of `standards' and `compliance' than others. The Framework for Higher Education Qualifications was regarded particularly in this light as demanding quite fixed compliance. It is also clear that academics in the departments are not expected to be conversant with every aspect of the infrastructure, although institutions are examined to ensure that their internal procedures are compatible and widely disseminated.
Although generally supportive of this increased regulatory formalism, many auditors were nevertheless concerned that it contained the danger of `mapping' at the expense of `understanding', and of reinforcing the `paper obsession' of the more recent processes. The image of the audit visit was outlined by one auditor as increasingly like a `ceremony' or `ritualised performance' in which the `altar' before which participants bowed was the `base room' containing all the institution's documents, as required to be available by QAA for the auditors. A minority of auditors, however, felt that paper systems were useful and enabled the right questions to be asked. In
15
`meta-regulatory' systems, in this view, it would be confusing matters to get too close to actual processes, such as observing classes in action.
It is clear that institutional standing and stratification plays a part in considering the issues of the academic infrastructure. A number of respondents who had been engaged on audits of high-reputation universities commented that these institutions were often falling quite short in their engagement with the codes and procedures of QAA's academic infrastructure. Nonetheless, this was not allowed to divert a `broad confidence' outcome to the visit, on the grounds that `it all had to be seen in the round that is, such institutions clearly have high quality staffing, excellently-qualified students exerting peer pressures on each other how could one fail to have confidence in their quality?' Another auditor, in similar vein, commented that it was natural to be influenced by the reputation of the institution being assessed `if they have the quality teachers they are not likely to have a lot of problems'. (These two auditors were from pre-1992 universities where such a view may be more prevalent than among the post-1992 universities). Clearly, even as regulatory formalism increases, more informal and `insider' knowledge retains its importance in the institutional audit process. For many auditors, the ultimate test was the quality of the student experience rather than absolute conformity to the letter of the academic infrastructure.
5) Risk and regulation in the institutional audit process In considering issues of regulatory intermediation as conducted by QAA auditors during an institutional audit visit, it is necessary that we have an
16
understanding of factors that influence, or are a consequence of, such audits. To what extent do universities enthusiastically accept the process of institutional audit, or do they largely go along with what is happening, or perhaps even display hostility? In the view of the auditors, has external quality assurance as expressed by institutional audit reinforced tendencies to managerialism in institutions?
For the most part, auditors felt that nearly all institutions took the process seriously (`it would be hard to pull the wool over the eyes of a visiting audit team for a whole week', an auditor claimed), although some examples of hostility could be cited. Institutions, it was felt, were more likely to be unhappy with the final report than with the visit process itself. Auditors regarded institutions, even high reputation ones, as having too much risk in an adverse decision not to take the process seriously. The consequences from a `limited confidence' outcome for overseas recruitment, and drawing adverse media comment, in increasingly competitive environments, were regarded as sufficiently compelling to prepare assiduously for institutional audit in nearly every case. Moreover, interviews in institutions indicated that managers felt it would consume too many resources if the institution did not take the exercise seriously and then had to undertake repair jobs.
Auditors felt also that that some Vice Chancellors, especially if new to the post, could use the institutional audit for their own internal management needs. They would indicate privately (perhaps at the pre-meeting) `where things need to change because of a vainglorious baron (Dean)', and, `they
17
give you feeds which we are obliged to investigate'. In comparison with professional body inspections and subject reviews, it is clear that QAA institutional audit is a quality assurance process that operates across the whole institution. Not only does that increase the potential risk for the institution (an adverse verdict cannot be confined to only a part of the organisation), but as one senior institutional manager indicated, there is a strong incentive to use the process `to set an executive agenda'. Nonetheless, some auditors had difficulty operating `meta-regulation' in cases where the universities (often, but not invariably, pre-1992) had devolved extensively to departments and had quite a weak central core.
The risk of an adverse outcome may be considered low (as we have noted above, few reports deliver other than the `top' category of `broad confidence'), and institutions profess that professional body assessment and the HEFCE Research Assessment Exercise carry more funding and status implications. Yet, to an institution about to undertake an audit, the risk appears high nonetheless, and there is no guarantee that they will avoid being one of the unlucky few. Moreover, auditors agreed that, broadly, most post-1992 universities (with longstanding traditions of external quality assurance) regarded institutional audit in a more routine and accepting manner than often found in the pre-1992 category. However, many older universities in the last few years, it was noted, have introduced the panoply of quality management and internal assurance structures and units that used to be the preserve of the ex-polytechnics, feeling obliged to respond to increased student social
18
heterogeneity and rising demands for improved pedagogy, as well as displaying regulatory compliance.
Regulators, such as the QAA, also face risk in what are highly politicised environments. A number of auditors noted that institutional reports following visits adopt a language that seeks to be as anodyne as possible in order to avoid controversy and to avoid risk. The normal outcome of `broad confidence' was considered to be largely harmless (deliberately so, as it was designed for public consumption) but a little `mealy-mouthed' and somewhat meaningless from an institutional developmental point of view. It seemed to suggest that there might be elements of the institution's provision that did not elicit confidence, even if this was not the case.
On the issue of whether external quality regulation enhanced corporate management within universities, the auditors' opinions were rather mixed. For most, managerialism within universities was seen as largely deriving from other factors, to do with funding constraints and increased competitiveness in the sector. If anything, external quality assurance was perceived as reinforcing bureaucracy, documentation, systems and formalism, rather than managerialism as such.
Etiquette and civil service language are two explanations why, in the view of auditors, the rules and verdicts of the audit process are so hard to decipher by outsiders, including politicians and consumers. One auditor claimed that she had to `decipher the Report for the Board of Governors' as they could not
19
understand what it all meant and whether it was good or bad'. Interestingly, too, although QAA has had the practice of using auditors from outside the sector (such as from business or from other public agencies), this rarely occurs now on the grounds that such auditors had difficulty understanding and coming to terms with the informal rules and processes of the assessment process. There is clear difficulty in using information and assessments, drawn up and operated primarily to aid interpretative understanding within the sector, in pursuit of wider consumer and public knowledge and awareness.
Importantly, however, all auditors felt that they had acted as `teams' with a collective and individual professional responsibility to do a good job. Generally, there is little `jarring' between the institutional and QAA sides on visits, according to one. Effectively, it was the audit `team' that `trained' the individual auditors and helped to weed out the mavericks, and from which members gained considerable support and sustenance. It was judged by the auditors, however, that auditors were chosen `because they were good team players they were pragmatic and flexible'.
Given the enhanced expertise needed for institutional audit in comparison with subject reviews, it is noticeable that many auditors hold, or aspire to hold, responsible managerial and similar positions within their own institutions. Virtually all opined that the quality of learning and teaching had improved over the years, despite the blemishes, and that fresh life had been injected by QAA institutional audits into what had become a rather tired collegiality in institutions. Research and publication competitiveness, and rising time
20
burdens, including from increased administration, were felt to be contrary processes that were debilitating older collegial ideals by encouraging heightened individualism.
In the interviews with the auditors it became clear that a conventional notion found in regulatory scholarship of the external regulator `walking a tightrope' between the demands of regulation and the sensitivities of the regulated (see Moran, 2002 for an account of this and similar issues in regulatory scholarship) is inappropriate. The relationships are not perceived to be as sharp as this metaphor implies. Rather, relational closeness and a culture of peer review reduce potential friction in the audit process, although a professional distance is judged by the auditors also to apply. As one auditor remarked `it is peer review rather than walking a tightrope and there is an empathy with the institution as we are all in the same boat: the better analogy is of walking a rather wide, tree-lined avenue'. Another auditor perceived the auditors to be `as much under scrutiny from the institutional team as they do from us the relationship is cooperative, non-confrontational and friendly, and it is peer-to-peer rather than regulator-to regulatee'. Another remarked that `we are not really cross-pressured as the permanent official is not there until the end (when we are interrogated on the evidence for our judgements rather than the judgements themselves) the team runs the show; we forget QAA when we are getting on with it'.
21
Conclusion It is clear from the findings that auditors display quite a high level of support for the process of institutional audit as conducted by the QAA. Predominantly this centres on its characteristics of peer review methodology and the relational closeness of the auditors to those they regulate. However, most auditors distinguish the audit visit process (which is seen as developmental, non-adversarial and professionally undertaken by both sides, and which has an in-built form of regulatory intermediation) from the final report. This latter is regarded as `the public accountability bit' and is draped in a `civil service' language that aims to reduce controversy and risk (to the institution and to the regulator). It also leads to what some auditors perceive as often anodyne outcomes and a customary verdict of `broad confidence', which itself is seen as somewhat undifferentiated and unsatisfactory.
In some respects, tension and conflict may arise more from the issuance of the draft report to the institution than from the audit visit itself. The QAA is punctilious in ensuring that the formal visit exchanges are left to the audit team, and the permanent official only appears at the end to ensure evidence for the judgements. Consequently, often it is subsequent to the visit that we find processes of full regulatory intermediation between the full-time QAA officials and the institution of the kind that we might find in other sectors that do not operate peer review. Moreover, although a little critical of the blandness of the subsequent report process, most auditors accept its inevitability as a worthwhile price for avoiding provocation to government and inviting further intrusion that would lead to a more over-bearing, draconian
22
and non-peer review methodology based on tighter external rules and `outsider' inspectors. It is also recognised that the media acts as a form of `secondary regulator' too, and that it is important not to provide it with information that may be used to `hang institutions with'.
A noticeable theme that emerges, too, is the rise of risk in more competitive university systems (Power, 2004). Some auditors felt that institutional leaders feel vulnerable to reputational risk over which they have little control. Whatever the priority of the institutional mission in terms of research, even those institutions with high reputational standing feel obliged to engage with the learning and teaching audit process in order to insure against unwanted media and other comment that may harm their overall standing and their position in markets, such as those for high fee-paying international students.
Finally, it is clear that the external quality assurance system in England, and probably elsewhere too, has developed a set of `rules' and language that are only fully interpretable by insiders those who understand the rules of the game (Black, 2001). The general public and other outsiders, including politicians and the media, generally unversed in such rules and processes, are sometimes regarded by those operating the quality assurance system as potentially disruptive flies-in-the-ointment. Consequently, reports are written in ways that help to protect institutions and the regulator, who form a common interpretative rule community, by applying the protective jargons of the governmental bureaucrat.
23
Acknowledgements We wish to thank the Chief Executive of the QAA, and his staff, and all the auditors who took part in the research project, and to those interviewed in the three universities. We are also grateful for the assistance offered by officials in the DfES, the Better Regulation Unit, and Universities UK, and to the funding provided by the Centre for Higher Education Research and Information at the Open University.
Roger King is visiting research professor at the Centre for Higher Education Research and Information at the Open University. Previously he was Vice Chancellor of the University of Lincoln, UK, and a visiting fellow at Australian universities, and at the Association of Commonwealth Universities. He is currently undertaking comparative research on the application of regulatory scholarship to higher education governance.
Paul Griffiths is currently a faculty dean at the University of Brighton, UK, and has held various teaching and senior management posts in higher education. He has a particular interest in quality related issues in higher education and was for a number of years a member of the Council for National Academic Awards. More recently he has experience as an auditor with the QAA and also contributes to the work of the Hong Kong Council for Academic Accreditation.
24
Ruth Williams is a senior policy analyst at the Centre for Higher Education Research and Information at the Open University. She has worked on many projects related to quality assurance in higher education, including internationally. She is currently involved in comparative research on regulation and governance in higher education.
References Alderman, G. and Brown, R. (2005) `Can quality assurance survive the market? Accreditation and audit at the crossroads', Higher Education Quarterly, 59, 4, October 313-28. Baldwin, R. and Cave, M. (1999) Understanding regulation (Oxford, Oxford University Press).
Black, J. (2001) `Decentring regulation: the role of regulation and selfregulation in a post-regulatory world', Current Legal Problems , 103-46.
King, R. (forthcoming) `Governance and accountability in the higher education regulatory state', Higher Education.
Moran, M. (2002) `Review article: Understanding the regulatory state', British Journal of Political Science 32, 391-413.
Parker, C. (2002) The Open Corporation (Cambridge, Cambridge University Press).
25
Power, M. (2004) The Risk management of everything: re-thinking the politics of uncertainty (London, Demos). Quality Assurance Agency for Higher Education (2001), The framework for higher education qualifications in England, Wales and Northern Ireland (Gloucester, QAA).
Lead author: Professor Roger King, Centre for Higher Education Research and Information, Open University, c/o Telham Lodge, Telham Lane, Battle, East Sussex, TN33 0SN (email: [e-mail]).
26
Abstract The paper takes the external quality assurance of English universities and colleges as an example of regulation in higher education as undertaken by the Quality Assurance Agency (QAA). Regulatory scholarship generally has been largely disinterested in higher education and the paper applies a `regulatory lens' to higher education quality assurance. It reports the findings of a research project on the role identities and perceptions of the auditors recruited by the QAA from the academic community for undertaking institutional audits. It suggests that such a group may be regarded as `regulatory intermediaries', facing both `upwards' to the regulator, and `down' to those being regulated. As such, they have an important function in the delivery of external quality assurance regulation and the paper reports on how they mediate and understand a range of frequently conflicting pressures.
27