Abstract: The introduction of a performance assessment model based upon the measurement of merit through explicit, standardized, and objective criteria of productivity has provoked significant changes in the academic profession within the public higher education in Portugal. Given that employment security was made contingent upon obtaining adequate positive scores and promotion upon achieving maximum scores, a new institutional culture framed by precariousness and competition seems to have emerged. Moreover, as a consequence of austerity and with it the freezing of the pay awards associated with a promotion, the positive effects of excellent performance have been suppressed, while punitive measures for inadequate performance have been maintained. Based on ongoing qualitative research consisting of analysis of union position statements, interviews with union representatives, and interviews with academic staff of a Portuguese higher education institution, this article advances the hypothesis that evolution has taken place from resistance to routinization and acceptance of assessment procedures.
Keywords: Performance assessment; higher education; NPM reforms; organizational change; bureaucracy; academic profession; teachers’ union
The recent development of public higher education (HE) in Portugal has been typical of the trends associated with New Public Management (NPM)-type reforms particularly with respect to the linkage of institutional development, competitivity, and success with academic output and, therefore, application of organizational strategies that link incentives to productivity with meritocratic reward based upon individual performance assessment (PA). Since 2007, organizational change in the context of public administration reform has provoked significant changes in the academic profession within the public HE in Portugal. The generalized introduction of models of PA based upon the measurement of merit through explicit, standardized and objective criteria of productivity has been exemplary of change in this context. Given that employment security was made contingent upon obtaining adequate, positive scores and promotion upon achieving maximum scores, a new institutional culture, framed by competition – and by risk, seems to have emerged. Moreover, as a consequence of austerity and, with it, the freezing of the pay awards associated with a promotion, the positive effects of high productivity and excellent performance were suppressed. Punitive measures for inadequate performance were, however, maintained. Thus, the application of PA is particularly symptomatic of how problems involved in the implementation of NPM-inspired reforms have produced organizational change with significant impacts upon the academic profession.
This article, therefore, proposes to place the development of individual PA in the Portuguese public HE system within the context of the institutional and organizational changes associated with NPM-inspired reforms. It will address the challenges that this assessment system has placed upon academics and their reactions to it. In this context, it looks at the discourses produced by union organizations and union activists and their evolution since the inception of PA. It also presents the points of view of teachers themselves through an analysis of interviews with union activists and teachers obtained from a case study at a representative Portuguese polytechnical institution. Its analysis is based upon the hypothesis that a general evolution has taken place in unions’ and teachers’ behaviour and attitudes from resistance to routinization and acceptance of assessment procedures. Accordingly, the article begins with a look at the implementation of NPM reforms in public HE and then reviews the application of the new assessment models for teachers in the Portuguese system.
Public HE systems have not escaped from international tendencies of management reform in public administration, such as organizational decentralization and autonomy, accountability and results-based management models. Indeed, the generalized influence and rising hegemony of NPM ideology, with its presumption of the superiority of private management models, together with the external pressures of context, provided fertile breeding ground for discourses regarding the ineffectiveness and indulgence of the traditional models of administration of public HE institutions (HEIs), supposedly founded upon the inefficient logics of academic development and collegiality (Barr, 2004; Lorenz, 2007). From these it has been easy to conclude—ideologically and not necessarily empirically—for the necessity of changing its existing management models as in the rest of public administration (Amaral, Magalhães, & Santiago, 2003; Anderson, 2008; Field, 2015; Kallio & Kallio, 2014; Olssen & Peters, 2005; Pollitt, 2003; Reed, 2002) in favour of organizational decentralization and autonomy and a convergence upon a results-based model of human resources and careers. Thus, the decentralization of the system of HE was justified theoretically by equating organizational autonomy from uniform public tutelage with efficiency.
As with the hospitals in public health systems (Stoleroff & Correia, 2008), decentralization and organizational autonomy give way to “marketization” and to increasing fragmentation of the system. Decentralized organizations produce discrete development strategies, each acting as if it were a “private” entity and going its own competitive way.1 Moreover, the long-term and crisis-induced decrease in public financing of HEIs has further stimulated such competition by increasing organizational dependence upon self-generated funding (in particular from increasing receipts from student tuition as well as private investment). Thus, although attenuated by the still public character of much of the system, highly competitive educational “markets” have emerged for the “products” of HE, thereby making necessary the exercise of explicit hierarchical control over results at all levels of their operation—with the additional consequence that internal hierarchies have been readjusted and reconstituted. In this context academic productivity becomes a key element to organizational competitivity and success and, consequently, systems of individual PA were almost universally introduced with the pretext of objectively measuring the productivity of teachers and researchers.
In short, the introduction of new management models in public institutions of HE has resulted in diverse and numerous organizational changes. However, the introduction of new managerial principles into practice has often gone ahead without sufficient attention being given to the limiting conditions for their application, and they may not adequately anticipate and respond to the complexity and limited rationality that characterize public organizations in particular (Reed, 2002). On the organizational level, a particularly relevant criticism holds that competition between decentralized organizations potentially jeopardizes the institutional cooperation that should be conducive to optimal academic and scientific development. Moreover, this competition may result in less efficient use of resources from a macro point of view, thus contradicting the goals of NPM itself, namely, an increase in the efficiency of the system as a whole.
Another line of critical reasoning involves the unintended consequences of the administrative measurement of merit. In the NPM ideology optic, both scientific and teaching quality require an organizational climate that promotes, at least in attenuated form, “market” competition (Van Dalen & Henkens, 2012) that is based upon human resource management (HRM) models. This pressuposes external control and monitoring of activity is taken on by administrative structures, whose responsibility it is to supervise and verify their production through standardized measures and instruments. There is, however, a risk in that such control procedures are located within a conception of merit conditioned by quantity rather than quality. Indeed, some authors argue that such quantified criteria of assessment result in the over-valorization of quantity over quality (Kallio & Kallio, 2014; Olssen & Peters, 2005; Stelmach & Von Wolff, 2011), to the extent that evaluation has become constrained by standardized scores based upon rankings and ratings (Lane, 2010). These may encourage conformism since publication norms tend to promote standard work rather than critical innovation (Coulthard & Keller, 2016; Sousa, 2011), potentially jeopardizing objectivity and integrity in publication choices (Fanelli, 2010) as well as individualism in research (Sousa, 2011). On the other hand, dependence upon sources of self-financing may constrain academic organizations’ decisions regarding their teaching and research to cater to the educational marketplace, leading them to approximate educational demand (Van Dalen & Henkens, 2012). Survival in an increasingly competitive and uncertain market thereby transforms ratings and rankings into the instruments through which HEIs measure merit and are themselves evaluated for merit (Stelmach & Von Wolff, 2011). Moreover, marketization may risk the independence of academic and scientific research and publication since projects, and the interpretation of results can be influenced by the priorities of private interests (Coulthard & Keller, 2016). These dynamics are a strong pressure on teachers and researchers and have a strong effect upon their motivations (Anderson, Johnson, & Saha, 2002).
Although the advocates of NPM and HRM may like to juxtapose two distinct models, one being a new more efficient flexible, meritocratic model with a preceding bureaucratic-administrative model, in practice the new models have tended to add layers of regulations upon increasingly standardized procedures and measures in a strongly bureaucratic manner. The devolution of powers—which sought to reduce bureaucracy in HE management, devolving greater autonomy to the organizations— has paradoxically lead to the creation of new bureaucratic control mechanisms due primarily to the new demands for accountability (Pollitt, Birchall & Putman, 1998) and explicit external supervision of results (Santiago & Ferreira, 2012). The establishment of accreditation processes (Amaral et al., 2003; Reed, 2002) as well as of procedures for PA based upon standardized, quantified criteria (Kallio, Kallio, Tienari, & Hyvönen, 2016; Vicente & Stoleroff, 2016) are two examples of such mechanisms. These lead not to the elimination of bureaucratic control but rather to its displacement within systems and individual organizations.
On the other hand, the introduction of managerial models within HE is frequently approached, in both the critical literature and professional discourse, as a rupture with a preceding supposedly democratic and collegial peer-based model of evaluation (e.g., Macfarlane, 2015). Collegiality here seems to refer to governance models dominated by peer relationships and in which decisions are taken on the basis of professional and scientific criteria in representative organs by colleagues who are normally elected to positions to serve roles for a limited period (Amaral et al., 2003). The question is whether this criticism is based fundamentally upon an idealization of a preceding organizational pattern purportedly characterized by looser professional regulation and is now, in hindsight, being remembered nostalgically as democratic.2
In order to get closer to discourses regarding change, in the following, we try to synthesize two different models (see Table 1) based upon an opposition between ideal types of models of administration that may hypothetically be extracted from discourses about governance models in public HE: a democratic-collegial model and a managerial-bureaucratic model (Bruckmann & Carvalho, 2014; Kallio & Kallio, 2014; Olssen & Peters, 2005; Santiago & Carvalho, 2008; Ter Bogt & Scapens, 2012; Van Dalen & Henkens, 2012; Vicente & Stoleroff, 2016). Although this opposition may appear somewhat simplistic, it does represent opposing discourses regarding the relevant changes in the institutions.
Table 1
Ideal types of administration models in HE
|
Democratic-collegial model |
Managerial-bureaucratic model |
|---|---|
|
Institutional Management |
|
|
Emphasis on the code of public administration/civil service and general framework for public HE system; |
Emphasis on internally generated statutes and regulations derived from a general framework for the public HE system; |
|
Dependence on public financing through the state budget; |
Mixed financing through the state budget, student tuition and self-generated receipts; |
|
Orientation towards academia; Governance based on the presumption of the common good and public interest; Aloof from the community, independent of companies; |
Orientation towards the educational market; Competition for students; Governance based upon market logic: with students as clients, teachers as service providers; Proximity with community understood mainly as “the economy;” |
|
Democratic participation in organs of governance; Collegiality; |
Top-down formal leadership. Responsibility and power located in Directors of Units (Faculties, Departments, Centers); |
|
Democratic-collegial model |
Managerial-bureaucratic model |
|
Organizational Structure |
|
|
Centralization at macro and supra-organizational level (i.e., Ministry); Underdeveloped internal structure; |
Relative organizational autonomy and managerial decentralization; Developed internal structure; |
|
Relatively flat organizational structure; The primacy of collegial organs; |
Pyramidal organizational structure Importance of centrally emanated managerial hierarchy (from Rector to Deans and Chairs) ; |
|
Human Resource Management |
|
|
Non-explicit self-control of outputs of academic production; |
Explicit external control of academic production; |
|
Remuneration according to category, scale, service; |
Remuneration according to category, scale, service and bonuses for productivity; |
|
Administrative management of academic staff; The primacy of the civil service statute; |
Hybrid regimes of human resource management, including civil service regime and private law contracts; |
|
Stability of academic employment in accordance with public service employment regime. |
Tenure and employment stability linked to performance assessment. |
While reforms in public HE have forged ahead, supported by NPM’s more or less successful hegemony, and pushed on under the pressure of austerity, the theory and its implementation have been subject to significant practical professional resistance, as well as the academic criticism, especially as they have had visible consequences for HEIs and teaching careers.
Given what has been at stake for teachers in public HE with the introduction of a purportedly meritocratic system of PA that includes awards and penalties, it should not be surprising if its implementation has led to significant adaptations and adjustments in their behaviour, such as the adoption of the management language (Carvalho, 2012). On the one hand, this applies in the positive sense, when the interested parties – especially when in possession of favourable resources – perceive opportunity and gain a hold on the game, foreseeing advantage and reward. However, this is even more valid in the negative sense since, employment security, for example, becomes dependent upon evaluation (Courtois & O’Keefe, 2015), especially in the polytechnic sector (Santiago & Carvalho, 2008). All of this further promotes an increasingly competitive environment and, yet, may lead, in particular, and ironically, to complacency.3 Such complacency may then lead to routinization of the new practices.
Fixed procedures fix expectations, and such routinization is partly what secures bureaucracy (Merton, 1957), permitting work to go ahead as planned. With time, routines and their underlying rules are internalized by the interested actors (Guidens, 1984) and their interests, such as promotion, will become dependent upon conformity—which may also hold back risk and innovation (Merton, 1957), which is particularly problematic in academics and science. In this light, the system of individual PA becomes a bureaucratic instrument for the control of results, and the simultaneous effects of expectations of reward or avoidance of sanction will be dependent upon the degree of conformity with the norms and behaviour defined by the management system. Thus, such routinization of behaviour may and should have significant impacts on really-lived professionalism. What we may hypothesize therefore is a dualization of professional strategies as a function of the individual academic’s perception of opportunity or disadvantage in the situation, but altogether a breakdown of resistance through routinization.
In the rest of this article, we will report upon research conducted in relation to our hypotheses, that is, the reflection of this process in union representatives’ and teachers’ discourses on organizational change and the consequences of PA.
In Portugal, until 1972 there were only four public universities (Lisbon, Coimbra, Évora and Porto) and a small, recently created Institute of HE (ISCTE in Lisbon) (Torgal, 2012). Following democratization in 1974, HE rapidly expanded in response to a massive increase in demand (Amaral et al., 2003; de Almeida, 2012). At present, the public system consists of 39 institutions (13 universities, a university institute, 15 polytechnical institutes, and five schools that are not integrated within either universities or polytechnical institutes (General Direction of Higher Education, 2017). These institutions are under the tutelage of the Ministry of Higher Education and Science. They all may confer Bachelors and Masters degrees while only the universities are entitled to confer doctorates.
During the dictatorship of 1933-1974, the regime tightly controlled the universities allowing them very little autonomy. Rectors were appointed directly by the government, and the organs of governance were hardly representative of the faculty (de Almeida, 2012). The democratization of the country resulted in concomitant democratization of the governance of HEIs, approximating a democratic-collegial model and obtaining a high degree of academic liberty and scientific and pedagogic autonomy. There was, therefore, a significant delegation of decision-making to the institutions themselves, extending even to financial and administrative matters, although many fundamental decisions, such as the creation of courses, the number and distribution of students to be enrolled or the creation of teaching lines for hiring, remained subject to ministerial approval. Nevertheless, by the beginning of the 1990s, managerial principles, at least at a rhetorical level, began to make their way into a political debate and effective practice (Bruckmann & Carvalho, 2014), preparing the way for important changes to the traditional model by the 2000s (Amaral et al., 2003). Subsequently, and in the wake of OECD recommendations for publicly financed institutions to operate within the private market (Kauko & Diogo, 2011), their administrative and management models were altered relatively rapidly along the lines of NPM (Bruckmann & Carvalho, 2014; Diogo & Bruckmann, 2015), with the passage of the Law Nr. 62/2007 creating the Judicial Regime for HEIs (RJIES).
The main changes in the new framework produced a top-down structure of power and authority and included numerous derivative changes in the organizations of HE (Araújo et al., 2014; Diogo & Bruckmann, 2015; Kauko & Diogo, 2011; Vicente & Stoleroff, 2016) as can be seen in Table 2.
Table 2
Main changes introduced by RJIES
|
Term |
Description |
|---|---|
|
Governance |
The concentration of decision-making power within a reduced number of organs and, especially, the strengthening of the powers of the university rectors and presidents of the polytechnical institutes; A significant decrease in the number of members constituting the government structures of the institutions, namely in the General Councils and the Management Councils; The introduction of external stakeholders into the management organs; The introduction of the modes of a selection of members of governing bodies, making it possible for some to be designated or co-opted from above; Professionalization of certain managerial functions and administrative leadership (supervisors, directors and the like). |
|
Management |
Creation of an option between two institutional models: the public institute regulated by public law and the foundation regulated by private law; Introduction and regularization of reporting and other instruments for accountability; Viabilization of formalized cooperation and consortia between institutions. |
|
Funding |
Linking institutional budgets to productivity through financing contracts with the state; Possibility of autonomous fixing of tuition, including the possibility of tuition increases. |
|
Human Resources |
Transformation of the civil service status of teachers and administrative staff to public employees; Introduction of Performance Assessment of academic personnel. |
Academic and scientific endeavour is necessarily subject to evaluation by peers, and academic careers are therefore inherently meritocratic (Ter Bogt & Scapens, 2012). Assessment of academics’ work, therefore, involves evaluation procedures that, to some degree, are familiar to the field and expected. However, previous systems of assessment in the Portuguese public sector tended to derive from bureaucratic systems of personnel administration review derived from the statutory conditions of the civil service employment contract and based upon loose, qualitative assessments made by supervisors—and even these were pretty much informally applied in academic contexts or not at all. In NPM-inspired discourse, this internal control of performance was over-determined by the job security attendant upon civil service status. Security purportedly functioned as a disincentive to productivity and excellence and weakened the meritocratic dynamic conjured up by the familiar expression publish or perish, perversely inducing accommodation and productivity fall-off, allowing academics who had acquired civil service status to rest upon their laurels (Vicente & Stoleroff, 2016). In this sense, and in the optic of NPM ideology, both scientific and teaching quality were in need of an organizational climate that promotes competition based upon human resource management models in which external control and monitoring of activity is taken on by administrative structures, whose responsibility is to supervise and verify their production through standardized measures and instruments (Vicente & Stoleroff, 2016). This NPM view of the positive function of meritocratic assessment was largely transported to a revision of the Career Estatutes of Teachers and Researchers in HE that necessarily followed the passage of the RJIES.
The revision of the Career Estatutes took place in 2009. Amongst other alterations, the revision entailed the institutionalization of evaluation based upon the assessment of pre-determined quantitative objectives distributed among the four areas representative of teachers’ activity: teaching, research, participation in organizational management bodies and services extended to the community. PA measures were to be put into effect through the elaboration, operationalization and implementation in each institution of a regime of regulations for PA (designated a RAD). The coordination of the process was to be led by the Scientific Councils in the universities and by the Technical-Scientific Councils of the Polytechnical Institutes and, according to the law, the RAD for each institution was to be initially negotiated with the teachers’ unions.
The decentralization and autonomy attained by the institutions led to diverse regimes throughout the system. Nevertheless, the following characteristics are common to almost all:
a certain uniformity in the weighting of scores amongst the four areas of teaching activity, with a tendency to value teaching and research activities;
similarities in the items of assessment that had been operationalized for each of the areas of activity;
a certain standardization of the criteria for evaluating teachers irrespective of their professional category, scientific area, degrees or seniority;
standardization of the scales for assessment results such as three or four levels for positive assessment and only one negative level;
a prevalence for assessment over three-year periods;
some sort of inclusion of students’ evaluation of teachers, usually based on annual or bi-annual questionnaires;
the presence of some form of self-assessment by teachers through reporting of activities or registering of results and its delivery to an organ (such as a department chair) with responsibility for its validation.
Diversity in the institutions’ regimes entails aspects such as the existence of quotas for the proportion of teachers who may obtain the highest gradings per unit or the application of assessment to part-time teachers.
Legitimated by the economic and financial crisis that swept through several European countries, such institutional and organizational reforms were, more recently, followed by austerity, which can be said to have simultaneously reinforced and distorted the logics of change, constraining the chances and conditions necessary for their consolidation (Lodge & Hood, 2012; Stoleroff, 2013). On the one hand, austerity increased the competitive pressures upon organizations for survival. However, austerity also threatened professional alliances, that would have been crucial for the transition to the new models, and jeopardized the levers and positive incentives for continued reform. If it was already problematic to introduce changes in the management of academic work and careers, austerity posed complex challenges to the meritocratic reconstruction sought by the reforms and, in particular, by the introduction of individual PA.
The austerity policies introduced in Portuguese public HE (Teixeira & Koriakina, 2016)—as in the rest of the public sector—ended up limiting the implementation of the PA system. Budget restrictions promoted the emphasis upon measurement of productivity, particularly, as a criterion for promotion (as promotions across the system involve significant costs). However, these budgetary restrictions have been invoked to suppress or freeze the positive effects, such as promotions and bonuses, for superior assessment. As a result, PA under austerity was transformed into a fundamentally punitive system, or at least could be perceived as such. This promoted, above all else, strategies for avoiding insufficient results in evaluation or, especially amongst those who become tired of the competition, strategies for sufficing and behaviour based upon just getting by, since better results produce no tangible career or immediate material advantages.
We are, therefore, faced with circumstances which permit us to advance hypotheses considering resistance and acceptance of this major reform in the academic profession and its career: an evolution has taken place from resistance to innovation to routinization and acceptance of assessment procedures 1) that is reflective of the underlying differentiation of academics to the new professional stratification processes in accordance with their varying adaptive capacities and that 2) is reflected in the discourses of teachers’ professional representatives.
Based on our research, we will now examine the perspectives of the teachers’ unions and their activists regarding the assessment system implemented in Portuguese public HE. We look at their evolution from the start of the process until the conclusion of the first cycle of evaluation. We will then report on the viewpoints of teachers themselves, taken from a specific case study, regarding the rules and process applied in their institution as well as assessment in general.
There are two representative union structures in public HE. The National Federation of Teachers (FENPROF) was formed in 1983, federating various regional teachers’ unions that were organized in the aftermath of democratization, aggregating teachers from all levels of the educational system. It is affiliated in the national CGTP confederation. The National Union for HE (SNESup) was created in 1989, following a wave of dissatisfaction with the performance of FENPROF in the negotiation of the Teachers Career Statute and the resulting aspiration of many teachers to have a union dedicated solely to HE (Lourtie, 2015). SNESup is an independent, national union. Each of the unions submitted opinions regarding the RAD conceived for each institution, explaining its position.
With respect to the positions of the unions, in addition to analysis of union documents, we interviewed a set of union activists in order to get a more general sense of the organizational dynamics involved in the implementation of the evaluation system nationally and in order to complement the information collected through the case study of a specific institution. We interviewed 18 union activists from diverse institutions (15 activists from SNESup and three from FENPROF).4
Table 3
Identification of union interviewees cited in the analysis
|
Interview |
Union |
Scientific area |
Holds leadership position |
|---|---|---|---|
|
A |
SNESup |
Social Science |
x |
|
B |
Fenprof |
Engineering |
x |
|
C |
SNESup |
Mathematics |
x |
|
D |
SNESup |
Social Science |
x |
|
E |
SNESup |
Sciences |
|
|
F |
SNESup |
Social Science |
x |
|
G |
SNESup |
Law |
|
|
H |
SNESup |
Sciences |
|
|
I |
SNESup |
Engineering |
|
|
J |
SNESup |
Management |
|
|
K |
SNESup |
Social Science |
x |
|
L |
Fenprof |
Sciences |
x |
Finally, as part of a case study of a HEI, semi-directive interviews were also conducted with a sample of 30 teachers (N=180) from a representative polytechnical institute that had implemented individual PA biannually since 2011. The option to do a case study in a polytechnical institute was due to an interest in developing sociological research specifically in this under-studied sub-system of HE in Portugal and an interest in researching the particularities in the application of an assessment regime in this sub-system, which is characterized by a high degree of precarity in employment relations (a higher proportion of teachers on non-permanent contracts), less experience in scientific research activity and significantly lower academic credentials among its teaching staff. The specific institution was chosen due to the opportunity afforded by the approval of the institution’s administration. The choice of using the interview method was adequate towards the goal of carrying out an exploratory and intensive analysis of the positions of the main actors involved in the assessment process, taking into account that, while research has been done on other measures of NPM reform in the academic context (e.g. Amaral, Tavares, & Santos, 2013 ; Bruckmann & Carvalho, 2014; Carvalho & Videira, 2017; Santiago & Carvalho, 2008; Santiago, Carvalho, & Sousa, 2015), to our knowledge, there do not as yet exist other studies of this object within the panorama of Portuguese HE. The sample of participants in the study was constructed intentionally based upon the following criteria: discipline, contract status, professional category, employment regime (full or part-time), and academic degree. The interviews were conducted between November 2015 and March 2017. The sample was diversified by both academic discipline (law, accounting, management, finance, languages, and computer science) and type of contract (full-time with exclusivity, full time and part-time).
This analysis will focus on four aspects of the issues at hand: 1) the positions and conduct of the teachers’ unions in relation to PA; 2) opinions of the interviewed union activists; 3) the opinions of the interviewed teachers regarding the implementation of the new models of PA applied in their institution; and 4) the possible sources of accommodation and acceptance behaviours with regard to PA.
An analysis of the unions’ positions is indispensable for an understanding, at the least, of discourses around the issue.
Analysis of the unions’ position papers leads to the conclusion that both FENPROF and SNESup, on the one hand, considered, at least officially, that the implementation of assessment would be inevitable due to the competitive and meritocratic character of the teaching and research careers and, on the other, challenged the legal basis of the system alleging, in a similar fashion to the critical sociological arguments, that it posed serious threats to academic professionalism, as can be understood from the following excerpts.
We do not refuse evaluation but consider that an exclusively individual evaluation, according to the models that have been proposed by the various institutions [based upon the quantification of results], is not justified,… since it will produce competition, fear and the destruction of any notion of collective work. (Position of SPN/FENPROF on the regulations of performance assessment, March 2010)
[O]ne of the fundamental tasks of our careers is evaluation: all of us evaluate, and we are, always have been, evaluated.… Before evaluating it is necessary to know what evaluation is for, what are the intended goals to be reached. (SNESup, Debate on Performance Assessment in Higher Education, 2010)
One of the most criticized principles was the quantification of teachers’ productivity (according to the four fields of evaluation: teaching, research, managerial roles, and extension) since it leads, in their view, to a decrease in academic autonomy and a weakening of professional powers as well as to a “proletarianization” of teaching and research work, subjecting it to managerial criteria for an assessment defined externally and hierarchically:
[m]any academics publish because they desperately need these publications to hold onto their job, so they can continue to research or be promoted and earn more money.… No serious and honest system of assessment can leave out a rigorous and thoughtful analysis of the content of the work of an academic,… which is contrary to the idea of permanent assessment that is based upon metrics, which, for its part, encourage bad practices. (SNESup, on Performance Assessment in Higher Education, 2010)
The analysis of the discourse of the activists in the first round of interviews revealed a significant variety of positions in relation to the system that was implemented, even amongst members of the same union organization, varying between resistance and full acceptance, as is demonstrated in the following excerpts.
Three of the interviewed activists expressed their rejection of the system of evaluation following the alteration of the teachers’ career statute without recognizing any value for the profession.
In contrast with the previous views, the majority of the interviewed activists recognized the importance of performance assessment for the teachers’ career, accepting the existence of a system geared towards this end. Nevertheless, they expressed disregard for some of the rules of the system as implemented as well as the manner in which it has been implemented in various institutions, such that their position is to be considered as of partial acceptance.
We understand that the existence of performance appraisal is very important.… But, one thing is to say that, and another is to say that any appraisal is ok. It is not that! There are principles that have to be followed in the assessment. (B)
I think that these regulations for performance assessment nowadays are indispensable. How can we assure the control, verification and validation of the work of others if we don’t have these tools? … at the level of putting them into practice the things end up with worse results.… These regulations always have a tendency that is for a certain uniformity. (D)
Activist E, on the other hand, expressed total acceptance of the implemented assessment system, stating that he had participated (as a teacher and not a union representative) in some of the phases of the conception of the assessment regime in the institution where he works. This participation may have contributed to the interviewee’s acceptance of the rules applied in his institution.
Before the new RJIES and the new teaching career statute, in this institution, there was no evaluation.… My opinion is that teachers should be evaluated and, therefore, I think it’s positive to introduce assessment.… I agree with this assessment in the form in which it is taking place … by objectives. (E)
The second round of interviews of the activists revealed a general posture of accommodation or acceptance in discourse, even amongst the present leaders of these organizations, as can be read in the following examples:
Performance appraisal right now stands more for conformism [on the part of academics] than for something else.… The Union also has been affected by that.… The theme of evaluation took up a lot of [time] in our board meetings. Now it no longer does. (A)
We could also detect outright acceptance as expressed in the interview with B:
We know what was the main reason [for P.A. implementation]—to prevent people to progress [in the career], which is motivating because it stimulates people not to rest. (B)
With respect to the remaining union activists, the results seem to point towards an evolution of positions, from initial fear with the introduction of assessment to either accommodation or acceptance following its application. This observation is reinforced by the positions expressed by even some union activists, such as E and F.
The 30 interviews conducted in the case study also revealed a diversity of positions amongst teachers in relation to the system of assessment introduced in their institution, similarly varying between resistance (Academic 1), partial acceptance (Academic 2) and acceptance (Academic 3).
People are not measured only by numbers. There are multiple dimensions through which an academic … can be evaluated and not only on quantitative terms. And this [system] is purely and systematically quantitative! (Academic 1)
The quantitative part of assessment of teaching to me doesn’t seem inadequate. It seems to me that they should make the criteria between evaluators uniform. (Academic 2)
I don’t see any inconvenience in the quantitative approach.… It is a way to guarantee that performance appraisal is more objective. (Academic 3)
The posture of accommodation/acceptance of the assessment system, observed in the opinions of the activists, is also shown in the discourse of some of the teachers interviewed in the case study, as is exemplified by Academic 3, a full-time teacher, with a fixed term contract who is not on permanent staff:
At the beginning, there was a little tumult, but then we saw how [the appraisal system works], the criteria, and since then it has been not a big problem. (Academic 4)
This result corresponds with our hypothesis, showing that the perspectives of the teachers evolved from initial resistance and fear to a position of accommodation or passive acceptance. Confronted with this evidence, we think it is important to identify the main causes of this phenomenon, through the analysis of discourse of all the participants in this study as described in the next point.
In the course of the interviews with union activists and the participants in the case study we identified various phenomena which, in their opinion, constitute possible causes for the accommodation and/or acceptance of the system of performance assessment. These will be described in the following.
The implementation of evaluation did not fulfil the expectations for positive effects—rewards—for the highest levels of performance, namely promotion and career advancement.
Although academics have been evaluated, there were no positive effects. That is, teachers with high enough scores to allow them to progress were not promoted. (C)
Although this phenomenon, derived from austerity, may lead to discouragement among teachers, leading them to question the ends of such assessment (Marsden, 2004), it may also serve to induce accommodation to the system in operation.
The representatives of the teachers indicate that colleagues could not understand the benefits of the implementation of a system of assessment, but that this question is not present in the immediate concerns of the teachers or the unions themselves.
In addition to the above, PA is coming to be seen by teachers as a mere, though mandatory, administrative procedure, that has already been discredited by various actions.
The routinization of behaviour associated with the implementation of the system of PA, which is manifested in the assimilation and internalization of expectations, is indicated by some of the interviewed as a factor promoting accommodation and reducing apprehension.
[Academics] have incorporated the mechanism and today they are generally prepared to fill in evaluation forms. (F)
The results also revealed the emergence of new practices and the acceptance of a new language of management (Carvalho, 2012), such as the frequent counting of the number of publications, the strategic selection of journals, in accordance with the scores attributed to them in the assessment regulations, to which to submit articles, the concern with taking on tasks that bring points, amongst others.
Another factor pointed out as a motive for accommodation to the system has to do with the relative absence of differentiation in the grading attributed to teachers following the application of the assessment procedures. This may create the perception amongst the assessed that competition amongst peers is less acute than it may actually be.
The results of the research reported here point to heterogeneous perspectives in relation to the implementation of the present system of PA of teachers in public HE. We repeat, this diversity of opinions is noticeable even among the union activists (themselves academics of course), whose organizations’ initial official positions were fairly critical of the introduction of the procedures. This may demonstrate the adoption of diverse strategies on the part of teachers and researchers in order to deal with the changes introduced to the profession (de Bruijn, 2002). However, the attenuation, until now, or even the absence, of positive effects for higher performance - due to austerity measures, does not make it easy to understand the purpose of such a system of teacher assessment (Behn, 2003), which is, as a result, perceived by many of those involved as a mere administrative procedure, required by law, but with not much credibility. With the institutionalization of the system, teachers’ and even the union activists’ reactions seem to have evolved from initial fear and rejection to routinization, which has in turn lead to the assimilation of the rules and their apparent integration into the expectations of daily academic life. In this context, the fulfilment of rules becomes a focal point of behaviour, as Merton (1957) pointed out in his seminal analysis of bureaucracy, taking precedence in relation to the substantive results. In a sense, therefore, PA becomes transformed for many into an instrument to assure continuity of employment, the maintenance of a status quo in an increasingly competitive and unstable environment, in which there are only negative effects for insufficient performance. At present, we still cannot project to what will be the reactions of the academic community to a post-austerity situation where positive effects for high performance are added to the negative, especially if such a situation brings quotas for the superior grading.
In light of this analysis, the application of systems of PA in public HE, based upon quantification of academic productivity through objective criteria and standardized instruments, seems to have become a routine practice in everyday life of HEIs and their academic staff. Therefore, the results of this institutionalization may not correspond to the ends that were originally intended for this management strategy, that is, as an incentive to excellence and as a compensation for merit. In other words, if despite this, academics continue to strive to show their merit and excellence in their work, for now, it is not principally due to this HRM device.
In spite of the relevance of the results of our research, the fragmentation of the assessment process (due to the autonomy of each institution in elaborating its model and regulations) and the different stages at each of the 39 institutions in the implementation of the system, there are important limits to the generalizations that can be made. For this reason, it would be important to engage in further research involving comparison amongst representative institutions within the two sub-systems of Portuguese HE.
This work is a part of a research project funded by Portuguese Foundation for Science and Technology, Public Institute (Reference: SFRH/BD/82126/2011).
[i] Rhoades & Slaughter (2004) refer to this phenomenon as “academic capitalism.”
[ii] Anderson et al (2002) argue that the term collegiality is abused as an evocation of an imagined climate of harmony and intellectual collaboration that they refer to as contrived collegiality, which in effect can be used as a counter-narrative or defensive tool in resistance to change (Fullan & Scott, 2009). Taylor (2008) argues that the fixation with collegiality is in part the result of a state of spirit and sense of loss that brings academics to reminisce upon a supposed (pre-NPM) golden era, thereby imagining that a previous collegial model has been presently supplanted by managerialism.
[iii] Complacency arises, in accordance with Anderson (2008, p. 264), when, given the impossibility of avoidance, academics may accept a situation pragmatically and strategically, even if minimally, which does not signify agreement.
[iv] The selection of the interviewees was made with the assistance of the unions. The first interviews were carried out between October 2014 and October 2015 and, in order to analyse the evolution of union and activists’ positions, a new round of interviewing was carried out between October 2016 and February 2017 with the same activists who had been previously interviewed.