Workloads #1 — Reforming our Departmental Workload Scheme

This is the first article in a series of three this year on workloads in academia and what we might do about them. This article deals with workloads at the department/school level. The next article will deal with a scheme for managing my own personal workload, and the final in the series will deal with workload management at the ‘whole of university’ level, and the economic thinking around this, as well as how the new UNSW EA would lead to changes in our existing school workload model (for good or for ill).

My aim in all of these articles is to push for systems that are more quantitative, more open/transparent and more fair, as this is a rarity in any university context, and perhaps even ultimately a unicorn at present.

In a post in February 2020 I wrote about a quantitative workload model, generally called the WorkLoad Unit or WLU scheme, that my school had been running for well over a decade with small modifications along the way. If you haven’t read that, it might be useful context for this article, as this post is about revisions to that scheme made later that year. In October 2020, I chaired a small committee in my school to look at a major overhaul for this scheme. The committee consisted of myself, the school manager, and three relatively new academics in the school to get some external perspectives and insights on the scheme we were running.

Issues we sought to address

At the time, there were five major issues we wanted to address:

• Investigate the bifurcation of load trajectories across the staff that led to some being inevitably overloaded (incurring progressively more negative balances over years) and others inevitably underloaded (incurring progressively positive balances). A factor in the latter was that large amounts of low-WLU-value teaching, what we will later call ‘small rocks’, needed to be allocated to some staff to achieve their load, and this caused significant inequities in the number of face-to-face hours that need to be spent to make the annual load.
  
• Investigate other sources of potential inequity and whether they might be resolved, e.g., by changing how tasks are counted or weighted most notably research contributions.
  
• Review the admin allowance structure to check for effectiveness and align values to a common system (some amounts were allocated back in 14-week semesters, some in 10-week terms).
  
• Look at whether the workload scheme was compliant with the current Enterprise Agreement (EA) at the time, and if it wasn’t, make changes to resolve that, since stipulations about workload schemes appear in that document (e.g., Section 24.2 of UNSW EA 2018).
  
• Look to some modernisation both in structure since the last review and whether the process might be streamlined/automated to reduce time-cost of administration.

The latter of the five led to a more automated model running in Excel, which I’ve included a stripped down version of here. I’ll talk a little more about how this model actually works after dealing with the major revisions we ended up making, but I will make an important note prior to that on how we quantify workload, and how I’ve come to think about workload management having been through this process.

How we quantify workload — getting the thinking right

Our school workload is counted in a thing that we call the workload unit or WLU. Anyone taking their first look at the scheme asks the obvious question, namely: What is the WLU conversion rate in hours?

The inevitable answer I give, which never satisfies, is that the conversion from WLU to hours is ‘of order of integers’, which means that 1 WLU = 1 hour is definitely not precisely true and we can’t split hairs between, say, 1.5, 2 and 2.5 hours in WLU necessarily either, but we certainly can see order of magnitude differences, e.g., 0.1, 1, 10 and 100 are clearly very different, and that both matters and is helpful to improving equity and workload equality.

There’s ultimately three reasons why it’s exceptionally hard to have a precise and constant WLU-to-hour conversion rate. The first is that different people take different amounts of time to do different tasks, and that varies between not just people but also tasks: Person A might be faster at Task A and slower at Task B, while Person B might be faster at Task B and slower at Task A. The second is that the same notional task can take different amounts of time sometimes, and even that’s not uniform, e.g. writing Lecture 3 in Course A might be quicker than in Course B, but it might be the other way around for Lecture 5. Lastly, there’s always more work to do than there are people to do it — if we account in real hours, a whole slice of work in the department just isn’t going to get done as there aren’t enough free hours to do it — This is the major issue of my third article in the series, so I’ll save this discussion for there.

There’s one other essential aspect of thinking about WLU, and that’s its nature as a currency in an economy. Just like money, WLU is a measure we use to link available resources, e.g., staff time, to outcomes, e.g. classes taught and admin roles performed. A way many initially see WLU (or money) is as their reward/remuneration for doing a task. At first sight, this is fine but it inevitably raises some issues if you let people mandate for themselves how long they will spend on a task to do it to whatever level/quality they choose and then demand payment for it. The best way to explain this is with a short parable.

Imagine you hire a builder to build you a backyard shed, you provide no further specifications, i.e., they have ‘academic freedom’, not even a budget limit — this is effectively how we allocate tasks in academia, right? You then return to find the Taj Mahal standing beside your modest 2-bedroom home and the builder standing there with a bill for $500 million dollars for you to pay. This is clearly not a good outcome for you as the home-owner. The smarter option here is to tell the builder that your budget is $1000, even if nothing else, because then the Taj Mahal is immediately off the table, and so is the shack with gold-plated zincalume. Instead you are going to get what can meaningfully be provided in your $1000 budget. Whether that’s sufficient and fit for task is a problem I’ll return to in my third article, but at least here you’ve done one thing, which is set a budget limit, which then sends a message that you don’t want the job overdone, and if that message is missed, it eliminates an incentive to massively overdo the job at least.

In my opinion, this shift from reward to expected resource allocation is vital to ever having a fair and functional workload scheme. A workload scheme exists to manage how you deploy resources as an oragnisation/team, and the allocations then act as a ‘price signal’ to individuals on how to divide their 1610 hours a year between those tasks. Some would say that’s impossible if there’s no precise WLU-to-hour conversion, but is that really true? An economy doesn’t work like this, otherwise the state would specify the exact price of every good in the economy from an algorithm, i.e., it would be a communist command economy on steroids. Instead, in a real functioning economy the prices float and there are few ‘rules of thumb’, one of them is that the relative pricing is sensible, e.g., a house costs more than a car, which costs more than a tv, which costs more than a pizza. Note that all modern currencies are fiat-currencies, which means the value of a dollar is no longer tied to a gold standard in the same way that a WLU isn’t tied to a time standard of exactly one work hour, and those modern currencies (largely) work fine because ultimately what matters is the relative values and the signals they send (more on this in the third article).

From this view, one should really consider a functional school work-load scheme as like a fiat currency for the school’s time-resource economy. The scheme’s job is to send price signals to its participants about the relative values of things that need to be done. It then takes all the things that need to be done, and divides them up between all the people available to do them. The key then really becomes the equality of that division process, which I guess one could consider to be something like a departmental Gini index. And just like in society, when the Gini index is low, your society functions well, and when it gets high, the likelihood of civil war is high. So the job of a school workload committee (or executive) is to design that fiat currency to minimise the Gini index and send the right signals to staff on how to balance the their personal time allocations to maximise the productivity of the school as a unit, ideally without the cost of managing that currency becoming ridiculous.

One last comment regarding WLU-to-hours convertibility. To the extent that this is an issue, this is an issue above the school rather than in the school. The only thing this conversion affects is the balance between a) what the school as a whole needs to do, and b) how many staff the school should have to do that, and both of these are Faculty/Central matters. I’ll come back to this in my third article, as it needs some insights from my second article, and the rest of the context of this first article to fully make sense of the whole argument. Ultimately though, WLU-to-hour convertibility only truly comes with a three-level work-load system. While there’s only one level it’s impossible to achieve exact convertibility (without massive problems in any route to achieving it), and even then, fixed-rate precision will never be achieved — it’s too prescriptive to ever have the flexibility to function in reality (see discussion of WLU noise later on).

Things we found as drivers for change to our WLU scheme in 2020

There were three significant issues we found that ultimately drove some very major reforms of the scheme:

• Education-focussed roles were poorly included: The original workload scheme was designed around traditional teaching & research (T+R) staff with the new education-focussed (EF) staff included ad hoc once we had our first EF in the school. The existing scheme made interpretation of load difficult due to the vastly differing time allocations between T+R roles (notionally 40% research, 40% teaching, 20% admin/service) and EF roles (notionally 80% teaching and 20% admin/service), which resulted in EF staff with an effective load of 166% FTE relative to T+R staff (i.e., they were implicitly assumed to work 66% more hours than any other academic in the school despite being on an equivalent contract — 1610 hours/year = 46 weeks at 35 hours/week), before accounting for research (see below).
  
• Research is a massive equity issue: This was a long-standing problem that was exacerbated but not caused by the EF issue above. The existing scheme allocated a capped reduction in workload based on a consideration of external grant income, publications & postgraduate supervision. We did some modelling and found that of the 26 academics in the scheme in 2019, 23 were T+R and 3 were EF with a range of fractional loadings to account for start-up for new hires, sabbatical, etc. Ultimately there was a strong skew in the research deduction (100 WLU max.) such that EFs and T+Rs as cohorts attained 2% and 94% respectively of the maximum possible deductions available.
  
  There are some significant implications of this. First, for an un-deducted 100% load of 484 WLU, a full T+R position achieving the maximum deduction (6 of 23 T+Rs – all male at Prof. or A/Prof. with >5 years on staff) had a load of 384 WLU while the EFs had a load of 797 WLU, corresponding to an effective EF load of 208%. Note that this is with a capped reduction at around 20-25% for research, were this uncapped like in many departments the inequality would be even more dire.
  
  Second, of the deductions not obtained by T+R staff, 93% were by new hires — essentially we were punishing our new staff simply for being new with loads that made it even harder for them to find the time to build a research program compared to those who had already established them.
  
  Lastly, the staff with maximal deductions tended to reach their annual load easily, i.e., with one or two courses worth of lecturing, while the staff with low deduction amounts struggled to make annual load, had a large number of small, low-WLU-value tasks as a consequence, and thus a comparative disadvantage at achieving deductions. This appeared to be the source of bifurcation in loads, with the 100 WLU cap simply constraining it to a slow trend rather than a rapid escalation (i.e., the effect is there but easily ignored to detrimental long-term result).
  
  There is no way to sugar-coat this — ultimately any research deductions scheme in a teaching allocation system acts to transfer load off senior T+R staff onto: a) a smaller number of EF staff causing them unreasonable workloads, b) new hires such that it makes it harder for them to do what you need them to do, i.e., build a successful research program, and c) staff with slightly lower research metrics such that it makes it harder for them to do what you need them to do, i.e., get their research productivity up. For us, entirely abolishing research deductions was the only clear solution, tweaking the deduction equation just moves the inequity around, i.e., shifts who is in the groups of ‘have it easy’, ‘have it tough’, and ‘being crushed’, and doesn’t ameliorate it.
  
• The scheme could account for some things better: The age of the scheme also meant other changes had been done ad hoc, e.g., load allocations for online courses, and that some activities were either not accounted for, e.g., a lot of hidden teaching in first year, or disincentivised, e.g., no load available to offset important contributions like designing new lab experiments, and other curriculum development.

Actions

Ultimately we made a range of reforms, some major ones that I’ll outline below, and a stack of very minor ones, often just changing workload formulas, that I’ll not mention here:

• Renormalised load to EF roles: We renormalised the scheme so that an EF has a load of 100% with 20% of their time to non-teaching tasks. This was partly for compliance with the EA, and partly to ensure that loads can be more meaningfully interpreted and understood (i.e., everyone is at 100%, all things considered). While the breakdown for a T+R is not stipulated in the EA, we settled on assuming 40% Research and 60% Teaching & Admin in line with the traditional notion of ’40/40/20′ splits and to balance that some staff carry more/fewer admin roles based on career-stage, preference, ability and opportunity. The importance of the 60% will become clear below.
  
• Research deductions abolished: There was no fair way to fix this other than to abolish the deductions scheme entirely. We replaced it with a flat 40% load reduction for all T+R staff relative to EF staff irrespective of output (e.g., grant income, publications, postgrad student numbers, etc). Essentially, all T+R staff implicitly get the same time opportunity for research, some might produce more or less in any given year and that gets rewarded in other ways, e.g., promotion, funding, awards, satisfaction, etc, but crucially it does not feed-back around in terms the time opportunity that enables that. This is what equity actually looks like — there’s lots of talk of it in the university sector, much of it hot air, but levelling the playing field on opportunity is where the rubber hits the road.
  
  Some might argue ‘but hey, my research or admin role is so important I shouldn’t have to teach.’ Fine, but if that’s the case, then someone will put up the cash to buy you out of some of your workload. If that’s research, then the funding agency will supply your salary through a fellowship (or teaching relief if you can convince them to), and if that’s admin, then it’d be the faculty or central buying you out. Either way, there’s cash on the table and we can use that money to hire someone to replace you rather than dump your load on others. And if there’s no cash to buy you out, is it really that important that you shouldn’t be subject to the same opportunity as everyone else? Probably not.
  
• Big Rocks & Small Rocks Accounting: This was a clever idea we came up with to mitigate the issue of bifurcating loads due to people being allocated differing ratios of a) tasks with comparatively large WLU allocations per hour of face-to-face teaching, e.g., lecturing, admin roles, etc, and b) tasks with comparatively small WLU allocations per hour of face-to-face teaching, e.g., labs, first year tutorials, etc. We took to calling these ‘big rocks’ and ‘small rocks’ accordingly, and in part because the ‘big rocks’ are usually allocated a year in advance as part of annual teaching allocations and the ‘small rocks’ are allocated during the year, often right before term starts, to fill the load.
  
  In the old workload scheme, tasks tended to be allocated without any consideration of the resulting big : small ratio, leading to some staff at close to 10:1, who could then meet their annual load with much fewer face-to-face hours, than others who ended up closer to a ratio of 1:1. This often came from biases (conscious and unconscious) in lecture course allocations (there aren’t enough that everyone can fill their load with them) and the fact that admin roles come in varying sizes and there aren’t enough of those to go around.
  
  Of the overall school load, approximately 75% is big rocks and 25% is small rocks, and so any staff member’s allocation should also be approximately 75% big rocks and 25% small rocks to avoid the bifurcation issue. The way we implemented this was to have the workload model calculate the ideal big rock and small rock loads for each staff member, and then stipulate that the teaching allocations committee cannot add any further big rocks tasks to any staff member who is already over their big rocks allocation (i.e. 75% of load already allocated). As you’ll see later, the excel model we built automatically calculates this and graphs it ‘on the fly’ so we can allocate teaching whilst being mindful of the load implications in real time. As long as the big rocks allocation limit is held to, everyone would by default be carrying some amount of the small rocks, such that it doesn’t become a crushing load on the few.
  
• Resource Development Projects: We added a new category to the scheme to provide a way to incentivise and better recognise the work that goes into teaching development projects in the school, e.g., developing new lab experiments, course pedagogy transformation (i.e., traditional lecture course to flipped), or curriculum development. We did this by reserving 200 WLU to be allocated at nominally 30 WLU per project (varied to expected size) to projects submitted by staff in the annual call for teaching preferences and assessed by the teaching allocation committee. All proposals needed a clear set of deliverables and the WLU allocation was made on demonstration that the deliverables have been achieved. This was designed as an anti-gaming measure to avoid recipients from over-promising and under-delivering, which would be incentivised if the allocation is given up front.

Drilling down into the detail

As mentioned earlier, part of the task was to build a more automated model for the calculations to streamline the handling. We did this in excel with an anonymised version of what the model looks like publicly available here. For the discussion that follows, a reader probably wants the excel sheet up on a second screen so they can follow along.

The Excel file has four admin tabs — Initial Load, Load Setup, Allowances and Outcomes — and thereafter each member of staff has their own page, I’ve preserved four of them here for a) a typical EF, b) a typical T+R academic with a low admin load, c) a typical T+R academic with a high admin load, and d) an ARC fellow. I’ll now work through the tabs one by one.

Initial Load Tab: This tab essentially tallies up all the teaching that needs to be done in the department by term and course in the rows and by type and ‘WLU cost’ in the columns. The first block of 5 columns accounts for lecturing of the first half (H1), lecturing of the second half (H2) and any convenor contributions (now removed from all the 1st year courses — tl;dr for here). The lecture contributions can be ‘new’ for the first time a person takes that course, ‘2nd’ for their 2nd year on the course, and ‘old’ for their 3rd and subsequent year. This ‘staircase’ of load accounts for the fact that the effort needed to give a course tapers off once you’ve given it a couple of times. The second block of 3 columns deals with online courses, with slightly different contributions for undergraduate (U) and postgraduate (P) level online courses. The ‘coord’ column accounts for course coordination on all of our higher year courses. All of these items, so far, are big rocks contributions.

The next four columns are small rocks, with two of them counting up laboratories and two of them counting up tutorials and problem-solving workshops (PSWs). These need to be adjusted a little ‘on the fly’ as some of these are taken by casuals, who are not in the WLU scheme, and others by continuing staff, who are in the WLU scheme. Ultimately, everything is designed to balance out with all of the load generated being allocated, and it will be clear how this works once we’re at the ‘Outcomes’ tab.

To do the annual teaching allocations, we only need to solve the big rocks in this tab, and we tune the lecture level (new/2nd/old) as we allocate. The sheet automatically updates for these things, with the goal being for everything to have balanced out by year end.

Load Setup Tab: This tab pulls together all the teaching WLU from the ‘Initial Load’ tab, all the honours and undergraduate research students admin allowances, resource development projects from the ‘Allowances’ tab, and tallies up the total load to be distributed. The WLU carried by fellows (essentially free teaching for the school) is deducted, and then we divide that by the staff to get the individual staff load (accounting for the differing load fraction for EFs and T+Rs). We also tally up the big rocks and small rocks and calculate those fractions. These naturally update through the year as the total number of small rocks refines in accuracy — our aim is to be quantitatively fair (minimise departmental Gini index) not infinitely precise in prediction. We adapt on the fly to even out these things and mostly they disappear in the ‘noise’ of the scheme, which I’ll discuss in my closing comments. The ‘assigned so far’ and ‘Smalls TBA’ quantities are there as a guide during the allocation process.

Allowances Tab: This tab carries the allowances for the various tasks teaching and admin that can be allocated in the school, and these propagate across to the two tabs previously discussed. The values are decided by the school executive with, notionally, an annual review, although they don’t vary much on that time-scale; changes tend to happen more at 5-10 year intervals in my experience.

The admin roles should make clear the folly of precision ever being achieved in WLU-to-hours conversion. The admin super-heavy roles, HoS, first year director and teaching director (250, 180 and 120 WLU respectively) are clearly more than 5.5, 4 and 2.5 hours a week, even if they’re done bare-bones. Having HoS as equivalent to 2.5 new half-term lecture courses a year is perhaps (slightly) more realistic, but nonetheless nearly unverifiable in comparison without strong data (see my second article). Ultimately, if you put real hours down against some of these things, then the number of staff multiplied by 1610 hours/year will be woefully insufficient to cover the load. Half your staff will be full-time admin, and the rest will be crushed by teaching unless Faculty/Central decides to increase your staff budget according to your calculations. Correcting this is essentially the same as a country in massive debt trashing its economy and starting a new one — the ensuing debt write-off comes at the cost of bankrupting your school. If you want to see how I think about this, I suggest reading Ray Dalio’s Principles for Navigating Big Debt Crises (available as free pdf), you’ll see where I’m coming from.

Ultimately, all you can do is operate WLU as a fiat currency and aim for relative values to minimise your departmental Gini index, it’s the least bad option without a whole-of-campus full-funded workload model.

Outcomes Tab: This tab is necessarily a stripped down version of what we really use in our school in the excel file provided due to confidentiality reasons, but I’ll capture the reality in a screenshot below.

The plot shows real data from our departmental model mid-way through teaching allocations for the following year (allocating the big rocks only), but with the names stripped and the positions randomised. Each set of three bars is one academic staff member. The left axis is in WLU. The three bars for each staff member is their allocated load in green (100% of annual share for EF, 60% of annual share for T+R and whatever a fellow is contributing for a fellow), their ideal big rocks allocation in blue (75% of load) and their currently allocated teaching in orange. You can spot all the fellows as they have no small rocks, you can also spot the two EFs.

This put a new dimension to the task for the teaching allocations committee, not only do they need to allocate all the big rocks tasks, but they need to do it such that as many people as possible have their orange bar above their blue bar but below their green bar. It’s clear completing this task is impossible, and there are some political complexities, it can’t all be numbers driven — every process in admin and management is messy, you can’t avoid it — but you can certainly get to a point of reasonably minimised Gini index, in that everyone has lecturing and to similar extents and everyone carries some minor tasks and to similar extents.

The key thing is that bringing a quantitative dimension to the whole workload and teaching allocation process means everyone on the committee can see the inequities and call them out — they are there right in front of you and cannot be called imaginary.

Lastly, the columns in this tab. Load % is 100% for EFs, 60% for T+R and autofills for whatever fellows offer or are cajoled by the HoS to teach. The next three columns fill automatically. The allocated column pulls from an individuals sheet to tally off their allocated load as we allocate it — this enables us to operate as close to possible in real time with allocations. The next three columns tell us how much load they have left to allocate, in various measures (e.g., WLU, % of load) to help us judge what might fit. The last column is the balance carryover from the previous year, which propagates to the load column — likewise the balance column becomes next year’s carryover. This part of the architecture enables us to carry forward loads from year to year, such that someone who underteaches or overteaches one year can be compensated for it in later years. This part is nice because you can ‘stack’ years and model loads over significant periods for departmental planning, if you wanted (I’m a datanerd, so trust me when I’ve said I’ve sunk more than one evening into doing this. 😀 ).

This brings me back to an essential point from earlier, which is ‘noise’. A typical balance carryover is 20-50 WLU, either positive or negative. If you run this model a while, you realise it is impossible to zero everyone out. Actually it’s nearly impossible to zero anyone out, the teaching just isn’t fine-grained enough. In each year some people have to teach more and some less, some get more small rocks, some less. But, you can build a system where this averages out if you’re smart about it. The noise becomes a feature not a flaw. It’s a reason why obsessive quantitative balancing is dangerous — the cost-to-benefit ratio is poor, aim for good enough and know the noise averages over time if you are attentive to it.

Individual tabs: Last bit, there’s a tab for each staff member, and this is where we take their load, subtract the allocated teaching, subtract the admin roles, and calculate the balance. In the lite model supplied there’s only four examples, but our departmental one has over 20. The nice thing here is we can at any time print and send to staff so they know their allocation (my ideal would be that they can just see in the file itself as a cloud file — there’s nothing to hide).

Closing Comments

This post is probably long enough already, but let me close with a few comments.

• Without workload models at the personal and campus level, you can never achieve sensible WLU-to-hours calibration in a departmental model without generating more severe departmental Gini index than you would by ignoring WLU-to-hours calibration and operating your scheme as a fiat currency solely devoted to minimising departmental Gini index and increasing transparency.
  
• Anything that’s not quantitative is bullshit. This doesn’t mean that tolerating some noise or uncertainty in a quantitative scheme is also bullshit, there is always a limit to how accurate any model or forecast can be, and aiming for perfect balance is folly from a cost-to-benefit perspective. Tolerate noise and uncertainty, aim as best you can to know how large these are, otherwise be ruthlessly quantitative — don’t let uncertainty and noise be an argument to not be quantitative.
  
• Always have Goodhart’s law in mind — when a measure becomes a target, it ceases to become a good measure. This means you need to actively look for anti-gaming measures in any quantitative system you build. Sometimes, the fact you have a quantitative system is an anti-gaming measure on its own. If there’s anyone who will bullshit on workload, given the chance, it’s an academic.

And with that… second article in the series coming soon.. what Adam’s real workload (now) looks like.