Three Terms Pt II — What should Attila do?

Attention, you have just entered a battlezone… It’s time for us to find out who’s, really the best, with the freshest crew, no time for words, we’re through with that, all there is left is pure combat.

I wrote a blog post back in June about the realities of the UTS and UNSW teaching calendars, with a study of actual term dates from 2014 through to 2021 to highlight the important differences between the calendars, and debunk aspects of statements made about their comparison.

Ever since that post, I’ve wanted to get around to a follow-up post on how I can see the current UNSW 3+ calendar might be turned into something functional that isn’t just a direct retreat to the old UNSW two semester system that predates UNSW-3+.

As a prelude to discussion, let me go back to the diagram from my initial post on the UNSW 3+ calendar and slice out the calendars for 2018 and 2019, as I think they highlight an option well worth considering going to.

The major problem with the UNSW 3+ calendar, if you look at the above, is actually the summer term. It is the term that puts the ‘plus’ onto the three and it’s actually, as a whole, a minus. You look around the campus, almost no one uses this term. It’s just too short to do anything in, and with most staff on campus ready to drop dead as Week 11 of Term 3 arrives, it’s little surprise!

If you offload that pointless summer term, you free up something in the vicinity of six to eight weeks, which might not initially seem like much, but actually makes a massive amount of difference, because what it enables you to do is rotate your calendar a bit so you can get to the UTS three-term calendars for 2018 and 2019 as a set of dates.

Before we get to the benefits, let’s just quickly unpack what this true three term calendar would look like. Just to avoid confusion, for the rest of this post, I’m going to call the UTS Autumn term as Term 1, the UTS Spring term as Term 2 and the UTS Summer term as Summer term. One thing I was fond of when I was working in Sweden was week numbering, so I’m going to use that here. Readers wanting to see 2018 and 2019 as week-counted calendars can see here.

Term 1 runs from 12th March (Week 11) to 8th June (Week 23), a total of 12 teaching weeks and 1 break week (late April) with a 1 week study break at the end (Week 24) and 2 week exam period (Weeks 25 and 26).

Term 2 runs from 23rd July (Week 30) to 19th October (Week 42), a total of 12 teaching weeks and 1 break week (mid Sept) with a 1 week study break at the end (Week 43) and 2 week exam period (Weeks 44 and 45).

Summer term runs from 19th November (Week 47) to 15th February (Week 7), a total of 12 teaching weeks and 1 break week (xmas to new years) with 1 week study break at the end (Week 8) and 1 week exam period (Week 9).

Some will immediately notice the decompression from the UNSW 3+ calendar that you get from killing the summer term. You now go back to 12 week terms, each with a 1 week break in the middle and a 1 week break at the end before exams. And this isn’t a Microsoft-style patch like the current ‘flexibility week’ in the UNSW 3+ calendar, or ‘inflexibility week’ as many have taken to calling it because not only does it remove a week of teaching from already desperately short terms, but it makes scheduling of labs, projects, assignments, etc. incredibly difficult to implement. There are more substantive breaks between terms too.

There’s an additional gain to be had here, and it comes from being smart about how you use that third term (summer), in particular, by giving the faculties & schools greater autonomy over how they spread their courses across the terms in the calendar.

For some degree programs, e.g., business or commerce, the content is pretty trivial, students want to come in and get out as fast as possible so they can get busy making money, and the school wants high turnover because it’s all about the money for them too. Fine, business is business. They can run courses in all three terms if they want — that’s ultimately a decision about whether demand drives enough income to cover the cost of running that high a course turnover. I’ll leave that to consultants with MBAs, it’s not a difficult problem.

For other degree programs, e.g., science or engineering, the content is far more technical and the pace that the material comes at you really matters. It might not seem like going from 12 weeks of 3 lectures a week to 9 weeks of 4 lectures will hurt much, but when it comes to coping with subjects like particle physics or higher differential geometry, the difference is brutal. The courses have lab components and research projects, which have a higher cost to run and are heavily constrained by things like lab bench spaces, number of equipment sets available, etc. Here, it makes sense for these faculties and schools to pull back most of their courses just into Terms 1 and 2, and we just accept that science or engineering takes time to learn properly — compression doesn’t bring benefits in the technical subjects. These faculties and schools can then use the Summer Term to deliver either selected ‘catchup’ courses where demand is sufficient or repeats of Term 1 or Term 2 courses where the student demand justifies the cost and effort of running them. Additionally, students can use the summer term off for summer vacation scholarships or internships, which have actually become harder rather than easier to access with how far out of alignment the 3+ calendar has put us with the traditional academic year. This summer break is quite long giving a solid period for those internships and research scholarships (i.e., they don’t need to be rushed or made into flimsy projects to fit a shortened break). A solid summer break with time to decompress and let your brain digest the technical content of the year will also lead to far better student outcomes and student experience.

Some other benefits to this calendar:

  • Term 1 doesn’t start until after the Australian Research Council Discovery Projects deadline, which means that staff have time to write better proposals and improve success rates in funding rounds.
  • In research intensive schools where Term 3 is not heavily taught in, the staff have a solid block of time to generate research outcomes and output. The lab and technical staff also have sufficient periods without students to perform maintenance and/or upgrades of lab experiments and projects to ensure they continue providing optimum student benefit and experience.
  • The admin overhead is reduced (3 terms instead of 4) but the intensity of that overhead is brought down also, simply because things are more spaced out in the transitions from teaching to exams, exams to results, and end of term to start of next term. This reduces stress, reduces fatigue, reduces risk of errors arising from tired staff racing under high time-pressure to complete tasks.
  • The campus is a nicer place to be around. The staff are happier, the students are happier. Everyone has time to do their job rather than struggle just to keep their head above water.
  • A pointless and largely unused term is removed from the calendar, reducing organisational waste.

I’ll let everyone else be the judge, but having thought about this for a while now, my opinion is: TKO.

Focus your anger and… demand better in your own space.

Thinking overnight about yesterday’s back’n’forth with @thesiswhisperer and whether I can give a good example of where our exploitation is a problem of our making and not easily blamed away… I can, so here it is.

A common complaint in academia is workloads. The blame is often attributed to the government (not enough money), the unions (not doing enough), senior management (aiming to exploit), but it can be amusing to watch how little academics below senior management actually do on this.

Last year I read a lot of university EAs for a blog post on salaries in academia, particularly at the top end (see here). There was an interesting section in one of the EAs…

“A workload formula will be in place in each unit, developed through a collegial process, generally supported by employees in the unit, and provide for the equitable and transparent allocation of workload within the unit.” It continues…

“The formula will be developed in a way that identifies a transparent correlation between the measure applied and the hours of work generated by each relevant academic activity” It then specifies all the aspects including teaching and admin.

I’ll spare it all, but two last relevant details: a) “The formula must contain a quantifiable maximum on required workload measured in hours and a quantifiable maximum on teaching contact hours.”; b) “The allocation of teaching contact hours will be consistent with the formula”.

At this point, clearly some admixture of the union and senior management have done their job properly — there is a requirement for a transparent and quantifiable workload allocation. And the government? Well, it really has no important role at this level.

One might expect every unit has such a scheme, right? Do they? Of course not! Some would say senior management is to blame if this isn’t the case, and perhaps there is an oversight failure there. But really, the failure is at the unit level.

Ultimate responsibility lies with the head of the unit. Have they not read the EA? Are they unaware that the Fair Work Commission could come after them for violating the EA? Or is it that not having this benefits them as they can preferentially award work to suit other agendas?

What about the staff in the unit, who should now be pressing their unit head about why a key legal condition of the EA is not being implemented? Some won’t speak up for fear of reprisals, same reason many would consider me writing this to be madness and asking for trouble.

Others do, and I know of an amusing instance where an attempt was made to take it up the ED&I chain, since that can feasibly bypass a head of unit if it wants/needs to. It is a fair ED&I issue because lack of such a scheme usually punishes junior or minority members in a unit.

The response, from ED&I rep to fellow academic, was ‘this is just how the system works, so just get used to playing it as it is’. Impressive ED&I leadership right there, hey! Some might snort ‘male Prof ED&I rep I bet’, nope, female & not prof. and a self-nominee to the role.

It’s interesting to think on why an ED&I rep would say such a thing? Do they benefit from the lack of a transparent system somehow? Is it the lack of action accountability in the ED&I system that means the value ratio of hot air or smoke & mirrors to real action is high? Fear?

After all, it’s not only one unit that’s a standout in lacking a workload scheme that meets the EA, there are many. How could middle-management level EA committees be so blind, silent or inactive on this? How could middle- and senior-management be in a similar position?

This is why I always find it amusing to watch academics blame the government, or unions, or society, anyone but themselves really, for various woes when many aspects of those woes can be fixed by good gutsy grassroots leadership action down at their own level.

Particularly when, having a solid quantitative, transparent, open workload model, you can then argue from a strong numerical standpoint about why and by how much the workload situation is unsustainable and in need of additional resourcing. Words are nice, numbers are power.

So as much as I’m all for ‘Focus your anger and vote accordingly’ as @thesiswhisperer suggests, you also cannot ignore the power of grassroots action. Academics have more power than they think, and to quote @midnightoilband ‘It’s better to die on your feet than live on your knees.

PS. This is not a gripe on my own management chain, as I’m in a unit that actually has a quantitative workload formula at unit level. It mightn’t be perfect, but at least it exists, and we continue to work to improve it.at least it exists.

But if you want these things, you have to fight for them. Start by making good trouble actually flow upwards because sadly, voting is designed by and for the people at the top to achieve change only if it benefits those already at the top. Good trouble is more powerful as a tool.

When is three terms really three terms?

For several years, my colleagues and I have been subjected to ‘the UNSW 3+ calendar’. A common ‘sales pitch’ has for years been that UTS has gone to three terms and it works wonderfully for them. The veracity of this was publicly questioned at the time (2018) and since, such that I would consider that there has been more than adequate opportunity for such statements to be corrected or nuanced to a position that has proper integrity by 2021.

The most recent incarnation of this statement was made in an all staff forum:

…except it says UTS tried terms and abandoned and dropped them. Now I actually talked, I rang a colleague at UTS to check this. It’s not true. UTS still has 3 terms: spring, autumn and summer, and they’re getting increasing enrolments in the summer semester, which is becoming a very big one for them. 14 Australian universities have extended academic calendars with three terms…”

The point has been reached where I feel compelled to put a sword through the spin and interrogate the facts, so let us do exactly this. As I would say in my video lectures, let’s bring up some slides

The image above is a comparative graphical representation of the competing calendars of UNSW (left column each year) and UTS (right column each year) for the years 2014 through to 2021. The start and ends are accurate to +/- 2 days (+/- 1mm) under an assumption that each month has 30 days so that it occupies 15mm of vertical grid to keep the graph easily drawable. The data has been obtained from the official UNSW and UTS calendars for those years by some mix of backwards link-smithing through UTS’s archived handbooks and UNSW’s academic calendar website using the Wayback Machine. What is presented in each instance is the full term from Day 1 of Week 1 to the formal last day of that term’s exam period to ensure the calendars are being mapped on a like-for-like basis (this explains the overlap between the summer and autumn semesters at UTS for 2021 — the summer term exams overlap the start of their autumn term this year).

The most obvious conclusion: Any attempt to say UNSW ‘has three terms’ in the same sense that UTS ‘has three terms’, or the other 13 Australian universities for that matter, is just plain wrong. The correct and honest interpretation/representation would be either:

  • UNSW has 3 terms plus a summer term and UTS and the other universities have 2 terms plus a summer term.

or

  • UNSW has 4 terms and UTS and the other universities have 3 terms.

To argue otherwise, including trying to obfuscate or articulate to a position where UNSW and UTS somehow both have 3 terms lacks the data-driven integrity that one would expect from a university.

An additional comment was made in the same all staff forum, namely:

The primary motive is that it provides more opportunities for students to study, and that in turn provides more jobs and better stability for staff… and it also aligns us better with society and it breaks down the notion that universities are ivory towers where staff teach for only 24 out of the 52 weeks in a year.

People can decide their own feelings on this comment, especially the end of it, but let us interrogate this statement briefly:

Going back to 2016, before the UNSW 3+ calendar commenced in 2019. There were 2 semesters, each with 12 teaching weeks, and a summer term of 6-8 weeks length, giving a total of 30-32 weeks total teaching in each calendar year.

In the UNSW 3+ calendar, after the introduction of flexibility week, we have 3 terms, each with 9 teaching weeks, and a summer term with 4 weeks, giving a total of 31 weeks teaching in each calendar year.

In contrast, UTS has 2 terms, each with 12 weeks, along with a summer term, which is also 12 weeks, giving a total of 36 teaching weeks a year. Sydney University, on the other hand, has two terms of 13 weeks, giving a total of 26 teaching weeks per year. It does not run a summer term presently. This has UNSW, with 31 teaching weeks a year, at slightly more than Sydney University and slightly fewer than UTS.

If we consider UNSW alone, before 3+ we had 30-32 weeks and after 3+ we have 31 weeks. There is no net gain. Even if we just focus on the main terms, we have 2 x 12 = 24 weeks before 3+ and just 3 x 9 = 27 weeks after 3+. I’ll let people decide for themselves if shuffling a few weeks around and adding a fancy new name shatters the ivory tower.

What is the price? The price, I’d argue, is twofold. First, our administrative overhead in terms of exams, enrolment, admin, etc. has gone up by at least 33%. Second, the physical and mental load on both staff and students has gone up by a commensurate amount (some would argue more) due to the compression of time available to teach & learn in — this is a longer argument that I will let people more expert in human performance aspects related to learning & teaching prosecute instead.

Efficiency? Effectiveness? That’s a separate debate. But, at the very least, we need to be fully honest about the comparative realities of university calendars because the entire basis of science and academia is that there are facts and objective truth. And if the leaders of our organisation cannot get to that objective truth, then how do they expect any trust from staff and students or any interest in following them — the truth always matters.

WAM Booster Courses: Magic or Myth?

There are three kinds of lies: lies, damned lies, and statistics” — Origin Disputed

If you look around student internet forums and other social media enough, you can’t miss the concept known as the ‘WAM Booster‘ or ‘GPA Booster‘ course. The basic idea is that you take one or two spare electives or general education classes, for campuses that have them, that are easy to get a high grade in, and use this to deliberately ‘inflate’ your WAM or GPA.

As generally happens with the internet, these ideas grow and spread, and start to build a mythology all of their own. At that point, students eager for an easy ticket to a high WAM/GPA will fully get on board, having never even thrown a critical eye at the reality. The idea is even being used by nefarious characters, i.e., contract cheating organisations, to draw in victims with nasty knock-ons including blackmail.

One of the skills that make physicists so employable is the ability to be skeptical about an idea and then use quantitative methods like an ultra-sharp sword to cut through the talk to get at the reality. And that is exactly what I will set out to do in this blog-post. Essentially, the question is:

Will the WAMBooster concept give you this?

Or something more like this?

I got interested in this problem when my colleague, Sue Hagon, was telling me about doing a simple back of the envelope calculation on this problem. Imagine a typical 3 year undergraduate degree with 24 courses in it, and let two of them be ‘WAM Booster’ courses — I’ll use WAM for the remainder of this post as WAM always sounds cooler.

Sue’s calculation was: Assume a student with a WAM of 65 from 22 courses, and they happen to get 75 for two courses, what will be their WAM? You can work this out pretty quickly, it is just (65 * 22 + 75 * 2)/24 = 65.83. Wait, those ten extra marks in two courses are worth only 0.83 on WAM? Yep, check it for yourself if you don’t trust us.

Seeing this immediately tempted me into testing a more extreme example — a student with a WAM of 50 who manages to get 95 in a pair of ‘WAM Booster’ courses. Surely this must take your WAM to the moon, right? It’s an effective ‘WAM differential’ — defined as the difference between mark in a course and your WAM – of a staggering +45… for two courses. Take a guess first, it’s always good to know what your intuition says, and then do the maths (answer down below).

In the end, there is no silver bullet, no substitute for actually knowing one’s subject and one’s organization, which is partly a matter of experience and partly a matter of unquantifiable skill. Many matters of importance are too subject to judgement and interpretation to be solved by standardized metrics. Ultimately, the issue is not one of metrics versus judgment, but metrics as informing judgement, which includes knowing how much weight to give to metrics, recognizing their characteristic distortions, and appreciating what can’t be measured. In recent decades, too many politicians, business leaders, policymakers, and academic officials have lost sight of that.” — Jerry Z. Muller

I spent a few hours this week marking computational physics exercises that our 3rd year physics undergrads do using Jupyter Notebook and the various libraries for data visualization, e.g., Matplotlib. I’ve always been a big believer in “It is not fair to ask of others what you are not willing to do yourself” (Eleanor Roosevelt), so I decided to sink a bit of Sunday into a bit of a personal refresher course on these aspects by doing a full workup of the statistics of WAM Boosting, with a few nice bits of DataViz to fully highlight how little edge there really is in this stuff.

I’m happy to share my model, but the basics are as follows:

  • I assume a set of 22 normal courses that give a ‘core WAM’ of x. For simplicity, I’ve assumed this distribution to be monodisperse (i.e., they’re all the same mark, which is x). The monodispersity shouldn’t massively affect the conclusions, but I’m happy for the curious to test this.
  • I assume a pair of WAM Booster courses, both of which get a mark of y, which I’ve called the ‘Boost’. I’ve assumed they get the same mark just to keep the model simple (doable in a few hours not days).
  • All the numerical modelling is done in Excel, simply because some clever tricks with absolute & relative cells and fill-right and fill-down mean I can MacGyver the primary data out quicker than by writing python for it, and see the numbers in real time for checking.
  • But excel sucks hard for plots, so I then kick the results off as .csv, and use Jupyter Notebook to handle the data from there, with pandas for the assembly and some of the nice 2D colourplot and contour map features of matplotlib to get all the visualisations out.

So without any further ado, let’s get into some visualisations and really dig into how this WAM Boosting nonsense works:

Fig. 1: 2D Colormap of Final WAM for 24 courses (color axis) vs Core WAM for 22 courses (x-axis) and score for the two WAM Booster courses (y-axis). The black solid contours are for Final WAM. The blue dot-dashed contours are for effective WAM differential, which is the difference between the Boost and the Core WAM (i.e., WAM differential of +x means Boost = Core WAM + x).

Figure 1 shows a 2D colormap of a students final WAM (24 courses) plotted as a function of core WAM (22 courses) on the x-axis and what I call ‘Boost’, which is just the score for the remaining two courses on the y-axis. To help with reading, I’ve added two sets of contours to the plot. The black solid line contours are just the final WAM, hence those lines running parallel to the gradations in the colormap. The blue dash-dot lines is a parameter that I’ve called effective WAM differential, which is the difference between the boost and the core WAM. A positive WAM differential means the boost marks are higher than the core WAM. A negative WAM differential means the boost marks ended up lower than the core WAM.

To walk you through the plot, consider the extreme case from earlier, which is a student with a core WAM of 50 who gets two scores of 95. This is a position on in the far upper left corner, essentially at 95 on the y-axis and around where the blue +45 WAM differential contour would be. You can see the black 55 final WAM contour coming in close here, which is consistent with the WAM of 53.75 that this student would get if you run the numbers (answering the question from earlier).

Yes, you read that right, even in the case where the student just barely passing 22 of their courses manages to get a totes amazeballs 95 their on two ‘WAM Booster’ courses, the most they get as an increase in WAM is a measly 3.75 marks. Wow. Such disappoint.

Figure 1 might be a little tricky to read for those who aren’t familiar with colormaps and contour plots, so it might be easier from here to calculate a new parameter called the ‘Effective WAM Boost’, which is the Final WAM for 24 courses minus the core WAM for 22 courses. In the case above, this would be 53.75 – 50.00 = 3.75. We can plot this as a colormap versus core WAM and Boost too, as we see in Fig. 2.

Fig. 2: 2D Colormap of Effective Boost (color axis), which is Final WAM minus core WAM vs core WAM for 22 courses (x-axis) and score for the two WAM Booster courses (y-axis). The red solid contours are for effective boost. The blue dot-dashed contours are for effective WAM differential, which is the difference between the Boost and the Core WAM (i.e., WAM differential of +x means Boost = Core WAM + x).

Figure 2 shows effective boost on a color scale that runs from pure yellow at an effective boost of zero, i.e., the two WAM Booster courses get the same score as the core WAM, to green for a positive effective WAM boost, i.e., the two courses increase final WAM above core WAM, and red for a negative WAM, i.e., the two courses increase final WAM below core WAM.

The first thing that’s evident in Fig. 2 is the limits on effective WAM boost, which takes its most positive value of +4.167 in the top left corner (student with core WAM 50 and 100 on their two WAM Booster courses) and -4.167 in the bottom right corner (student with a core WAM of 100 and 50 in their two WAM Booster courses).

The symmetry in this plot is interesting because what it implies is that almost all of the benefit of WAM Booster courses goes to the students with the lowest core WAM, which will immediately raise the hackles for all the meritocratic elitists out there — except that the gain made from this small in real terms, at most 8.3% (4.167/50) for a student with core WAM of 50. The effect drops off pretty quickly, for example, a student with a core WAM of 65 can’t get any more than 4.9% gain (2.917/65), a student with a core WAM of 80 can’t get any more than 2.1% (1.667/80), and a student with core WAM of 95 can’t get more than 0.44% (0.417/95).

The other interesting aspect of Fig. 2 is that WAM Booster courses can present risk rather than reward to students with high WAM. What I mean here is that the effective boost for a student with high core WAM will be negative unless the score on those two courses is at least equal to the core WAM. This means that the philosophy that we used to take on such courses when I was an undergrad in the early 1990s, which was that ’50 was a pass and 51 was a sign of misplaced effort’ would be a grave tactical error in these modern ‘WAM Booster’ times (i.e., you can’t opt out of the game if the education system makes WAM everything). Students who have a high core WAM and want to hold it cannot afford to do anything but look to knock such courses out of the park as well.

At this point I want to return to what I see as the origin of this whole WAM Booster nonsense, which is as I’ve pointed out in my last two blog posts here and here, that the uncertainty on a statistical quantifier such as WAM is at the integer level, and much of the pathology of WAM arises because it’s a) often presented to several decimal places and b) as though there is no uncertainty in the measure at all. As anyone who has done a first year physics course before will know, this is essentially scientific malpractice.

So how large is the uncertainty here? For the core WAM, since it’s monodisperse, it is zero, but, the final WAM is not monodisperse at all, and so the standard deviation, i.e., uncertainty, is far from zero. Crucially, how does it compare to the boost?

Fig. 3: 2D Colormap of WAM uncertainty obtained as the standard deviation for the 24 courses (color axis) vs Core WAM for 22 courses (x-axis) and score for the two WAM Booster courses (y-axis). The green dashed contours are for uncertainty. The blue dot-dashed contours are for effective WAM differential, which is the difference between the Boost and the Core WAM (i.e., WAM differential of +x means Boost = Core WAM + x).

Figure 3 plots the statistical uncertainty, which ranges from 0 in the case where no effective boost has been obtained to a maximum of 14.116 in the top left and bottom right corners. At these two maximal points, indeed everywhere on the entire 2D map, this uncertainty exceeds the effective WAM boost by a factor of 3.39. The reason why will be evident in Fig. 4. To put it completely bluntly, lest it might be missed — in a proper treatment of WAM as a statistical quantity, any boost obtained by the gaming of two courses is less than 30% of the resulting statistical uncertainty from doing so.

In other words, if you are a true professional and always quote WAM with an uncertainty and a number of significant figures commensurate to that uncertainty, then the whole WAM Boosting thing is no longer a thing. For our student with a WAM of 65 who gets two courses with 75, the correct WAM to write is 65 +/- 3 or perhaps 65.8 +/- 2.8, and in that context, a boost of 0.83 is meaningless. Likewise, for our student with a WAM of 50 who gets two courses with 95, the correct WAM to write is 54 +/- 13 or perhaps 53.8 +/- 12.7, and in that context, a boost of 3.75 is also meaningless.

Breaking the monodispersity of the core WAM, which is what gives zero uncertainty for WAM differential of zero, should only add to the uncertainty everywhere. And this should always occur to a greater extent than breaking the monodispersity would affect the resulting effective boost. In other words, the finding that our uncertainty on the average is always greater that any boost obtained should be a universal result.

I will end with one last visualisation of the data, which is just to reinforce how much of a tiny-gains game this WAM Boosting stuff is, more or less across the board.

Fig. 4: Animated GIF series of plots of Effective Boost (red solid line), Uncertainty (green dashed line) and WAM differential (blue dot-dashed line) vs the boost value for the 2 courses at core WAMs of 55, 65, 75, 85 and 95. The black dot-dash line is to flag WAM differential of zero for each plot. There is a horizontal dotted line to highlight the zero, and another two sets of horizontal/vertical dotted lines to highlight the cases of WAM differential of +10 and WAM differential of +15.

Figure 4 shows ‘slices’ of the full datafield obtained at 5 different core WAM values (55, 65, 75, 85 and 95) plotted against the magnitude of boost for the 2 remaining courses. The point of this plot is to present what should be evident by comparing Fig. 2 and Fig. 3, which is that while the effective boost and uncertainty both rise with WAM differential, as you’d expect, the uncertainty is always larger. Indeed, these trends are both linear, and their constant difference in slope is the reason behind my statement earlier that the uncertainty is always 3.39 times larger than the boost irrespective of core WAM and boost values.

I’ve chosen to highlight WAM differentials of +10 and +15 in Fig. 4 as a return to what are more sensible/likely WAM differences in real terms compared to the more extreme examples, e.g., core WAM 50 and boost of 95, earlier. As the graph shows, the boost for these two cases is the same, across the board, with the exception of very high WAM, where the boost simply cannot be achieved because a course has a maximum score of 100.

For a WAM differential of +10, the maximum boost in final WAM that can be achieved by this approach, at any core WAM, is 0.83, and for +15, it is 1.25. Someone on a core WAM of 50 would move to final WAM of 51.25, someone on a core WAM of 75 would move to a final WAM of 76.25, and someone on a core WAM of 95 would move to a final WAM of 95.42 as they’d hit 100 on both courses and only see 0.42 of the 1.25.

Ultimately, we really are just talking about a tiny lift here, that somehow by mythology and most people’s weak intuition on statistics, has been totally blown up into an advantage that it’s really not. But that’s not surprising, most people also have a weak intuition for probability, and that’s why online gambling is such a ridiculously lucrative ‘industry’ (whether you’d call wrecking lives for profit an industry is up for debate), as is speculation on things like Dogecoin.

To quickly recap, my main points are:

  • Yes, there is a boost, but the boost is relatively small in real terms. A heavily weighted mean is always very resilient to outliers.
  • The boost is less than 30% of the statistical uncertainty in the final WAM anyway, so if we simply reported it with the statistical uncertainty and to the correct number of significant figures, the effect would become irrelevant.
  • We are totally looking at a problem caused by our obsession with single figure metrics, Goodhart’s law and perverse incentives, and I’d still advocate for getting rid of WAM/GPA entirely, as discussed in my last blog post.

Having done the maths now, I think the whole WAM boosting thing is best ignored by students. The better path is just to a) choose courses that interest and excite you and b) work hard on doing well in all your courses, and the rest will just take care of itself. I certainly wouldn’t take the risk of getting busted for academic misconduct, or worse, sending business the way of notorious thugs & organised crime syndicates, trying to squeeze out such miniscule advantage.

Acknowledgements: Sue Hagon for getting me started down this rabbit hole and matplotlib for some very excellent free plotting software that does awesome quality data visualisations.

Death by summative over-assessment — When are students supposed to learn?

The assessment load is manageable for students, teachers and support services.” — Unnamed university assessment policy (publicly available)

Gotta love the part of term where every course decides to give an assessment with 1 week to do it, all due at the same time.” — Unnamed undergrad student

The road to hell is paved with good intentions.” – exact source unknown

I am now a little over a year into the role of having responsibility for my school’s overall teaching program. It always feels like there’s pressure to get right down to action in a new role, but unless there’s an impending crisis, which there was — COVID-19 — albeit in a different space, then the wise money is to sit on your hands for a bit, and take the luxury of a nice long observe & orient phase, before getting to work. Your decisions and actions are much more likely to be in a sensible and ultimately correct direction this way.

In taking it all in, it’s way too easy to focus on the what and how, and forget the most important question of all — Why? I’ve possibly annoyed the hell out of some of my colleagues in the last 12 months with my ‘why’ questions, applied to everything from ‘Why do we have exams?’ and ‘Why do we have lectures?’ to ‘Why do we even care about market share or WAM differential?’ But in my view the ‘why’ is essential to working out what truly matters and what doesn’t. And when it comes to assessment, my digging & thinking on why this past 12 months has only convinced me that the modern higher education system’s real raison d’etre is nothing short of disturbing and pathological. To put it bluntly:

… there is one path to ultimate happiness — having money — that in turn comes from attending prestigious colleges.” — Michael J Sandel

Sandel’s Tyranny of Merit is an excellent read and should be on the reading list for every academic in the modern higher education system, particularly the chapter titled “The Sorting Machine“, but let me explain how Sandel’s thesis becomes my point in this blog post.

Sandel’s central theme ultimately is that “the meritocractic ideal is not a remedy for inequality; it is a justification of inequality.” The role that the modern university plays in that is twofold: a) it is the gatekeeper that provides the ‘ticket of merit’ required to access a limited supply of high-status/salary jobs, and b) it is a filter or ‘sorting machine’ for taking in a large number of people, deciding their relative merit through a large series of measurements, and appending that determination to their ‘ticket of merit’.

We can argue about the merits of ‘meritocracy’ (pun intended) as a philosophy separately, but in the context of trying to be an educator, what are the inevitable outcomes of such a heavily ‘meritocratic’ system.

From the organisational perspective:

  • Everything tends to focus on measurement, i.e., assessment, over actual education.
  • All assessment tends to become summative rather than formative.
  • There is a tendency towards ‘single-figure’ metrics to enable easy/rapid (i.e., lazy) ranking.
  • There is a tendency to ignore uncertainty in measurement and present absolutes.
  • There is a tendency to measure continuously from start to finish, ignoring that this entrenches academic privilege (a cynic would suggest this is a stealth design feature, and to an extent, be correct).
  • There is a tendency to obsess over ‘cheatability’. Assessment design becomes dominated by measures to ‘cheatproof’ the assessment over all other aspects.
  • There is a tendency to tolerate bad teaching and assessment as long as ‘position’ remains rigorous, i.e., it ensures ‘more smart’ students stay near the top and ‘less smart’ students stay near the bottom, as defined by statistics applied to existing single-figure metrics (i.e. WAM/GPA differentials).
  • There is a tendency to view courses where everyone succeeds or the gap between top and bottom is small, with immediate deep suspicion on rigor and then question whether it is just a lazy lecturer offering ‘an easy ride’ rather than a dedicated lecturer providing effective course-wide teaching.

From the student perspective:

  • WAM/GPA becomes the priority: I addressed this in my last post Markissism, so I’ll keep it short. Everything from decisions to take easy courses for higher marks rather than ‘push the envelope’ through to gaming the number using contract cheating services. As Campbell’s law says:

The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” — Donald T. Campbell

  • Student experience plummets: The whole experience becomes about fighting to cope with the crushing weight of endless assessment, which amusingly can start as early as Week 1 of term, i.e., before a student has even learned anything yet. And because these tasks contribute directly to your grade & WAM/GPA, which you will carry with you on your ‘ticket to work’ for life, there is no room to not put ridiculous efforts against it or make any mistakes. Given making mistakes is learning, there is essentially no room for learning, your whole experience is about demonstrating performance.
  • Assessment ranges from adversarial to punitive: When the whole assessment structure is essentially summative, there’s no room to make mistakes and learn from them or to experiment because those will all be black marks on the fine-grained measures that determine your future (some like WAM/GPA presented as though accurate to 3 decimal places!). Every assessment becomes an exercise in finding faults to set you in the pecking order of life. The elaborate designs to make assessments ‘cheatproof’ and ‘rigorous’ are not invisible to those who do them, and are seen as an assumption that all students are cheats or out to unfairly climb up some invisible ranking list.
  • Equity and fairness becomes extinct: When every week of every term is endless crushing assessment, it is pretty clear the system is pointless and stacked against you. Students with rich parents who can support them to devote full time & effort to study are obviously at a massive advantage to students who need to work to cover living costs (rent/food). And that’s before we consider any educational disadvantage from public vs private school systems and access to private tutoring services, which is stacked in the favour of students from rich families by virtue of 1st & 2nd year often counting equivalently to WAM/GPA as 3rd & 4th year.
  • Misconduct can become attractive: Most students in the right circumstances and with honour codes in place would be happy to not cheat. In their view, they come to university to be educated, to be exposed to new ideas, to have new experiences, to be challenged and expected to grow as people. Assessment is part of that, and when it’s done in the right way and for the right reasons, it is healthy.

    But, when you start to build up the pathologies above, e.g., crushing assessment that’s purely summative from day one, systems that appear adversarial and which demonstrate an implicit lack of trust, charging fees that result in significant debt, treating education as a profitable business, then you build an ideal environment for the temptation to cheat. And there are plenty of unscrupulous operators out there who will happily take advantage of this for financial gain.

Pointing out problems is all well and good, but it doesn’t achieve a lot unless you pitch some solutions, so let’s get get straight into it. Here are my top ten actions I think any university should be taking in this space as a matter of urgency.

  1. Accept change is needed & start the process: The system is obviously broken, it is unfair, it produces poor outcomes and it is currently being picked apart by contract cheating organisations and poor assessment design (i.e., tasks that are easily gamed via past assessments posted to the internet, e.g., coursehero, chegg). Rome wasn’t built in a day, and neither is a university’s assessment system. There are hundreds of moving parts that extend from governance documents and approvals by university administrative structures through to properly teaching academics how to design strong assessment that measures fairly, broadly and in a way that lifts the performance of all students during their time at university. The discussion and consultation processes need to start now, and be driven well by strong organisational leadership at the top level. There is also a massive cultural shift required as part of this, and cultural shifts are slow and also require very strong organisational leadership.

  2. WAM/GPA should be consigned to the dustbin of history: If the reasons in my last post weren’t sufficient, let me add a personal anecdote. One of the amusing aspects of my career is that my most cited paper was a side-project — we basically took the fractal analysis software I was using for work on quantum devices and deployed it on artwork by the American abstract expressionist painter Jackson Pollock. I grew to have a love-hate relationship with the project ultimately. The analysis was a fun challenge but the idea of taking paintings with so much in them and reducing them to little more than a single number (fractal dimension) felt hollow to me. We would talk about Pollock trying to increase the chaos with time, using that single number to justify it, but then I would look to the actual works, and would see so much more going on in them that was entirely hidden by that number and the reductionist analysis that generated it.

    As an educator, I see our current obsession with WAM/GPA in much the same way. We take students with different skills, different strengths and weaknesses, different personalities and we try to reduce them down to a single number, one given to a number of decimal places entirely incommensurate with the associated certainty of measurement, and unethically so. And as educational organisations, we do this simply so that lazy employers can excuse themselves from treating people on their full spectrum of ability in their employee selection processes (including the universities when they recruit to Ph.D. programs and offer scholarships for them). We do this under the foolish notion of ‘meritocracy’, when in reality, all we largely do is entrench and justify pre-existing inequality. It is our choice to do that, of course, because we could also choose not to; there was a time before WAM/GPA existed and there could always be a time after WAM/GPA exist too. We just have to be brave enough to admit that this measure is flawed and no longer fit for purpose, and replace it.

    So what do we replace it with?

  3. See students as a spectrum not a number: Most modern universities have a list of ‘Graduate Attributes‘ or similar. They are essentially the highest level of learning goals that underpin the learning goals of the various degree programs and subjects that fall underneath them. Like any good set of goals they should be ‘SMART’, and what I mean by this is Specific, Measurable, Attainable, Relevant and Time-bound. If that’s true, then we should be able to rate graduates on their attainment of these attributes at completion of their degree. After all, we claim to do this for the learning goals of the individual courses that make up that degree — if it’s good enough for the parts, then it should be good enough for the whole. An interesting way to do this (hat tip to Kane Murdoch & Garth Pearce for this idea), would be with a spider plot. Any home brewer/winemaker would be well aware of them, but for those who aren’t, I’ve included an example below.



    One could then meaningfully assess to what extent a student has mastered the various attributes, and in particular, which of them they are strongest and weakest in. It would encourage students to find and build their strengths and work on enhancing their weaknesses. It would also increase the likelihood that they end up in jobs following their studies that are a good fit for them and where they are a good fit for the organisation. Having this influence assessment design at program and course level would also improve the training and experience for students through their degree.

  4. Test them properly at graduation: The most important measurement is the state of the student at the point of graduation. This is notionally where all of the student’s training and studies are complete in a specific degree and you can fairly and sensibly expect them to have the full set of graduate attributes and knowledge for the degree program they enrolled in.

    The ‘graduation exam’ should be a rigorous and separate process from the standard ‘end of course’ exams in any usual year of course including the final year. It should seek to test the full set of graduate attributes and also look broadly at technical knowledge, both from the perspective of the broad norms of a graduate in the field and their chosen specialization in the degree program. It might have a written component and/or a short presentation, but the ultimate would be for it to include a substantive panel oral exam (a.k.a. ‘exit viva’). There is no better way to work out whether someone really gets a subject properly and has certain key graduate attributes than to spend 2 hours in a room with them talking about that subject and making them do tasks associated with it. It is a key reason why the world’s best Ph.D. exams are all oral exams with a panel and opponent (yes, I think Aussie Ph.D. exams are rubbish).

    Such a ‘graduation exam’, although being an ordeal at the time as all such things are, would also be a major achievement that adds to the sense of value in graduating from a degree in a way that just a last set of written exams and the grades in the mail a few weeks later doesn’t.

  5. Make course final exams about progression: Items 2-4 above mean that the final exam for a course can return to its original focus, which is simply to determine whether a student has adequate competence in the course and is ready to move on. This might seem like a small change, but it is rather significant when you think about exams more broadly than the university & upper high school environment.

    If I think about courses that I’ve taken outside the university or high school environment, they have all been significantly different in four ways: 1. A comprehensive set of learning goals was presented right down at lecture level (not just 4 token goals for a whole term course), 2. Educational aspects were squarely focused on and driven by those goals in an obviously clear way, 3. The final assessment was always of the learning goals, i.e., fair, and 4. the pass grade was often very high, e.g., 80% or above, or competence within 3 attempts. Having a class where almost everyone passed was an expectation and a demonstration of a competent educator. Having a class where the mean was 50% or 60% and only a handful got above 80% was a sign that the educator was ineffective, the course was poorly designed, or both.

    Somehow both in universities, and probably by osmosis over the years, the upper end of high school, we ended up with an entanglement between assessment for demonstrating competence and assessment as an ‘intelligence test’, which really is just an educational privilege test for the most part. I’ve always found it strange that we’ve designed and tolerated a system where the final test of competence sees getting half of the demonstrated performance wrong as an acceptable threshold for progression — if you can only do half of what’s in the course, then how the hell are you ready to move on? It is even more bizarre that someone demonstrating 95% or 100% competence with the material is considered ‘rare’ and ‘the sign of an exceptional student’. Why on earth do people as intelligent as academics supposedly are, think that this is a sensible, effective and fair way to teach and assess?

    Every course at university should have clear expectations (learning goals) not just at the course level but at the individual class level, and the final assessment should be demonstrably connected to proving mastery of those goals. They should not be intelligence tests aimed at determining who is better than who — leave that to entrance exams and aptitude tests for whatever they do after their studies — universities are there to train not be a meritocratic sorting machine. We should expect the passing grade to be high in courses, and if the cohort isn’t making that high grade, we should be questioning the course design or effectiveness of the educator. Some academics will claim this just means courses will be made ‘soft’ (i.e., easy), but anyone who has trained or taught in a serious course outside a university will know that’s absolutely not true. And the best university educators know that’s untrue as well. It’s really just the excuse that the mediocre use to justify poor learning outcomes and avoid having to teach to a higher standard.

  6. Throw numerical course marks in the bin too: In many universities the course marks are given on a hundred point scale to integer accuracy. It’s the same issue with WAM/GPA given to 3 decimal places, do we really know performance in an individual course to that level of accuracy? Imagine a report with two or even three markers, the standard deviation on that average has to be much greater than 1. Imagine an exam, there’s a finite number of questions, Student A might be lucky and got a question they liked and Student B wasn’t and got a question they didn’t. But, let them sit another years’ exam, and it might be Student A that doesn’t like the question and Student B getting the luck. There is no way that a 1 mark difference is meaningful and therefore fair.

    Courses should only carry final grades, and although numbers ultimately have to be used to determine that grade, there’s a long stretch between using a number and presenting it, which implies that it’s known at least as accurately as the presented significant figures it is specified with. Grades should be representative of how well you can know performance, including being able to tease it out from educational privilege in early years of the program. For example, grades in first year could be Fail/Pass/Superior scaling upwards to a more banded structure in higher years, e.g., Fail/Pass/Credit/Distinction/High Distinction, where the cohort has been on a (more) level playing field for several years than they might be at intake. But just having Fail/Pass/Superior right through the program would be arguably sufficient and possibly more fair.

  7. Put sensible measures/discussion of uncertainty in assessment on transcripts: Eliminating WAM/GPA and hundred-point grading lessens the need for this, but probably doesn’t eliminate it entirely. Any assessment carries uncertainties. Students have good days and bad days, and even good-bad days — one of my best exam performances was a hungover Monday morning after a large weekend. Some students thrive on certain types of assessment, e.g., me on written exams. Others are possibly equivalently good at the actual stuff in the course, but just don’t perform well for certain assessment types, and so their grades ‘under-represent’ them.

    The important thing here is that universities have an ethical responsibility to properly represent how certain they are of the performance they measure and put on a transcript that follows a student around for the rest of their life. Putting courses to a grading accuracy of 1% and WAM/GPA as a ‘single KPI’ with an accuracy down to 0.001% is clearly ridiculous given what we know about how those numbers are derived. And it should be considered anything on the range from unethical to clearly fraudulent given a transcript is an official document.

    This aspect is important because the misrepresentation of grading accuracy drives severe perverse incentives from a student perspective that range from absurd focus on minutiae to misconduct, all of which are destructive to learning.

  8. Make well-designed formative assessment compulsory: The benefits of formative assessment in higher education are well known, but despite that, its actual use in higher education is far from as wide-spread as many would suspect. Our constant obsession with performance rating means that the students are instead merely subjected to what are actually just summative assessments that are claimed to be formative in intent. A common example is a weekly quiz where, one is given unlimited attempts, but the final score of right vs wrong answers contributes to the marks — ultimately, your whole aim here is performance measurement, it is really just summative. The students pick up on this, and it just adds to the summative assessment load pressure across the term — they no longer feel able to make the mistakes that enable them to learn because every mark is sacred. The focus shifts from learning to earning, and the benefit is gone.

    A challenge with formative assessment is providing an incentive to actually do it. The easy solution here is to simply award some small component of the course marks to these tasks in a way that is not tied to performance (right vs wrong) but meeting a demonstrated threshold for a sensible attempt (engagement). This provides the space for students to be wrong without penalty, or even admit to just not knowing, which is useful for an educator to know if it’s a decent portion of the class. It also enables questions that are designed to properly advance learning on key or difficult aspects rather than with summative intent, including some with ambiguous answers deliberately aimed at forcing errors or questions with no single correct answer. The subtle question design shift here can substantially enhance effectiveness of the learning experience for students by forcing them to confront head-on the ambiguities that arise in learning new material.

    However, our obsession with performance-based marking in universities means that formative assessment with marks as incentive is almost impossible without breaking the rules. Using my own institution as an example (since I know the policy, it’s publicly available, and probably similar to many places), the assessment design procedure states pretty clearly that “…the overall course result will be calculated from the marks of all summative assessment tasks.“, “Participation in an assessment task in itself is insufficient grounds for awarding marks or grades.“, “Assessment marks will not be used to reward or penalise student behaviours that do not demonstrate student achievement…“. The word ‘formative’ appears four times in the relevant document, summative double that, and twice in the definitions and acronyms table. And while formative assessment is clearly put as an option, any reading of the policy pretty clearly makes a solid implementation of it impossible. If it carries no marks, why would a student do it, let alone spend a useful amount of time on it, especially when they are under an otherwise crushing summative load across multiple courses? I know what my priority would be under such circumstances.

    The obvious solution here is to have mandated formative-to-summative assessment ratios with clear guidelines on how marks can/cannot be allocated in each category (even if it’s just ‘up to’ caps). In the interests of learning, the ratio should probably be as high as 50:50 in Years 1 and 2, tailing back in higher years, e.g., 30:70 in Year 3 and 15:85 in Year 4, for example. And if there’s concerns that this gives ‘an easy ride’, it is easily solved by making the final exam a hurdle task, i.e., it must be passed in its own right to pass the course, with the final grade being made up of the overall course components to enable performance levels to be gauged.

  9. Stop showing fails and withdrawals on transcripts: Students fail courses for a variety of reasons, many that are outside their control including the fact that the course, the lecturer, or both, are poor. Withdrawals are similar, and recently, often driven by the student’s perception at census date on whether the course will help or hinder increases in their WAM/GPA.

    University is supposed to be an opportunity for finding your place in a particular study area or even just as a person. Sometimes you take a subject with the best of intentions and find that it ‘just isn’t you’. Why should that be held against you for the rest of your life if you withdraw or fail? Likewise, university should encourage you to push yourself. It is the pinnacle of the educational experience, students should want to go after challenging courses. Why should taking on a hard course and failing be seen as a bad thing? There’s still a lot learned along the way, even if you can’t claim at the end to be fully competent in that particular course.

    Transcripts should be about what a student has achieved in competence across their studies, not about tarring them for life with some fails or withdrawals.

  10. Use all of the above to get the culture right: Actions in any organisation are driven by culture, which to some extent is set by the way that environment operates. If you measure performance incessantly and to the point where a person’s whole worth is driven by a single performance metric, then you create severe perverse incentives that drive pathological behaviour where performance becomes the myopic focus of all efforts, even if that is not your core organisational mission (and I’d argue it is not for any university).

    If you want students to focus on learning, you need to create an environment that operates in a way that incentivises learning. Some academics will claim here ‘but that’s what we’re doing, we’re using performance measurement to incentivise learning’. Are you? Really?

    Take a close look at the system, and tell me we aren’t at the point of Goodhart’s law, namely “When a measure becomes a target, it ceases to be a good measure.” What we originally designed as a system for measuring learning has become a target to such an extent that not only is it no longer a good measure, but for the vast majority of students, it is producing sub-optimal outcomes. And if it wasn’t, we wouldn’t have to make the pass mark only 50 to get people through degrees!

    Deciding that endless summative assessment is good for learning in the modern era is no different to deciding that ‘the beatings will continue until morale improves’ as a strategy for managing a workforce. It will only be through a massive change in our approach to assessment in higher education that we will manage to put the ship back on an even keel and create a culture where students are encouraged to learn, enjoy the experience of doing so and feel positive about themselves in the process.

It is probably good to finish with looping back around to ‘why’. Why would you go to the trouble of doing all the things above? There are several reasons:

  • It’s more fair: Would you decide on the quality of a cake based on the taste of a finger-dip before the dough is even mixed? Of course not, you judge it on the final product. In much the same way is it fair to judge the final learning outcomes in the middle of a course or even in the first few weeks? Of course not. All you do with summative assessment during a course is punish students for not yet mastering things they’re supposed to be working on mastering for some point in the future, which is just destructive to learning.

  • Assessment is more reliable: You still measure students, but you measure them in a more sensible and reliable way at a point where you can fairly expect outcomes. Essentially you stop Goodhart’s law from becoming quite so pathological to your aim of measuring learning.

  • You spend less time on warfare: If all your courses are is endless assessment under enormous pressure, of course cheating is going to be an option. Designing assessments that are uncheatable for a few strategic endpoints is easy and sustainable, the sector has done it for decades. And if you get the culture right, the incentive to cheat is low anyway. However, if you want to assess like mad, all the time, and the culture is such that outcomes are everything, then you should expect rampant cheating and you should expect to be in an endless arms race that sees both sides devoting ever more effort to outsmarting the other. Higher education will run out of resources before the black market does, trust me, the black market always wins (just look at the ‘war on drugs’ if you don’t believe me).

  • Student experience will rise: Yes, we can have our cake and eat it to… we can still measure student performance, but we do that in such a way that it doesn’t get in the way of the learning process, which should be fun and engaging and not oppressive or punitive. If students feel that they’re being given completely fair opportunity to learn, including with assessments where they are encouraged, and even given course marks to deliberately to engage, fail and learn from failure, with the expectation that at the very end they will be fairly and rigorously measured on what they’ve learned, you’ll find the student experience will rise. It’s what they’re paying for, not funky cafes and outdoor foosball tables, and if you give it to them, they will be happy.

  • Staff will be happier too: There’s nothing like having an inspiring mission, which is to help people through that same learning experience you’ve had yourself. Crushing assessments inflict just as much pain on the instigators as on the victims. After all, the endless assessment has to be written and it has to be marked. And as the cheating arms race spirals forward, such that the time required for ‘countermeasures’ escalates, one can only expect staff to have less time, less interest and less engagement in education as a result.

  • The staff-student interactions are better: Getting the rapport you really want with students to teach well is very hard when you’re always having to assess them. This is because the ‘staff persona’ has to be different between teacher and assessor. In the former, you are there to help, in the latter, you are there to impartially judge. If you are always having to judge, then it’s very hard to build the sort of relationship that enables you to teach well. Switching between these personas takes time. Separating learning from assessment gives that space — one can build that teaching relationship with students during the term, and then switch to impartial judge during the study break to determine performance at the end. Even better would be to have someone else play the final judge on each course, in addition to improving staff-student relationships, it would also reduce the built-in biases in the system.

Ultimately, it doesn’t matter whether you’re in a higher education system where you charge high fees, i.e., the business of monetising intellectual respectability, or one that is entirely a public service, i.e., fully government funded education system for public good, getting the assessment right is a no brainer. So I’ll end with one last why question: Why the hell aren’t we doing it?

Markissism — Have grade point averages made higher education completely lose the plot?

“I would like to take this course because the subject is highly interesting to me but I am worried about the risk. Suppose I scored 100% in some component, is there any chance my marks could be adjusted down just because other people have performed well? I have worked really hard in my academic career to build up a WAM [weighted assessment mark, see also grade point average or GPA] of XX and I’d rather take another course if this has any chance of dropping.” — Paraphrased from numerous anonymous late year students asking about a first year course.

Of all the years to start a new admin role, 2020 was a rather interesting one. Notwithstanding all the coronavirus uncertainty over international student arrivals early on, the urgent decamp from campus to fully-online teaching in the space of weeks around late March, and the rapid adaptation to change in everything from lab classes to final exams thereafter, probably the biggest eye-opener for me this entire year has been the looming train-wreck that our recent obsession with the minutiae of metrics in the higher education sector holds for the future of student learning.

A favourite story from my undergraduate years is my first tutorial for Higher Complex Analysis in second year. Our first tutorial started with the tutor stating point-blank “Welcome to higher complex analysis. This is a hard course. The typical fail rate is 50%. Look at the person next to you… One of you is going to fail.” To my right, was Mr Sydney Grammar with his ATAR (Australian Tertiary Admission Rank then TER) of 99.95 and to my left, was Miss Elite Private School with a TER not far off. Half the room was the cream of Sydney’s elite private and selective public schools, and I was not from one of them. As Hunter would say…

A modern student (and some follow my twitter, so may read this) would probably ask why on earth I would risk a scraping pass or a credit when I could just walk a HD in the ordinary level course. What about my WAM? The answer is that no one cared about WAM back then, it didn’t even exist, that we students could see at least. What more, we barely cared about marks even, at least in the way a modern student does. A HD was always nice, and a credit was a sign that you could have done better. Besides, the marks arrived by snail mail in the middle of the long university breaks, when our minds were on better things.

Back to the question of why though? Because I wanted to push myself and see if I could hack the pace in higher maths courses. That was the culture, if you took yourself seriously as a student, you set out to take courses because they a) seriously challenged you and/or b) were interesting. Doing easy courses was just boring and ‘being soft’. And in the courses that didn’t meet a) or b), often the mantra was something along the lines of “50 is a pass, 51 is a waste of effort that could have been spent elsewhere”. There were courses that had legendary status in terms of difficulty and they attracted serious interest. Taking a shot was respected, and coming out with a lot less than a HD was totally fine, and definitely not cause for major anxiety.

I think about what would happen if the first tutorial for higher complex in 2020 started like it did in 1994. Some would probably drop the course, but that likely happened in 1994 too. I’m sure the tutor was ‘putting the fear’ into us so we wouldn’t underestimate the course or lack solid commitment (Garth Gaudry was the lecturer, really nice guy but he absolutely did not ‘pull punches’, it was fierce). Nowadays, there would probably also be a stack of emails like the one at the top to deal with, questioning how exactly the marking will work to protect their WAM, possibly even a stack of formal complaints about the tutor as well for intimidation, causing anxiety, or suggesting unfair marking practices.

“Their mask reflects what you seek and that is what makes it so nice at first. A manufactured mirror of your dreams” — Tracy Malone

But why are we not surprised? When we ourselves fill our funding proposals with meaningless journal impact factors and obsess/boast about our h-indices. When in a year full of important crises to solve, key amongst them the ever declining funds available for teaching and research, some see high importance in coming up with yet another global university ranking scheme to measure by and boast about. When our students receive an email soon after their exams with their marks, and a key line is their term and overall WAMs… to several decimal places! And it’s not just my own campus — every campus in Australia and many around the world are no different. Look at any modern graduate CV and more likely than not there is a WAM or GPA or equivalent, prominently placed, given to decimal places. With all of this, we all soon become like an academic version of Narcissus, staring into our pool, constantly all-absorbed by our own key performance indicator, letting it dominate our whole existence.

I find it personally amusing, particularly for science and engineering students, because when you come into a first year physics course, what is the first thing that you learn? Uncertainty and significant figures. You start by asking how accurately you can measure things, and then how to present your results so that your level of certainty is clear. For example, if a quantity is 110 to plus or minus 15, then writing 109.4857843 with no stated uncertainty is probably a glaring misrepresentation of your level of trust in that number. All those decimal places are meaningless garbage with an uncertainty at the +/- 15 point level, and we dock the students marks accordingly in their assessments for bad use of significant figures.

Meanwhile, here we are, presenting grade point averages and weighted assessment marks to several decimal places. The question begs asking: What is our real uncertainty on such a number? From a statistical perspective alone, it’s at the very least the standard deviation in the numbers making up that average. Yet, how many students have all their grades such that the standard deviation is less than 1? The decimal places are obviously moribund from the outset, and we haven’t even got down to the uncertainties in the underlying grades yet either. Amusingly, I can go look up my own transcript, which now includes a WAM (the printed transcript with my testamur from 1997 has no such number). My WAM is a touch over 85, and the standard deviation… 8.7. Yet, there is my WAM in the system as 85.733, presented as though that 3 in 1000 is a truly meaningful digit that an employer can bank on. Am I better than someone with 85.732 and worse than someone with 85.734, truly?

And at least locally, the problem is exacerbated by the culture that’s set in high school, where the fixation is on another single numerical score — the Australian Tertiary Admission Rank (ATAR). However, as quantities they are on very different levels in terms of uncertainty and statistical significance. The ATAR is a number between 0 and 100, given to 2 decimal places, that reflects a ranking rather than a statistical average mark. For example, an ATAR of 95 means you are in the top 5% of your age group (based on age not on year level). The statistical set underlying the ranking has approximately 55,000 students in it, and there is due consideration for the course set taken, relative difficulty, etc. with a rigorous and reported process. It may well be sensible at the smallest increment (0.05) albeit incremental, and certainly at integer level.

The easy mistake to make is that a WAM or GPA is similar. But WAM is not a ranking nor is it statistically underpinned by all the courses of 55,000 students. It is little more than the raw average of maybe 30-odd scores between 0 (ideally 50) and 100, often with no weighting for course difficulty or year level, with an inherent lack of statistical significance or good certainty, i.e., low standard deviation. To treat it as meaningful as the ATAR is would be a huge mistake, yet, for the most part, that’s exactly what most people do.

The disturbing thing is that we as a sector present WAM/GPA on official documents as though it has any real meaning at all — we pull the same bullshit charade that we do with h-index, impact factor and university rankings — with our students as the victims this time rather than ourselves. Given a number so prominent and delightfully trivial for employers to discriminate on the basis of, it would be utterly miraculous if employers didn’t do this and do so en masse. It may not be the thing that decides between two final candidates (I’d hope not, I’d seriously disrespect as utterly negligent any employer who did so), but it almost certainly discriminates the first cut with nary a deeper look at the strengths and weaknesses that make for the standard deviation in the marks. With WAM determining your foot in the door to higher levels of recruitment processes, it’s entirely clear why students live in absolute terror of this number, with its whims influencing their every decision. Particularly given that nowhere have I ever seen a clear statement on transcripts and other such documents about the uncertainty in such a number or warning about the extreme care that should be exercised in its interpretation.

“That was one of the problems with the Narcissus figure. Here is a face looking at a face, and the problem is the image of the thing is never actually the thing. You try and grab it and it’s not there. It’s water. It disappears.” — Jane Alison

There’s a destructive assumption inherently built into WAM, which is that it somehow accurately measures student merit. This is a furphy for a number of reasons. First and foremost is the Matthew effect, namely that students from better socio-economic backgrounds and access to elite private schools are better prepared to come into university, hit the ground running, and score highly in their first 1-2 years of studies. Many students go to schools where they were lucky to be adequately prepared for the higher school certificate (I taught myself both my HSC chemistry electives, plus other components of other courses to optimise my performance chances), let alone prepared in advance to excel at university. For many students, that first or two year of adjusting just entrenches the privilege disparities of high school into their WAM and chances after university, effectively turning meritocracy into stealth-cover for aristocracy. And that’s before we look at other factors influencing ability to excel at university, such as having the luxury of parents who can support you to work full time on studies vs those who need to carry a job to pay the rent and eat.

Second is the lack of any real difficulty weighting. For example, whether you get a 95 in complex analysis or a 73 in higher complex analysis, those 6 units of credit count the same in many WAM or GPA schemes. Yet, who would you consider the better student — the one who took a shot at the hard subject and did well, or the one who didn’t push themselves at all and banked the super-high grade? Here we see our first perverse incentive — it discourages students from doing harder subjects in the quest of fully extending their knowledge. It is like an olympic diving contest were every dive has a difficulty score of 1 — why would you risk a front four-and-a-half when you can just execute a perfect pin-drop?

The third follows from second. Many of the WAM/GPA schemes I’ve seen don’t account for the year level of the course either. This opens an enormous perverse incentive known as ‘WAM-gaming’. The idea being that you invest your electives in taking subjects at low levels (1st year) focusing on those with a reputation for having easier assessment or higher grades, since they numerically improve your WAM. This incentivises behaviour like we see in the quote at the start of this post — students taking courses solely for their impact on a numerical score. Is this what we want to see in higher education? I’d argue it isn’t, yet, that’s what we achieve.

Fourth there’s also not really any factor for the type of assessment either. A course could be almost all small pieces of assessment, such as labs or online quizzes, it could be dominated by a whole-of-term major work, like a portfolio, or it could almost be entirely a tail-loaded pressure assessment, for example, a final exam. All these assessment types carry different levels of both difficulty and access to invigilation (i.e., ability to avoid cheating). And within some assessment types there are also issues with measurement of assessment type aptitude over real ability, as well as poor assessment design.

Regarding aptitude versus ability, the classic example is heavily weighted final exams. There’s already been much written on this, see here or here, but a big issue is that they often tend to test a students aptitude for doing exams more than they test actual ability in the taught material. Like many students, I learned to become good at exams, to put the pressure aside and do all the things that enabled me to snaffle up marks like a dog chasing treats. I’m not convinced they tested my knowledge and understanding that much, even the well designed ones (I’ll get to the poorly designed ones in a moment). They were almost entirely about short-term cramming of huge volumes of algebra, and I was good at the mental tricks required for that. I’ve also seen enough cases of the inverse — perfectly good students who clearly understand the material, perhaps even better than I did and still do, but who seem to collapse in a heap under exam pressure and massively under-perform. And the most ridiculous part of the whole lot — a real workplace is nothing like a final exam, so what is the actual point of this? My job, even as a professional physicist, is not dumping algebra onto a page from my brain under time pressure. Ultimately, we use a contrived and pointless exercise to develop a meaningless number with high uncertainty that then determines career prospects, it’s literally insane.

And that’s before I even get to the highly variable quality of assessment in higher education. How truly poor the assessment design can get is best illustrated by another example from my undergrad student days. A lecturer, who I won’t name, gave a horrifically hard exam the year before us, it was carnage, on raw marks for just that exam the whole class is sure they failed spectacularly. We spent weeks with the previous years’ paper, teasing out reasonable solutions to all the problems — putting students under some time pressure is one thing, but giving them a 3 week assignment to solve in 2 hours is another. Anyway, said lecturer made the amateur error of not knowing we had access to the past paper and giving the same exam two years in a row. Needless to say, we were prepared — we knocked the damn thing out of the park. The lecturer was stunned and told us all he suspected we had cheated as we couldn’t all be that smart. No shit, Sherlock. But the point is, unlike, e.g., ATAR, there is sometimes very little accounting for continuity of quality of the assessment between years or across cohorts in a single institution, let alone between one university and another. There isn’t even standardization of the curriculum and assessment at national level in real terms comparable to ATAR. All of these WAMs and GPAs measure different things in different rules in sub-statistical ways, even within a single university, let alone between them.

I wrote earlier about the uncertainty arising from the statistics of averaging being at the 5-10 point level, that’s before we even account for uncertainty in the underlying 100 point grades for each course. Those too are potentially no better than to +/- 5-10 points in real terms at best, perhaps more given an exam can be a good day or a bad day entirely at random over the cohort. By the time you get to the bottom of the four effects above, you’re probably looking at a real uncertainty up at the +/- 25 level, and at that point, you really can’t say a lot more than not competent, competent and very competent. Ultimately, a measure like WAM or GPA is about as predictive of performance for any job you’d like to choose, including being an academic, as height is for performance in basketball. There is some correlation but if you want to make accurate predictions from the integers or decimal places, you are demonstrating nothing but your own foolishness…

“There is no innovation and creativity without failure. Period.” — Brené Brown

Yet, despite this all, WAM or GPA has come to be the dominant factor motivating student decisions across in many universities in Australia and internationally. It makes students into risk-averse, highly-stressed individuals focused solely on chasing a number, much like the academics who teach them, who are endlessly chasing citations and h-indices, and the organisational leaders that are entirely beholden to whatever it takes to jump one spot in some ranking table or other. From the bottom to top, our obsession with the minutiae of single metrics has made us totally lose the plot in the last two decades, and in particular, totally lose sight of our actual mission — producing a next generation of young professionals that are not just highly trained but well adjusted individuals ready to contribute to society.

We profess to value innovation, creativity, leadership, resilience, and in particular, an ability to think critically in a world awash with knowledge and information. We expect them to think about uncertainty, statistical significance and the search for true meaning in noise. And then what do we do? We construct them a totally bullshit metric that we know should earn you a fail in any proper academic course, and happily turn a blind-eye while everyone buys into it. We sell the first few grips up a greasy pole full of stupid sub-statistical performance measures that are about as much a reality as true measures of achievement as pixies and bats in the desert.

So what is the solution?

I am becoming a huge fan of abolishing WAM/GPA and numerical grading on all courses entirely. Throw the whole lot in the bin, it’s largely meaningless, detrimental to education, and as we’re seeing now in the 2020s, driving a whole massive online industry of contract cheating and proctoring wars that I think can only lead the sector down a path to its own destruction. Contract cheating is as much our stupidity being weaponised for private profit as we see our obsession with impact factor and ‘prestige’ journals being similarly weaponised.

A three-point grading system for each course should be more than sufficient, something like not competent, competent and competent with merit*. The first one is only permanently recorded if it’s not ultimately substituted by one of the other two grades. The third one gives the students something to stretch themselves for. This is more than enough to give any employer an initial triage on competence and ability and enable proper management of degree progression. We can still have marks for individual assessments, it’s a good thing for students to have a measure for how they are performing. But there’s no need for those to be tallied and documented on a transcript.

A breakdown into three-point grades gives students room to push themselves and be adventurous with their learning. It gives them the space to sometimes even fail or underperform on a task in a way that’s invisible, thereby enabling all the life lessons that failure brings, e.g., resilience, ability to be positively self-critical, etc. to be achieved without life-long punishment. The simplified grading system also lends itself to assessment that is more continuous across a course, producing less negative stress for the students as a result, with the ability to implement diversity of assessment type and, where required, ‘hurdle requirements’, to enable proper confirmation of competence.

Ultimately, we have to make a decision about what higher education is about and focus on doing it. Is it about developing students or is it about taking money for poorly measured credentials? I would argue our mission is the former, and if we aren’t careful, our obsession with the latter is going to get entirely in the way of our mission and render us ripe for disruption.

Footnotes

*An alternative would perhaps be a four-point grading system, with not competent, competent (equivalent to current PS/CR) and perhaps two bands of competent with merit, which would cover off the current DN and HD grades. This would in some senses be the minimal change that essentially is just a dropping of all insignificant numerical digits, rebadging of the grades, and proper admission of the true uncertainties involved in assessment.

The ‘mental game’ of exams.

I did a revision session for one of my undergrad courses today, and I like to talk a little in these about exam technique, as it’s a topic that isn’t formally taught and often under-appreciated by undergrad students as an important element of performance.

I teach a 2nd year course, and exams in 2nd year are hard in an interesting way. The exam style for 2nd, 3rd & 4th year is similar but often quite different to 1st year. The answer path is often not as straightforward and technique can have more bearing on your outcomes than it does in 1st year exams. The problem is that’s not obvious as a 2nd year, many students don’t realise it until it’s too late, e.g., they get to 4th year. Certainly, in higher year undergrad exams, simply studying the notes and doing the quiz problems isn’t the complete package.

I talked in the revision session about making sure your background maths skills are tight, learning to both read the question and read ‘into’ the question what the examiner is looking for, how to get part marks well, etc. I might write a separate post on those aspects another time.

In this post, I want to focus on what happens outside the normal thinking about exams a bit, what I like to call the ‘mental game’ of exams. Two points on this:

1. Don’t study right up until the exam: As an undergrad, I wouldn’t study the day before the exam or on the day of the exam.

If you’re a racer (triathlon, marathon, cycling, whatever) one thing you learn is that you don’t train the night before a race. If you do, your performance sucks. Your body is still recovering, it can’t give you full performance. Instead, you ramp your training down and don’t train at all the few days before the race. It gives your body time to replenish supplies, repair muscles, refresh and reset your mind. You wake up on race day firing on all cylinders, mentally and physically.

Exams are like races in many ways. They’re the bit at the end of the training where it really counts. And you want to be on your peak performance for it (more on that in Point 2). If you study right up to the exam, your mind is tired. And it’s still trying to digest all this stuff you’ve been jamming in there for days on end right when you want it to shift focus to calmly working through the problems. If you study like crazy until the last minute, you inevitably get mental blanks and disordered thinking much like you get stitches and breathless patches if you train right up to a race.

I would do other things within a day or so of the exam. It might be some light study on another exam further out if I had a crowded exam timetable but more often I’d just read a book or watch tv or go for a walk or go out with friends. Anything to get my mind off the exam and studying and the topic of the course.

An interesting thing happens when you do this. Occasionally your brain throws messages at you. For example, you’re watching GoT and Tyrion is in the middle of his latest scheme and suddenly ‘hey, I don’t get this thing about the inertia tensor, why was one of the terms on the diagonal zero for that plate thing? Does that mean something?’ If I was in the middle of something, I’d just note the question and sort it out later. If I wasn’t, I’d go away and find the answer. Don’t use this as an excuse to get back into intense study (e.g., ‘oh man, I’m not ready, I have to study more’), just fix the disconnect your brain told you about and get back to relaxing until it happens again.

If I ever looked at my notes on the day of an exam, it was only to answer the questions by brain threw up. Nothing you put in your head on exam day is going stay. I would literally just let my brain say ‘I want you to check x for me please’, and I’d check x. And then I’d find something else to do until it hit me with something else to check.

Sometimes your brain also does the opposite, rather than asking it tells. It might throw some weird logical connection at you, and when you follow it, you realise you suddenly understand a connection in the subject better. Likewise, let yourself follow these ‘brain farts’ just to where they are resolved, don’t let them lull you back into intense study. Your brain doesn’t want more to chew on, it want’s time to chew on what it already has.

I would also do things like go on walks or do mundane things like wash the car (the more boring the better) to try and encourage this because it’s what you want to happen. It’s your brain processing what you studied earlier. But you need to stop with enough time before the exam to let this happen. If you study until the last minute, there’s just no chance.

One warning, if you go out with friends to relax the day before, keep it contained. There’s relaxing and going nuts. Doing an exam with a hangover is strongly not advised (trust me).

2. Prepare physically and mentally for the exam: Another aspect of ‘race management’ in sports is pre-race and post-race. And many of these ideas hold for exams too.

After 13 years teaching, I’ve seen my fair share of students unravel in an exam, it’s an unfortunate reality and no fun for anyone (and I came close to doing so myself once as a 1st year, so I do know the feeling). You chat to these students later, and you find out they crammed for 18 hours the day before, crashed at 2am, slept terribly, woke late, were in a mad rush to make the exam, couldn’t find parking… no wonder they fell apart. Some have been sleep deprived for days, binging on caffeine to extend their study hours. Their mind is screaming away with anxiety, they can’t focus… they’re completely paralyzed by the first question that requires a little stretch in mental effort, and it spirals downhill from there.

This is obviously no good, you simply cannot operate like this.

The important thing to realise is you can overcome this, much like any fear. But… it takes conscious effort and practice. You have to work at it. You mightn’t get it the first few times, and sometimes you have it cracked and there’s relapses, but with work you can get there.

At the very least, you can turn exams from terrifying and paralysing (distress) into just nervous energy (eustress), but this is good, that little buzz of adrenaline that comes from ‘it’s time for an exam’ or standing on the beach in a wetsuit waiting for the starter’s gun is the energy that you can learn to turn into a laser-like focus on the task.

Some tips you might want to think about:

  • Make sure you get a proper night’s sleep before the exam. This obviously is not going to happen if you are studying in the evening, your brain will just be buzzing. It’s another reason I wouldn’t study the day before the exam. Instead I would do what I needed to make sure I got a good solid night’s sleep. Being well rested has a massive effect on performance and boosts you much more than some last minute cramming.
  • Get up at a sensible time before the exam. You don’t want to have to rush and panic, that just makes you stressed and anxious, which sets your whole day up like that. Instead, kick off the day with something that puts you in a good mood. Find something funny to watch. Catch up with some friends you can talk some other stuff with. Play with the dog. Whatever it is that does it for you, put it in the start of exam day. If you set out first thing with a good day, then you are off to a really positive start.
  • Have a proper breakfast. If caffeine is your thing, have some and not too much.
  • Less relevant for online exams, but be there early. Again, this is anxiety management, you want to reduce stressors in the exam lead-up. For an online exam, make sure your space for the exam is ready well ahead of the time and in a set-up you like.
  • Leave some time to ‘get in the zone’ before the exam. Put everything after the exam out of your head; you can’t afford to think about that. As extreme skier Doug Coombs says in Warren Miller’s Journey: “If you’re scared, the world is shaking, you’re thinking about the future, thinking about the consequences… that’s not right, that’s no good.” You’re just not going to be able to function at your peak mentally on a crucial task if you’re preoccupied with something other than that task! Same deal with the swim leg in a triathlon, if you’re thinking about sharks or drowning in the sea, you’re obviously not going to swim well, your navigation is off, panic messes up your breathing, it’s no good. Learn to put bad talk out of your head. Often it’s a matter of just steering your thoughts away from it. Going back to the swimming example, when my brain screams ‘but what about those sharks, hey’ during a race and I feel that sort of pre-panic happen, I just start thinking about my stroke, is my hand entry good, is my breathing pace right, etc. I basically talk my brain back to calm by giving it something else to think about. Work out how to do the same in exams, even if it’s something as dumb as thinking about how cool your calculator is and taking a few slow breaths. On exam day, no amount of positive self-talk is too much. Trust your preparation. Back yourself. If music is your thing to get in the zone before an exam, do that. If talking to people about anything but the exam is, do that. Work out what makes you feel comfortable and get your mind off the anxieties of the exam.
  • Find an easy question to start on. If you get stuck, just pass and go to the next question. Like Point 1, sometimes your brain just needs to process the question for a while.
  • Mentally reward yourself for good answers in the exam (‘yeah, nice work’), especially early on. The positive self-talk before the exam should continue during it. Remind yourself that you don’t need to get every question 100% right to make it through. It’s easy to think you haven’t done enough when you probably have. Don’t be overly tough on yourself, encouraging yourself is better.
  • If you need to stop a minute or two, take a mental break and calm yourself down, just do it. That’s better than continuing on and fighting the panic. This is actually how I bailed myself out of the panic in my 1st year exam way back in the past. I stopped, took a few moments to just get some perspective (‘look, it’s just one exam, it’s not the end of the world, work out what you can offer against the question, grab all the part marks you can, and we’ll just hope that’s enough.’) and then pushed on. Don’t be afraid to do this. There’s no rule saying you have to be writing every second of the exam time.

And lastly… make yourself a time to ‘debrief’ after each exam. It’s one thing that never really comes up in science for some reason, but in sports it happens after every race/game and it’s common in the military after missions too, which often have an ‘exam-like’ stress profile and the same mental management games are required.

  • Try to do your review dispassionately. Look both at what you did well and what you could do better (note the language, not what was bad, not what was dreadful, what you can do better — see next point below). There *will* always be both sides so look for both sides. Take note of those lessons for next time, they are exactly how you get better at this. Remind yourself before your next exam what you wanted to do better last time.
  • Don’t be down on yourself. You did your best, you can’t change how it went, and beating yourself up about it certainly isn’t going to help anything. Focus on what you did well and what the lessons are, it will make you better at exams and it will make you happier too.
  • Accept that sometimes you have a bad day. We all have them. It’s not the end of the world. You’re still alive, you still have friends, you still have a place to live and food to eat. Retain your perspective. Many of your professors have failed exams before, failed courses before, had other things go wrong (I fell off a stage during a talk at a conference once, for example). Same in sports, even the best have come dead last in races, stopped in the swim leg freaking out about sharks, crashed a bike. You definitely aren’t the only one. All you can do is pick yourself up, dust yourself off, learn the lessons and fight on.
  • Irrespective of how you did, reward yourself. You survived! If you can take the rest of the day off, do it. Find something fun to do. Whatever you do, don’t just jump straight into study again.

Happy examing!

Game of Gowns — The spoils of #ponzidemia

We live in interesting times… the worst nightmare for a university sector heavily financed by foreign-student revenues has arrived: A global pandemic. Borders closed, revenues declining precipitously, and a government reluctant to bail the sector out.

Salaries in Australian higher education have been a topic I’ve wanted to get to for some time but I’ve avoided it a little as it’s depressing and inflammatory. There are definitely winners and losers. And like any corporation, the winners are almost universally at the top. Now, the crisis, resulting quarantine lock-downs, and two articles in the media this week have converged to push me over the precipice on a long delayed project.

Let me begin with the two articles. It started for me with Merlin Crossley’s opinion piece in the Sydney Morning Herald this week. I agree with some parts of the article, but one part in particular got my ‘yeah, nah, that doesn’t square up’ going, it was:

2020-04-26_2127.png

That’s not untrue, but if you think deeply about it, that picture is not completely true either. Sure. there’s not a ‘profit’ in the corporate balance sheet sense, where you divide up the revenue minus costs and return it as dividend to shareholders who bought equity in your company on the stock market. But… some people sure make a hell of a lot more money out of this new system compared to the standard of an Australian public university of the 1990s and earlier! They might not be share-holders per se. They’re more employees that get serious returns on the business turning ever greater fee revenues.

The self-interest of increasing pay packets in a climate of reduced regulation, spread across dozens of executives over two dozen institutions and 20 years, has driven what anyone with a pair of eyes and some critical assessment skills can see is a massive corporatisation of a public service once run efficiently to minimise costs to the public who use it. Gone are the days when an entire bachelors degree would set a student back maybe $10,000 at most (or even be free going far enough back). Now we charge them several times that, all for super-flashy campuses that look like Westfield malls and courses so compressed as to be almost impossible to teach and impossible to learn.

Now imagine you’re stuck at home in quarantine, without work. And the education minister suggests you might want to retrain. Since huge numbers of Australians now have an undergraduate degree, those are inevitably going to be postgraduate options without commonwealth support. So, you go look at a Grad. Cert. or a Masters, and see that an online degree is going to cost you $30,000/year minimum. Ouch, that’s a lot of upfront when you just lost an income. The appeal in this article to being simply ‘a registered charity’ and all about the community is going to grate with many, who will have a response ranging from a snort to spat coffee, I suspect.

The other was Michael Sainsbury’s article on michaelwest.com.au a few days earlier. This made an attack commonly thrown against the modern higher education sector, namely that VCs and other senior executives earn way too much for institutions that are public sector organisations. And that if they want to run them like corporations, and earn corporate salaries in the process, they need to accept some of the responsibilities of risk management in the corporate sector, like not putting all your eggs in one basket. The article has some fair points amongst a sea of trying to whip sensationalism at the salary numbers. Some colleagues told me this article was unfair; another told me to go look at the top ASX100 CEO salaries and see if I still think the article is valid. I’ll get to that below…

The truth, in most things in the media these days it seems, is somewhere in the middle. And so I figured I’d try to find the reality, and then use some numbers to point at where things really are. Two days of data mining follow below, much of it raking through 25 years of university enterprise agreements on the Fair Work Commission Website.

Back into history: At the core here is the ‘corporatisation’ of the modern university. No one knows how or when it was decided, it just sort of happened. Historically, the major universities in Australia were public organisations, with staff essentially an arm of the public service. You can see this as late as 1995, where the salaries across the universities were standardised and the academic pay charts of universities matched those in the Australian Public Service Enterprise agreements (EAs). It didn’t matter if I worked at UTS or Monash, Murdoch or Melbourne, a Senior Lecturer (B6) earned $50,111 in 1995 and a Professor (E1) earned $80,176. Higher positions were just loadings, for example, in the University of Wollongong EA from 2000, you can see a Head of School’s loading of $4,648 and a Dean’s loading $20,359 on a $92,968 E1 salary. I’m pretty sure back in the 80s & 90s the executive branch (Dean & above) was nowhere as large as it is now. The faculty was a small office, there was maybe a DVC or two.

The regulation of salaries ended about 1997, and the universities all went their own way soon after. At least at the standard levels B6 through E1. You can easily map them as a function of time. Executive salaries are much harder, there’s little data available, even in the few ‘senior staff EA’s that were popular in the mid-00s. I’ll just do some numbers on them later. There’s also the oft-forgotten workforce of universities — the Ph.D. students — I will get to them also.

Does International Student Percentage Affect Salary? The first thing I wanted to look at was the effect of international students, since it’s something Sainsbury’s article alludes to. There was a nice plot from the Centre for Independent Studies recently, which I show below. International student percentages range from below 5% to as much as 45%, surely that should have some effect on salary.

92816862_10158698826770832_7999876956784951296_n.jpgI chose as my set to analyse 8 from the top 11 in the graph below: RMIT, Wollongong (UoW), Monash, Murdoch, Melbourne, UNSW, UTS and UQ. A mix of Go8 and non-Go8, spread geographically. Some in capitals, some not. I also took three with low internationalisation levels: University of New England (UNE), University of Western Sydney (UWS — rebranding, bah, I’m a Campbelltown kid, it’ll always be UWS to me 🙂 ) and University of Tasmania (UTas). I went for more established universities here, mostly as I really want to get back to 1995 with all of them, so I can see how they all evolved from a common salary level.

To keep this sensible, I went for the top rung of Academic level B, to capture what would be a relatively junior academic but almost certainly permanent to have made Step 6. The other obvious one to go after was Professor (E1); I will deal with the mysterious ‘super professors’ later on. Let’s not waste any more time, onto my first graphs.

B6 Salaries.jpg   E1 Salaries CPI.jpg

The universities with high international fractions are blue diamonds, low international fractions are red-orange circles. The trend lines are average in green and CPI projection of 1995 salary in purple (using RBA’s online inflation calculator). I don’t care much for people picking which campus is which, it’s less relevant, but you can ask me if you want the data (happy to share it). Ultimately, it’s not clear that being at a campus with a higher international fraction confers much pay advantage on the salary at a given level for the non-executive staff. But there’s a missing piece in the puzzle here that requires a quick look at another graph, this one on the breakdown of the academic workforce.

Demographics.png

The data above is national and includes both permanent and casual academic staff. I can see two things in it. The first is the huge growth in Lv A, which is essentially just a big increase in casualisation in the higher education sector. Causalisation keeps Lv A’s at Lv A either through churn (Ph.D. students come, teach, move on and are replaced by new students) or by preventing rising in the system (side effect of churn, you are easily replaced if you don’t like your level). The other is the huge growth in Lv E and to some extent Lv D. My guess, and I can’t support it with available data, is that the fattening of Lv D and Lv E has happened more strongly at the campuses where the international fraction is higher. I can back it with anecdata (i.e., talking to lots of colleagues at lots of different campuses about what their campus demographics look like), but to be solid on this, someone really needs to get the same data as above, comparing between 1996, mid-00s and recent but fine-grained enough to see individual universities.

Let me be clear, what I’m pointing to here is that the profit for staff from increased internationalisation comes less in the rise of the salary at a given academic level and step but more in the ease of access to higher academic levels. The salary tables are always going to be pinned by inter-university competition for staff — it’s the thing a new hire can see when you are courting them during recruitment. Promotion processes are less transparent. And it is not as simple as them just ‘being easier’ at some places than others, there’s also a strong component of Mathew effect involved. But either way, there’s a trend.

If nothing else, it’s much as a colleague at another campus quipped when I congratulated him on getting to Prof. many years ago: “It’s nice, but Level E is really just the new Level B.” And given this, I think for any staff member that’s moved up the system into the ever fattening Lv D and E to say they haven’t personally gained from internationalisation of the system is simply untrue, because not only are you sliding up an exponential but you are jumping up the cascade of exponentials, it’s a double-whammy. This is not insubstantial as a systemic profit because a small win integrated over a large number of people has a pretty big cost. But you certainly can’t miss it when you travel — when people in countries with strongly public (non-corporatised) university systems ask what the salaries are like here, just watch their eyes when you give the answer.

What about the public sector then? To further explore the growth in a given academic level/step, I went and did some comparison to what’s left of public sector science & technology here. The obvious choice is CSIRO of course. In the deep past the universities and CSIRO were closer counterparts in the ecosystem of science & technology in Australia. To do this comparison, I looked at the closest levels in salary in 1995 to B6 and E1. These are CSIRO Levels 6.1 and 8.2, I also tracked their highest level 9M in my data, but only present it in my final graph at the end. To get a ‘control sample’, I also looked at the Australian Public Service system, which is tricky, as it restructured in the late 90s and some of the EAs aren’t available in the early 00s. So I tracked the bottom and top of the APS executive levels (EL1.1 and EL2.7) in recent years, just to see what government non-science salaries are doing.

The results knocked my socks off at first, and I suspect I may instigate an insurrection in CSIRO with the graph below (hold onto your hats kids).

Sector comparison B.JPG

The CSIRO salaries tracked well early. Taking my academic hat off, I’m not surprised, they’re somewhat closer to the applied/industrial side so the salaries should be higher (just the old money-freedom continuum of academia). Something alarming happens in 2012 though. If you look back at the university data, you can see the same inflection in the average line but it’s less severe. I suspect this is the late days of Gillard/Rudd and the quest to regather a surplus followed by the austerity of Abbott/Hockey. Either way, CSIRO staff are losing ground lately, they aren’t even keeping up with salary progression rates for the APS any more!

One hypothesis for this could be that government is withdrawing funding for science across the board — both CSIRO and universities — and the universities are doing a better job of filling the gap by selling educational services internationally. Still, there isn’t a massive divergence here to suggest that internationalisation is massively beneficial at fixed level/step. It’s still the level-promotion aspect I mentioned earlier where I think the big gains come.

I might come back to staff down below, because if I’m getting into promotion, I need to look at the top end. I want to look the other direction for a moment, to the true workhorse of the university research sector: the Ph.D. students.

Corporations love cheap labour: One of the long-standing ethical issues I’ve had with higher education for some time now has been the way it uses Ph.D. students. A lot has changed in 25 years, let’s go back in time before I look at some data.

I was a university student of the 1990s. I did my first year in 1993, completed a B.Sc. (Hons) in Physics in 1996, and then did a Ph.D. from 1997 to early-mid 2000 before leaving for the US for a postdoc. The only reason I could afford to do this was that university was relatively well subsidised by the public of Australia. I grew up a few blocks from one of Sydney’s notorious nasty suburbs (Airds) in a family of 7. We weren’t in total poverty, but we were hardly well off either. It was only Whitlam’s legacy that meant I even stood a chance of getting where I am now. I could suspend the fees and not end up in massive debt in the process. I finished my 4 years with a HECS debt of $9,700 ($16,742 in 2019 dollars). Were I to do the same degree today, I would be up for $38,100 ($22,000 in 1996 dollars) with domestic fee support. And without that, I’d be in debt to the tune of $191,000 at the age of 21! Nonetheless, owing $38,000 and owing $9,700 are a big difference at that age, especially when you look at the massive differences in employment opportunities, wages growth and the housing sector between 1995 and 2020.

A Ph.D. was also a different prospect back in the 1990s. The stipend was reasonable, the cost of living in Sydney was even more reasonable, and the future prospects were quite strong on the opportunity side. Postdoc positions were easily obtained, and it was clear that plenty of academic positions would be opening in the future too. I found a $150/week apartment in Bondi Beach, a place where no one of sane mind wanted to actually live in 1997 it should be said (it took the olympics and reality tv to bring them back). I kept my finances tight, and got to work.

If I look to 2020, I really do have questions about whether I would make similar decisions to the ones I did in 1995. The ratio of RTP to cost of living is now, quite frankly, poor. The job prospects are similarly dire, to the point where the postdoc in Australia is becoming rapidly an extinct species (see my earlier post on this). And as for academia, why would you want to get in on the bottom level of an obvious Ponzi scheme, especially one this ripe for financial collapse.

That said, I’m totes recruiting, kids, because… well… do I have any choice? Generating the ever-growing output metrics required to compete in this system on my own, with my teaching and admin load, yeah right. And we can forget hiring postdocs in the not-too-distant future, they’re becoming a luxury you cannot afford without multiple grants (plugging my post again, so shameless 🙂 ). What one really needs is Ph.D. students, they are the labour force of modern academic science after all. They truly are the ones who get the real work done, and very cheaply too. To quote a certain past PVC “Why hire a postdoc? They’re too expensive. No, ask for the postdoc, but when you get the salary, hire 4 students instead, it’s so much cheaper”. I’ll leave debates of efficacy for the pub, but let’s drill into the economics on Ph.D. students. I’ve wanted to do this for a long time now, and I keep putting it off as it’s incredibly depressing. Let’s do this in two graphs.

Wins at the top log.jpg

The plot above has E1 and B6 data from earlier, along with APA/RTP in green and the salary for an Australian Public Service Cadet in red. I’ve gone with the log axis to make this less embarrassing (you’ll see why if you read that other post I keep plugging), and I’ve dashed some of the CPI lines for distinguishability. Before I unleash years of pent up bile, let me drop my second graph on the table.

Life on PhD stipend.JPG

In the second graph I go from annual to weekly and I dug back into my archives for some of my budgets from that era. In 1996, when I started, my rent was $150/week. There was no way I could do a Ph.D. with a 1.5 – 2 hour commute each way each day. I had already spent a decent part of my honours year sleeping on the couch of the Physics student society common room (and made good friends with the cleaners and security to get away with it; people tight for cash can always spot people tight for cash). My living costs were about $100/week, tinned spaghetti on toast was a regular meal. That left $40 discretionary, which was a reasonable amount in 1996 dollars. Thankfully, there was casual teaching on campus and my supervisor gave me a top-up, the same $5,000 amount we have now actually, which hasn’t moved partly to prevent financial arms races for students and partly, more recently, because funding is always tight. It is one of many things associated with Ph.D. student funding that shifts at truly glacial pace (travel scholarships are another), especially when compared to how rapidly salaries across the board, and particularly at executive level, have grown. Did someone say we were reinvesting the revenue back into the system? Because if we are, the Ph.D. students certainly aren’t seeing much of it! (n.b. some universities are shifting on this, to their credit, but it’s hardly universal).

It’s interesting to project my scenario from 1995 forward. I’ve had the APA data needed to do it for a while, it was just knowing how ugly the numbers were likely to be that stopped me. The black diamonds are the base income, and the tricky bit is rent. I’ve taken the rent and done three things: a simple CPI projection is the purple line, a proper analysis using rental yield history data (ABS, happy to supply, but it’s a distraction to show) in yellow, and the real rental in red. I just use CPI for living costs (it’s what CPI is after all 🙂 ) and then discretionary is APA minus rent and living costs. It looks ok if you assume the projected rent, except when you realise that $40 in 1996 dollars is $70 in 2019 dollars, yet the discretionary spend is $40 still in 2019. The APA/RTP really has been shaped to ensure it’s ‘just enough’. To highlight that, I’ve put in the poverty line (50% of median salary) for both 1995 and 2020, and yes, I’ve already had one person on twitter comment that they are glad to see it there as they always thought it felt that close.

The problem is… the projected rent data is a national figure, and as we all know, universities are typically in cities and often close to the CBD. Rents there grow much more explosively. I first saw this just before I moved out in 2000 — the olympics were coming, the landlord wanted to increase my rent by 20%, but preferred if I moved out so they could charge even more (letter was something like, if you intend to move out by date, we will forego the increase). It got smashed again in 2016 when it got sold (possibly as deceased estate) to an investor. Either way, rents in Sydney are nothing like the yellow projection and it’s easy for an APA/RTP to evaporate on housing costs in the major capitals. Casual work for Ph.D. students in the modern era is a necessity rather than a way to buy discretionary spend.

But let me come back to the first graph for a second, because when you look at it after the second graph, there’s something sinister. Firstly, the APS Cadet salary, which started well below the APA, has actually risen above it with time. The thing to realise here is that the cadet salary is, essentially, a short-term paygrade in the APS for interns and temporaries (e.g., vacation students). No one is meant to be on cadet pay very long. But when you take an RTP, you’re on it 3-5 years. There are Trainee and Graduate Trainee levels above cadet in the APS that, at 2018 rates are $43,750 and $58,231, i.e., 162% and 215%, of an RTP.  To put it bluntly — in salary terms you would be over two times better off going to work for the Australian government as a trainee instead! And that’s before we even get to the private sector. The fact that the APA/RTP has tracked like that on real world terms is remarkable, the only thing I can think to be as bad is Newstart. The people who generate most of Australia’s scientific productivity get paid peanuts to do so folks…

And before people tell me that the lower pay during an RTP pays off later as a post-doc and academic, let me point out that it did, once, back in the late 90s and early 00s. But then we used internationalisation to build an oversupply of Ph.D. graduates because we were addicted to them as a cheap labour force. It enabled us to funnel money into academic salaries and fancy buildings when the one thing we absolutely should have been doing was building a sustainable and equitable sector. Sainsbury does have a point in his article, he’s just missing some parts of the dereliction of duty of higher education management collectively in Australia. This is not just one VC or university alone. Stating names and numbers is a sideshow. This is over two decades of endless short-sightedness and ‘get-rich-quick’ schemes and league-table games from an entire sector of people demanding the sort of salaries people pay to have things managed properly and then utterly failing to do so. Maybe academia does need to be burned to the ground, because I really don’t know how we fix it in the state it’s currently in.

Back to the top: I want to come back around now onto a final graph to finish up the discussion. The thing that caught me with Sainsbury’s article is that it’s full of series of large numbers, and numbers are hard to accurately gauge as just numbers because emotion can become your ruler and emotion has a non-linear scale. I prefer to see them on a graph with a bunch of other things, and so I set out to make that graph.

My colleague of the ‘E is the new B’ quip saw me post the Sainsbury article somewhere and threw the top 10 ASX 100 CEO salaries at me, said ‘Are the salaries that large? They are very modest by industry standard.’ Sure, but last time I checked, we still worked for the public sector. Our job is to train the next generation and produce science not maximise revenues and generate growth. It’s exactly as Merlin says (although the registered charity thing I still find a little bit of a stretch given we put students into $38,000 of debt at age 21 as part of our operation).

The thing about CEO salaries is that people only see the big ones. Those big salaries are the exception rather than the rule, and using them is a convenient way to make the VC salaries look small and shrink the appearance of greed and self-interest involved. The data in the figure below is in part my own, and some from the Australian Council of Superannuation Investors (ACSI) Annual Survey of ASX200 Chief Executive Remuneration. It does a good job of covering the true CEO salary spectrum. Note that below there is a ‘fixed salary’, which is what they get paid even if bonuses are zero, and a ‘realised salary’, which is the whole package, some of which may be equity (whether it is liquidated immediately or not is a separate question).

Barplot.JPG

To explain the colour-codes: green is corporate, blue is public service, red is academic and yellow are just benchmarks for context (my control samples). I had to put a log scale on this as well, same reason as before, it’s less embarrassing (can share the non-log version later perhaps). I’ve also included what I call a ‘super professor’, these are a growing breed in the internationalisation era — people of sufficient ‘merit’ that they sit above the publicly listed pay scales. I’ve had to make an estimate for what they get paid, and this might be towards the top end of it across the sector (folks can comment, it’s hard to get good info on this for obvious reasons). The ‘super-professor’ is essentially the core of my colleague’s comment about ‘Lv E is the new Lv B’, Lv E1 is just the entry point to a new range of clandestine pay-scales one can access. The transparency on them varies — some campuses just don’t advertise them others will try to deny they even exist. In some ways, you can consider these as returning an extra share of revenue to a special class of employees — more what a corporation does than a public sector organisation. Then there’s all the executive salaries, they are unknown and are not just academics — I’ve heard several cases of non-professorial admin staff on upwards of $0.5M in this sector. The VC is always the top, obviously, and if you look at that graph, they truly are in amongst the corporate salaries now. The VC minimum there is an exception rather than a rule too — it is one single VC that’s below the PM (see here — also I don’t consider the University of Divinity to be serious, sorry.)

The interesting thing is if you extrapolate the VC being the top salary in a university to the public sector. Clearly in the public sector the top of the pyramid is the PM, right, and even the education minister would outrank any individual vice chancellor. Yet, the minimum VC package is nearly double the education minister’s salary, and the highest has a multiplier exceeding 4! You can possibly see why the Education Minister might be a little unhappy about the salary situation given the graph above… and laugh at any suggestion we’re still a public sector organisation in dire need of bail-out. Executive pay cuts of 20% are barely a sip of the glass when you look at the real numbers!

You’ll also note that the VC salary bracket sits neatly amongst the ASX 101-200 salaries and well above what’s typical for CEOs outside the ASX200. As Sainsbury points out, we’re way beyond even the maximum end of the charity sector on CEO salaries, let alone the typical (don’t forget the log scale!).

For some final context, let’s consider VC salaries in a historical context. Let me choose two examples here. Prof. Spence at U. Sydney has a 2017 salary of $1,445,000, which translates to $845,000 in 1995 dollars. I’m pretty sure VC salaries were nowhere near that large back in 1995, given an E1 was $80,000 and a Dean’s loading would have been less than $20,000. Maybe someone long retired can go on the record and tell us what the VC salaries were really like back then. And there’s always the old chestnut that you need to pay good money to get good leaders who give good results. If that’s true, then clearly Sydney University should be well ahead of ANU, given Brian Schmidt does the same job for reportedly $618,000 in 2017. This is $361,500 in 1995 dollars, and might be somewhat more realistic to an actual VC salary in those days (albeit a fat one perhaps). Amusingly, ANU is ranked well above U. Sydney — so much for money buying performance.

I think it’s quite clear there are definite winners and losers in the 2.5 decades of internationalising the higher education sector and that some have profited very handsomely from the whole exercise. It makes any claim that this is ‘all for the community’ likely to be received a little distastefully I think.

It’s clear that the closer you are to the top and the earlier you entered the better your wins are. The academic system was historically easier to navigate. Grants were easier to get and more likely to be both funded and fully funded. The publishing system was easier and the productivity demands in terms of volume and rate of output were not as severe and more easily resourced. This fed heavily off ramping international enrolments and investment in the 00s and easy access to cheap labour in the form of Ph.D. students. Much like CO2 in the atmosphere, in the early days, the swelling numbers of Ph.D. graduates and young postdocs seeking fellowships didn’t matter — lots of the old guys were retiring, particularly since the old super system was so generous. There was still room in the funding systems and department structures. Positions were filled, and then things got tighter and tighter as the years went on, early 10s were hard, late 10s got really tough. It’s been glaringly obvious for some time there’s a big resourcing problem coming. I can’t work out whether the folks running this show are ignorant, negligent or willfully blind to it. Do they not know? Are their heads in the sand instead? Is it the next VC’s problem? Where are the opinion pieces pointing to a solution rather than trying to shove the bill for the expensive dinner back onto the taxpayer?

One thing’s for sure, higher up in the system, the Mathew effect tends to protect you. Lower down, it’s rapidly becoming carnage. You’ll have seen Darren Saunders talk about the issues on the funding front, and he isn’t the first or the last of the researchers now in their 30s & 40s who will end up getting closed out of the funding system and need to put their research efforts on ice. Everyone I know on campus in their 30s & 40s is sure it’s coming for them. For all the bold talk about revenues being invested back into research and teaching operations, it always seems to be the funds that keep labs going when the external funding runs dry that get cut first. Meanwhile, certain labs flush with cash get more funneled in, so much that they sometimes barely know what to do with it. We all know where the best equipment is! And the pads and plastic pens and plastic rulers with the research centre logo on them are when the stationary supply runs dry. And it’s not like the 80s where you probably got funded unless your proposal was really crappy; now your proposal can be supremely awesome and still miss out again and again. It’s only the upper 2-sigma tail that gets any external funding these days. The reality of ‘reinvesting back into research’ is that most young researchers are running on fumes, burning themselves out and destroying their mental health in competitive processes that are massively stacked against them.

Nah, we (and I mean the sector as a whole) built a Ponzi. We got told to look to international opportunities to help supplement the extra money we kept asking for. We aren’t responsible for being asked to do that, sure. But we sure as hell are responsible for the way we responded in the follow-up.

We could have done that maturely, with an eye to sustainability, without being like hungry pigs to a money trough. We could have done it in such a way that we didn’t jack up the fees on our local students to saddle them with five-digit debt levels at age 21. We could have done it in such a way that students weren’t being admitted despite insufficient preparation just to rake in money on fees (and without then nuking people for calling it out as a problem). We could have done this in a way such that the people who generate the results at the coalface and do the painful parts of the teaching — our Ph.D. students — don’t have to live just above the poverty line while watching the VC’s annual salary grow to exceed the median house price in our most expensive capital cities. We could have invested in sustainable research funding and management structures so that individual researchers don’t have to ride a rollercoaster of lab bankruptcy when a grant proposal fails and then having to try and reactivate when some money finally rolls in and then look at going bankrupt again, all the while watching more and more new ‘strategic hires’ get helicoptered in to fight for the same shrinking pool of cash. We could have not treated the system like some mix of get-rich-quick scheme and league-table pissing contest.

And now, as a sector, I think we are all going to pay a very handsome price for our folly. All the gnashing of teeth and pleas to not our fault are not going to get us very far. Some of us will walk away with some very fat bank accounts. Others? Well… Death Amway.

Working from home: How to get sh*t done.

Coronavirus has forced a lot of people who normally don’t work from home to start working from home. I even have academic colleagues who, for the first time, have had to buy a desk and a chair and get a proper home office running.

I have been working from home for years now. What started as bad (workaholism) evolved into a practice of working from home 1-3 days a week on a regular basis depending on my tasks. My typical is probably 2, I will come down to 1 if there’s a lot requiring me on campus and I will come up to 3 when I need a higher density of solid blocks of focussed time (writing grants or a new course). I’ve learned a lot of lessons along the way, so here are some tips to those new to this game.

Initial disclaimer: Yes, I don’t have kids, and yes, I know not all of the things below work or can even be implemented by everyone. This is just what works for me, take what you want from it. I think the main point below is to find your own way…

No particular order on these points, mostly because they kind of tie in with each other in multiple ways.

1. Environment matters: Probably top of the pile for me, if your environment sucks your productivity will suffer. It’s hard at short notice, but try to build a working environment conducive to work. I long ago invested in a proper desk and chair, get the ergonomics right. If you can, try to get your IT set-up similar to work too. I have the same keyboard and trackball at home as at work, I also run the same monitor setup (2 x portrait side by side plus small landscape on the right — laptop at home, old monitor at work) and I run the same file system at both too (C: OS and D: my files, which I sync bidaily using FreeFileSync and a flash HD — nice side-effect, I always have 3 backups done daily!!). I run the same software on both systems too. Moving from home to work and v.v. is completely seamless for me and not reliant on net-stability. I can literally stop at work, sync, ride home, sync, and get started again.

Thinking beyond my desk, make sure you’re in a quiet area, good light, good airflow, nice outlook if you can get it, and most importantly, a place where you can minimise distractions. If you like silence, do what you can (noise cancelling headphones, earplugs, whatever). If you like music, get that set-up well.

It might sound like I invest a lot in this… yes absolutely. If your environment is unworkable, your productivity suffers and eventually your income will too. This really is spending money to make money, and we all know you need to do that sometimes.

2. Plan & prioritise: You should have a good plan even ‘at work’ but it becomes more important when working from home because if you don’t plan your time then no one else will. In the current Coronavirus ‘remote working’ world, with many things becoming asynchronous, your calendar will stop planning your time as well. This can be dangerous in many ways. Firstly, working at home it’s hard to keep tabs on your time — some will work a lot less, some (like me) will just end up working more, even to the point of working to burnout. Also, working at home, it’s easy to lose focus and devote time to things that don’t carry impact. Ultimately, you can easily be like the truck stuck in the mud, wheels desperately spinning, but getting no traction and going nowhere.

When you work from home, make sure you have day, week and month plans. Have ‘to do’ lists at all three scales and prioritise them. Tick things off when they are done. Not only does this help you know what to spend your time on, but it helps you realise how much you are getting done. One thing about working from home is that you save some time in your day from the commute to and from. For me it’s about 1 hour a day, possibly a little more. I tend to reclaim half of this as personal time and use the other half as planning time. One clever trick here: if you can, put your planning time on the end of your exercise time. You can then use your exercise time to work through all the thinking, and then just empty your mind onto the page once you get home.

3. Big blocks: I like big blocks and I cannot lie… in fact in normal times, my work from home days are my ‘big blocks’ days since they are the days where I can knock out most of my distractions. The need to do this in a coronavirus remote working scenario is even greater — with everything going asynchronous it means there’s always stuff flying all over the place and your time is always chopped to pieces. Without big blocks, you just cannot get major tasks done. You spend your whole life being reactive rather than proactive.

For me, on a ‘big blocks’ day, I have only two tasks scheduled: a main and a reserve. The main can take up the whole day, and my big blocks days are the days I’m most willing to work ‘over time’ because of the way ‘flow’ works. It takes time to build momentum into big block tasks, often 1-2 hours, and I don’t want to lose that investment while I’m still getting high output from it. I will attack that main task until it hits a wall, which will either be: a) I finished it, b) I’ve hit a roadblock I can’t solve today, or c) I’ve gone all day, my output is waning, and I need another solid block of time to finish the task. The reserve task is there for when this happens — if I still have enough time left in the day to put solid hours to the reserve I will switch to it and dig in. I won’t always switch to reserve though, if I can’t do it justice, I will often just turn the rest of the day into mopping up pieces (delayed emails, planning, etc).

4. Getting in the zone: Basically this is knowing how to find ‘flow’ and is particularly important on big blocks days. For those who don’t know what I mean by ‘flow’, it’s that mental state you get in when you are heavily engaged in a task — all the distractions fall aside, there is just you and the task and you smash away at the task. It is particularly useful for any writing task, almost to the point now where I don’t want to write unless I know I really do have several hours to push through the ‘flow finding’ phase to proper flow. What you need to do is learn how you find that place for yourself quickly. For me, it’s most effectively done with a) push out all distractions, b) 2-3 minutes somewhere quiet to prep myself for the task — what am I going to do, remind myself of the important bits, etc., c) get the right music going, settle in at the desk, d) either edit the few paragraphs before where I need to write (or if it’s a blank page write some rough rubbish close to the topic), and then e) hopefully I just slide on in to where the work’s needed and get rolling.

Finding the zone is like good running form or swimming stroke. You have to work on it. Find what works for you. Critically analyse your approach, work on developing it.

5. Cluster the small stuff: As much as possible, cluster all the little tasks together and most importantly, don’t let them chop up your big blocks. That doesn’t mean I completely ignore anything small in a big block, sometimes a tiny easy task is a nice ‘mental break’ but do them on your terms strictly — they are breaks not distractions. I usually keep at least one day for small stuff and usually that’s the day with the most meetings in it (currently Friday for me). I also keep a little block of it on Monday morning, partly to ‘shovel snow’ from the weekend and partly because easy tasks are a good way to ‘restart the engine’ after the weekend. The other place I like small stuff clusters is in a big blocks day when the reserve task isn’t viable.

6. Schedule start and finish: Perhaps obvious, but no, you probably shouldn’t sleep to 1pm working from home and you shouldn’t be going until 5am either. Try to keep somewhat normal hours for yourself. On a work from home day, I’m always ‘at the desk’ by 9am without fail and I will have a finish time for myself as well that sometimes depends on my plan for the day. I am in bed by midnight without fail also and rarely work right up until then, it has to be some really exceptional flow on a task that warrants running late. As a famous football coach once said, ‘nothing much that’s good happens after midnight’ it holds as much for work as it does being on the town.

7. Take breaks & get exercise: Still important and you should see working from home and the flexibility of schedule it affords as an opportunity rather than a cost. For example, I do some mix of running and swimming to stay in shape. In the normal 9-5, swimming is hell, the lanes are always most busy before 10am and after 4pm. I make my work from home days my swim days and my office days my run days. I then put my swim either 10:30 to fit between the morning crowd and lunch crowd or 2:30 to fit between the lunch crowd and afternoon crowd. Aside from getting a relatively peaceful swim in, I have the added benefit of being able to break up my day, sometimes use the exercise as thinking time, etc. I sometimes also even jam a nap into a work from home day, something I could never really do in the office. Set a 30 min timer on the phone, crash on the lounge, get back to work after. A key thing in working from home is learning how to maximise your energy and effectiveness in the process. Little things like building exercise and rest in well really matter.

8. Eat properly: Easy when working from home to not keep to meal times, snack all over the place, eat half the pantry as a procrastination tool… Might seem obvious and trivial but if you don’t stick to sensible eating, your energy and focus will lurch all over the place making you less effective not more. Stick to routines and keep some discipline on this aspect. I have my usual breakfast on a work from home day, and a scheduled lunch and dinner. I usually don’t have open snacks in the house to reduce temptation on this front — the open is important, if there’s nothing open it’s less tempting, still need something around if friends pop over.

9. Shut the world out: Really essential to minimising distractions and getting flow going. I never have the television on when working, ever, no radio or podcasts either. If it’s music it’s albums or streaming with no adverts and it’ll be stuff that doesn’t chew too much mental effort up, i.e., music I’ve listened to a lot before so the novelty of it isn’t drawing neurons away from task. If someone is talking about stuff in whatever I listen to, boom, the focus is gone. It’s why people have to walk into my office and scare the crap out of me at work — my noise-cancelling headphones are my tools to shut the distracting voices out.

I sometimes even take this to extreme levels as a focus strategy, putting a single album on endless repeat through noise-cancelling headphones just to ‘lock in’ to the zone. For example, my review on 0.7 anomaly was almost entirely written to Guns’n’Roses’ ‘Chinese Democracy’ album [high rotation for several months, I know every note of that album] and one of my ARC grants this year to Hole’s ‘Live Through This’, which I’m also listening to while writing this. Whatever album it is, it is on only when I’m working on that task, and it’s almost to the point where I train my brain like Pavlov’s dog that that music means ‘focus all to writing task’.

Blocking out the world also means the internet. Hide your phone in the back of the lounge, turn your email browser off, ban yourself from social media. Shut down the things that send pointless notifications (Teams is especially bad for this, I hate Teams). I’m sometimes a little relaxed on this, it depends on the task because it can also make a good mental reset for me, but it needs to be short doses. Sometimes you want none at all. Writing this post I haven’t looked at anything but the text on this page…

10. Brief & debrief: The first half is kind of obvious, give yourself 5-10 min at the start of the day to think about what your strategy for the day is. What are your main tasks? What are your priorities? What type of day is it? Big blocks, lots of little things? When are your meetings (if any)? What’s the smartest way to assemble the day?

The second half is often not. People get to the end of the day, and they just stop working without looking back at the day. When you finish up, find 5-10 minutes to ‘debrief’ your day. How did it go? What is unfinished that needs to be put into a future day? What did you get done and what was the most effective part of the day? What was the least effective part of the day and what lessons are in that to get better at working like this? Celebrate your little wins, just observe what didn’t work perfectly without being down on yourself for it.

Some of this is managing your own morale and expectations. Seeing that you achieved stuff at the end of the day keeps you feeling ok about yourself. It also helps you become more realistic about what you can and cannot achieve in a day.

I put this one last because this is going to be particularly important in the Coronavirus hellscape where we have to work from home every day, again and again. This is working from home ‘for the long haul’ and it requires some extra effort in mental management. It’s going to be really easy to feel demotivated, unengaged, unproductive and unhappy if you really don’t have good methodology for working from home. But it doesn’t have to be this way if you focus on evolving to a situation that you can work with.

Bonus 11. Try not to punish yourself when working at home fails: A big suggestion for beginners is to forget perfect on this, it never happens, even for seasoned ‘work from home’ folks like me. Some of my work from home days in the past have been epic. I’ve started 9am, get to 10pm and written half a paper, and jammed in my meals and a swim. I find my zone easily, the flow is fast and strong, everything seems to hang together. I wind down with an hour and a half on the bass and head off to bed feeling awesome about a massively productive day.

Others are bloody awful, the main task ends abruptly at 10:15am, there’s a book I need and it’s on my shelf in the office. Continuing without it is a waste of time. It’s super-annoying as I was primed for days to smash that task. I go for a swim to try and get over it, come back, start on the reserve task, get an hour in and I’m just not feeling it. Nothing’s working, can’t find the zone, want to smash something because my brain keeps stewing on the main task for the day. I can’t make my brain refocus, I ask nicely, it calls me a jerk. I go to small tasks instead and just pack it in at dinner time as the day is a write-off. Sometimes it just doesn’t work, the mojo isn’t there. Not much you can do.

Don’t rip yourself up about working from home being hard, there’s not much good in it. Just treat each day as a separate day and look at how to get better at it with time. Try to avoid the unrealistic expectations generated by HBR and inc.com and stuff on this… the management gurus and associated meritocrats are always looking to turn your productivity into their easy profit. Ignore them as much as you can for anything but tips that seem easy to try and abandon if they fail. The better approach is to just hold to a process of plan, brief, put in a day’s work, debrief, take your wins, sleep, rinse, repeat. Focus on the long game, try to put the little daily ups and downs to the side. You know what you want to get done, pick the important parts of that, divert as much productive time as you can to them, and try to stay positive along the way.

In the end, all you can do is all you can do, right?

The real merit in emeriti…

Life has just reminded me that I’ve been wanting to write this one for a while. I won’t go into that, but a question I’ve long contemplated is: What is the point of emeritus professors? Why have them? What value do they even bring?

Most of us have crossed our fair share of emeritus professors. The old guys (they’re almost universally male) who, on reaching forced retirement at 65 or thereabouts, decide the last thing they want to do is stop being a professor, and so in exchange for a salary to redeploy on someone much younger and, in many cases, working on something that’s important now rather than 30 years ago, the university gives a title and a desk and an institutional affiliation.

Many are quite benign. They come and go for a little while, slowing down as the grant money runs out and they gradually find more interesting things to do. They probably don’t add a lot to the department as a whole, but they don’t cost it a lot either, so why not? It’s like the academic equivalent of a golden handshake, I guess, since many of us seem to see more work as a reward somehow (strange, I know).

Some range from being mildly annoying to a right pain in the neck. Continuing to pursue their own interests, even if they’re sometimes 100% anti-parallel to the interests of the department, via some mix of Machiavellianism, threats, intimidation or militant resistance. I will spare said emeriti the indignity of having their sins laid bare here, but I’m sure enough of us have seen it for my earlier words to be justified. Others do their best to keep the department culture and/or demographics trapped in the 1960s or 70s or whatever decade they consider to be the golden years of academia.

And the trouble with an emeritus position is that it’s easily granted from above and then conveniently forgotten, while the holder lives on another 20-30 years, popping out of the shadows occasionally for a short spree of trouble like a departmental cold sore.

And then, more rarely, there’s the really good emeritus professor. Some of us know examples, often on other campuses as they aren’t nearly as common. And when I’ve talked to people on this topic in the past, it’s interesting how a small handful of names always get mentioned. These are the ones who bring true value to the role, which becomes more like that really nice grandparent you sometimes met as a child. They know everyone in the department, and not only do they get along with them well but they see it as their role to advance their colleagues’ interests as much or even more so than their own. They are helpful with advice, not in a smug or arrogant or condescending way, not chewing your ear off or pontificating loudly at every morning tea, but more in the way that people all feel like they can go seek their council when needed, without judgement or self-interest getting in the way. They realise that their job, as an emeritus professor, is to build legacy, not by pushing to grow their own research, that time has passed, but by being there for the department they worked in, helping to advance the next generation by offering them their one key advantage — years of experience at having to work through many of the issues that their junior colleagues are encountering for the first time. And that’s often less a job of telling people what to do than hearing them out and then guiding them to their own smarter decisions bearing in mind that modern contexts can often be quite different to 20 or 30 years ago.

I think universities could be far more discerning in their decisions of who to give this role to, and more the case, they should completely change their definition of ‘merit’ from thinking about just E for academic excellence and more to thinking about the other two letters ‘us’. Yep, it might be a terrible pun, but the merit in emeritus should be about excellence for us, namely the department that will be hosting them for however many years. There should be wide consultation from the department on who gets given the role, from the top to the bottom. There should be references sought, selected by people other than the candidate. There could even be an anonymous vote within the department that requires more than 2/3 to vote yes for the role to be granted. The role and requirements should be more carefully specified, and more tightly focussed on ability to humbly serve others using outstanding interpersonal skills and impeccable ethics and morals than on past research performance alone.

Start looking at merit the right way, and we might start seeing more of the good ones and a lot less of the bad ones.