Wednesday, 5 April 2023
Alongside a longstanding interest in research methods, I do research on strategy, leadership and change. I am delighted to share that the second edition of Strategists @ work is released tomorrow and you can access a free sample chapter here. I'd be interested in your views on the book and hopefully you can see how the research that underpins the book was conducted.
Monday, 15 November 2021
I created the PhDBlog in 2009 which seems positively ages ago. At the time I was responsible for an Executive PhD Programme at the University of Glasgow and spent a fair amount of time on the phone (remember those) to potential applicants talking through the challenge of doing a part-time doctorate. Those conversations tended to follow some familiar themes. What should you study? How would you study it? How would you find a supervisor and how much time will it take? A former colleague suggested that I capture some of these topics in the form of a blog.
Having never heard of a blog, I was sceptical to say the least, but I found my way to the Blogger.com tool and spent a few afternoons creating the early posts on this website. I then used the blog as a useful resource, a form of FAQs, for potential applicants. "I'll send you a link, read the material and then call me back" I would say when my phone rang about the PhD programme. Beyond this, I forgot all about it until one day, years later, at a conference someone said "I love your blog". Unaware that I had a blog, it took some time to figure out what they were referring to.
By that point I was working at Heriot-Watt and had to provide Google with a scan of my passport and a 1 cent transaction on my credit card to prove I was genuine in my claim that I had forgotten the log-in details to my own blog. When I did regain access to the site, I was amazed to find how often it had been read. I was more amazed, and a little embarrased to find a stack of unanswered queries from readers. With the help of another new colleague, it was rebranded as ThePhDBlog, I bought a URL for it and commissioned a lovely logo. I still can't quite believe that at one point is was Google's top answer to "what's the difference between ontology, epistemology and methodology?" ... this is still, by a country mile, the most popular post on the site. Over the last 5 to 10 years I have been more disciplined in answering queries from people and peeked behind the curtain to read the stats every now and then.
Last month, the site passed through half a million reads making it the most read thing I've ever produced. Thank you to those who have read the contents and reached out to say that they found it helpful. There are far fewer comments these days and the vast majority of those are from people offering "thesis writing" services so they get blocked and deleted by me. However, I remain passionate about research and about the joys / challenges of undertaking doctoral level work. I hope that the blog remains a useful resource and thank you for helping me to clock up 500,000 reads.
I am now based at Northumbria University in Newcastle and still speak regularly to doctoral students in conferences, events and training sessions. If you have completed your docotorate, well done. If you are studying toward your doctorate ... good luck and don't give up.
Thursday, 1 April 2021
April Fool’s Day 2021 marks the start of a new cycle for UK universities. Yesterday saw the deadline for submissions to the national census of research effectiveness, now branded as the Research Excellence Framework or REF. This is the sixth such census dating back to 1992. This time around the sector was granted a short, Covid-related extension and now the long wait for feedback begins. That feedback will impact individual careers and institutional bragging rights as well as being a key mechanism by which billions of pounds of public funding flows into the sector. If you aren’t familiar with the process, every few years, each academic discipline is asked to report on its best research in the form of papers, research grant income, doctoral completions and impact. The last exercise took place in 2014 meaning that this cycle which closed yesterday covered 7 years for each of these categories.
Alongside some simple facts and figures (how many people, how much grant income, how many PhD graduates) there is a lengthy narrative about research strategy, the organisational support and environment which accounts for 15% of the overall grade. There are then two other more contentious and subjective elements.
First, are research outputs. REF 2021 is the first time that universities have been compelled to return everyone that they employ to do research. Up until 2104, universities employed careful selection processes to find the sweet spot between the highest rated individual researcher and the financial reward which was made available on a per person basis. Gaming the system in this way was frowned upon but everyone did it. The regulatory response was to insist that everyone must be returned and each individual returned must put forward their best work. Those items (journal articles, books, reports, etc.) are reviewed by judging panels and rated. Such judgements are made by leading scholars but remain open to concerns about subjectivity. After all, how easy is it to classify an individual paper as a primary or essential point of reference (4 star), an important point of reference (3 star) or a recognised point of reference (2 star)? In my own discipline of management research there are stark differences within and across a diverse range of sub-disciplines; these are further exacerbated by methodological differences meaning that defining “good” is not unproblematic. Your “essential” reference point might only be “recognised” to me. This is challenging given that outputs account for 60% of the overall grade. Whilst a REF panel, consisting of recognised experts will allocate the eventual score, most universities have spent the last several years evaluating and revaluating outputs in an attempt to second guess the nuances around a subtle scoring system. This represents a significant investment of time fretting about research which has already been published and, in the vast majority of cases, peer-reviewed. By extension, that time is not being spent on the doing of new research.
The second subjective element of REF 2021 is that of impact. Using case studies, each discipline in each university is asked to present evidence that their research has made a difference in the world. Perhaps a theoretical model has informed policy or practice such that it has been adopted widely. This accounts for the remaining 25% of the overall outcome but again is subject to interpretation and opinion. Referring to my own discipline of management again, it is reasonably clear to see how ideas, models and concepts could influence the ways in which people actually manage in practice. Arguably a less straightforward translation process might face those in some other disciplines. Even in management impact is a complex, temporal phenomenon which, with colleagues, I have written about at length (see here).
Over the last three decades, REF and its predecessor exercises (the Research Assessment Exercise or RAE) has profoundly changed the behaviour of academics and universities. Peer-reviewed publication had long been a key activity, testing as it does, the quality of ideas and contributions. The top rating in REF terms is referred to as four star and producing a four star paper is arduous. It typically involves multiple rounds of double or triple blind review and high rejection rates. Success in management journals rests on demonstrating a novel theoretical contribution often through recourse to theories which are difficult to follow. Using management as an example, the kind of four star work which appears in top-rated journals isn’t easy to read unless you are another well-read and informed academic researcher. Elliot Green’s excellent blog piece sets out the most heavily cited works in social science. Most professional management researchers would find this list familiar and could probably guess a fair number of entries on the list were it a game show format. Most practicing managers would probably not be familiar with the entries on this list. Why does that disjuncture matter?
Take the global pandemic as an example. It has been tragic for each and every family that has lost a loved one but it has also wrought rapid and radical changes to the world of work. Organisations in a variety of sectors are rethinking whether they need all that office space. Employees are rethinking whether they can face all the time and cost associated with commuting. Given this real world problem, it would seem sensible to do some research on who might do what over the next few years since such findings could inform both policy decisions and the choices for individual firms. A survey, by sector, geography, organisation size, etc. would be very helpful and indeed many such surveys have been undertaken by me and others. The kind of insight required to help decide whether to renew your office lease is not however, the kind of insight required to get published in a top-tier journal. Few policy makers or business leaders would thank you for extensively theorising the social processes underpinning the decision they need to make but this is what will help secure a spot in a prestigious journal.
Taking this issue of mismatched incentives a little further, let’s consider the similarities between academic and sporting excellence. Tokyo has finally set the Olympic torch alight. Packed stadia may not fill our screens this summer but we can expect coverage of crowning achievements, gold medals and world records. Amidst the sweat, tears and elation, elite athletes will hope that their victory inspires more participants to take up the sport. The public health benefits of mass participation in sport is a laudable aim but only loosely related to the specialist training, infrastructure and skills required to top the medal table. Similarly in disciplines such as management, education, nursing or medicine elite, peer-reviewed research may push back the boundaries of knowledge but those on the ground delivering services are just as important when it comes to making a positive difference in the world. Educating managers, teachers, doctors and nurses is closer to mass participation in sport than to Olympic gold medals. Knowledge doesn’t always have to be world-leading to be world-changing. Reaching a wide cross-section of practicing professionals depends on the knowledge concerned being relevant and digestible. Since these are not the primary criteria by which elite peer-reviewed publication operates, these are not always the criteria that incentivise academics.
The basic premise of measuring research performance, especially in connection to the dispersal of public monies, is a good one. After all, some means of dispersing £14bn is needed and there seems little appetite for a flat, per capita arrangement though this would be radically simpler. The problem that REF faces is that it is labour intensive, focuses too heavily on a narrow, specialist form of research activity and has become institutionalised in ways which are not particularly helpful. As the panel members for REF2021 gear up to read thousands of pages of research it would seem timely to pause. Rather than turning the handle of what will probably be REF 2027, we should really consider a radical rethink of the purpose, process and unintended consequences of the current system. To not do so would seem (April) foolish.
Tuesday, 12 May 2020
Tuesday, 9 July 2019
A second theme was introduced by the historian and Stanford Professor Niall Ferguson who pointed to the enabling importance of the rule of law, especially in relation to trade. Ferguson noted that the fundamental ideals of socialism and capitalism, which have shaped political discourse for more than a century, are poorly understood by today’s electorate with the result that traditional labels of left and right are increasingly being usurped by a new politics. With adherence to the rule of law in place, the key decision facing entrepreneurs and the economies in which they sit is one’s attitude to redistribution. A generation of entrepreneurs, regulators and voters, who have grown up with the pervasive presence of information and communication, are beginning to make their way into the workforce. The high school and university curricula offered to the so-called i-Generation have not routinely offered a thorough grounding in our traditional political and economic labels. Worse, this same generation appear to have lost faith and interest in our political process and institutions. Rather than pledging allegiance to particular political parties, they are more likely to connect to issues and individuals. Greta Thunberg is not aligned to traditional political groupings of left or right, rather she is an example of challenge or issue based campaigning that are replicated in the #metoo phenomenon and many others. Modern entrepreneurs and economies face concerns of sustainability and responsibility that not in the forefront of Smith’s mind and a new focus on redistribution is required.
Thursday, 11 April 2019
Meet the new boss, same as the old boss ?
Do your homework
Control, Alt, Delete
Actions speak louder than words
Wednesday, 18 July 2018
This blog post first appeared on the Times Higher's website and can be found here.
The full text is ...