The problem with research in mentoring

by David Clutterbuck, Copyright Clutterbuck Associates 2003

If you have any comments to add to this topic or would like to be in communication with other people exploring this field then please email us

One of the remarkable aspects of mentoring is how extensively researched the topic has been. However, extensively-researched isn’t the same as wellresearched.

Having had to trawl through hundreds of papers and a fair pile of dissertations for my own current doctoral research, I soon came to echo the thoughts of an anonymous business school faculty member who said; 'When I was a journalist, I thought journalism was just badly-done academic research; now I’m an academic, I realise that research is often just badly-done journalism!'

Over recent months, I have been trying to establish what valid research in this area would entail. I have been less interested in issues such as sample size (though this clearly is an issue – the original research by Kathy Kram 1, on which so much subsequent research has been based, had a sample size of just 28 pairs) or the accuracy of the mathematical analysis, as in the overall logic and structure of the research. I’ve also been concerned with that critical, but so often neglected question, how relevant and useful is this to the practitioner? What follows is to a large extent a summary of my own (painful) learning about research method in this field.

In a review of formality and informality in mentoring, David Megginson and I found an almost totally divergence between the conclusions of academic papers and actual experience in the field. We concluded that this divergence was at least partially the result of failings in the structure and definition of much of the research.

So how does one test the quality and value of research in this field?

Like the UK’s Chancellor of the Exchequer, Gordon Brown, in his approach to joining the Euro, I have been using five tests. These are:

1. Definition Is it clear what kind of relationship is being measured? Some research mixes participants in structured programmes with those in informal relationships and some even with relationships, where one party does not realise they are part of a mentoring duo. Some papers mix in-line relationships with off-line (leaving aside the argument as to whether it is possible to be a mentor in a boss-subordinate relationship).

There are, of course, dozens of definitions of mentoring, yet many studies fail to be precise about which definition they are following. Many, mainly US-originated definitions, emphasises sponsorship and hands-on help by the mentor; others, mostly European and Australian in origin, see such behaviours as unacceptable within the mentor role. Unless it is clear, which model is being followed in a particular piece of research, it is often impossible to draw conclusions with confidence, or to make comparisons with other studies. Meta-studies and literature reviews may compound the problem, because they tend to begin from the (false) assumption that everyone is measuring the same phenomenon.

The issue is made even more complex by the recognition by some researchers in the area that multiple, simultaneous mentoring relationships are also a common factor. Clearly, the dynamics of one relationship within a web of others may be different from those of a single, intensive mentoring dyad.

To increase the validity of research in mentoring, it is necessary in my view to provide a precise definition of exactly what kind of relationship is being measured and to ensure that all the samples lie within that definition. Some research has attempted to get round this problem by asking people about broad helping relationships, but then the data is too general to apply meaningfully to specific types of mentoring relationship.

Recognising that mentoring is a class of phenomena and that each phenomenon needs to be investigated in its own right, would be a major step forward in research quality in this field. (An interesting analogy is in the field of medical research, specifically into the origins of autism. Almost no progress towards an understanding of this condition had been made until recently, when researchers began to recognise it as a number of related and interacting sub-conditions.)

2. Context A wide variety of contextual actors can affect the relationship and the scheme. At a minimum, these will impact upon the intent (their own or that of third parties, such as the organisation) mentor and mentee bring to the relationship.

Other contextual variables include the level of training participants receive, the way in which they are matched (with or without an element of choice) and whether the relationship is supported as it develops (for example, by additional sources of learning and/or advice). Other contextual factors might include differences in race, age or gender.

Trying to account for all the contextual variables that might apply, especially when a research sample is drawn from many organisations or schemes would be very difficult to do without vast sample sizes. This suggests the need for relatively narrow selection criteria – for example, senior managers, in company-sponsored mentoring relationships of at least six months duration with a paid external, professional mentor; or young males 12- 15 from deprived backgrounds at risk, paired with male role models between 10 and 20 years older. The more variables subsequently introduced (eg gender variation), the larger the sample size will need to be to draw conclusions with confidence.

3. Process provides another set of variables. It is clear, for example, that ementoring differs in some fundamental aspects from traditional face-toface mentoring. Simple process factors, such as frequency of meeting, can have a major impact on outcomes. At the very least, studies need to allow for or try to eliminate such variables. Studies attempting to link personality to success of mentoring relationships, for example, would be better grounded if they also investigated the degree, to which personality factors resulted in specific behaviours, perceived as helpful or unhelpful to the maintenance of the relationship and to the achievement of its goals. (This classification into maintenance and achievement oriented behaviours appears to be very relevant across the whole area of mentoring relationship dynamics.)
4. Outcomes Much of the research literature uses Kram’s functions of a mentor (or the subsequent recasting of the functions by Noe 2) as measures of outcomes. Yet the functions are a mixture of behaviours, enablers and outcomes and so for the most part unsuitable for this use. (Kram herself did not intend them to be used in this way, I am sure.) Moreover, outcomes are almost never related back to goals/ intent. The reality is that different types of mentoring relationship have different expectations of outcomes; and even different dyads within the same scheme. Failure to recognise these means that the purpose of the relationship is ignored – which suggests the research fails the fifth test, that of relevance.
5. Relevance The so-what test is a standard element in guidance on research design, but it seems often to be honoured mostly in the breach. My own experience has been that I struggled to get co-operation from companies until I was able to articulate very clearly the practical value both of the expected research outcomes and of participating in the research process itself. Even then, maintaining commitment for a longitudinal study has proven very difficult. I recommend anyone designing future studies to convene at any early stage of research design a panel of practitioners – those, who the research is intended to inform and benefit – to help shape and ground the project.

There are many other failings in the general literature on mentoring – for example, the paucity of longitudinal studies, with a few exceptions 3 (I sometimes despair of ever completing mine!). However, these many holes provide many opportunities for useful research and it is possible – with care – to mine the literature for useful indicators that can be tested in well-defined contexts. In the future, I am convinced that our understanding of mentoring will be enhanced by making the same shift of emphasis as the autism researchers, focusing on specific definitions and contexts to begin with and gradually building a richer, more complex model than currently exists.

Footnotes

1 Kram herself makes the valuable point that sample size has to be relevant to the issue being investigated. So for a small sample, qualitative study may be appropriate to initial investigations of a topic, but less appropriate when there is already a body of accepted theory and practice. In addition, small samples investigated in depth may be more revealing in multiple complex relationship dynamics.

2 For example, see Noe, R.A (1988), 'An investigation of the determinants of successful assigned mentoring relationships', Personnel Psychology, 41, pp.457-479 It is also remarkable how few studies attempt to measure outcomes for both parties. Yet mentoring is an interaction between two partners, with the outcomes highly dependent on the motivation of both.

3 See, for example, Hunt, D, 'A longitudinal study of mentor outcomes', Mentoring International, volume 6, no’s 2/3, Spring 1992, and Seinert, S. 'The effectiveness of facilitated mentoring: a longitudinal quasi-experiment' Journal of Vocational Behaviour, no. 54, pp.483-502, 1999

David Clutterbuck is a an acclaimed author on mentoring and organisational learning, a member of the European Mentoring & Coaching Council and head of Clutterbuck Associates. He can be contacted on 01628 661667 or via david@clutterbuckassociates.co.uk