«The Unwritten Rules of PhD Research Gordon Rugg Marian Petre THE UNWRITTEN RULES OF PHD RESEARCH THE UNWRITTEN RULES OF PHD RESEARCH Gordon Rugg and ...»
Cynical supervisors have been known to give students explicit advice about which sources to read, but not quote, as a source of an initial overview so that they understand the area. Alleged examples range from How to Lie with Statistics (almost certainly true) to The Ladybird Book of Computers (surprisingly, perhaps true to some extent).
Although literature reviews are a useful way into the literature, they are not infallible, and were not written with your particular needs at the forefront of their writers’ minds. You therefore need to do your own trawls through the literature, to see what’s out there and ﬁnd bits of the literature that are relevant to you.
Supervisors and externals are not allowed to kill students who include in their literature reviews a sentence starting, ‘A search on the internet found no previous work on this topic’. They are, however, allowed to fail students, and to write elegant, cutting comments on the offending page, which goes some way towards remedying this shortcoming in the legal system.
Why do supervisors and externals get so worked up about that sentence?
Answer: because it’s equivalent to writing in large letters, ‘I am either ignorant or lazy or both’. That is not a signal that you want to send out to the reader.
and about other relevant issues such as the physiology of the brain. If someone claimed to be a leading brain surgeon and then appeared to be unsure of the difference between a clamp and a retractor, that would not be an encouraging sign.
Anyone in the academic system ought to know the tools of the academic trade. The amount of detail required will vary with the academic level – for instance, undergraduates will not normally be expected to know as much as PhD students, who in turn will not be expected to know as much as leading professional researchers in the area. However, if you know more than you are expected to, this is usually viewed as a very encouraging sign.
Academics deal with knowledge and information, and should know how to ﬁnd, interpret and present knowledge and information. An important part of this is ﬁnding the best possible sources so that your assessment of the problem in question is based on the best information and knowledge available. The academic literature has a pecking order, ranging from publications which are accepted on sufferance through to publications which are treated with considerable respect. Some of this pecking order is quite possibly based on snobbery, but most of it is based on the quality control that the publication uses. The more rigorous the quality control that a publication uses, the more prestigious the publication is. It’s a simple and sensible concept, and it makes life a lot simpler and more reliable for everyone involved. If you are about to spend months or years of your life, and perhaps sizeable amounts of money researching a topic, then it’s very reassuring to know that your initial assumptions are as solidly based as they can be.
At the top end of the pecking order come encyclopaedia articles and the top journals. Encyclopaedias usually choose the leading international experts in an area to write their articles – it is a considerable compliment to be asked to write one. Anything submitted to a top-quality academic journal for publication will normally be checked in detail by several leading international authorities on the topic before being accepted for publication. Anything which is not of suitable quality will be rejected.
Further down the pecking order come the middle-range journals, which also use refereeing, but which normally use less eminent referees. Towards the bottom of the scale come specialist newsletters and professional trade magazines, where articles may be reviewed by the editor rather than specialist referees.
The precise status of a publication will be affected by individual factors – for instance, some specialist newsletters will be edited by very eminent authorities, have very high-level contributions and be higher on the pecking order than some journals. Books are also very variable in their status. As a fair rule of thumb, textbooks are low on the pecking order, because they usually present simpliﬁed accounts for students. Specialist books may be extremely prestigious.
The observant reader will by now have noticed that this description of the
60 THE UNWRITTEN RULES OF PhD RESEARCHpecking order contains absolutely no mention of the internet, of newspapers or of popular magazines. There is a good reason for this. The internet has absolutely no quality control as regards the content of the sites accessible through it. If you ﬁnd an interesting-looking site relating to your chosen area, it may possibly have been written by a major authority on the area, but it could just as easily have been put together by someone who believes that they are being controlled by devices put in their brain by aliens, and who has a degree from a college based above Joe’s Pizza Shack in Peoria. Newspapers and popular magazines at least have some quality control, but if you think that reading a newspaper sends out the signal that you are a professional with considerable expertise, then you might be better advised to transfer your registration to that college based above Joe’s Pizza Shack. Remember that ‘online’ includes things like using library databases and CD-ROM indexes: you don’t have to be on the internet to be online.
Online searching: overview One of the main reasons for performing an online search is to ﬁnd out what has been done before, so that you don’t reinvent the wheel and make it square. If something has been done, then you need to get a clear overview of that previous work. If you can’t ﬁnd any sign that anything has been done previously, then you need to be pretty sure of your ground before saying, ‘No previous work has been done in this area’. At best, you might look a bit silly if there is a major literature which has been missed; at worst, you might be accused of academic malpractice in claiming priority over a previous researcher in this area (not too likely if you are a ﬁnal year undergraduate, but more of a worry if you are a PhD student aiming for a career in academia). For this reason, saying, ‘No previous work has been done in this area’ is simply asking for trouble, and most sensible professionals use expressions such as, ‘This area appears to have received little or no attention in the past’, which allows them to wriggle out with some face saved if a previous literature does exist.
So, how do you set about ﬁnding out what has been done previously? There are three main things you need to think about: sources, strategy and tactics.
‘Sources’ involves where to look; ‘strategy’ involves ways of structuring the search process; ‘tactics’ involves things such as the search engine features that you use.
Sources The amount of information in the world is enormous. For instance, the number of physics journals alone is so huge that reading the physics journals which are published each year would take more than a year of non-stop reading – you simply couldn’t keep up with the current publications, let alone the previous issues stretching back to the nineteenth century.
READING 61 To make life simpler for everyone, librarians and professional indexing bodies thoughtfully index the contents of journals (and other sources of information too, for that matter). This means that you can look up a term in an index of this type and ﬁnd out when and where something about it was previously published. For obvious reasons, indexes usually cover journals relevant to the index topic, so physics journals will be indexed together in one index, chemistry journals in another and so on. The old indexes were printed;
more recent ones are on CD-ROM or accessible on specialist sites, usually password-protected (librarians will have passwords to many relevant sites, though some sites will charge you for searches).
This means that if you want to ﬁnd out what has been done previously in a particular area, then your friendly campus librarian will probably be able to direct you towards an appropriate index covering the relevant topic. This in turn means that (a) you will have a good chance of ﬁnding anything worth mentioning on the topic, and (b) that if nothing turns up in your search, then this probably indicates that nothing has been published previously – an important consideration if you’re doing something like a PhD where originality is important. (It might also mean that you’ve mis-typed a keyword, which is why the next two topics below are important too.) An added advantage of the indexing process is that there is a certain amount of quality control – only reputable sources are usually indexed. If you ﬁnd a fact or a claim via an index of this type, then it will probably be sound.
Strategy Once you have found the right index, you need to have a strategy for searching it. The usual novice strategy is to type in two or three keywords and see what happens; the usual system response to this is to say either that no records have been found, or that 231,768 records have been found. A more sophisticated strategy offers some advantages over this.
There are various good articles about conducting effective searches, and time spent reading them is time well spent. This section is just an introduction to the topic and you would be well advised to read some of the specialist articles.
One useful strategy is to plan in advance what you are going to do during the search. Another is to keep a written record of the things you have done, so that you don’t end up going round in circles.
There are different types of search. The standard information science literature has the useful concept of the ‘known item’ search, where you are looking for one speciﬁc item or fact – for instance, trying to locate a copy of J.R. Hartley’s Fly Fishing, or to ﬁnd out its date of publication, when you already know that the book exists. For this type of search, as soon as you have found the answer, you can stop.
With other types of search, however, you will not always know when to stop.
If you are trying to get an overview of the main previous work in an area, for instance, you will need to do a fair bit of searching and you will almost
62 THE UNWRITTEN RULES OF PhD RESEARCHcertainly encounter problems with the system either claiming that there are no relevant records or claiming that there are millions. You therefore need strategies for improving your hit rate.
One simple but effective strategy is to use a keyword search and then wade through the list until you ﬁnd a relevant record. You can then look through the relevant record for other potential keywords to use in your next search.
Authors’ names are well worth considering for this (unless they are extremely common ones such as Smith) – someone who has published one relevant article on a topic will probably have published more, and you can then start adding their co-authors’ names to your list. The same is true of technical terms, where you might ﬁnd different names for the same concept, or more specialist names (if the number of hits was previously too large), or broader terms (if the number of hits was previously too small).
Tactics and commands Different search engines operate in different ways: it is an instructive experience to type in the same keyword to different search engines on the internet and see what results you obtain from each. Underlying them all, though, are a few basic concepts, and understanding these can make your searching a lot more productive and efﬁcient.
Two key concepts are weighting and Boolean search.
Weighting Weighting involves assigning different weights (in the sense of e.g. importance or relevance ratings) to something – in this context, usually keywords or records. This allows the search engine to list in a systematic order the records which you ﬁnd. One popular way of doing this is to use ‘inverse frequency weighting’ – the rarer a term is, the more weighting it is given, on the assumption that it will be more speciﬁc and information-rich. So, for instance, a search on ‘low entropy systems’ would result in low weightings for ‘low’ and ‘systems’ on the grounds that there would be millions of records containing these terms, and a much higher weighting for ‘entropy’, which is a much rarer term.
It is worth bearing this in mind when choosing your keywords: more speciﬁc terms usually produce lower numbers of hits, but a higher proportion of relevant hits.
Boolean searching Boolean searching involves using the operators ‘AND’, ‘OR’ and ‘NOT’ on the keywords which you enter. So, for instance, ‘repertory’ AND ‘grid’ would ﬁnd only records which contained both the words ‘repertory’ and ‘grid’. A search for ‘repertory’ OR ‘grid’ would ﬁnd records which contained either ‘repertory’ READING 63 or ‘grid’ or (usually) both. A search for ‘repertory’ NOT ‘grid’ would ﬁnd records which contained ‘repertory’ but which did not also contain ‘grid’. This approach can be very useful when you are trying to exclude records on a topic with a similar name – for instance, if you are trying to ﬁnd out about repertory theatre, but keep ﬁnding records about repertory grid technique.
Most online search engines on the internet use Boolean ‘OR’ searches as the norm, but offer ‘AND’ and ‘NOT’ in the ‘advanced search’ option. The same is true of most library online search systems.
In addition, ‘advanced search’ usually offers other features, such as being able to treat two or more words as a phrase (for instance, by enclosing them in inverted commas). In the case of repertory theatre, for instance, you might be able to search using the key phrase ‘repertory theatre’ in inverted commas, which would then ignore the phrase ‘repertory grid’.
It is highly advisable to learn to use advanced search. Librarians are usually supportive if you ask for help with this – they have a hard time from many users, so it is a welcome change for them to encounter someone who wants to learn how to do it right.
Other sources of information
As usual, a cup of coffee with a friendly expert can save you an enormous amount of effort.
It is also a good idea to get an overview from a textbook, which will list relevant articles in its bibliography, and an even better idea to get an overview from a review paper or from a recent encyclopaedia. Review papers and encyclopaedias are usually good things to quote in your bibliography;
textbooks are usually not a good thing to quote in your bibliography, since they are saying to the reader: ‘I’ve read the simpliﬁed account for beginners, not the professional account’.
It is also worth being pleasant to librarians – they have a wealth of information which they are usually happy to share with polite, appreciative people.
How a seasoned referee reads a paper
The most interesting things in a paper are usually written between the lines.
In the stereotyped picture of the good old days, this was something which your supervisor would teach you over a glass of sherry (and a very pleasant way of operating it was too for both parties, as we can testify from personal experience). Nowadays you usually have to pick this up the hard way.
64 THE UNWRITTEN RULES OF PhD RESEARCHThis section describes some of the things that an experienced professional (such as your external examiner) will be looking at.
One point worth mentioning at the outset is that it is not a safe strategy to hope that the reader won’t notice mistakes if they’re tucked away late in the text. There really are readers out there, particularly external examiners, who actually do read every single line of a thesis – for instance, by moving a ruler down the page a line at a time to make sure they don’t miss anything, and noting every mistake, question or comment that they want to draw to your attention. It’s also worth mentioning that experienced professionals can skim-read and spot errors at speeds you might ﬁnd hard to believe.