«The Unwritten Rules of PhD Research Gordon Rugg Marian Petre THE UNWRITTEN RULES OF PHD RESEARCH THE UNWRITTEN RULES OF PHD RESEARCH Gordon Rugg and ...»
Have comfort food and bandages ready The world is not fair. Sometimes people will be gratuitously and unnecessarily rude to you even when you’re in the right. Sometimes you won’t think of the brilliant and correct riposte until years afterwards. (For instance: ‘Prospect theory is descriptive, and assumes that people fail to make the normatively correct decision as a result of heuristics and biases; this version of possibility theory shows that there are justiﬁable normative solutions very different from those advocated by the heuristics and biases school, and which correspond closely with those which are actually used by people, and which are described in prospect theory’. Not exactly a line that trips readily off the tongue, and several years too late, but at least it’s been said now, and one of the authors feels better for it.) Sometimes the comments are justiﬁed – for instance, the occasion when one of us was writing a fairly esoteric paper about an obscure aspect of repertory grid technique and didn’t bother to cite Kelly (the seminal text) on the grounds that nobody could have become familiar enough with the technique to write that paper without having read Kelly. One of the referees pointed this out as a mistake, and they were right. The change was made and that mistake not made again.
On other occasions the fairness of comments is more debatable. Here are
some examples which happened to other people:
• Opening line in question from audience at conference (witnessed by one of us): ‘That was the most ignorant and ill-informed talk I have ever heard’.
• Alleged opening line in PhD viva from external examiner who was allegedly taken away for psychiatric treatment soon afterwards (possibly urban myth): ‘Can you give me one good reason why I shouldn’t use this thesis to wipe my backside?’
• Suggestion from audience in conference to someone else in the audience who complained with gratuitous rudeness that the speaker’s recommended approach hadn’t worked for them (witnessed by one of us): ‘Perhaps you should try doing it right.’ A good strategy (once you’re out of the danger zone) is to feel utterly sorry for yourself for the rest of the day and seek solace in comfort foods and your personal equivalent of bandages – a small sherry, chocolate, watching a movie with a high body count, or whatever. That gives your psyche a chance to sort itself out. Then, the next day, you ask yourself what you are going to do about it and how you are going to move on. Were the comments a fair hit? If so, you need to work out how to ﬁx the problem. If not, what are you going to do to PRESENTATIONS 143 reduce the risk of similar unfair hits in the future? Remember that the search for revenge can do you just as much damage as the initial wrong – quite often the best strategy is to get on with seeking fortune and glory, and leave your assailant behind you. (There’s the added comfort that if you learn from the experience you might be able to wipe the ﬂoor with your assailant next time you meet...) A closing point about strategy and fairness: although it isn’t a fair world, there are quite a lot of fair people in it. If you’re perceived as a nice person who does good work, rather than an embittered seeker after petty revenge, then more experienced researchers will be likely to talent-spot you and to put opportunities your way. This is something which doesn’t usually happen to people who spend their lives in pointless wrangling.
A brief checklist for presentations
• Have you checked the level of detail at which to give the talk?
• Have you checked what the audience will already know?
• Have you rehearsed the talk?
• Are your slides readable?
• Do you know how to use the audio-visual equipment where you will be presenting?
• Have you looked at the room where you will be presenting?
• Do you have a master sheet showing when you should be at which stage of the talk?
• Do you have a backup plan in case of equipment failure?
Presenting a paper at a conference – some tips
Dealing with nerves
• Preparedness: being really well prepared won’t stop you being nervous, but it will give you something to rely on as you overcome your nerves. If you’ve given a practice talk that was well received and that allowed you to sort out any glitches, then you are likely to be more conﬁdent in the conference presentation.
• Crib sheets: if there’s key information you want to remember (e.g. key papers and who wrote them), then write yourself a ‘crib sheet’ (i.e. list of the key facts and points) that you can take along for reference in case you need it.
144 THE UNWRITTEN RULES OF PhD RESEARCH
• Anticipate your fears: think through the things that worry you most.
What’s the worst thing that could happen? How might you deal with it? Ask someone else how they’d deal with it.
• Find a friendly face: it’s easier to make the talk warm and conversational if you can view it as a conversation with someone – especially someone who is interested.
• Pause for breath: pauses during a presentation feel to you like they last for years, but the audience hardly notices them. Allow yourself pauses to think, to collect yourself, to catch your breath. Your talk will be better for it.
• Introduce yourself as a student: if you’re really terriﬁed, you can slip in the information that you’re a student in a bid to get the audience to treat you gently. For example, you can credit your supervisor, and you should credit your funding body, if appropriate.
• Study the question patterns in preceding talks: many people ask the same sorts of question of all speakers (e.g. methodology, statistics, application, relationship to particular theories). So, if you have a chance, pay attention to what sort of audience is in attendance and consider how you’d answer similar questions focused on your talk.
• Dress comfortably: you’ll have enough to think about without being distracted by shoes that pinch or clothes that feel inappropriate. So wear something that makes you feel good.
What sort of script?
Different people use different sorts of scripts or notes to guide their talks.
Think about what sort of script will support you best.
• Overheads: your overheads should be a distillation of the key ideas in the talk. They can themselves provide the cues for your narrative.
• Notes: people keep notes of points they want to make, in the order in which they want to make them. Some annotate a photocopy of the overheads to include fuller information and an indication of the ‘story’ they want to tell about the overheads. Others keep a separate ‘text’, with indications of how the overheads relate to the notes.
• Full script: some people write a full script for the talk – not necessarily to read it (which is not a good plan), but to have a set of words to fallback on if they ‘dry up’.
• Time line: it’s a good idea to know how your talk ﬁlls the available time, so that you know how much material you should have covered by the halfway mark and so on. If you have a script, or a set of notes, or just a running order, you can annotate it with elapsed time. Then you can check your progress during the talk.
PRESENTATIONS 145 Handling questions
• Practice: if you’ve given the paper as a seminar, you’ll already have met some questions and had the experience of being ‘on the spot’. This will help.
• Question patterns: during seminars and other people’s talks, pay attention to the sorts of question that people ask. See if you can discern patterns in what people ask about. That gives you a basis for anticipating questions that might arise after your talk, and you can prepare answers for those.
• Fending off references to unfamiliar literature: if you don’t know the paper you’re being asked about, you can throw the question back to the asker: ‘I’m not familiar with that paper; what point does the author make?’ Or you can ask the questioner to relate it to literature you do know: ‘I’m not familiar with that paper; does it fall into the AI camp or the empirical studies camp?’ or, ‘Is that anything like the travelling salesman problem?’ Don’t fake it. Make a note of the questioner and ask for the citation after the session.
• Divert overly technical questions to private discussion. (‘That’s an interesting point, but it would take a while to answer. Could we discuss it at the break?’)
• Missed questions: if you’re not sure you’ve understood the question, then paraphrase it back to the questioner: ‘If I’ve understood you correctly, you’re asking me if...’ and then answer your version. If you haven’t followed the question at all, ask the questioner to repeat it – he or she may ask a simpler version.
• Long questions: have paper and pencil ready. If someone asks a multi-part question, or passes off an essay as a question, then making some quick notes will help you keep track of what you want to say in response.
• Bizarre questions: treat similarly to overly technical questions; something along the lines of, ‘That’s a very interesting point, and one to which I hadn’t previously given much attention. I’ll look into that once I return to the ofﬁce’. Don’t offer to discuss it in the break.
Get someone else to record the questions asked, preferably with the names of the askers. You’re unlikely to just remember them and you may not have time to make notes. In general, it’s a good idea to be prepared to take notes – of people to catch later, or of particularly good points (theirs or yours), or of things you want to follow up.
13 Research design
This is a topic which requires a book in its own right, and excellent books exist on this topic, but it would be hard to write a book such as this one without some coverage of research design, so here goes.
The research question This is not the same as a question you ask people (such as experimental subjects): in many disciplines, such as metallurgy, research is performed on things, not people. Asking the right research question is a key academic skill.
Bad research questions are a common cause of (at best) wasted time and (at worst) failed research, or (occasionally) of tragedy when a mistaken result is used for public policy making. A good research question reduces the problem space in an area. This means that the answer, whatever it is, eliminates one set of plausible possibilities. The next research question will then further reduce the possibility space, and so on, until there is only one sensible explanation for the problem which corresponds with the facts. Good examples of this approach can be found in the history of medicine – for instance, Pasteur successively eliminating possible answers for the cause of decay in foodstuffs.
Bad research questions come in various forms. The most common are listed below.
RESEARCH DESIGN 147 Seeking supporting evidence for a preconceived idea It is surprisingly easy to ﬁnd large amounts of evidence for even the most silly ideas. One exercise we use to demonstrate this to students involves dividing them into groups, then giving each group the name of a living thing which may or may not be a human (for instance, it may be a kangaroo or an ant).
Each group then has to list as many arguments as possible for their living thing being a human being (for instance, that it has two legs, or that it constructs homes), with the other groups trying to guess whether or not they are describing a human.
A frequent version of this problem involves setting out to measure an effect, without thinking about (a) whether that effect exists or (b) what the wider context is. For instance, a lot of research by computer science students involves setting out to measure how much better their software is than the previous industry standard; if their software doesn’t perform better, they end up with several years’ worth of embarrassment and wasted effort staring them in the face.
Asking an unanswerable question A question may be an important one, but unanswerable. For instance, do different Palaeolithic tool assemblages reﬂect (a) different activities within the same group or (b) different groups of people, such as different tribes or cultures? Both explanations ﬁtted the facts equally neatly for a long time, until techniques were invented for identifying how tools were used.
Asking a useless question Just because a question can be answered, that does not mean that it is of any use. For instance, discovering that a particular group of people (e.g. those with low scores on the Smith & Wesson dance test) have particular difﬁculty in learning foreign languages is unlikely to help anyone who is trying to teach them a foreign language – the teacher will be much more interested in ﬁnding out about better ways of teaching them. A well-formed research question will usually have very clear practical implications for someone.
An improbable-sounding instance involves research into ﬂamingo breeding.
Captive ﬂamingos are often reluctant to breed. Research indicated that this was because ﬂamingos would only breed when they believed that the ﬂock size was large enough. The practical implication of this was that putting mirrors beside the ﬂamingo enclosure would make the ﬂock appear twice the size, which did in fact encourage the ﬂamingos to breed.
It is a good habit to work out the possible answers to your research question before you start your data collection, and to make sure you know (a) why each possible answer would usefully reduce the problem space and (b) what the practical implications would be. The latter is also very useful if you are looking for funding.
148 THE UNWRITTEN RULES OF PhD RESEARCH Useful questions about your question There are some simple useful questions you can ask yourself about your research question. If the questions (or the answers) make you angry or nervous, then you should rethink your research design. (Hint: the answer to each question should be ‘yes’.)
• Are you trying to ﬁnd something out, rather than prove something?
• Do you ever ﬁnd yourself being surprised by what you ﬁnd in your data?
• Do you ever decide, on the basis of your data, that your previous ideas about an area were wrong?
Choice of method Most research uses the traditional methods for that ﬁeld. The number of researchers who take the time and trouble to learn the tools of their trade thoroughly is woefully small. A useful test is to list in detail the steps in your plan of research and then ask yourself, ‘Why?’ for each. For instance, why use a questionnaire rather than any other method (and what other methods are there)? Why use a sample size of 300 rather than 298 or 154? Why is a p value of 0.05 treated as statistically signiﬁcant, whereas one of 0.051 is not? What does ‘statistically signiﬁcant’ mean anyway? All these questions and more have been extensively researched for decades, and the more you know about research methods the more likely it is that you will be able to carry out good research with minimum wasted effort. This section looks brieﬂy at these issues, as an illustrative example rather than an exhaustive list.
Data collection method This is usually treated as synonymous with ‘questionnaire’ or ‘interview’.
Both of these are very easy to do badly and very hard to do well. They also both encounter serious problems with external validity (i.e. how well the answers obtained correspond with reality in any sense of that term) and usually also encounter problems with the representativeness of the sample (particularly in the case of questionnaires, where it is completely usual for over 80 per cent of recipients to throw away the questionnaire). Trying to argue that (a) analysing the results from the 20 per cent which were returned is in any way a meaningful exercise or (b) that everyone else in the ﬁeld does the same, so it must be acceptable, are both pretty shoddy arguments.