How to write a Research Methodology Chapter in Computer Science without fluff and nonsense?

<<…First dirty draft of the article – draft under continuous updates…>>

This article will attempt to explain what Research MethodologyResearch Methods and Research Instruments/Procedures means in Computer Science (CS).  It will explain the difference between a Research Methodology and a Research Method.

The article will cover also research paradigms in Computer Science. I aim from this article to detail the different research methods employed in Computer Science. I know it is not an easy task at all by the way and you will know why later! I aim, in similar vein, to explore what are the general type of research problems and research questions, each one of these methods is suitable to address. Remember always that your research questions dictates what research philosophy, overall research methodology, research methods and research instruments you end up adopting in your research work.

In any research work, you must always have a cogent rationale of why you have chosen the research method X and not the research method Y. It is a classical assessment question.

Please refer to the article that I wrote on how to write literature reviews without fluff and nonsense since you need to  ALWAYS take into consideration what research methods have been used by the relevant works  in the literature  and how and why you have followed or NOT followed them in this regard. With each literature review you make, always remember to look at the research methodologies and methods employed by relevant studies.  This process is called a methodological review. If you are writing a PhD thesis, examiners expect to see these rationales in your Methodology Chapter and will definitely ask you about that in your Viva Voce.

This article will be an exposition of Research Methodologies, Research Methods and Research Instruments or techniques used in many divers fields of Computer Science (CS).

For brevity, I will refer to Research Methodology, Research Method and Research Instruments as the three Rs throughout the article whenever needed. The three Rs in Computer Science are never legitimate children of the discipline.

Computer Science is a very young science (compared to other sciences) and  by nature it is interdisciplinary. The three Rs in CS are usually borrowed from the social sciences especially the fields of  Psychology, Behavioral Science and Sociology (examples: the Computer Science fields of Human Computer Interaction,  Software Engineering, Software Architecture, Usability engineering, Accessibility, Quality of Experience,  etc… ), from mathematics (you can see this clearly in many Computer Science sub-fields just to name a few like Computational Complexity Theory, Database Theory, in formal languages theory, vel cetera), from natural sciences and engineering disciplines.

The interdisciplinary/multi-disciplinary nature of Computer Science [1,15]
The research methods taken from mathematics were in the form of axioms, postulates and proofs whether the ones taken from the engineering fields were actualized through quantification, measurements and comparison.

Both Quantitative and Qualitative methodological approaches will be explained. Also the article will contain a wealth of suggested resources in the form of books, online video courses that are specific or relevant to our field and others which are more general and cross-disciplinary that can help us in some suitable fields of CS.

It is pointed out throughout the literature [1, 2, 4 ,15] that Computer Science has a serious identity crisis. It inherited from many fields and did not find itself yet nor established or formalized or canonized yet its methodologies and methods.  I will avoid nonsense and fluff as much as possible and give the gist of what was gained from reading over 40 papers on the topic, in addition to few books.

NB: I am using a plural form of the term “methodology” despite that some scholars advocate that the term has no plural, I will explain why later.

Computer Science is new and hip science and I am using both terms in a “negative and derogative way“, not in a positive praising way. The moment you begin to search for Computer Science Research Methodologies and Methods, you immediately fall into a state of confusion and frustration.

You might find a book or two on research methods in Human Computer Interaction (HCI) which is a sub-field of Computer Science, and this is because the methods are borrowed from research methods in social sciences and the authors recycled the methods for HCI. You might also find a book from the early nineties on empirical methods in AI. On this note, a huge thank you for  the authors of these books that I am alluding to: Jonathon Lazar et al. and Paul Cairns. Their endeavors aim to put in single textbooks, essential HCI research methods in the objective of helping research students and practitioners not go crazy. Chapeau bas! THANK YOU SO MUCH!

In addition, if you search the web or scholarly search engines, you find will a lot of nonsense tutorials and few lecture slides of courses from some universities and a set (not too big) of somehow useful academic papers with information scattered all over the place.

Of course, you probably have been told about Raj Book the holy grail if you are doing research in system performance, a book from 1991 that the Networks folks are still completely obsessed with and keep it near their desks. Similar books are this and this but Raj book is still the best despite the age.

That’s it my friend!!! That is it!!! Beside that there is nada, nothing, rien, nichts!!!! (at least at the time of writing of this article).

Please go ahead and do your search and then come back to this article. I am waiting here!!! Now go again and search for  research methods for any social/natural sciences: education, health, economics, business or medicine or  physics, chemistry, astronomy, psychology etc… and you will find literally hundreds of books and millions of academic papers.

So what is happening with Computer Science? Where the hell are all the research methods and research methodologies books in the field???? Why aren’t these methods, if they exist, not formalized yet? Is Computer Science  really a science on its own?

It is quite funny to see how this latter question was considered a painful question [17] for Computer Scientists which has been discussed and debated it all the history of CS. Are we really scientists?

I believe this condition (i.e. lack of canonical books and courses) is really disrespectful and degrading to our discipline. Our discipline should not be about “programming monkeys” or writing “cool” applications.

Programming monkeys are the slaves of the new world system. In the old world,  monarchs needed a lot of farmers, a lot of stone masons, and a lot of soldiers. In the new world, the “system” needs more and more “programming monkeys and slaves” to accommodate the information age. Now the question is: do we want computer science students to become actual future scientists and researchers or do we want them to become future programming slaves and monkeys? This is a question that should be asked by all computer scientists or IT specialists.

Research methods in Computing fields have historically been passed from supervisors to students in the form of apprenticeship [11]. This constitutes a clear disadvantage for someone outside academia.

Why no one has compiled a book of one volume or two on all types of methodologies and research methods used in Computer Science fields similar to this article but expanded a lot? and then name the damn book(s) Research Methods in Computer Science!!!  It is not difficult but is definitely a cumbersome work since it requires that the good Samaritan academic(s)  to scan a huge literature for papers/books and check the methodologies/methods used and then create taxonomies (i.e. classifications) and put those in a book (even spanning two or three volumes). In addition, the book (s) would discuss what each method is good for in terms of adequate research problem and space. For God’s sake?!!! I will pay top dollars for this encyclopedia. I do not think this is brain surgery in the sense of being a difficult task! It is tedious task that is sure but not difficult at all!  Till now I did not found any descent and satisfying answer from anyone in our field! It is shameful on all of us!!!.

Actually I have discussed the idea with academics in the school of Computer Science. We have a lot of expertise in the school and some very reputable names (authorities), so why not create a book of two volumes where folks from HCI, from software engineering, from networks, from AI and Machine Learning, from Usability, etc.. participate in writing and thus we  would create our OWN huge CS Research Onion similar to the lovely works written by others such as Saunders and Tosey [9].

After a very considerable time, I managed to gather material suitable for this article. I have tons and tons of material from other disciplines but there is a scarcity of material germane to Computer Science in particular (other than what was borrowed from the social and natural sciences).

I am sure you want to get to the gist of things quickly so here it is:

To write a good methodology chapter in a PhD let us say, you start by stating your research questions Yep! (which you should have done in introduction chapter anyway) and then you discuss them briefly (a kind of a quick reminder) of how they were actualized in your thesis chapters, in addition you discuss their nature and working space (very important!). then you go on to discuss the research philosophy.   The choice of it would focus on the “nature” of each of your research questions since the philosophy deals with the nature of knowledge.

Some CS supervisors avoid letting their CS PhD candidates even talk at all about research philosophies which is funny  🙂 Some allow them to only mention it briefly. This is because frankly speaking, in many cases the whole CS PhD thesis is one colour (only empirical experiments or only bunch of developed tools or algorithms), in addition, as usual I say things as they are, this is an extremely dangerous ground for CS supervisors to put their students in since it requires a whole level of intellectualism to defend it well in a Viva Voce and supervisors know that a lot of their CS PhD candidates tend to be in general “programming monkeys” instead of “thinking monkeys” 🙂

The next step which is the first step in many theses (for non PC reasons mentioned above) would be to talk about the overall research methodology of the thesis linking it always to the research questions so this is the next step deep in the research onion if we are following it right. Then the student move on to discuss the different employed research methods in the thesis and discuss why they have been used and not others. Here a good quality chapter involves a mini synopsis of literature review (better to call it in this case “a methodological literature review” since it cares more about surveying research methods of germane studies than research outcomes) showing the research methods employed by relevant studies.  Then it boils down to say after the why,  the how. You move on to state and discuss the research instruments/procedures which are directly linked to what you have used as research method(s).

Per example if you used experiments/empirical measurements as a research method, you would state in this chapter also all the experimental procedures if any (such as how the experiment was done and controlled, the devices specifications, instruments used, software tools used to capture metrics etc…). Of course expect examiners to ask you the why of every single step. Please before I forget  here, avoid using magical numbers in per example saying “we measured X for a duration of 3 minutes” etc.. or if you want to use them really that is fine but have the why answer ready under your armpit as we say in my country.

Do you know that a big number of CS postgraduate students I have asked about this, think actually that research instruments/procedures (last step really) are actually “The methodology” (of course this is WRONG) and many of them even do not know that research methodology is NOT a synonym to a research method? Who to blame? not them really! There should be mandatory courses/seminars/workshops, call it whatever you want, on research methodologies and methods in Computer Science given as part of research/taught Masters and PhD degrees  given at the start of the degrees!!!!

This article is a work in progress akin to all my articles in this blog. I don’t always have time to blog so please be patient …. but please also return back, I will definitely add new material.

Conventions in this article

I would do something extremely “Samaritan” in this article. So what I will try to browse all the  PhD theses and Distinction Masters Dissertations in the School of Computer Science, University of St Andrews that can be found and downloaded here: St Andrews Research Repository (as much as I can and as much as my time allows) and then try to give these theses as examples for you to look at when I discuss specific research philosophies/ methodologies and methods.

I will be using “CSST” (that stands to Computer Science St Andrews) to mean I am giving an example of a good PhD thesis that follows the research method I am discussing. I will  not state the name of the student/candidate, title of PhD or Masters but only give the URL link for you folks to download the thesis or dissertation.

Please rectify me if I am wrong in classifying in the comments below since this will be an extremely tedious task. I also urge any good Samaritans out there to help out also in this regard.

The following is an exposition of the common research philosophies that are applicable to Computer Science fields obtained from surveying the literature.

Important Definitions

Etymological definitions are taken verbatim from a lovely online tool which is the Online Etymology Dictionary. I believe it is a worthy website to bookmark.

Research

Etymology Dissection of  the termResearch”: from 1570s, “act of searching closely” from Middle French recerche (1530s, Modern French recherche), back-formation from Old French recercher (see research (v.)). Meaning “scientific inquiry” is first attested 1630s. Phrase research and development is recorded from 1923.

1590s, from Middle French recercher, from Old French recercherseek out, search closely,” from re-, intensive prefix (see re-), + cercherto seek for” from Latin circarego about, wander, traverse” in Late Latin “to wander hither and thither” from circus “circle”.

Definition 1: “A careful, systematic, patient study and an investigation in some field of knowledge, undertaken to establish facts” [37]

Definition 2:

Definition 3:

Definition 4:

Definition 5:

There are different types of research [37].

Exploratory Research: “explores”, “structure” and “identify” new problems. It digs into the unknown realms in order to improve our basic knowledge.  It find new problems for researchers to investigate. Usually exploratory research aims to open new dimensions of research.  To tell people, hey!, we have a problem here! or to say we do not have a problem so don’t bother”.

Constructive Research: develop solutions (i.e. think constructions) to specific problems. The term “construct” is used to refer to new contributions being developed (new theory, algorithm, framework).

Empirical Research: test the feasibility or do-ability of a solution using empirical evidence. It aim to gaining new knowledge by indirect observation or experience. These observations can be analised quantitatively or qualitatively. This answers empirical questions that should be well-defined. Usually with a mixture or assemblage of both empirical evidence types (qualitative and quantitative) many questions are answered.

Etymology of  the term “Empirical”: from 1560s, originally in medicine, “pertaining to or derived from experience or experiments,” from Latin empiricus (n.) “a physician guided by experience,” from Greek empeirikosexperienced,” from empeiriaexperience; mere experience or practice without knowledge,” especially in medicine, from empeiros “experienced (in a thing), proven by use,” from assimilated form of en “in” (see en- (2)) + peira “trial, experiment,” from PIE *per-ya-, suffixed form of root *per- (3) “to try, risk.” With -al (1). In a general sense of “guided by mere experience” from 1757.

Research Paradigm

Etymology Dissection of of the term Paradigm”: late 15c., from Late Latin paradigmapattern, example,” especially in grammar, from Greek paradeigmapattern, model; precedent, example,” from paradeiknynai “exhibit, represent,” literally “show side by side,” from para- “beside” (see para- (1)) + deiknynai “to show” (cognate with Latin dicere “to show;” from PIE root *deik- “to show,” also “pronounce solemnly”). Related: Paradigmatic; paradigmatical.

So what we can deduce quickly from the etymological definition  above is a research paradigm involves with patterns of research or models of research. If we want to be more fancier and take all the constituting words etymologies, research paradigm means the patterns or models of ways of doing scientific inquiry.

Definition 1:

Definition 2:

Definition 3:

Definition 4:

Definition 5:

Research Methodology

Etymology Dissection of of the term “Methodology”: 1800, from French méthodologie or directly from Modern Latin methodologia; see method + -ology.

 Etymology Dissection of suffix “-ology”: word-forming element indicating “branch of knowledge, science” now the usual form of -logy. Originally used c. 1800 in nonce formations (commonsensology, etc.), it gained legitimacy by influence of the proper formation in geology, mythology, etc., where the -o- is a stem vowel in the previous element

So what we can deduce quickly from the etymological definitions above that the two terms “research methodology” simply means the study or science or branch of knowledge that deals with research methods.

If we want to be more fancier and take into account all etymological definitions: research methodology would be the branch of knowledge  that deals with studying the different “ways of doing scientific inquiry”.

Definition 1: “refers to the study of research methods” [48].

Definition 2:

Definition 3:

Definition 4:

Definition 5:

Research Method

Etymology Dissection of of the term “Method”: from early 15 century, “regular, systematic treatment of disease,” from Latin methodusway of teaching or going,” from Greek methodos “scientific inquiry, method of inquiry, investigation,” originally “pursuit, a following after,” from meta “in pursuit or quest of” (see meta-) + hodos “a method, system; a way or manner” (of doing, saying, etc.), also “a traveling, journey,” literally “a path, track, road,” a word of uncertain origin (see Exodus). Meaning “way of doing anything” is from 1580s; that of “orderliness, regularity” is from 1610s. In reference to a theory of acting associated with Russian director Konstantin Stanislavsky, it is attested from 1923.

Definition 1: “refers to the manner in which a particular research project is undertaken” [48]

Definition 2:

Definition 3:

Definition 4:

Definition 5:

Research Technique/Procedure:

Etymology dissection of the term “Technique”:

Definition 1: “refers to a specific means, approach, or tool-and-its-use, whereby data is gathered and analysed, and inferences are drawn” [48].

Definition 2:

Definition 3:

Definition 4:

Definition 5:

General Research Philosophies

Ah! yayyay!! you must love and appreciate philosophy. Philo Sophie (love of wisdom) is the highest and most refined intellectual endeavor a person can ever seek and contemplate. Philosophy is everywhere and of course it is everywhere in research, in PhDs, in Masters etc.

There are many research philosophies suitable for different scenarios, and different research questions. From a philosophical perspective, how do you know as a researcher observing an experiment unfolding in front of you per example that your senses or your brain are not tricking you (think of optical illusions, mind tricks etc…)?, so when people normally read the research philosophies for the first time they immediately think that one philosophy is right or the others are wrong.  I mean look at Quantum Physics experimental level, the mere fact of observing an experiment changes its behaviour !!!

So saying a research philosophy is better than the others is a fallacious thinking, actually it is a stupid thinking to be frank with you, so avoid falling into this please, all of them are right for specific scenarios and research questions.

One thing to put in mind for the “programming monkey” out there (i.e. the average Joe who panics when he is asked by his/her supervisor to think a little bit more abstract, to think in the intellectual realm), is that when talking about what research philosophy fits your Computer Science work, you need to look at the nature of your research questions.  I think I mentioned that before but it worth repeating the idea again. The term nature is a key here.

The Positivism Philosophy

The positivism philosophy or the positivist school of thought believe that “posited” or given  knowledge is based on the natural world and its phenomena, informed through sensory devices and  is interpreted via logic and reason, for this school this is what solely form the true source of knowledge. A researcher is concerned with observing and predicting outcomes like in a laboratory scientist. Questions such as what are the cause and effect of things? The scientific empiricist method is adopted here to propose and test theories and yield pure data and facts outside the values and any interpretation given by the researcher or anybody else.

The positivism school is not one variant but actually many [47]. That has been said for the sake of convenience and this is what Computer Science usually concern itself with

What it means for Computer Scientists?

Convenient CS Research Methods for Positivism

….

As a side note, is democracy a best system to use? NO and NO and NO  you must be saying what!? democracy is the best of what is available currently but it is an extremely flawed system, democratic systems are made to serve only rational people, who always think of pros and cons. It is a system not made to be used for the mentality of the flock who rush and vote through fear, superstitions, and ignorance. Democracies produce terrible rulers especially when they are used in times of fears and crises. Terrible rulers always comes from the mentality and the current mood of the flock.

If there are a bunch of god-like philosophers, they would rule themselves through direct or indirect democracy, this is a good case to use democracy, anything else has sever flaws.

Back to research philosophies,

The realism philosophy

Like positivism, realism rely on using the scientific method and scientific inquiry. To sate what realism better I will quote Saunders and Tosey [16],  “realism states that reality exists independent of the mind and that what a researcher’s senses show her or him is the truth, although the researcher is influenced by world views and their experience”.

Direct Realism

Critical Realism

….

What it means for Computer Scientists?

Convenient CS Research Methods for Realism

….

The Interpretivism philosophy

….

What it means for Computer Scientists?

Convenient CS Research Methods for Interpretivism

….

The Pragmatism philosophy

….

What it means for Computer Scientists?

Convenient CS Research Methods for Pragmatism

….

Computer Science Fields

Research Methodologies in Computer Sciences

A – Formal Methodologies

This category of methodologies is used by the folks in our field that proves facts about algorithms and systems . These Methodologies are geared more toward Theoretical Computer Science which is formal in the sense of mathematical for the lay man and is generally concerned with modelling and abstraction. Theoretical Computer Science used mathematical techniques to answer questions like: given a problem X, how much hard is it to solve? (question that deals with computability) ; How much time/space it takes? (Question that deals with complexity) ; Given a computation model Y, what are its limitations? ; Given a formalism what can it express?[10]

Research in theoretical computer science aim to discover algorithms in areas like Combinatorial problems, computation geometry, cryptography, parallel and distributed computing among many others

B – Experimental/Empirical Methodologies

C – Build Methodologies

…. it deals with building software

D – Process Methodologies

E – Model Methodologies

….

Software Engineering

Identity Crisis of Software Engineering

Research Methods in Software Engineering

Case Studies

Case Studies are quite common in Software Engineering according to Runeson and Host [5]

Exploratory Case Studies

Example School

Feasibility Studies

Comparative Studies

….

Research Methods in Computer Science

A-  Formal Research Methods

Mathematical Proof techniques

Algorithm Design and Analysis

Complexity Analysis Methods

These methods pertain to Complexity Theory

Computability Analysis Methods

Research Instruments in Computer Science

Methods Borrowed from social sciences

Questionnaires

Interviews

One to One Interviews:

Focus Group Interviews:

Ethnography

Diaries

Content Analysis

Computer Science Methodologies/Methods Books

There is a scarcity of books that cover research methodologies and methods in Computer Science as a whole (i.e. books that play the role of an encyclopedia of research methods or a textbook that “rule them all” if we can say that). Nevertheless, there are few awesome people who wrote books on research methodologies/methods for their respective sub-fields of Computer Science. “Chapeau Bas” and a big thank you for them and for their great job!!

Human Computer Interaction

  1. Research Methods in Human Computer Interactions by Jonathon Lazar et al. – a amazing book recommended by the majority of HCI supervisors in UK. A must read if you will be doing research in HCI. It also helps a lot people who are investigating topics such as Quality of Experience (QoE), Usability/Accessibility and other research topics that are similar in nature.
  2. Research methods for human-computer interaction by Paul Cairns and Anna L. Cox
  3. Doing Better Statistics in Human-Computer Interaction by Paul Cairns

Artificial Intelligence

  1. Empirical Methods for Artificial Intelligence by Paul Cohen

Software Engineering

There are more academic papers that cover experimental software engineering than actual books but one book  stand out of all the literature and is well recommended by many folks I know who are doing research in experimental software engineering (if you have any other suggestions, please leave them in the comments):

  1. Software Metrics: A Rigorous and Practical Approach by Fenton and Bieman

System Performance, Simulation and Quality of Service Methods

Many supervisors in UK universities recommend Raj book to their students to read. If you are doing a PhD in Computer Networks’ Quality of Service (QoS) or any system performance studies, the following list is for you:

  1. The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling by Raj Jain – This book is the holy grail on the topic.
  2. Measuring Computer Performance: A Practitioner’s Guide by Lilja
  3. Systems Performance: Enterprise and the Cloud by Brendan Gregg
  4. Performance Modeling and Design of Computer Systems: Queueing Theory in Action by Mor Harchol-Balter

Information Technology (Management & Technology)

Three volumes that are very detailed set of academic paper from the Journal of information Technology (JIT)  – just waiting for a good Samaritan to digest, compact them and put them in a textbook for students.

  1. Enacting Research Methods in Information Systems – Volume 1  (covers Critical Research, Grounded Theory, and Historical Approaches)
  2. Enacting Research Methods in Information Systems – Volume 2  (covers Interpretive Approaches and also explores Action Research)
  3. Enacting Research Methods in Information Systems – Volume 3  (covers Design Science Approaches and discusses Alternative Approaches including Semiotics Research, Complexity Theory and Gender in IS Research)

Good Books that are cross disciplines (not CS)

You can of course buy one of the following general high-level books (they are good!) but I believe if you are doing a specific field of research, it is better for you to buy (or borrow from the library) a book that covers directly your research field or even your particular sub-field instead of reading a book that is too general when it comes to covering research methods. Ask your supervisor/adviser, other academics or PhD candidates to suggest a good research methodologies/methods book on the topic:

  1. Research methodology: Methods and techniques by Kothari
  2. Research methodology: A step-by-step guide for beginners by Kumar [Covers health, education, psychology, social work, nursing, public health, library studies and marketing research]
  3. How to research by Loraine Blaxter
  4. Introduction to Research – Understanding and Applying multiple strategies by Elizabeth Depoy
  5. The Good Research Guide – For small-scale social research projects by Martyn Denscombe (6th Edition) – a very good book for the social sciences folks
  6. Quantitative Methods in Social Science by Stephen Gorard [cross discipline book but only for social sciences such as economics, sociology etc.]
  7. Keywords in Qualitative Methods by Michael Bloor et al.
  8. Your Research Project by Nicholas Walliman

There are many books tailoring specific research fields in many sciences. For completeness purpose, and to show you the big amount of high-quality books and articles we have in other disciplines when it comes to research methods. The below suggestions were given to me by friends of mine doing PhD research in those fields.

Business, Management and Marketing Research Methodologies/Methods Books/online resources

The best and greatest book on the topic which is the classic of the classics is the book of Bryan & Bell – a must read for the folks in Business and even economics. The following list elucidates few recommended books for Business research Methods:

  1. Business Research Methods by Emma Bell, Alan Bryman and Bill Harley – in the new edition Bill Harley wrote few chapters – if you are in a hurry you can buy the old 2011 version.
  2. Research Methods for Business Students by Saunders et al. is a GREAT  book on research methods. If half of the supervisors in UK recommend Bell & Bryman book, the other half definitely recommend this one. Actually I found Saunders book absolutely amazing and even better than Bell and Bryman which I have also read!
  3. Mathematics for Economics and Business by Ian Jacques – Now this book, according to a friend of mine who is doing a PhD in Economics, was strongly suggested for reading by her supervisor together with Bell and Bryman book.
  4. A dictionary of business research methods by John Duignan (kindle version and paid online version)
  5. Research methods for managers by Gill and Johnson
  6. Handbook of qualitative research methods in marketing by Russell W. Belk.

Online Video courses/YouTube Channels

  • Research Methods for Business Students Udemy course – is somehow of average quality. The literature review module of the course is too primitive and dumb. This is only a small taster video course when it comes to PhD candidates and is not authoritative in anyway as the instructor clearly states it is for a Masters and Undergraduates. I think it is also a taster for Masters. The course misses many things including research philosophical stances. You still need to read one of the classical textbooks I suggested previously.
  • YoutTube channel called Meanthat has a playlist on research methods for business.

Finance, Economics and Accounting

  1. Research Methods and Methodology in Finance and Accounting by Bob Ryan
  2. Research Methods in Accounting by by Malcolm Smith
  3. Handbook of Research Methods and Applications in Empirical Finance by Bell et al.
  4. Research Methodology in Applied Economics: Organizing, Planning, and Conducting Economic Research by Ethridge
  5. Mathematics for Economics and Business by Ian Jacques – Now this book, according to a friend of mine who is doing a PhD in Economics, was strongly suggested for reading by her supervisor together with Bell and Bryman book.

Human Resource Management

  1. Research methods in human resource management by Valerie Anderson

Psychology Research Methodologies/Methods Books

The lovely psychology discipline has a plethora of books on research methodologies and research methods. Ask your supervisor to suggest a book. For what is germane to the main topic of this article, Computer Science borrowed a lot (I mean a lot!) from psychology research especially in the fields of Human Computer Interaction, Quality of Experience, Accessibility and Usability. Few suggestions on research methods in the field of psychology are stated below (please if you have any suggestions, write them in the comments):

  1. Research Methods and Statistics in Psychology by by Hugh Coolican – Buy the new edition (2018) – it is amazing!
  2. Research Methods and Statistics in Psychology by by Haslam and McGarty.
  3. Statistical Methods for Psychology by David C. Howell.
  4. Statistics and Research Methods in Psychology with Excel by Verma.
  5. Research Methods in Psychology: Investigating Human Behavior by Paul G. Nestor and Russell K. Schutt.
  6. Research Methods in Psychology: Evaluating a World of Information by Beth Morling.
  7. Research methods and data analysis for psychology by  Stuart Wilson and Rory MacLean
  8. Discovering research methods in psychology : a student’s guide by Sanders

Information Studies, Knowledge/records Management and Library science

  1. The Facet LIS Textbook Collection: Research Methods in Information by Alison Pickard – this book is great. I learned a lot from it. It has very easy explanation of the research philosophies. It has a great chapter on reviewing the literature which contains good advice. This book teaches you how to choose your sample, teaches statistical tests among many other goodies.
  2. Research Methods in Library and Information Science by Connaway and Radford.
  3. Research methods in information by Alison Jane Pickard

Political Science

  1. Political Science Research Methods by by Johnson and Reynolds
  2. Political Research: Methods and Practical Skills by by Sandra Halperin and Oliver Heath
  3. Research Methods in Politics by Burnham et al.

Geography/Human Geography

  1. Research Methods Geography: A critical introduction by Gomez and Jones
  2. An Introduction to Scientific Research Methods in Geography and Environmental Studies by Daniel Montello and Paul Sutton
  3. Qualitative research methods in human geography by Iain Hay

Sociology

  1. Social Research Methods by Bryman
  2. A dictionary of social research methods by Mark Elliot et al. (kindle version and paid online version)

Anthropology

  1. Research methods in anthropology: qualitative and quantitative approaches by H. Russell Bernard

Law

  1. Research Methods in Human Rights by Bard A. Andreassen

Education/Languages

  1. Research Methods in Education by Cohen et al.
  2. Research Methods and Methodologies in Education by Coe et al.
  3. Introduction to Research Methods in Education by Punch and Oancea
  4. Research methods in language and education by Kendall A. King
  5. Visual research methods in educational research by Julianne Moss and Barbara Pini
  6. Research methods for applied language studies by Richards et al.
  7. Research methods for English studies by Gabriele Griffin

Theology and religions

  1. The Routledge handbook of research methods in the study of religion by Michael Stausberg and Steven Engler

Encompassing many social sciences fields

  1. Research Methods in Indigenous Contexts by Arnold Groh.
  2. Research Methods in the Study of Substance Abuse by VanGeest et al.
  3. The SAGE handbook of social media research methods by Sloan and Luke
  4. Handbook of qualitative research methods on human resource management : innovative techniques by Keith Townsend et al.
  5. Research methods for creating and curating data in the digital humanities by Matt Hayler and Gabriele Griffin
  6. Research methods for reading digital data in the digital humanities by Gabriele Griffin and Matt Hayler

Health and Medicine

  1. Introduction to health research methods  by Jacobsen
  2. Research methods in health: investigating health and health services by Ann Bowling
  3. Clinical research methods for surgeons by David F. Penson and John Wei
  4. Handbook of health research methods : investigation, measurement and analysis by Ann Bowling and Shah Ebrahim

Pharmacy

  1. Pharmacy practice research methods by Zaheer-Ud-Din Babar

Few examples of books for other fields

The following books  stated here for completion sake only and are suggested by few friends and acquaintances from the University of Dundee, I have never managed to look at them at all and I should mention the last two books showed up in a search I made myself since I could not find books on biology and engineering. A disclaimer: I am not sure if these books might be helpful to us even, they might, I don’t know. I just need more time to look into them.

  1. Research Methods for Cognitive Neuroscience by by Aaron Newman
  2. Research Methods In Health: Investigating Health And Health Services by by Ann Bowling
  3. Nutrition Research Methodologies by Lovegrove et al.
  4. Research Methodologies and Practical Applications of Chemistry by Pogliani et al.
  5. Research Methods and Applications in Chemical and Biological Engineering by Pourhashemi et al. – This book is not released yet and I found it in an online search.
  6. Engineering Research Methodology: A Practical Insight for Researchers by Deb et al. – This book is not released yet and I found it in an online search.

Books that cover in detail only one research method (across many disciplines)

These types of books are targetted to a specific research method across many discplines, in other words, not really tied to a particular field.

Grounded Theory

The bible on grounded theory is that of Strauss and Corbin:

  1. Grounded Theory in Practice – an amazing book and a legendary classic suggested as a reading by many supervisors.

References

[1] Demeyer, Serge. “Research methods in computer science.” ICSM. 2011.

[2] Perry, Dewayne E., Adam A. Porter, and Lawrence G. Votta. “Empirical studies of software engineering: a roadmap.Proceedings of the conference on The future of Software engineering. ACM, 2000.

[3] Zelkowitz, Marvin V., and Dolores R. Wallace. “Experimental models for validating technology.Computer 31.5 (1998): 23-31.

[4] Shaw, Mary. “What makes good research in software engineering?.International Journal on Software Tools for Technology Transfer 4.1 (2002): 1-7.

[5] Runeson, Per, and Martin Höst. “Guidelines for conducting and reporting case study research in software engineering.Empirical software engineering 14.2 (2009): 131.

[6] Andonie, Răzvan, and Ioan Dzitac. “How to Write a Good Paper in Computer Science and How Will It Be Measured by ISI Web of Knowledge.International Journal of Computers Communications & Control 5.4 (2010): 432-446.

[7] Chandra, Shalini, and Raees Ahmad Khan. “Implementing Availability State Transition Model to Quantify Risk Factor.Advances in Computer Science, Engineering & Applications. Springer, Berlin, Heidelberg, 2012. 937-952.

[8] Rüger, Stefan. “How to write a good PhD thesis and survive the viva.Open University (2016).

[9] Saunders, M. N. K., and P. C. Tosey. “The layers of research design.Rapport Winter (2013): 58-59.

[10] Amaral, José Nelson. “About computing science research methodology.” (2011).

[11] Holz, Hilary J., et al. “Research methods in computing: what are they, and how should we teach them?.” ACM SIGCSE Bulletin. Vol. 38. No. 4. ACM, 2006.

[12] Hassani, Hossein. “Research methods in computer science: The challenges and issues.” arXiv preprint arXiv:1703.04080(2017).

[13] Yin, Robert K. Case study research and applications: Design and methods. Sage publications, 2017.

[14] Jain, Raj. The art of computer systems performance analysis: techniques for experimental design, measurement, simulation, and modeling. John Wiley & Sons, 1990.

[15] Serge Demeyer – University of Antwerp slides found at http://win.ua.ac.be/~sdemey/Tutorial_ResearchMethods/ResearchMethds01_MethodsOvervw.pdf

[16] Saunders, M. N. K., and P. C. Tosey. “The layers of research design.Rapport Winter (2013): 58-59.

[17] Denning, Peter J. “Is computer science science?.” Communications of the ACM 48.4 (2005): 27-31.

[18] Easterbrook, Steve, et al. “Selecting empirical methods for software engineering research.Guide to advanced empirical software engineering. Springer, London, 2008. 285-311.

[19] Johnson, R. Burke, Anthony J. Onwuegbuzie, and Lisa A. Turner. “Toward a definition of mixed methods research.Journal of mixed methods research 1.2 (2007): 112-133.

[20] Goede, Roelien, and Carina De Villiers. “The applicability of grounded theory as research methodology in studies on the use of methodologies in IS practices.Proceedings of the 2003 annual research conference of the South African institute of computer scientists and information technologists on Enablement through technology. South African Institute for Computer Scientists and Information Technologists, 2003.

[21] Kitchenham, Barbara, et al. “Systematic literature reviews in software engineering–a tertiary study.Information and software technology 52.8 (2010): 792-805.

[22] Kitchenham, Barbara, et al. “Systematic literature reviews in software engineering–a systematic literature review.Information and software technology 51.1 (2009): 7-15.

[23] Tichy, Walter F. “Should computer scientists experiment more?.Computer 31.5 (1998): 32-40.

[24] Wegner, Peter. “Research paradigms in computer science.Proceedings of the 2nd international Conference on Software Engineering. IEEE Computer Society Press, 1976.

[25] Glass, Robert L., Iris Vessey, and Venkataraman Ramesh. “Research in software engineering: an analysis of the literature.Information and Software technology 44.8 (2002): 491-506.

[26] Ramesh, Venkataraman, Robert L. Glass, and Iris Vessey. “Research in computer science: an empirical study.Journal of systems and software 70.1-2 (2004): 165-176.

[27] Mackenzie, Noella, and Sally Knipe. “Research dilemmas: Paradigms, methods and methodology.Issues in educational research 16.2 (2006): 193-205.

[28] Johnson, Chris. “What is research in computing science.Computer Science Dept., Glasgow University. Electronic resource: http://www. dcs. gla. ac. uk/~ johnson/teaching/research_skills/research. ht ml (2006).

[29] Kitchenham, Barbara A., et al. “Preliminary guidelines for empirical research in software engineering.IEEE Transactions on software engineering 28.8 (2002): 721-734.

[30] Budgen, David, and Pearl Brereton. “Performing systematic literature reviews in software engineering.Proceedings of the 28th international conference on Software engineering. ACM, 2006.

[31] Zelkowitz, Marvin V., and Dolores Wallace. “Experimental validation in software engineering.Information and Software Technology 39.11 (1997): 735-743.

[32] Feitelson, Dror G. “Experimental computer science: The need for a cultural change.Internet version: http://www. cs. huji. ac. il/~ feit/papers/exp05. pdf (2006).

[33] Kitchenham, Barbara Ann. “Evaluating software engineering methods and tools, part 7: planning feature analysis evaluation.ACM SIGSOFT software engineering Notes 22.4 (1997): 21-24.

[34] Kitchenham, Barbara Ann, and Lesley M. Pickard. “Evaluating software engineering methods and tools: part 9: quantitative case study methodology.ACM SIGSOFT Software Engineering Notes 23.1 (1998): 24-26.

[35] Kitchenham, Barbara, et al. “Evaluating guidelines for reporting empirical software engineering studies.Empirical Software Engineering 13.1 (2008): 97-121.

[36] Robinson, Hugh, Judith Segal, and Helen Sharp. “Ethnographically-informed empirical studies of software practice.Information and Software Technology 49.6 (2007): 540-551.

[37] Research Methods – Empirical/Experimental CS Research Methods slides of a course given by J. Gamper – Free University of Bozen-Bolzano found at https://www.inf.unibz.it/~calvanese/teaching/2017-02-PhD-RM/RM-2017-M4-gamper.pdf

[38] Wohlin, Claes, Martin Höst, and Kennet Henningsson. “Empirical research methods in software engineering.” Empirical methods and studies in software engineering. Springer, Berlin, Heidelberg, 2003. 7-23.

[39] Wainer, Jacques, et al. “Empirical evaluation in Computer Science research published by ACM.Information and Software Technology 51.6 (2009): 1081-1085.

[40] Hagen, Penny, et al. “Emerging research methods for understanding mobile technology use.Proceedings of the 17th Australia conference on Computer-Human Interaction: Citizens Online: Considerations for Today and the Future. Computer-Human Interaction Special Interest Group (CHISIG) of Australia, 2005.

[41] Dittrich, Yvonne, et al. “Editorial: for the special issue on qualitative software engineering research.Information and software technology 49.6 (2007): 531-539.

[42] Newell, Allen, and Herbert A. Simon. Computer science as empirical inquiry: Symbols and search. ACM, 2007.

[43] Glass, Robert L., Venkataraman Ramesh, and Iris Vessey. “An analysis of research in computing disciplines.Communications of the ACM 47.6 (2004): 89-94.

[44] Kantner, Laurie, Deborah Hinderer Sova, and Stephanie Rosenbaum. “Alternative methods for field usability research.Proceedings of the 21st annual international conference on Documentation. ACM, 2003.

[45] Vessey, Iris, Venkataraman Ramesh, and Robert L. Glass. “A unified classification system for research in the computing disciplines.Information and Software Technology 47.4 (2005): 245-255.

[46] Harrison, Rachel, and Moira Wells. “A meta-analysis of multidisciplinary research.Conference on Empirical Assessment in Software Engineering (EASE). 2000.

[47] Crotty, Michael. The foundations of social research: Meaning and perspective in the research process. Sage, 1998.

[48] Clarke, Roger. “Appropriate research methods for electronic commerce.Canberra: Xamax Consultancy (2000).

Leave a Reply

Your email address will not be published. Required fields are marked *