Key Concepts Blog: Data Collection Techniques Readings

Wildemuth (2009)

Chapter 18 Transaction Logs

Transaction Log Analysis (TLA)

TLA is used to study user system interaction (like browse behaviors) and system-mediated interpersonal interaction (like reference and support chat services).  There are server side and client side TLA.  Server side TLA is cheaper and unobtrusive, but has ethical concern.  Client side TLA is expensive and needs contact with participants.

Advantages and Disadvantages of TLA

Advantages are Accurate because data is captured as they occur, draws on large volume of data, and it is appropriate for both experimental and field study Types.

Disadvantages are it does not record context of users like motives, difficulty to distinguish between individual users and very large amount of data.

Wildemuth (2009) 
Chapter 19 Think-Aloud Protocols

Think-aloud Protocols

Subjects speak aloud what they are thinking while performing tasks during an experiment.  It is used to understand subject’s cognitive processes.  It is used in LIS to study search tactics, processes and strategies.  Some kinds of think-aloud protocols are concurrent versus retrospective and individual versus collaborative.

Advantages and Disadvantages

Advantages

·       Relatively easy to collect data.

·       Makes it possible to investigate reactions, feelings and problems of subjects – difficult to do this with other methods.

·       Shows sequential steps of subject’s cognitive process over a period of time rather than obtaining a general description at the end of the process (detailed data)

Disadvantages

·       Slower task performance speed during experiment because they have to speak.

·       May influence the thinking process on the task (may disrupt or improve task).

Wildemuth (2009) 
Chapter 20  Direct Observation

Direct observation is watching actions or behaviors of participants in a particular setting and making field notes and/or times, counting or rating behaviors or events.  There are two types – participant and nonparticipant observation.  May be combined other methods like interview.

Advantages and Disadvantages

Advantages - Better accuracy than asking questions

Disadvantages:

·       Problem of observing behaviors that occur infrequently and at unpredictable times.

·       Obtrusive (people may change their behavior when observed)

·       Ethical issues – privacy and confidentiality of participants.

Procedures

·       Gaining access to the setting

·       Gaining consent of individuals to be observed

·       Plan a sample (eg. Particular time intervals.)

·       Decide which people to observe.

·       Collect data using observation schedule or checklist.

·       Sometimes, video and audio recordings are used (may be more obtrusive)

·       Pre-analysis of data

·       Analysis of data – qualitative or interpretive approaches may be used.

·       Leaving the field.

Wildemuth (2009) 
Chapter 21 Participant observation

What is Participant Observation

·       Observer is participant in the setting

·       Such participation leads to better understanding of topic of study

·       This understanding can lead to better theories about social processes.

Usually uses interpretive research paradigm – reality is socially constructed.  Usually researcher needs permission from gatekeepers of the culture to be in the setting.

Types of Participant Observation

Passive participation (being physically present in the setting)

Complete participation (full membership or active participant - true member of the culture)

Wildemuth (2009)

Chapter 22 Research Diaries

Research Diaries

Research diaries are usually solicited diaries.  Types - unstructured, semi structured, or structured (eg. logs).  The advantages of Research diaries are they captures ordinary events that might be neglected by other methods and information recorded at or close to the time of the event.  However, they impose significant burden on participants and researcher. 

Considerations:

• ·       Amount of structured on diary entries
• ·       Record making triggers.
• ·       How long should the diary be kept?
• ·       Frequencies and intervals (between an event and recording)
• ·       Technologies for capturing diaries (eg. Audio visual, web-based)
• ·       Using with interview method.

Wildemuth (2009)
Chapter 23 Unstructured Interviews

Unstructured Interviews

Also called informal conversational interview, in-depth interview or non-standardized interview or ethnographic interview.  Question and answer categories are not predetermined.  It is usually part of participant observation.  Ideally, the researcher follows the respondent’s narration and generates questions based on his/her reflection of the narration.  Aide-memoire may be used – agenda/broad guide to topics/issues that might be covered – open ended and flexible.

How to conduct unstructured interview:

• ·       Accessing the setting.
• ·       Understanding the language and culture of the interviewees.
• ·       Presenting oneself (researcher is learner in the conversation trying to understand the interviewee’s experiences from the interviewee’s point of view).
• ·       Gaining trust
• ·       Capturing the data – note taking, audio recording, memory techniques

Wildemuth (2009)

Chapter 24 Semistructured Interviews

Semistructured Interviews

·       Has predetermined questions (interview guide)

·       The order of questions can be modified, wording can be changed, a question might be omitted or added as necessary.

In contrast, structured interview is like survey except it is conducted personally.

Interview guide for semi-structured interview

It may have four types of questions - essential questions, extra questions, throw-away questions and probing questions (asking elaborations to elicit more information).

Questions to avoid are affectively worded questions, double-barreled questions (two issues in one question) and complex questions (instead use brief, concise and to-the-point  questions).

Wildemuth (2009) 
Chapter 25 Focus Group

Focus Group interview is a group of individuals selected and assembled by researchers to discuss and comment on the research topic based on their personal experience.  Members are encouraged to talk to each other as well as the researcher and data collected includes their discussion with each other as well.  Researcher should pay attention to the consensus as well as opposing views.

Steps/Issues

• ·       Decide the objectives of the research and issues of discussion in advance.
• ·       Select moderator – keeps the discussion on track.
• ·       Identifying and recruiting appropriate participants – decide the characteristics of the people you want to participate in advance.
• ·       During the session – select a site that is quite, comfortable, private enough and has minimum interruptions.
• ·       Let participants to get comfortable with other before the session starts.
• ·       Understanding your results – since this will be qualitative data, it needs transcriptions/detailed notes and coding data based on themes.

Wildemuth (2009)

Chapter 26 Survey Research

Survey Research is a popular method, but needs extensive planning.  You have to carefully consider your objectives when choosing participants, design and administer survey and data analysis methods.

Steps

• ·       Design the survey instruments – prepare set of questions. 

o   The questions have influence on the number of responses and validity of responses..

o   May be closed ended questions or open ended questions (for exploratory research)

o   Organize questions appropriately.

o   Consider physical appearance.

• ·       Pretesting (by experts) and pilot testing (by a small subset of the sample)
• ·       Administering the survey – by mail, e-mail, website, online chat, phone …
• ·       To increase response rate

o   Contact individuals multiple times, thank you notes and reminders and collaborating with organizations, incentives for participating.

• ·       Analyzing responses.

o   For descriptive and comparative purposes.

o   Open ended questions needs categorizing responses.

Wildemuth (2009)
Chapter 27 Measuring Cognitive and Affective Variables

Measuring cognitive and affective variables

These are variables like attitudes, beliefs, feelings, interests etc.  These variables need to be operationalized.

Steps

• ·       Identifying appropriate inventory (standards of measurement) from previous researches – checking from other researches how they measured that variable.
• ·       Assessing reliability and validity of an inventory.  It has to be consistent

Wildemuth (2009)
Chapter 28 Developing new measures

Cognitive and affective variables cannot be directly observed.  The objective is to quantify the amount or level of such variables

Likert Scales and other similar scales use a five point scale like from strongly agree to strongly disagree.

4-point scale – does not allow a neutral response.

We have to be careful not to use ambiguous wording.

Connway and Powell (2007)

 Chapter 5

Advantages and disadvantages of Questionnaire

Advantages

• ·       Encourages frank answers
• ·       Eliminates interviewer bias due to verbal presentation of questions.
• ·       Eliminates variation in the questioning process.
• ·       Can be completed at the convenience of respondents (esp. mail questionnaire) which gives time to respondents and thus more accurate answers.
• ·       Suited for quantitative data collection.
• ·       Enables to collect large amount of data in a short time.
• ·       Relatively inexpensive.

Disadvantages

• ·       Persons who are highly motivated about the topic are likely to return.
• ·       Difficult for uneducated people, which results in a biased return.
• ·       High non-response rates.
• ·       Electronic questionnaire can reach only those who have access to the technology.

Observation

Observational study is done by watching attentively in a systematic manner.  Data is collected not by asking but by observing.  Observation can be considered as research method or data collection method to be used in another research method.  As data collection technique, it may be us
  
Sources

Connaway, L. S. & Powell, R.R. (2007). Chapter 5. In Basic Research Methods or Librarians (5th ed.). California.

Wildemuth, B. M. (2009). Chapter 18-28. In Applications of Social Research Methods to Questions in Information and Library Science. Connecticut.