creating.pdf

Linda Amankwaa, PhD, RN, FAAN

Abstract: Experienced and novice researchers, plan qualitative proposals where evidence of rigor must be provided within the document. One option is the creation of a trustworthiness protocol with details noting the characteristic of rigor, the process used to document the rigor, and then a timeline directing the planned time for conducting trustworthiness activities. After reviewing several documents, an actual plan of conducting trustworthiness as not found. Thus, these authors set out to create a trustworthiness protocol designed not only for the dissertation, but a framework for others who must create similar trustworthiness protocols for their research. The purpose of this article is to provide a reference for the trustworthiness plan, a dissertation example and showcase a trustworthiness protocol that may be used as an example to other qualitative researchers embarking on the creation of a trustworthiness protocol that is concrete and clear.

Key Words: Trustworthiness, Research Protocols, Qualitative Research

Creating Protocols for Trustworthiness in Q ualitative

Research

Anything perceived as being of low or no value is also perceived as being worthless, unreliable, or invalid. Research that is perceived as worthless

is said to lack rigor. This means findings are not worth noting or paying attention to, because they are unreliable. To avoid this argument, proof of reliability and validity in qualitative research methods is required. However, some researchers have suggested that reliability and validity are not terms to be used to explain the usefulness of qualitative research. They believe that those terms are to be used to validate quantitative research (Altheide & Johnson, 1998; Leininger, 1994). Morse (1999) expressed concern about qualitative research losing value by em­phasizing when qualitative researchers fail to recognize crucial importance of reliability and validity in qualita­tive methods, they are also mistakenly supporting the idea that qualitative research is defective and worthless, lacking in thoroughness, and of unempirical value. Guba and Lincoln (1981) stated that, "All research must have 'truth value', 'applicability', 'consistency', and 'neutrality' in order to be considered worthwhile. They concluded that the end result of establishing rigor or "trustworthiness," (the analogous for rigor in qualitative research), for each method of research requires a differ­ent approach. It was noted by Guba and Lincoln (1981),

Linda Amankwaa, PhD, RN, FAAN, is an Associate Professor in the Department of Nursing at Albany State Uni­versity in Albany, GA31705. Dr. Amankwaa may be reached at: 229-430-4731 or at: Linda.Amankwaa@asurams.edu.

within the rationalistic paradigm, criteria to reach the goal of rigor are internal validity, external validity, reli­ability, and objectivity. They proposed use of terms such as credibility, fittingness, auditability, and confirmability in qualitative research to ensure "trustworthiness" (Guba & Lincoln, 1981). Later, these criteria were changed to credibility, transferability, dependability, and confirm­ability (Lincoln & Guba, 1985).Lincoln and Guba (1985) suggested that the value of a research study is strengthened by its trustworthiness. As established by Lincoln and Guba in the 1980s, trustwor­thiness involves establishing:

• Credibility – confidence in the 'truth ' of the finding

• Transferability – showing that the findings have applicability in other contexts

• Dependability – showing that the findings are consistent and could be repeated

• Confirmability – a degree of neutrality or the ex­tent to which the findings of a study are shaped by the respondents and not researcher bias, motivation, or interest.

For purposes of this discussion, this classic work is used to frame trustworthiness actions and activities to create a protocol for qualitative studies. Nursing faculty and doctoral nursing students who conduct qualitative research will find this reference useful.

Journal of Cultural Diversity • Vol. 23, No. 3 Fall 2016

Credibility ActivitiesLincoln and Guba (1985) described a series of techniques

that can be used to conduct qualitative research that at­tains the criteria they outlined. Techniques for establishing credibility as identified by Lincoln and Guba (1985) are: prolonged engagement, persistent observation, triangula­tion, peer debriefing, negative case analysis, referential adequacy, and member-checking. Typically member check­ing is viewed as a technique for establishing the validity of an account. Lincoln and Guba posit that this is the most crucial technique for establishing credibility.

Transferability ActivitiesOne strategy that can be employed to facilitate transfer-

ability is thick description (Creswell & Miller, 2000; Lincoln & Guba, 1985). Thick description is described by Lincoln and Guba as a way of achieving a type of external valid­ity. By describing a phenomenon in sufficient detail one can begin to evaluate the extent to which the conclusions drawn are transferable to other times, settings, situations, and people. Since, as stated by Merriam (1995) it is the responsibility of the consumer of research to determine or decide if and how research results might be applied to other settings, the original researcher m ust provide detailed information about the phenomenon of study to assist the consumer in making the decision. This requires the provision of copious amounts of information regard­ing every aspect of the research. The investigator will include such details as the location setting, atmosphere, climate, participants present, attitudes of the participants involved, reactions observed that may not be captured on audio recording, bonds established between participants, and feelings of the investigator. One word descriptors will not suffice in the development of thick description. The investigator in essence is telling a story with enough detail that the consumer/reader obtains a vivid picture of the events of the research. This can be accomplished through journaling and maintaining records whether digital or handwritten for review by the consumer/reader.

Confirmability ActivitiesTo establish confirmability Lincoln and Guba (1985)

suggested confirmability audit, audit trail, triangulation, and reflexivity. An audit trail is a transparent description of the research steps taken from the start of a research project to the development and reporting of findings (Lincoln & Guba). These are records that are kept regarding what was done in an investigation. Lincoln and Guba cite Halpern's (1983) categories for reporting information when develop­ing an audit trail:

"1) Raw data – including all raw data, written field notes, unobstrusive measures (documents); 2) Data reduction and analysis products – including sum­maries such as condensed notes, unitized information and quantitative summaries and theoretical notes; 3) Data reconstruction and synthesis products – includ­ing structure of categories (themes, definitions, and relationships), findings and conclusions and a final report including connections to existing literatures and an integration of concepts, relationships, and interpretations; 4) Process notes – including method­ological notes (procedures, designs, strategies, ratio­nales), trustworthiness notes (relating to credibility, dependability and confirmability) and audit trail notes;5) Materials relating to intentions and dispositions –

including inquiry proposal, personal notes (reflexive notes and. motivations) and expectations (predictions and intentions); 6) Instrument development informa­tion – including pilot forms, preliminary schedules, observation formats" (page#).

Using multiple data sources within an investigation to enhance understanding is called triangulation. Researchers see triangulation as a method for corroborating findings and as a test for validity (Lincoln & Guba, 1985). Rather than seeing triangulation as a method for validation or veri­fication, qualitative researchers generally use this technique to ensure that an account is rich, robust, comprehensive and well-developed (Lincoln & Guba, 1985).

Denzin (1978) and Patton (1999) identify four types of triangulation: methods triangulation, source triangulation; analyst triangulation; theory/perspective triangulation. They suggested that methods triangulation involves check­ing out the consistency of finding generated by different data collection methods. Triangulation of sources is an examination of the consistency of different data sources from within the same method (i.e. at different points in time; in public vs. private settings; comparing people with different viewpoints).

Another one of the four methods identified by Denzin and Patton includes analyst triangulation. This is the use of multiple analysts to review findings or using multiple observers and analysts. This provides a check on selective perception and illuminate blind spots in an interpretive analysis. The goal is to understand multiple ways of see­ing the data. Finally, they described theory/perspective triangulation as the use of multiple theoretical perspectives to examine and interpret the data.

According to Lincoln and Guba (1985) reflexivity is, "An attitude of attending systematically to the context of knowledge construction, especially to the effect of the researcher, at every step of the research process." They suggested the following steps to develop reflexivity: 1) Designing research that includes multiple investigators. This fosters dialogue, leads to the development of comple­mentary and divergent understandings of a study situation and provides a context in which researchers' (often hid­den) – beliefs, values, perspectives and assumptions can be revealed and contested; 2) Develop a reflexive journal. This is a type of diary where a researcher makes regular entries during the research process. In these entries, the researcher records methodological decisions and the reasons for them, the logistics of the study and reflection upon what is hap­pening in terms of one's own values and interests. Diary keeping of this type is often very private and cathartic; 3) Report research perspectives, positions, values and beliefs in manuscripts and other publications. Many believe that it is valuable and essential to briefly report in manuscripts, as best as possible, how one's preconceptions, beliefs, values, assumptions and position may have come into play during the research process.

Dependability ActivitiesTo establish dependability, Lincoln and Guba (1985) sug­

gested a technique known as inquiry audit. Inquiry audits are conducted by having a researcher that is not involved in the research process examine both the process and product of the research study (Lincoln & Guba, 1985). The purpose is to evaluate the accuracy and evaluate whether or not the findings, interpretations and conclusions are supported by the data (Lincoln & Guba, 1985).

Journal of Cultural Diversity • Vol. 23, No. 3 Fall 2016

Creating a Protocol for Qualitative ResearchersThe creation of a protocol for establishing trustwor­

thiness within qualitative research is essential to rigor. Further, we note that researchers rarely document how or what their trustworthiness plan or protocol consisted of within research documents. Thus, we posit here that creating such a protocol prior to initiation of the research study is essential to revealing trustworthiness within the research process. By creating this plan a priori, the rigor of qualitative research is apparent.

This history and purposed need for this article heralds from a doctoral dissertation search to find examples of trustworthiness protocols for direction to complete trust­worthiness within doctoral qualitative research. Since none could be found, discussions lead the researcher to create a table that could used by those who are planning qualita­tive studies. Another interesting point is that qualitative researchers, unlike quantitative researchers, rarely create protocol guidelines.

The establishment of trustworthiness protocols in quali­tative research requires the use of several techniques. This protocol will be detail specific so those researchers have a guideline for trustworthiness activities. Such a protocol guides prospective qualitative researchers in their quest for rigor. Several tables are presented here. The first table outlines the main topics within the trustworthiness proto­col. The remaining tables outline the suggested activities within trustworthiness protocol and for those creating a trustworthiness protocol.

Table one is the basic criteria for a trustworthiness pro­tocol using Lincoln and Guba (1985). However, researchers may use other models of rigor. Creating a table aligned with the planned model of rigor is the recommendation. The following five table are examples of a "created" protocol with examples of very specific activities related to each trustworthiness criteria.Summary

In summary, trustworthiness is a vital component within the research process. Attending to the language of trustworthiness and the important activities of reliabil­ity, add to the comprehensiveness and the quality of the research product. This discussion heralds the new idea that trustworthiness must be planned ahead of time with a protocol. This protocol must include dates and times trustworthiness actions. We contend that researchers can use the protocol by adding two columns which specify the date of the planned trustworthiness action and the date the action was actually completed. This information can then be included in the audit trail thus authenticating the work qualitative researcher and the rigor of the research.

REFERENCESAltheide, D., & Johnson, J. (1998). Criteria for assessing interpre­

tive validity in qualitative research. In N. K. Denzin, & Y. S. Lincoln (Eds.), Collecting and interpreting materials, 283- 312.

Creswell, J. & Miller, D. (2000). Determining validity and qualita­tive inquiry. Theory Into Practice, 39(3), 125-130.

Denzin, N. (1978). Sociological Methods. New York: McGraw-Hill.Guba, E. & Lincoln, Y. (1981). Effective evaluation: improving the

usefidness of evaluation results through responsive and naturalistic approaches. San Francisco, CA: Jossey-Bass.

Leininger, M. (1994). Evaluation criteria and critique of qualitative and interpretive research. Qualitative Inquiry, 1, 275-279.

Lincoln, Y. S. & Guba, E. G. (1985). Naturalistic Inquiry. Newbury Park, CA: Sage Publications.

Morse, J. (1999). Myth #3: Reliability and validity are not relevant to qualitative mquiry.Qualitative Heath Research, 9, 717.

Patton, M. Q. (1999). "Enhancing the quality and credibility of qualitative analysis." HSR: Health Services Research. 34(5), Part II, 1189-1208.

BIBLIOGRAPHYBitsch, V. (2005). Qualitative research: A grounded theory example

and evaluation criteria. Journal of Argibusiness, 23 (1), 75-91.Carpenter, R. (1995). Grounded theory research approach. In H.

J. Streubert & R. D. Carpenter(Eds-), Qualitative research and in nursing: Advancing the humanistic imperative, 145-161.

Cohen D., Crabtree, B. (2006). Qualitative Research Guidelines Project. July 2006. http://www.qualres.org/HomeRefl-3703. html

Giacomini, M. & Cook, D. (2000). A user's guide to qualitative research in health care. In Users' guides to evidence-based medicine. Journal of the American Medical Association, 284(4), 478-482.

Morse, J. Barrett, M., Mayan, M., Olson, K., & Spiers, J. (2002). Verification strategies for establishing reliability and valid­ity in qualitative research. International Journal of Q ualita­tive Methods, 1, 2, Article 2. Retrieved April 30, 2010 from http: / / w w w.ualberta.ca/-ijqrn/

Neuman, L. (2003). Qualitative and quantitative measurements. In Social research methods: Qualitative and quantitative approaches, fifth edition, 169-209.

Plack, M. (2005). Human nature and research paradigms: Theory meets physical therapy practice. The Qualitative Report, 10(2), 223-245.

Polit, D. & Hungler, B. (1999). Research control in quantitative research. In Nursing research: Principles and m ethods, sixth edition, 219-238. Lippincott.

Rubin, H. & Rubin, I. (1995). Qualitative interviewing: The art of hearing data. Thousand Oaks, CA: Sage Publications.

Siegle, D. (2002). Principles and methods in educational research: A web-based course from the University of Connecticut. Re­trieved April 30, 2010 from http: / / www.gifted.uconn.edu/ siegle / research/qualitative / qualitativeInstructorNotes.html

Tobin, G. & Begley, C. (2004). Methodological rigour within a qualitative framework. Journal of Advanced Nursing, 48(4), 388-396.

Table 1. Basic Trustworthiness Criteria (Lincoln & Guba, 1985)

Criteria Technique

Credibility Peer debriefing, member checks, journaling

Transferability Thick description, journaling

Dependability Inquiry audit with audit trail

Confirmability Triangulation, journaling

Journal of Cultural Diversity Fall 2016

Table 2. Credibility

Credibility Recommended activities/plan

Peer 1. Write plan within proposal.debriefing/debriefer

2. Commission a peer to work with researcher during the time of interviews and data collection.

3. This person must complete an attestation form to work with researcher. Plan to meet with this person after each interview.

4. During visits with the peer debriefer, research and peer discuss interviews, feelings, actions of subjects, thoughts, and ideas that present during this time. Discuss blocking, clouding and other feelings of researcher. Discuss dates and times needed for these activities. Will meet once a week for 30 minutes to an hour.

5. Journal these meetings. Write about thoughts that surfaced and keep these dated for research and evaluation during data analysis.

6. Need to be computer files so that you may use this information within data analysis.

Member Checks 1. Outline different times and reasons you plan to conduct member checks or collect feedback from members about any step in the research process.

2. Member checks will consist of communication with members after significant activities.

3. These activities may include interviews, data analysis, and other activities.

4. Within two weeks of the interview, send members a copy of their interview so that they can read it and edit for accuracy.

5. Within two weeks of data analysis completion, member will review a copy of the final themes.

6. Members are asked the question, “Does the interview transcript reflect your words during the interview?”

7. Choose negative cases and cases that follow pattern.

8. Be sure these check are recorded and are computer files so that you may use this information in data analysis.

Journaling plans 1. Journaling will begin with the writing of the proposal.

2. Journaling will be conducted after each significant activity. These include each interview, weekly during analysis, after peer debriefing visits, and theme production.

3. Journals will be audited by research auditor.

4. Journals will include dates, times, places and persons on the research team.

5. Journals need to be computer files so that you may use them in data analysis.

Protocol Create a timeline with planned dates for each activity related to credibility before commencing the study. This protocol with dates and activities should appear in the appendix.

Journal of Cultural Diversity • Vol. 23, No. 3 Fall 2016

Table 3. Transferability

Thick Description Actions for this activity include:

1. Reviewing crafted questions with Peer reviewer for clarity.

2. Planning questions that call for extended answers.

3. Asking open ended questions that solicit detailed answers.

4. Interviewing in such a way as to obtain a detailed, thick and robust response.

5. The object is to reproduce the phenomenon of research as clearly and as detailed as possible.

6. This action is replicated with each participant and with each question (sub-question) or statement.

7. This continues until all questions and sub-questions are discussed.

8. The peer reviewer along with the researcher review responses for thickness and robustness.

9. There are two issues related to thick description here. The first is receiving thick responses (not one sentence paragraphs). The second is writing up the responses of multiple participants in such a way as to describe the phenomena as a thick response.

Journaling Actions for this activity include:

1. Planning journal work in advance is an option. Such that the researcher could decide what dates and how often the journal will occur.

2. Journaling after interview is common.

3. Journaling after peer-review sessions.

4. Journaling after a major event during the study.

5. Journal entries should be discussed with peer reviewer such that expression of thoughts and ideas gleaned during research activities can be connected to participants’ experiences.

6. Journals can be maintained in various formats. Information for the journal can be received in the form of emails, documents, recordings, note cards/note pads. We recommend that the researcher decide on one of the options.

7. Journaling includes dates of actions related to significant and insignificant activities of the research.

8. Journal may start on the first date a decision is made to conduct the research.

9. Journaling ends when the research is completed and all participants have been interviewed.

10. As with each of the concepts here, create a timeline with a date-line protocol for each activity before commencing the study.

Protocol Create a timeline with planned dates for each activity related to transferability before commencing the study. This protocol with dates and activities should appear in the appendix.

Journal of Cultural Diversity • Vol. 23, No. 3 Fall 2016

Table 4. Dependability

Audit Trail Components of the audit trail include:

1. Make the list of documents planned for audit during the research work.

2. Commission the auditor based on plan for study.

3. Decide audit trail review dates and times.

4. See auditor information below

5. Write up audit trail results in the journal.

Journaling Actions for this activity include:

I . Planning journal work in advance is an option. Such that the researcher could decide what dates and how often the journal will occur.

I I . Journaling after interview is common.

12. Journaling after peer-review sessions.

13. Journaling after a major event during the study.

14. Journal entries should be discussed with peer reviewer such that expression of thoughts and ideas gleaned during research activities can be connected to participants’ experiences.

15. Journals can be maintained in various formats. Information for the journal can be received in the form of emails, documents, recordings, note cards/note pads. We recommend that the researcher decide on one of the options.

16. Journaling includes dates of actions related to significant and insignificant activities of the research.

17. Journal may start on the first date a decision is made to conduct the research.

18. Journaling ends when the research is completed and all participants have been interviewed.

Auditor 1. The auditor is reviewing the documents for authenticity and consistency.

2. The auditor may be a colleague or someone unfamiliar with the research such that activities can be questioned for clarity.

3. The auditor should have some comprehension of the research process.

4. Planning in advance for the time commitment as an auditor is crucial.

5. Should provide constructive feedback on processes in an honest fashion.

6. Auditor, researcher, and participants should speak the same language.

7. Must be able to create and maintain audit trail documents.

Protocol Create a timeline with planned dates for each activity related dependability before commencing the study. This protocol with dates and activities should appear in the appendix.

Journal of Cultural Diversity • Vol. 23, No. 3 Fall 2016

Table 5. Confirmability

Triangulation 1. Determine triangulation methods

2. Document triangulation plans within journal.

3. Discuss triangulation results peer-reviewer

4. Decide if further triangulation is needed

5. Write up the triangulation results.

Journaling Actions for this activity include:

2. Planning journal work in advance is an option. Such that the researcher could decide what dates and how often the journal will occur.

19. Journaling after interview is common.

20. Journaling after peer-review sessions.

21. Journaling after a major event during the study.

22. Journal entries should be discussed with peer reviewer such that expression of thoughts and ideas gleaned during research activities can be connected to participants experiences.

23. Journals can be maintained in various formats. Information for the journal can be received in the form of emails, documents, recordings, note cards/note pads. We recommend that the researcher decide on one of the options.

24. Journaling includes dates of actions related to significant and insignificant activities of the research.

25. Journal may start on the first date a decision is made to conduct the research.

Journaling ends when the research is completed and all participants have been interviewed.

Protocol Create a timeline with planned dates for each activity related confirmability before commencing the study. This protocol with dates and activities should appear in the appendix.

• Vol. 23, No. 3 ( E 9Journal of Cultural Diversity Fall 2016

Copyright of Journal of Cultural Diversity is the property of Tucker Publications, Inc. and itscontent may not be copied or emailed to multiple sites or posted to a listserv without thecopyright holder's express written permission. However, users may print, download, or emailarticles for individual use.