EPISCOPAL MEASURE OF FAITH TRADITION: A CONTEXT-SPECIFIC APPROACH TO MEASURING RELIGIOUSNESS

Abstract

Precise measurement of religiousness remains a vexing problem. In addition to relying almost exclusively on self-report, existing measures of religiousness pay little attention to the specific context of religious belief, and this may override distinctive norms of particular faith traditions and potentially confound the conclusions drawn from such research. To address these limitations, the authors describe a modified form of narrative content analysis that could eventually sort respondents into distinct theological traditions. A pilot test among Episcopalians demonstrates encouraging reliability (kappa 0.74, 95% LCI 0.47, p<0.0002), and tests for convergent and discriminate validity suggest that the context of religious belief is both relevant and insufficiently assessed by the existing paradigm of religious measurement. If validated in a religiously diverse sample, this approach could be combined with existing, context-free measures of religiousness to generate more meaningful findings.

Precise measurement of religiousness remains a vexing problem that currently limits empirical research (Peter C. Hill & Pargament, 2003). Over 100 measures of various aspects of religiousness or spirituality already exist, but these attempts to measure faith have met only limited success (P.C. Hill & Hood, 1999). In order to address some of these limitations, this paper describes a new approach to measuring religiousness and presents preliminary data that uses a modified version of narrative content analysis to sort respondents into distinct groups. This approach could be extended to sort respondents into distinct theological traditions.

BACKGROUND

One of the greatest strengths of the existing measures of religiousness is that the best measures are phrased in such a way that respondents can load their own particular context onto the scale. For example, both Jews and Christians can answer the question, “How often do you feel God’s presence in your life?” Because respondents can read their own religious tradition into the questions, several of the existing measures have been validated in different religious contexts such as American Christianity (Holland et al., 1998) or Israeli Jewry (Baider et al., 2001). Such studies can generate powerful, context-specific findings as long as the study population is religiously homogenous, and in fact, some of the best research is designed for samples of single faith traditions such as Judaism (Kark et al., 1996) or Presbyterian Christianity (Krause, Ellison, & Wulff, 1999; Krause, Tarakeshwar, Ellison, & Wulff, 2001).

In the commendable attempt to achieve broad applicability, most existing measures of religiousness are designed to measure “religiousness-in-general” (D. E. Hall, Koenig, & Meador, 2004). That is to say that they locate subjects on a spectrum of intensity ranging from “extremely religious” to “not religious at all.” A driving question has been to understand if and how “religious” people (independent of any specific religious context) share certain traits that distinguish them from “secular” people who have no religious or spiritual commitments. This theoretical framework assumes that what a subject believes is not nearly as important as the fact that the subject does believe; and that the specific theological context of religiousness is not as important as the fact that it is distinctly “religious.”

As noted by several critics (Kirkpatrick & Hood, 1990; Moberg, 2002), the problem with measuring religiousness-in-general is that it is difficult to interpret the results when such measures are applied to populations drawn from more than one faith tradition. What a Jew and a Christian mean by “feeling God’s presence” depends entirely on their religious context, and religiousness-in-general cannot distinguish particular religious contexts within religiously diverse samples. By lumping together widely different faith traditions, such “context-free” measures of religiousness-in-general override distinctive norms of particular faith traditions and potentially confound the conclusions drawn from such research (Moberg, 2002).

Despite this critical limitation, the existing context-free measures of religiousness-in-general might be fruitfully applied to religiously mixed populations if there were some reliable way to sort respondents into appropriate groups that share theologically similar perspectives. One approach might be to sort respondents according to religious affiliation using widely available denominational taxonomies(Fetzer-Institute/NIA-WorkingGroup, 1999). This would allow researchers to analyze the data from context-free measures of religiousness according to denominational subgroups, and in fact, several studies suggest interesting differences between Jews, Protestants and Catholics (A.B. Cohen, 2002; A.B. Cohen, Hall, & Koenig, 2009; A.B. Cohen, Hall, Koenig, & Meador, 2005; Adam B. Cohen, Keltner, & Rozin, 2005; A.B. Cohen, Rozin, & Siegel, 2003). However, although religious affiliation is sometimes assessed according to standard denominational taxonomies, the primary endpoints and statistical interactions are rarely compared between denominational groups.

To complicate matters further, even if findings were analyzed according to denominational self-report, it is not clear that the findings would be meaningful because at least within the American context, denominational taxonomies no longer describe the theologically relevant differences that distinguish different streams of the predominantly Christian faith traditions (D. E. Hall et al., 2004; Wuthnow, 1993). For example, liberalism is a worldview that seems to cut across religious traditions such that Liberal Jews, Christians and Muslims may share more in common with each other than with conservative members of their own faith tradition (Jensen, 1998). Likewise, an Evangelical Presbyterian is likely to share more in common with an Evangelical Methodist than she is likely to share with the Charismatic or Liberal Presbyterian who sits next to her in the same pew. It will not be easy to define the relevant theological categories of faith context, but it is not unreasonable to expect that investigators might reach some general consensus about broad categories of faith tradition; and once described, it may be possible to devise strategies for sorting people into these theologically relevant categories.

Finally, given that people are not always self-consciously aware of the assumptions that make up their worldview (religious or otherwise), any strategy for sorting subjects into relevant faith categories will likely require some aspect of observational technique that might complement self-reported affiliation (T. W. Hall, 2002). Indeed, other critics have noted that most measures of religiousness rely exclusively on self report in the form of psychometric questionnaires (P.C. Hill & Hood, 1999). As a result they have no capacity to distinguish those respondents whose self-identified religious affiliation would be acknowledged by other members of that faith tradition and those respondents whose self-identification does not correspond to the norms usually recognized by and within that tradition.

To address these specific limitations we describe an observational and context-specific approach for measuring faith tradition that could eventually be used to categorize respondents into primary faith traditions or worldviews such as Christianity, Judaism, Islam, or a non-religious worldview such as Secular Humanism. The sorting of respondents is achieved through an adapted form of narrative content analysis that makes quantitative assessments of faith tradition. The pilot study presented here was designed to test proof of concept for this novel approach within a sample of Episcopalians.

AN OBSERVATIONAL, CONTEXT-SPECIFIC MEASURE OF FAITH TRADITION

As argued elsewhere (D. E. Hall et al., 2004), religion can be conceived as a cultural-linguistic phenomenon where learning a specific faith tradition is analogous to mastering a language. Fluent use of language depends on grammar and syntax to hold vocabulary together in meaningful ways. Similarly it is not sufficient to study only the “vocabulary” of religious traditions (e.g. the content of belief or the varieties of religious experience) because such vocabulary remains incoherent if divorced from the grammatical context that holds the elements of faithful belief and practice in meaningful relationships. The process of appropriating the cultural-linguistic context of a religious tradition is commonly called “formation” by religious leaders. Formation may include formal catechesis, but more frequently, it references the many passive and often subconscious ways that individuals are formed into a religious cultural-linguistic context through the daily practices of faith communities.

We suggest that formation within a religious tradition can be measured as a form of cultural-linguistic fluency. The formative influences on a person’s worldview (religious or secular) are frequently manifest through subtle clues in the ways they use language to discuss even mundane issues. It is precisely these clues that inform pastors, rabbis and secular counselors about the formative influences on a person’s life. These clues are available in almost any subject matter, be it a conversation about a grandmother with cancer or a daughter who is soon to be married. Therefore, experienced listeners well-formed and “fluent” within a specific faith tradition should be able to recognize other people who are “speaking their language”. The authors hypothesize that such expert listeners will be able to reliably sort narrative responses into “fluent” and “non-fluent” categories corresponding to how each narrative response is reflective of someone formed by their religious tradition. Such a method could be used to sort a diverse sample of subjects into one of several faith traditions.

Before attempting to validate this method in a diverse sample of faith traditions, the general approach was pilot tested within a single faith tradition. The specific aim of this study was to assess the reliability and validity of an adapted form of narrative content analysis designed to sort respondents into categorical faith traditions. We hypothesized that within any main line church, it would be possible to identify at least two distinct populations: 1) those members who were both identified with the denomination and deeply formed by its distinct theological context; and 2) those members who, despite identification with the denomination, were not recognizably formed by the distinct theological context of that denomination. We further hypothesized that “expert listeners” deeply formed by that denomination (clergy and lay leaders) could use the concept of “cultural-linguistic fluency” to reliably sort subjects into one of these two groups by analyzing samples of their speech. Finally, we hypothesized that these two groups of subjects would respond to existing context-free measures of religiousness in distinctly different ways.

METHODS

Using the professional contacts of the first author, we identified a convenience sample of three Episcopal priests across the country who agreed to serve as expert listeners (Los Angeles, Wichita, rural Maryland). After discussing with each priest the concept of cultural-linguistic fluency, they identified specific parishioners who agreed to be interviewed. Half of these parishioners were people they considered to be paragons of solid Christian formation. The other half were people they considered to be identified with the church but not deeply formed by it. For each subject, the priest who knew them personally specified a “formation status” as “well-formed” or “not-so-well-formed” by the Christian tradition. This formation status was recorded and set aside.

The first author contacted each subject by telephone and, after obtaining informed consent, asked a series of nine open ended questions (See Table 1). The spontaneous verbal responses were recorded with little or no prompting after each question. The recordings were transcribed and stripped of identifying information, and the de-identified transcripts were then distributed to the three priests. Each “expert” priest analyzed all twenty-three transcripts, thereby ensuring that a maximum of eight transcripts were from subjects potentially recognizable by the priests. The priests were asked to read each transcript, and rather than extracting qualitative themes, they were asked to make a single, quantitative judgment, yes or no, indicating whether or not the transcript reflected someone deeply formed by the Christian tradition.

Table 1:

Question Tree for the Episcopal Measure of Faith Tradition

Directions: We will ask you several questions about the way you perceive the world. There are no right or wrong answers. We want to learn how you understand the world and your place in it. Please respond freely in your own words as best you can. Any one of these questions could be the subject of a book. We are not looking so much for complete answers as just your initial approach to making a response. We expect the entire interview will take about 10–15 minutes.

In what ways, if any, are you a religious or spiritual person? Who or what is God for you? Please describe your practice of prayer, if any. In light of what you have just said about your spiritual life, how do you understand illness? In light of what you have just said about your spiritual life, how do you understand death? How is the Holy Spirit revealed in your community? Describe what the Eucharist (Holy Communion) means to you. Describe how you make major decisions. What authorities do you trust? If you can, please describe a decision in the last year that was influenced one way or the other by your faith commitments.

After all transcripts were evaluated by each priest, the initial assessment of “formation status” for each subject was compared to the assessment of the corresponding subject’s transcript. Kappa statistics were calculated for the assessments of each individual priest as well as a composite assessment of “formation status” that resolved disagreements between the priests by simple majority (i.e if two of three priests considered the transcript “well-formed” the composite assessment considered the transcript “well-formed”). Interclass Correlation Coefficients (ICC) were also calculated for agreement between ratings.

To assess for the potential bias attributable to priests rating the (de-identified) transcripts of their own parishioners, Kappa was calculated using the ratings of the two priests unknown to the subject. This strategy required censoring three subjects (one from each location) for whom the two priests disagreed regarding their formation status. To further test the sorting method, we recruited six additional “expert listeners” unknown to the subjects of this study to sort the transcripts (three Episcopal priests and three well-formed Episcopal lay leaders). Kappa was calculated for each individual rater as well as for the composite scores.

In addition to the recorded telephone conversation, each subject completed a sixty item pen and paper questionnaire with items drawn from three established context-free measures of religiousness. The Brief Multidimensional Measure of Religiousness/Spirituality (BMDMRS) is one of the most promising multidimensional measures of religiousness (Fetzer-Institute/NIA-WorkingGroup, 1999). It assesses several dimensions of religiousness (see Table 2) with both brevity and power, and it was chosen because it is the only measure of religiousness validated in the General Social Survey (Idler et al., 2003). The Duke Religion Index (DUREL) is a brief and widely used measure that includes three items that assess Intrinsic Religiousness (Koenig, Parkerson, & Meador, 1997). Although not measured by the BMDMRS, Intrinsic Religiousness (IR) is one of the most prevalent and highly developed constructs for measuring religiousness, and for this reason, we include the IR items from the DUREL. Finally, the Spiritual History Scale (Hays, Meador, Branch, & George, 2001) was developed to assess exposure to religion over time, and as such it tests a distinct and theoretically important aspect of religiousness not assessed by either the DUREL or the BMDMRS. Because formation in a faith tradition likely correlates with education, the questionnaire assessed education as a covariate along with basic demographic information including age (measured continuously in years), sex (male/female), and race (Caucasian, Black, Asian, South Asian, Arab, Hispanic or Other).

Table 2:

Summary Statistics and Correlation

Question (Range of Responses)	Mean (SD)	Well-Formed	P value
Demographics
Age (24–73)	45.5 (14.5)	0.453*	.0299
Sex, (female=1)	0.52	0.045	.8368
Ethnicity (Caucasian=1, Black=2)	1.04 (0.2)	0.223	.3071
Education (Grade School =1; High School= 2; College=3; Masters=4; Doctoral =5; Postdoctoral=6)	3.7 (1.1)	0.212	.3307
Private Practices (1= More than once a day; 8=Never); (Range 3–24)	11.9(4.6)	0.711 ***	.0001
How often do you pray privately in places other than at church or synagogue?	2.3 (1.4)	0.740***	<.0001
Within your religious or spiritual tradition, how often do you meditate	5.2 (2.4)	0.544**	.0073
How often have you read the Bible or other religious literature in the last year?	4.3 (1.8)	0.484*	.0192
Coping (1=A great deal; 4=Not at all)
Positive Coping (Range 3–12)	5.5 (2.1)	0.572 **	.0043
I think about how my life is part of a larger spiritual force.	1.7 (0.8)	0.298	.1669
I work together with God as partners.	2.2 (0.9)	0.623**	.0015
I look to God for strength, support and guidance.	1.6 (0.7)	0.523*	.0104
Negative Coping (Range 2–8)	7.7 (0.6)	0.188	.3893
I feel God is punishing me for my sins or lack of spirituality.	3.9 (0.3)	0.371	.0815
I wonder whether God has abandoned me.	3.8 (0.4)	−0.020	.9279
Coping NOS
I try to make sense of the situation and decide what to do without relying on God.	3.2 (0.9)	−0.169	.4418
To what extent is your religion involved in understanding or dealing with stressful situations in any way?	1.6 (0.7)	0.438*	.0368
Support (1=A great deal; 4=None)
Congregation Benefits (Range 2–8)	3.1 (1.6)	0.502 *	.0147
If you were ill, how much would the people in your congregation help you out?	1.6 (0.9)	0.466*	.0252
If you had a problem or were faced with a difficult situation, how much comfort would the people in your congregation be willing to give you?	1.5 (0.8)	0.381	.0731
Congregation Problems (Range 2–8)	6.7 (1.1)	−0.419 *	.0467
How often do people in your congregation make too many demands on you?	3.1 (0.6)	−0.355	.0968
How often are the people in your congregation critical of you and the things you do?	3.6 (0.6)	−0.456*	.0287
Intrinsic Religiousness (1=Definitely true of me; 5=Definitely not true); (Range 3–14)	5.1 (1.9)	0.549 **	.0067
I try hard to carry my religious beliefs over into all my other dealings in life. (1–4)	1.7 (0.6)	0.536**	.0084
My religious beliefs are what really lie behind my whole approach to life.	1.8 (0.8)	0.652***	.0008
In my life, I experience the presence of the Divine (i.e., God).	1.6 (0.9)	0.180	.4106
Public Practice (1= More than once a week; 8=Never); (Range 2–16)	4.7 (1.9)	0.571 **	.0044
How often do you attend to religious services?	2.1 (0.6)	0.296	.1709
How often do you take part in the activities or organizations of a church or place of worship other than attending services?	2.7 (1.4)	0.620**	.0016
Self Assessment/Intensity (1=Very religious; 4=Not religious at all); (Range 2–8)	3.3 (1.0)	0.387	.0654
To what extent do you consider yourself a religious person?	1.8 (0.6)	0.501*	.0149
To what extent do you consider yourself a spiritual person?	1.5 (0.6)	0.164	.4554
Beliefs & Values (1=Strongly agree; 4=Strongly Disagree); (Range 4–16)	6.4 (1.6)	0.501 *	.0148
I feel a deep sense of responsibility for reducing pain and suffering in the world.	1.8 (0.6)	0.539**	.0080
I believe there is life after death?	1.5 (0.5)	0.267	.2303
I believe God watches over me.	1.4 (0.6)	0.409	.0526
In my life, I experience the presence of the Divine (i.e., God).	1.6 (0.9)	0.180	.4106
Forgiveness (1=Always or almost always; 4=Never); (Range 3–12)	4.7 (1.6)	0.176	.4211
I have forgiven myself for things that I have done wrong.	1.7 (0.6)	0.219	.3159
I have forgiven those who hurt me.	1.7 (0.6)	0.308	.1524
I know that God forgives me.	1.3 (0.6)	−0.137	.5330
Daily Spiritual Experiences (1=Many times a day; 6=Never or almost never): (Range 6–36)	16.3 (5.0)	0.343	.1095
I feel God’s presence.	3.1 (1.2)	0.272	.2101
I find strength and comfort in my religion.	2.7 (0.9)	0.390	.0661
I feel deep inner peace or harmony.	3.2 (1.2)	0.271	.2103
I desire to be closer to or in union with God.	2.3 (0.9)	0.549**	.0067
I feel God’s love for me, directly or through others.	2.7 (1.1)	0.469*	.0238
I am spiritually touched by beauty of creation.	2.3 (1.0)	0.144	.5112
Spiritual History in Four Dimensions (SHS-4)—Subscales
God Helped (1=Strongly Agree; 5=Strongly Disagree);(Range 9–45);	13.8 (3.9)	0.226	.3003
God has helped me this far through my life.	1.3 (0.7)
Some of the good tings that have happened to me were blessings from God.	1.6 (0.9)
Overall, god has answered my prayers.	1.9 (0.8)
I have trusted God to take care of me through the years.	1.7 (0.8)
Through the years I have prayed for my health or for the health of others.	1.2 (0.4)
Overall, my religious life has taught me to have a positive attitude.	1.4 (0.5)
Overall, my religious life has helped me to reduce stress	1.7 (0.6)
Overall, my religious life has helped me to persevere (go on when life gets hard).	1.3 (0.5)
I have lived this long because God’s time for me to die has not yet come.	2.2 (1.0)
Lifetime Religious Social Support (1=Strongly Agree; 5=Strongly Disagree);(Range 3–15)	9.5 (3.5)	0.420 *	.0516
For most of my life, my social life has revolved around the church.	3.3 (1.5)
For most of my life, I have known many of the people in my church.	2.7 (1.4)
An important part of my religious life has been inviting and taking people to my church.	3.4 (1.2)
Family History of Religiousness (1=Strongly Agree; 5=Strongly Disagree); (Range 6–30)	16.2 (5.5)	−0.026	.9049
When I was a child, I was very involved in the church.	2.3 (1.5)
When I was a child, the church was like a family to me.	2.7 (1.5)
When I was a child, religion was a natural part of my life.	2.2 (1.4)
When I was a child, my parents left my religion up to me.	4.0 (1.4)
When I was a young child, I had a religious or spiritual role model.	2.6 (1.5)
My family passed down their religion to me.	2.4 (1.4)
Cost of Religiousness (1=Strongly Agree; 5=Strongly Disagree); (Range 3–15)	10.2 (2.8)	0.361	.0902
At times, my religious life has caused conflict between myself and other people.	2.7 (1.3)
At times, my religious life has caused me stress.	2.9 (1.3)
I have suffered physically because of my religion.	4.6 (0.9)
Conversion and Contribution
Did you ever have a religious or spiritual experience that changed your life? (Yes=1)	0.74	0.224	.3040
If yes, how old were you? (Continuous in years, N=15)	31.3 (13.3)	0.373	.1711
During the last year, how much money did you and the other family members in your household contribute to your local congregation. (Continuous in thousands of dollars per year, N=22)	4.1K (5.4K)	0.569**	.0057
What is your average annual household income? (1=Less than $20,000; 4=More than $100,000, N=22)	3.0 (1.0)	0.031	.8928

^*p<.05,

^**p<.01

^***p<.001

Note: The first column details the actual question as well as the range of possible answers, most of which are Likert scales. The second column presents the mean score and standard deviation for the entire sample. The third and fourth columns contain the correlation between formation status and the specified question along with the associated p value. Bolded results show the composite scores for each of the factor structures validated for the BMDMRS in the GSS (Idler, et al, 2003).

Convergent and discriminate validity were examined by correlating these context-free measures with the composite assessment of “formation status”. Spearman correlations were calculated for non-parametric data including the Likert scale questions. Pearson correlations were calculated for continuous data like age or income. This research was approved by the Duke University Medical Center IRB, and all statistics were calculated using the Enterprise Guide overlay of the SAS 8.2 software package.

RESULTS

Twenty three subjects were identified from three Episcopal parishes across the country (8 Los Angeles, 8 Wichita, 7 Rural Maryland). All subjects completed both the written and verbal questionnaires. The average age was 45.5 (SD 14.5) with a range from 24 to 73 years. Fifty-two percent were women, and all but one subject were Caucasian. Most had completed college (91%) and more than half had completed a graduate degree (52%). The means and standard deviations for each of the written questions are listed in Table 2.

A composite assessment of formation status was calculated by simple majority (i.e. the three priests (i.e if two of three priests considered the transcript “well-formed” the transcript was analyzed as “well-formed”). Kappa for this composite assessment of formation status was 0.74 with a 95% lower confidence interval (LCI) of 0.47 (p<0.0002). This corresponds to sorting 20 of 23 subjects into the proper classification. Only 3 subjects were misclassified, and there was no statistical bias in the type of error in misclassification. Kappa statistics for each priest were 0.91 (95% LCI 0 .75, p<0.0001), 0.65 (95% LCI 0.34, p<0.001), 0.47(95% LCI 0.12, p<0.02). This corresponds with accurately sorting 22, 19, and 17 of the 23 subjects respectively. The ICC for any single rating by any single rater was 0.54, and very little of the error was attributable to bias between the raters. However, the ICC of the composite score was 0.75 which, as expected, corresponds with the Kappa for the composite score.

In order to test for potential bias due to priests rating transcripts from their own parishioners, Kappa was recalculated using only the ratings of the two priests unknown to the respondent. This required censoring the data from three subjects (one from each site) about whom the two priests disagreed regarding their formation status. Using the remaining 20 transcripts, Kappa increased to 0.80 (95% LCI 0 .53, p=0.001) corresponding to sorting accurately 18 of 20 transcripts. We also recruited six additional “expert listeners” who were strangers to the subjects of this study. Individual Kappa statistics for these additional raters ranged from 0.21 to 0.57. Three of these additional raters were Episcopal priests from North Carolina, Oregon and New York, and the composite of their assessments of formation status yielded a kappa of 0.65 (95% LCI 0.34, p=.002) corresponding to sorting correctly 19 of 23 subjects. The other three supplemental raters were all lay persons and their composite assessment of formation status yielded a kappa of 0.56 (95% LCI 0.22, p=.01) corresponding to sorting correctly 18 of 23 subjects.

There were many statistically significant correlations between the composite assessment of “formation status” and the previously validated context-free measures of religiousness (See Table 2). There were also many trends that would likely reach statistical significance in a larger sample. As expected from previous data suggesting that older people are more religious (Idler et al., 2003; Princeton Religion Research Center, 1996), formation was significantly correlated with increasing age (0.45, p<0.05). However, contrary to the widely published data that suggest that women are more religious than men (Idler et al., 2003; Princeton Religion Research Center, 1996), there was no correlation between sex and formation status. There was very little variance within both the ethnicity and education of this sample, and consequently, there was no significant correlation between these variables and formation status.

Private religious practices were strongly correlated with formation status (0.71, p<.001). The more a subject prayed (0.74, p<0.001), meditated (0.54, p<0.01), or studied of scripture (0.48, p<0.05), the more likely that subject was to be considered a well-formed Episcopal Christian. Well-formed subjects were also more likely to expect help from their religious community (0.50, p<0.05), and less likely to find that religious community demanding or critical (−0.42, p<0.05). Well formed subjects gave more money to the church, regardless of income (0.57, p<.01), and they perceived a deep “responsibility to reduce pain and suffering” (0.54, p<.01).

The concept of intrinsic and extrinsic religiousness was developed by an Episcopal psychologist (Allport, 1950), and it is therefore not surprising to find that the three questions tapping intrinsic religiousness show a strong correlation with well-formed Episcopalians (0.55, p<0.01). Religious coping also followed predictable patterns. There was no correlation with negative coping strategies, but there was strong correlation with more “positive” coping behaviors (0.57, p<.01). The well-formed subjects were also more likely to involve their religion when coping with stress (0.44, p<0.05).

Although well-formed subjects were more likely to take part in organized religious activities other than worship (0.62, p<0.01), there was surprisingly no correlation between formation status and attendance at worship services. There is also an interesting distinction between questions framed in language of generic spirituality and questions framed in the more traditional language of religion or God. Whereas there was a strong correlation between formation status and a self assessment as a “religious person” (0.50, p<.05), there was no correlation between formation status and a self assessment as a “spiritual person” (0.16, p=.46). Questions designed to tap the distinct concept of generic spirituality did not significantly correlate with formation status even though the correlation coefficients were all in the expected direction (i.e. increasing “spirituality” was positively, but insignificantly correlated with formation). For example, there was no significant correlation between formation status and questions regarding “life as a spiritual force”, “experiencing the divine”, being “touched by the beauty of creation”, or sensing “deep inner peace/harmony”. Two of the questions from the Daily Spiritual Experiences scale do, in fact, correlate strongly and significantly with formation status (0.55, p<.01; 0.47, p<.05). However, both questions directly reference “God” instead of the “Divine” or the more abstract concepts of beauty and harmony. Furthermore, these two questions tap clearly orthodox Christian yearnings for union with and love of God.

DISCUSSION:

These preliminary data tentatively support the reliability and validity of this new approach to sorting subjects into categorical faith traditions using a modified form of narrative content analysis. The technique described in this study demonstrated reliability statistics (kappa=0.74) within the accepted range for similar psychometric instruments (Cramer, 1998; Landis & Koch, 1977), and this supports the hypothesis that cultural-linguistic fluency can be used to recognize and categorize different formative worldviews, both sacred and secular. Furthermore, the specific pattern of correlations between formation status and several previously validated, context-free measures of religiousness supports the hypothesis that there are recognizable and potentially relevant differences between subjects deeply formed by a religious tradition and other subjects who identify with the same tradition without being well-formed by it.

One of the most interesting findings exposed a stark difference between the generic language of spirituality and the language of religiousness. Although there were many strong correlations between formation status and other measures of religiousness, there were no correlations with any of the previously validated items that explicitly tapped the concept of spirituality—though the mean scores on these items suggest that both “formed” and “unformed” Episcopalians consider themselves to be spiritual. Many people claim to be “spiritual” but not religious (Fetzer-Institute/NIA-WorkingGroup, 1999), and this data suggests a similar pattern where the “unformed” Episcopalians are “spiritual” but not religious, and the “well formed” Episcopalians are both “spiritual” and “religious”. This data lends limited support to the popular theoretical view that the generic language of spirituality taps a broader and more universal concept. However, it appears that the broad concept of spirituality is not enough to discriminate between “well-formed” and “unformed” Episcopalians. Therefore, these data suggest that the more generic language of spirituality used by several newer measures of religion/spirituality (Hatch, Burg, Naberhaus, & Hellmich, 1998; Peterman, Fitchett, Brady, Hernandez, & Cella, 2002; Underwood & Teresi, 2002) cannot entirely supplant the distinctly religious language necessary to distinguish between some faith traditions.

Forgiveness is another dimension of religiousness that is thought to cut across theological and spiritual traditions (McCullough, Pargament, & Thoresen, 2000), and given the importance of forgiveness in Christian theology, it is surprising that none of these items correlate with formation status. However, the concept of forgiveness tapped by these questions is not particularly “religious”. Although forgiveness may trace its roots to a Christian worldview, the concepts of forgiveness described and recommended by secular mental health professionals may be sufficiently decontextualized from their religious origins that they no longer correlate exclusively with that faith tradition. This might support the opinion that forgiveness is more “universal” than religiousness, but at the same time, it undermines the opinion that forgiveness is a uniquely religious or spiritual variable. The influential Christian theologian, Paul Tillich (Tillich, 1951–63) attempted to recast theological language in culturally current terms (e.g “God” becomes “Ground of Being”). Despite Tillich’s obvious brilliance and deep faith, critics suggest that Tillich’s theology becomes no longer recognizably Christian as the culture in which it is defined embraces secular pluralism (Frei, 1975; Lindbeck, 1984; Taylor, 1987). Likewise, attempts to reduce religiousness to functional or dispositional notions like forgiveness, well-being or coping may end up losing any recognizable religiousness.

Alternatively, the absence of a correlation between formation status and forgiveness might suggest that the wider American culture retains some rudimentary “Christian” legacy regarding the virtue of forgiveness. Indeed, the widely shared concepts of forgiveness, life after death, vague spirituality and a generically benevolent God may constitute the content what some call “cultural Christianity”. In fact, the so-called “less-formed” Episcopalians from this sample might actually represent subjects deeply formed by such “cultural Christianity”. This interpretation is only suggestive, but these data demonstrate that there are empirical differences between two distinct populations within this sample of Episcopalians, and it remains an empirical challenge to determine if and how they may relate to other theological traditions.

Strangely enough, these data show no correlation between religious attendance and formation status. This finding may be an anomaly of the sampling frame because the research design deliberately recruited subjects who attended church, but asked that some of them be well-formed, and others not so well-formed. That religious attendance did not correlate with formation suggests that the priests did a good job selecting people who differed in formation status, but not in attendance. Further supporting this interpretation is the observation that the well-formed subjects were more likely to take part in organized religious activities other than worship (0.62, p<0.01).

It is interesting and ironic to note that formation status was strongly correlated with both a sense of “responsibility to reduce pain and suffering” (0.54, p<.01) and the amount of money given to the church by the subject, regardless of income (0.57, p<.01). It may be that these factors are important criteria by which the priests assessed formation-status to establish the original sampling frame. The optimist might be encouraged to see the correlation between formation and a responsibility to reduce suffering. The cynic might suggest that, at root, the priests assessed formation status according to how much money a subject gives to the church. Even so, the strong correlation with monetary contribution is not surprising given that in our capitalist society, people tend to spend money on the things they value most, and such financial commitment to a community should likely correlated with deep formation within that community’s practice and belief.

Finally, it is puzzling to note that only one of the four subscales of religious history correlated significantly with formation status (0.42, p=.05). We expected well-formed Episcopalians to demonstrate strong and consistent involvement with their religion over time. However, the data show no strong correlations, and the correlation with family and childhood religious history is particularly weak (−0.026, p=.9). These findings may reflect the general trend in the Episcopal Church that many adult members were not raised Episcopalian, but rather came to the church from some other tradition, usually in reaction against that tradition (Roman Catholicism, Evangelicalism). Therefore, the subjects in this sample may be distinctively independent from their childhood roots--a hypothesis directly opposite to the assumptions within the SHS-4 that postulate that childhood history is a continuum and precursor to the adult history. Regardless of the interpretation, there appears to be discontinuity in this sample between childhood and adult practice of religion.

Despite these encouraging findings, this study has several significant limitations. Foremost is the inherent subjectivity of our modified method of narrative content analysis. All qualitative research involves a necessary element of subjectivity, and the strength of qualitative research depends on the ways in which that subjectivity is systematized (Mays & Pope, 1995; Pope & Mays, 1995). Narrative content analysis usually focuses on extracting relevant themes through expert analysis and coding. This primarily descriptive technique is frequently extended to make quantitative assessments of narrative data. For example, after carefully defining the content of interest, it is possible to train naïve coders to identify that content reliably within narrative passages (King, 2003; King, Scollon, Ramsey, & Williams, 2000). By way of contrast, our modified method of content analysis deliberately refuses to specify the content of interest for three reasons.

First, as argued elsewhere (D. E. Hall et al., 2004), the central concept of cultural-linguistic fluency suggests that formation within particular faith tradition is not defined so much by its “content” as by the cultural-linguistic relationships between elements of that content. Cultural-linguistic fluency in a faith tradition cannot be judged by mere content for the same reason that fluency in French cannot be judged by simply tabulating the presence or absence of French words in sample of text.

Second, even though they may not be able to diagram the grammar and syntax of a sentence, native French speakers can recognize fluent French when they hear it. In a similar way, we contend that people well-formed by a faith tradition (such as priests) are “already trained” to assess fluency in that tradition. Furthermore, even though they can recognize cultural-linguistic fluency in their tradition, they may not be able to accurately abstract their method of recognition. Indeed, their method may not be constant, adapting to specific contexts. Yet we contend that such well-formed “expert listeners” will more accurately recognize cultural-linguistic fluency than any external researcher.

Finally, although two of the authors are themselves formed by Episcopal Christianity and therefore potentially capable of defining the content of interest for narrative content analysis, this coincidence will not always be the case. It would be presumptuous for a researcher to define the relevant concepts of fluency for a faith tradition in which that researcher is not deeply formed, and therefore, our method leaves determination of cultural-linguistic fluency in the hands of those best equipped to assess it.

Although Episcopal priests may be uniquely equipped to assess formation status of Episcopal parishioners, the priests chosen for this study were biased to the extent that they were drawn from a convenience sample of the first author’s colleagues. Additionally, the priests were potentially biased by the fact that a sizable minority of the transcripts evaluated by each priest came from their own parishioners, and were therefore potentially recognizable by the priests. We examined this potential bias in several ways. First, we calculated Kappa using only the ratings of the two priests unknown to the subject, and the statistic increased from 0.74 to 0.80 because we were forced to censor though three subjects for whom the two priests disagreed regarding formation status. We also asked each priest if they could identify any of the transcripts, but only three were accurately identified (the identified transcripts were not the same transcripts that were censored in the recalculated Kappa). We also recruited six additional “expert listeners” who were strangers to the subjects represented by the transcripts, and although their level of accuracy was not as high as the original three priests, the sorting strategy was, in fact, statistically significant in each case. The composite Kappa was better for the clergy (.65) than it was for the lay persons (.56), suggesting that clergy may, in fact, be better equipped to assess formation. However, in support of our theory of fluency the lay persons were also quite accurate in their composite assessment, accurately sorting 18 of 23 transcripts. In sum, these sensitivity tests suggest that the success of the sorting strategy was not, in fact, biased by the fact that the priests knew some of the subjects of this study. In future studies, this bias could be eliminated by using four or more priests to allow three ratings of each transcript without requiring any priest to rate transcripts from their own parishioners. Future studies could also implement more representative sampling of Episcopal clergy as “expert listeners”.

The variance of the validation sample was deliberately maximized by asking the priests to identify only subjects at opposite ends of a broad spectrum of formation, thereby artificially improving the power of this method to discriminate formation status. Further validation in more representatively diverse samples will be necessary before this approach could be widely applied. However, such optimal sampling with artificially enhanced variance is commonly employed in the initial stages of test development (Crocker & Algina, 1986).

Finally, the utility of this technique may be limited by its expense. Collecting and evaluating narrative interviews takes time and money. However, the authors contend that such extraordinary effort is necessary to move this field of inquiry beyond expedient convenience to a new level of depth. Feinstein notes the tendency in medical research to “avoid and divert,” preferring “an exact answer to a diverted question rather than an imprecise or approximate answer to the direct question” (Feinstein, 2002). This cultural-linguistic approach endeavors to get at the direct question of interest with as much precision as possible. Future work might be directed at ways to expedite the process and reduce the expense.

Given this initial success, the next step is to broaden the technique to include several different theological traditions. It will not be possible to develop a cultural-linguistic measure of all of the many different faith traditions, but it is not unreasonable to start with the largest and most common religious traditions to include Judaism, Islam, Secular Humanism, and several broad streams of Christianity. (Hindu and Buddhist measures could also be developed, but in the American context, these traditions describe such a small proportion of the population that they may not yet justify the effort required to develop specific scales). To that end, we are currently using consensus methodology to develop questionnaires suitable for exposing formation within each of several broad faith traditions, and we hope to validate these questionnaires in a method similar to that described in this pilot study.