Reliability and Validity Statement

Test-Retest Reliability
Test Re-Retest Over Long Periods Of Time
Validity
Validity as an Educational Instrument

 

NOTE: The following text is from Dr. Porter’s original study. The results of a subsequent validation study (N=564) in 1988-89 confirm the validity of the Strength Deployment Inventory.® Details on the study are provided in the Relationship Awareness Theory Manual of Administration, Ninth Edition.

SOME GENERAL CONSIDERATIONS

Having neither evidence nor belief to the contrary, in constructing the Strength Deployment Inventory (SDI®) it was assumed that when things are going well for people, one third of them will score highest on the Altruistic-Nurturing (A-N) scale, one third on the Assertive-Directing (A-D) scale and one third on the Analytic-Autonomizing (A-A) scale. The items on each scale were written, tested and rewritten until successive samples yielded approximately equal distributions of populations among the three scales. This manipulation brought the means for each scale to approximately 331/3, the center of the Interaction Triangle, under the conditions of “when things are going well.” The standard deviations for each scale were also quite similar (A-N SD=12.33, A-D SD=15.03 and the A-A SD=11.88) and judged to be close enough to establish rules-of-thumb for interpretive purposes.

As it became clear over time that the Motivational Value System of persons scoring relatively equally on all three scales differed from the orientation (if not the behaviors) of persons scoring higher up on one of the A-N, A-D or A-A scales, it became necessary to establish a “boundary,” as it were, to define the “Hub” area. We set the boundary more or less empirically at 11 points above and below the mean (331/3) on each side (approximately 1 standard deviation above and below the mean). We have since learned that this may have been too loose since some studies have indicated statistically discriminable differences between INNER-HUBS (1/2 – standard deviation above and below) and BLUE-HUBS, RED-HUBS and GREEN-HUBS (from 1/2 SD out to 1 full SD from the mean.) At present, however, we see no reason to press for such greater precision in what can at best be an arbitrary boundary setting exercise.

No assumptions were made as to where the means of the scores ought to be under conditions of conflict and opposition, since the handling of conflict is so culturally determined. As one might very well expect, there is a big drop on the Altruistic-Nurturing scale (9 points) and increases on the other two scales (A-D up 6 points and A-A up 3 points). To the social psychologist interested in identifying cultural differences, these results may have meaning, but to the trainer or to the educator, they are of little or no assistance in helping an individual gain insight into the way they act as an individual.

TEST-RETEST RELIABILITY

One hundred subjects were retested within a two-week period. The Pearson correlation coefficients were calculated for test re-test scores for each scale: A-N, r=.78; A-D, r=.78; and A-A, r=.76.

TEST RE-RETEST OVER LONG PERIODS OF TIME

Over the past two decades enough experience has been gained with people who have retaken the SDI after a span of 2 to 20 years to raise the question, “Do scores change over time?” There can be no doubt about one thing: it is easy to change one’s scores, if one wants to play games. Work with the SDI has shown that when taken seriously without intent to manipulate scores, a high degree of consistency is found no matter how long the intervening length of time between test and retest.

The amount of fluctuation in scores over time is usually less than 6 points, which is not statistically significant. When scores from initial and later testings are plotted on the same triangle, more often than not it is clear that both vectors would be interpreted in essentially the same manner.

There are times, however, when changes in scores are real. They are a result of improper instructions given by the trainer. The SDI scales for “when all is going well” (items 1-10) are intended to get at how one acts that makes one feel worthwhile about one’s self. Some trainers, not understanding that the pattern of the answers is used to infer underlying motivations, have asked participants to limit their answers to how they behave “on the job.” This emphasis on “the behavior related to the job” rather than “the behavior related to when things are going well for me” causes people who are unhappy on their jobs to describe what is probably a Mask Relating Style. Later, under proper instructions, retaking the SDI gives a different set of scores more in line with the person’s Motivational Value System.

Another reason for a change in scores involves a Mask Relating Style. If a person first takes the SDI when they are wearing a “mask,” the scores will not reflect their true Motivational Value System. On retaking the Inventory, if the person has dropped the mask, the true Motivational Value System will be demonstrated. The converse is also possible. Someone, due to an environmental demand begins wearing a mask, and may, upon retesting, answer the items in the SDI based on their Mask Relating Style.

Experience has shown that one whose score has changed is likely to report that they have gone through a major life change, or experienced a significant trauma that has caused them to re-evaluate their life. In any event, exploring with a trainee what has been going on in their life and what was going on in their life when they first took the SDI may shed light on the shift.

VALIDITY

In considering the matter of validity, there is one very important matter to take into account. The Strength Deployment Inventory was not designed to be a test even though it is in the traditional format of a test. These inventories were designed to be educational instruments and must be judged by that standard. An inspection of the format shows immediately that no effort was made to avoid any halo effect. There is no doubt in our minds that anyone with any insight could manipulate his or her answers to achieve any profile of scores he or she wanted. This does not mean, however, that the scores of a person who answers the items honestly have no validity.

VALIDITY AS INTERNAL CONSISTENCY OF THE SCALES

Since many Inventory users like to discuss individual items with their clients, it will be of interest to them to know the level of confidence to be placed in each. Each item ending was analyzed to determine the extent to which it discriminated between high scorers on a scale and low scorers on a scale, using the Chi-square method (N=100). The level of confidence with which each item ending discriminated are listed in the table below.

VALIDITY AS CONGRUENCE WITH EXTERNAL REALITY

In Table 1, it is quite clear that the items in each scale have a high degree of internal consistency; that is, what each scale measures is being measured with high consistency.

The next questions are, “Does the Altruistic–Nurturing scale measure altruistic–nurturing behavior, does the Assertive–Directing scale measure assertive–directing behavior and does the Analytic–Autonomizing scale measure analytic–autonomizing behavior?”

To answer such questions, one must go to those spots in nature where the phenomena are supposed to occur and then determine whether or not the instrument yields scores congruent with the presence of the phenomenon. When one does just that, one sometimes finds that the phenomena don’t occur where one thought they would.

We went to the nursing profession where we expected to find a lot of Altruistic–Nurturing behavior. Except for some nurses who were in administrative positions, the Inventory scores were quite congruent; the great majority scored highest in the Altruistic–Nurturing scale.

We administered the Inventory to number of social workers. Again, the Altruistic–Nurturing scores were mostly congruent with a helping profession.

We administered the Inventory to a group of students majoring in business administration. Again, the scores were congruent, i.e., tending toward the Assertive–Directing scale.

We administered the Inventory to a group of engineers and found them scattered but, as a group, highest on the Analytic–Autonomizing Scale.

One of the surprises came when we gave the Inventory to a group of cadets in a police academy and to a group of police sergeants with 3 to 5 years of police experience. Most members of both groups were clearly in the Hub. We had, as most people do, predicted they would be highest in the Assertive–Directing scale. One group of women deputy sheriffs who were on the staff of a women’s jail, did have very high scores on the Assertive–Directing scale.

We administered the Inventory to a group of about 80 young men (17 to 21 in age) incarcerated in a state school for boys. The average of their scores charted in the Cautious–Supporting area when all is going well and near the Judicious–Competing area in the face of conflict and opposition. The director, a professional social worker, affirmed that this was a very accurate description of his charges. Shortly thereafter we gave the Inventory to a group of youths in the same age range chosen by the Catholic Diocese to be group leaders at a summer retreat. The average of their scores was clearly in the Altruistic–Nurturing region when all is going well and, in the face of conflict, moved into the Cautious–Supporting area almost to where the incarcerated youths’ scores fell.

We found another kind of validity as well: cases where people were in jobs which called upon them to behave in ways that were incongruent with their inventory scores. We have had reports from some people who subsequently changed to jobs more congruent with their Inventory scores and who claimed to be much more gratified in their work.

VALIDITY AS AN EDUCATIONAL INSTRUMENT

Since the Inventory was not designed to be a test, but to be an educational instrument, we can ask whether or not it is effective as an educational device. We have only experience to go on as to its effectiveness. In our experience of other users of the Inventory, it is quite common for participants to report:

  • a sense of exhilaration and personal gain
  • understanding themselves and others better
  • liking and respecting themselves and others more
  • feeling freer to be themselves and try new ways of relating to others
  • feeling less locked-in to behaving according to how “They” say one should behave
  • an increased ability to be open and honest with others, to give and receive feedback from others.