Somebody Paid Money for This UPDATED!

A Devoted TOFling yclept Guy Stewart writes:

"While reading io9 (which I enjoy a lot), I came across this article: “Are Religious Beliefs Going To Screw Up First Contact?”

Based on: “...[clinical neuropsychologist Gabriel G. de la Torre]…sent a questionnaire to 116 American, Italian, and Spanish university students.”

Reached this conclusion: “…he found that many of the students — and by virtue the rest of society — lack awareness on many astronomical aspects. He also learned that the majority of people assess these subjects according to their religious beliefs.”

As a statistician, what do you make of the significance of the conclusion of this survey?

To which TOF responds: "Not Much."



Social "science" researchers or (if they are neuro-psychologists) "phrenologists," rely a great deal on questionnaires, which they refer to as "instruments" as if they were actual, you know, instruments. This questionnaire was used on a trifle of 116 students -- college students comprising the vast majority of "subjects" in social studies. Since the details are behind the usual paywall that protects Science!™ from the profane eyes of the people who pay for it, TOF does not know how the sample was selected or indeed even if it was selected (rather than composed of haphazard volunteers hoping for a good grade from their psych professor). Three countries are mentioned; but were these from universities in three countries or all from an Italian university where there were some foreign exchange students? If anyone wants to shell out $35 to find out, let TOF in on the gnosis. [NOTE: This has been done. See below.]

Consequently, we do not know what questions were asked, except that "Among other things, the students were asked if God created the universe, or if they believe contact with ETIs is likely." It is unclear what either answer to either question portends so far as our readiness for first contact, save that a "no" to the latter question would entrain a vast surprise at ET's advent, and a "no" to the former would entrain a vast surprise at the existence of a universe.

Nonetheless, we are assured that "many of the students — and by virtue the rest of society — lack awareness on many astronomical aspects."  It is unclear what an "aspect" is or what constitutes "awareness" of it; nor what quantity "many of the students" would be of the 116 participants? Nor in what manner can we legitimately infer from those college boys and girls to "the rest of society"? It is illegitimate to draw conclusions about a population if the sampling frame does not cover that population. (Recall how the Literary Digest poll of 1936 did not access voters lacking telephones.) Were these college students a random sample from "the rest of [Western?] society"? Based on the track record of the social "sciences" in discovering either the obvious or the wrong, TOF is not hopeful.

The article touting the article makes statements like:

• Despite the eerie Great Silence, there's good reason to suspect we'll eventually make contact with aliens.
• Clinical neuropsychologist Gabriel G. de la Torre [the author] now argues, we're not quite ready to make contact just yet. The problem, he says, is that most of us haven't come to grips with our place in the universe.
• He calls this the "cosmic consciousness" — a term that Canadian psychologist Richard Bucke defined as "a new evolutionary step beyond self-consciousness."
• [citing Bucke] by virtue of self-consciousness, man is not only conscious of trees, rocks, bodies of water, and his own limbs and body, but he also becomes conscious of himself as a distinct entity apart from all the rest of the universe.

But the "Great Silence" is "eerie" only if one assumes a particular answer. If there is no other intelligent communicative life elsewhere, the Great Silence is actually a Great Big Piece of Evidence. There is nothing eerie about hearing nothing from nobody. The other possibility is that the universe is so vast that no messages will have time to reach us before we have evolved into a species of community organizers, literary critics, and other advanced lifeforms with no interest in icky STEM subjects.

The amount of question begging combined with credulous woo-woo is astonishing. The second bullet makes sense only if de la Torre actually knows "our place in the universe." But that stuff about cosmic consciousness is a definite hoot because, the fourth bullet notwithstanding, it would be hard to locate a time when Western philosophers were not already aware of man "as a distinct entity apart from all the rest of the universe." In fact, it is the Late Modern who believes that man is merely a bag of chemicals responding to the random impacts of atoms and molecules, and even that there is no consciousness at all, let alone a cosmic one.

De la Torre announces that "On the basis of the results of our study, most transcendent factors for existential concerns continue to be related to religion instead of ecology or cosmology."

It is unclear what transcendent factors ecology or cosmology have at hand, since they both deal in measurable and mundane things like quasars and quaggas. What relevance exactly does ecology have for human readiness to encounter ET, given that ET is unlikely to fit into our ecology?

Further: "Preliminary results are discussed with regard to current neuroscience, factor analysis, and possible implications for the understanding of contact with extraterrestrial intelligence. The roles of education, new search strategies, and possible contact scenarios are also discussed."

"Possible contact scenarios" indicates how far Science!™ has drifted from its roots in actual evidence. We now write Serious Papers about alien contact where we used to write much more entertaining fictions. But how these "scenarios" relate to a survey of 116 college kids is unclear. They were not asked about them.

The argumentum ad neuroscience may or may not mean the students were wearing tin-foil hats while answering the questions so that their EEGs or MRIs or bumps on their skulls could be interpreted. The argumentum ad factor analysis is surely a reference to the conjuring of ghostly "factors" out of the messy, if sparse, data. (See "The Fourth Uncertainty: Conjuring Factors from the Vasty Deep" for details.)

As for the poignant question in the title: “Are Religious Beliefs Going To Screw Up First Contact?” we must simply have faith that the religious beliefs of the aliens will be robust enough to keep things unscrewed and they will not hold us in contempt for the lack of same among so many of us.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Addendum

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Thanks to the good offices of Ian Thompson, TOF now has access to the De La Torrey paper, and he is appalled to report that it is worse than he had feared. There seems to be no discernable relationship between the introduction and conclusions and the actual questionnaire and "experimental results." A few comments follow.

The prime characteristic of cosmic consciousness is, as its name implies, a consciousness of the cosmos

Whoa! Stop the presses!

Recently, quantitative similarities and calculated solutions for the intensities of magnetic fields associated with cerebral function and those that exist within intragalactic and extra-galactic Space suggest that the energetic conditions associated with consciousness and its many variants may be more universal than anticipated.

Ummm.... Magnetic fields in "intragalactic and extra-galactic Space" [which pretty much covers the possibilities] are similar to those associated with cerebral function. Right. So the universe is a giant brain, and we are inside it? Or what?

Space, time, denial, and asymmetries between technology and levels of consciousness may play a modular role in the event of contact with an EC.

All those modulars: space, time, denial,.... It's not clear what the difference is between playing a role and playing a modular role.

We do not understand cosmic consciousness as a special mystic-state achievement [that's a relief -- TOF] but as a conscious perception of reality beyond the major effect of the modular aspects (fear, religion, denial, etc.), which is related to attention–intention cognitive-behavioral patterns and mediated to some extent by learning and education.

See here. But... "fear, religion, denial, etc." the mind boggles at the potentialities of that "etc."  TOF supposes that if we get "beyond" all that modular fear, religion, and denial, we will become conscious of the cosmos.

Some individuals, small groups of people or scientists, may actually possess the required level of awareness to be involved in an event such as contact with an EC

Yes, of course. The Scientists in their ceremonial white lab coats possess a more elevated state of consciousness. Apparently, they are distinct from "people."

To date, Space and time are somehow variables out of our control

Yes. "Somehow." But he likely means the time and place of an encounter with an extraterrestrial civilization.

Religion appears to be a detour from self-consciousness to apprehend to some extent a cosmic consciousness.

So, that's what religion is.  Or "appears" to be. Yet earlier we were told we had to get beyond religion [as well as fear and denial] to reach Kosmic Konsciousness™.

Now for the Statistics Part

Getting down to the nitty gritty, or at least as nitty and gritty as "students filling out a questionnaire" is ever likely to get, we learn that:

The main objective of this study was to do a screening for general opinions of university students regarding the search for extraterrestrial intelligence, SETI, active SETI (SETI strategy of sending a message to Space), and the characteristics associated with cosmic consciousness as well as other related domains of interest such as religious beliefs, environmental aspects, and astronomy.

Which certainly sounds real tightly focused. 

Our hypotheses were that

1. religion is one of the main modular aspects,
2. general knowledge among university students on Space science to be significantly poor,
3. environmental concerns to be high, and
4. pseudoscience interest to be higher in those less religious subjects.

What any of these "hypotheses" have to do with a potential ET meet-and-greet is not especially clear. What they might have to do with "cosmic consciousness" is also unclear; unless cosmic consciousness has something to do with getting a  good grade in astronomy class.

That (1) is really specific. One of the main modular aspects. Really. Social scientists do not seem to frame "hypotheses" very neatly.
Regarding (2), TOF suspects general knowledge among university students on just about any topic will be significantly poor.
Actually, (4) is interesting: it seems to imply that someone who is religious is somehow immunized against interest in pseudoscience woo-woo. Chesterton quipped that those who don't believe in God wind up believing in anything.

Sample size

In this study, 116 subjects were from the United States (n=13), Italy (n=23), and Spain (n=80). Subjects were all university students from Rome in Italy, Chicago in the United States, and Cadiz in Spain.  ...[P]reliminary results are presented here, focusing on the Spanish group because it is the most numerous. Descriptive analysis was performed for the total sample, while varimax factorial analysis and correlation analysis was performed with the Spanish sample only.

IOW, for most of the analysis, the sample size is quietly reduced to 80 Spanish college kids. And from this we will make "preliminary" assessments of Earth's readiness to host the Galactic Federation.

The metric

[Students] responded using a computerized 5-point Likert-type scale ... where 1=completely disagree and 5=completely agree for scales A, B, and D. Scales C and E were 5-way multiple choice. 

The first is called an ordinal scale. Because it is not a ratio scale -- i.e., because a rating of "4" does not mean "twice as agreeable as '2'" -- it is strictly illegitimate to calculate averages and standard deviations, or to do regressions or hypothesis tests. That is, labeling a response with a numeral does not make it a number.

the data obtained were later analyzed using SPSS software for Windows.

Whew, that's a relief. He used a computer. All is well.

Analysis

Separate factor analyses were performed for the A, B, D, and E scales only for the Spanish sample (n=80). Factors were transformed using varimax rotation (orthogonal). The largest number of factors that met the standard criterion of having an eigenvalue of >1.0 was extracted.

Factor analysis has been mentioned already. But doesn't this make you feel all warm and fuzzy? And eigenvalues, no less.

One single factor was extracted from the 11 items assessing A scale, religious beliefs (7 items loading at >.40), explaining more than 45% of the variance. The factor was re-encoded and called the believe in God construct.

IOW, he took the 11 questions in this category, found that on 7 of them the responses sorta correlated, then "rotated" to remove the correlation. Using 80 data points. On an ordinal scale. The Believe in God "construct" is simply a name he gives to a cluster of seven items, but an inspection of the list shows that only two of them (3, 5) actually require belief in God:

1. The essential core of all religions is the same. (.780)
2. I believe in reincarnation. (.869)
3. The Bible is inspired by the word of God. (.840)
4. I believe that demons and angels exist. (.728)
5. I think that God made us in His image and likeness. (.826)
6. Spirituality and religion are the same. (.734)
7. There is life after death. (.461)

Several other factors from the other scales are also listed, but you may consult the paper itself for the details.

bivariate Pearson's correlations were calculated for different comparisons between extracted factors...

Note: he is not doing correlations between pairs of Xs but between pairs of "factors" extracted from those Xs. A Pearson's correlation between ordinal values is, let's say, procedurally challenged.

Results were also interesting for believe in God with contact (r= -.400; p<.001). This clearly indicates that people who are believers in God or religious persons had more difficulty accepting the possibility of contact with extraterrestrial civilizations, although, interestingly, they did not have problems accepting the existence of life beyond in the universe.

A correlation coefficient of -0.4 is nothing to write home about, although social studies folk seem to get all hot and bothered by them. The plea that such values are as good as R gets in social studies does not mean that they are good or useful. Even if R is legit, it would mean that the factor explains only 16% of the variation in contact.

Notice also the semantic shift between believe in God, which is merely a name given to a cluster of questions with similar response patterns and "believers in God or religious persons," which are people who actually, you know, believe in God. The magic was performed without, so far as can be told, ever asking respondents if they believed in God and correlating this to an actual belief in the likelihood of contact with ET. [A simple χ², no?]  Another term for people who are more skeptical about the possibility of alien contact is "realist." Or perhaps "data-oriented." Whereas those who do believe in the possibility of contact might be dubbed "faith-based."

Conclusions

We consider that the possibility of real contact implies the development of a wider scope of perception, to some degree beyond the regular framework of logic in terms of physics and psychology.

Let's get beyond that framework of logic. Psychology is apparently already there, we better get physics on the stick.

Other studies with Spanish samples found results opposite to ours regarding attitudes toward the possibility of contact, but others did find that religion was an important factor as well.

IOW, "our study seems to be a fluke." Statisticians have a word for studies where some support and others do not the same hypothesis. It's called "random."

The relationship between consciousness and the universe is not clear, but we suspect it could be greater than we previously thought.

"Not clear". Got it. "Suspect it could be". Good thing these are "preliminary conclusions."

Some studies, especially those in quantum physics, are trying to close the gap. Penrose proposes that consciousness depends on quantum biological computations, perhaps within or among brain neurons.

Close the gap we suppose between consciousness and the universe, whatever that may mean.

Our empirical [sic] study shows a deficiency of astronomy and Space-related knowledge among subjects of our sample, but we suspect that this is a generalized effect.  

That's it? The kids in his sample didn't learn enough space science in school?  But he suspects that this is a generalized effect. Stop the presses.

This effect together with other factors such as religious beliefs and self- and global consciousness levels as a species may represent key modular aspects to higher levels of awareness of our cosmic nature.

"May" represent? "Key modular aspects"? What has any of this to do with the questionnaire? TOF saw only a mild correlation reported between an artifact formed of a gallimaufry of questions and labeled "believe in God" and a skepticism about alien contact any time soon.


This is a common theme in social studies statistics: announcing a correlation between two things that were never actually measured. Matt Briggs, the Statistician to the Stars, has reported numerous sightings of this sort of critter.

Good news.
Support for the study came from MINECO grant AYA2010-18573. (El Ministerio de Economía y Competitividad). The US taxpayer, at least, is off the hook on this one. Why the ministry of economics and competitiveness is funding such a study escapes TOF's otherwise keen insight. Perhaps if aliens come it will severely affect economic competitiveness. Are there people actually worried about this?

edited for clarity 5/12/14