Fun With Statistics

Once upon a time, a Customer called to complain that our beverage cans held too much beer. This could be a problem because the Benevolent Government, ever alert to our well-being, or at least to their own revenues, taxes alcoholic beverages on the nominal package volume. A 12-oz. can is presumed to hold 12 oz. of beer at 3.4% alcohol by weight and is taxed accordingly. If it holds more than 12 oz. then the brewer is giving away free beer and (more importantly) not paying taxes on the extra fraction of a percent of an ounce. ATF has never reconciled with the end of Prohibition.

The primary control measurement is called "Overflow Capacity," which measures the volume of fluid the container can hold up to the tippy-top. The ASTM "Wet Fill" method of making the measurement proceeds as follows:

1. Weigh the can empty with a special clear plastic lid in place.
2. Fill the can completely with distilled water at 20°C (68°F) 
3. Top off with eyedropper so that no air bubbles can be seen through the clear plastic lid.
4. Weigh the filled can-and-lid.
5. The difference in weights is the weight of the contents.
6. Together with the temperature, this can be converted to fluid ounces of volume. (In those days, this was done using a table.)

So Laszlo calls and says, "Your overflow capacities are too high."
So TOF, in an earlier incarnation, contacts Metallurgy, which has custody of the measurement. (It is controlled by the diameter of the punch.)
So Metallurgy looks upon its data, and sees that it is good.

Glass bottles suck air

Upshot: the customer, disinclined to get wet, was using a different method to measure the volume in receiving inspection. You see, they also bought glass bottles, and to measure this volume they attached the bottle to a vacuum pump and sucked all the air out. Not only did the tech stay dry, but the instrument gave a digital readout, and everyone knows that a digital readout is always more scientificalistic. So (they reasoned) why not fabricate a fixture for cans and use the vacuum pump!

OK. Maybe not THIS flat!

However, an aluminum can will do something when you pull a vacuum on it that a glass bottle will not. The entire weight of the Earth's atmosphere will stomp the sucker flatter than a possum on a county road, thus producing a spuriously larger volume.

Take-away: Two different methods of making a measurement will generally give two different results. Which is one the "True" value is a matter of judgment and choice. In this case, the vacuum pump is suspected of being a less desirable method of flexible packaging. The wet fill method is the standard.

But TOF has encountered other metrological disagreements that were less obvious than the Wet Fill/Vacuum Pump example.

Hand-held gauge

Bench-top gauge

A customer used a hand-held dial indicator from which the part dangled on a pin. The supplier used a dial indicator on a granite slab bearing a "nest" to hold the part at an angle. The customer's quality engineer (named Wally) flew out from Chicago and measured the offending lids in our face. See? They're out of spec!

In the spirit of reasoned discourse, we said, "Gimme that!" and took the lids and placed them in the nest on our bench top gauge. Right in mid-spec! Wally sez, "Your gauge must be out of spec." TOF said, "It's got a calibration sticker on it." (TOF did not expect that to fly, but could not resist.)

"Let us verify the calibration!" So the call went out for the metrologist. Trumpets sounded and the vault door of the calibration lab swung open with a puff of positive air pressure. The metrologist stepped forth in his robes, bearing the Gauge Block on a satin pillow while his acolytes made sweet his path with incense. Upon reaching our lab, he put upon his hands the gloves of calibration, lest the Block be profaned with vulgar bodily oils. He placed the Block in our gauge.

Dead nuts on.

Now Wally was concerned, as he had flown all the way from Chi-town to resolve the problem! The Block was measured with his gauge.

Dead nuts on.

Can lid geometry is complex even without Chinese ideograms

Now the situation became interesting. Previously, it had been only the usual urinary Olympics in which suppliers and customers engage. Now there was an intriguing problem to solve. The two gauges gave the same result on the Block but differed by 0.003" on the Part. The resolution was that the two gauges held the Part at slightly different angles and so they did not quite measure the same dimension. The Gauge Block being a simple rectangular solid, the angle of the dangle did not enter into it.

In this case, unlike the Overflow Capacity, both methods were perfectly valid and employed properly calibrated instruments. Yet they gave different results. What the true Curl Width of the lids were depended on which method was designated the Standard Method.

Now, TOF told you that to tell you this:

Back in 2009, the current administration announced its intention to shift the Census Bureau from the Department of Commerce to report directly to the White House. This led then-Sen. Judd Gregg to withdraw as the putative trans-partisan member of the Cabinet. (He had been sounded out for Commerce.  The actual purpose of his nomination was to remove a member of the opposition party from the Senate so that the Governor of New Hampshire could appoint a Senator of the proper party and move the Senate one seat closer to being filibuster-proof. )

Statisticians have regarded the statistical work at the Census Bureau as the "best practice." Cochran had worked there, and Deming. They feared that by making the Census Bureau directly subordinate to the White House, its statistical work would become subordinated to politics, which TOF will call "polistitics." For example, in the run-up to the 2012 election the bureau put out labor statistics that showed unemployment dropping from 8.1% (Aug) to 7.8% (Sep); but the New York Post's John Crudele reported that "according to a reliable source" the data "were manipulated." Maybe so, maybe no; but why was the administration so determined to bring the Census directly under its control?

Now the White House Census Bureau has changed the "instrument" (it's actually a questionnaire) that is used to estimate insurance coverage. And we all know what happens when a different instrument is used, don't we? The % uninsured measured the new way will not be comparable to the % uninsured measured the old way. Since the New Way produces numbers about two percentage points lower than the old way,* figures for 2014 will be lower than for 2013, even if there was no actual change in insurance coverage, making it appear as if % uninsured has decreased.

(*) "The percentage of people without health insurance was 10.6 percent in the content test [of the new way] and 12.5 percent in the production instrument [the old way]."
--- Carla Medalia, Carla, et al. "Changing the CPS Health Insurance Questions And The Implications On The Uninsured Rate: Redesign and Production Estimates" Proceedings of the 2013 Federal Committee on Statistical Methodology Research Conference

The reason for the "annexation" of the Census Bureau and the redesign of the instrument may not have been to ensure a lower percentage for Obamacare. Work on the redesign started ten years ago, during a previous administration. It may however be one reason why folks were so anxious to get the act implemented in this time frame. If the "number" of uninsured was going to decrease anyway due to a technical correction to the metric, why not piggyback and make use of that? 

But the reason given for the revision was that "research suggest[ed] that the calendar year estimate of the uninsured is higher than it should be and that estimates actually reflect a mixture of current and past year coverage." This in turn suggests the magnitude of the problem was exaggerated -- but that does not signify. TOF does not recall any instances during the public debates when any actual statistics on magnitude were ever provided on the subject. 

The usual practice when changing instruments is to maintain both metrics for a period of time to assess the difference, as in the case of a shift in a Greenland temperature station shown here:

The new instrument at Egedesminde runs slightly colder than the old
instrument at Jakobshavn, but gives largely comparable results.

So, since it would not be feasible to travel back in time and produce figures for 2012 and 2013* using the new method, it would seem reasonable to estimate 2014 using both methods so that the before and after comparison can be made using the same metric rather that using the high measure for Before and the low measure for After.

(*) The new instrument was tested in March 2013, so there is only a one-month point rather than the whole of 2013.