Select your localized edition:

Close ×

More Ways to Connect

Discover one of our 28 local entrepreneurial communities »

Be the first to know as we launch in new countries and markets around the globe.

Interested in bringing MIT Technology Review to your local market?

MIT Technology ReviewMIT Technology Review - logo


Unsupported browser: Your browser does not meet modern web standards. See how it scores »

Part I one of this article can be found here.

Another job that Smeed gave me was to invent ways to estimate the effectiveness of various counter­measures, using all the evidence from a heterogeneous collection of operations. The first countermeasure that I worked on was MONICA. MONICA was a tail-mounted warning radar that emitted a high-pitched squeal over the intercom when a bomber had another aircraft close behind it. The squeals came more rapidly as the distance measured by the radar became shorter. The crews disliked MONICA because it was too sensitive and raised many false alarms. They usually switched it off so that they could talk to each other without interruption. My job was to see from the results of many operations whether MONICA actually saved lives. I had to compare the loss rates of bombers with and without MONICA. This was difficult because MONICA was distributed unevenly among the squadrons. It was given preferentially to Halifaxes (one of the two main types of British heavy bomber), which usually had higher loss rates, and less often to Lancaster bombers, which usually had lower loss rates. In addition, Halifaxes were sent preferentially on less dangerous operations and Lancasters on more dangerous operations. To use all the evidence from Halifax and Lancaster losses on a variety of operations, I invented a method that was later reinvented by epidemiologists and given the name “meta-­analysis.” Assembling the evidence from many operations to judge the effectiveness of MONICA was just like assembling the evidence from many clinical trials to judge the effectiveness of a drug.

My method of meta-analysis was the following: First, I subdivided the data by operation and by type of aircraft. For example, one subdivision would be Halifaxes on Bremen on March 5; another would be Lancasters on Berlin on December 2. In each sub­division I tabulated the number of aircraft with and without MONICA and the number lost with and without MONICA. I also tabulated the number of MONICA aircraft expected to be lost if the warning system had no effect, and the statistical variance of that number. So I had two quantities for each subdivision: observed-minus-expected losses of MONICA aircraft, and the variance of this difference. I assumed that the distributions of losses in the various subdivisions were uncorrelated. Thus, I could simply add up the two quantities, observed-minus-expected losses and variance, over all the subdivisions. The result was a total observed-minus-expected losses and variance for all the MONICA aircraft, unbiased by the different fractions of MONICA aircraft in the various subdivisions. This was a sensitive test of effectiveness, making use of all the available information. If the total of observed-minus-expected losses was significantly negative, it meant that MONICA was effective. But instead, the total was slightly positive and less than the square root of the total variance. ­MONICA was statistically worthless. The crews had been right when they decided to switch it off.

I later applied the same method of analysis to the question of whether experience helped crews to survive. Bomber Command told the crews that their chances of survival would increase with experience, and the crews believed it. They were told, After you have got through the first few operations, things will get better. This idea was important for morale at a time when the fraction of crews surviving to the end of a 30-­operation tour was only about 25 percent. I subdivided the experienced and inexperienced crews on each operation and did the analysis, and again, the result was clear. Experience did not reduce loss rates. The cause of losses, whatever it was, killed novice and expert crews impartially. This result contradicted the official dogma, and the Command never accepted it. I blame the ORS, and I blame myself in particular, for not taking this result seriously enough. The evidence showed that the main cause of losses was an attack that gave experienced crews no chance either to escape or to defend themselves. If we had taken the evidence more seriously, we might have discovered Schräge Musik in time to respond with effective countermeasures.


5 comments. Share your thoughts »

Tagged: Computing

Reprints and Permissions | Send feedback to the editor

From the Archives


Introducing MIT Technology Review Insider.

Already a Magazine subscriber?

You're automatically an Insider. It's easy to activate or upgrade your account.

Activate Your Account

Become an Insider

It's the new way to subscribe. Get even more of the tech news, research, and discoveries you crave.

Sign Up

Learn More

Find out why MIT Technology Review Insider is for you and explore your options.

Show Me

A Place of Inspiration

Understand the technologies that are changing business and driving the new global economy.

September 23-25, 2014
Register »