Process Improvement and the Census

In any process improvement effort, observing a process is an important step. I recently came across an interesting article about how the Office of the Inspector General was observing the door-to door counting process of the Census Bureau.

Just a few items of interest in the article.

1. The observation was done late in the game. While it is hard to determine the exact timing of the study, the article states that 93% of the follow up to the census is complete. I imagine that the door-to-door activities 10 years from now at the next census will be substantially different. Key Point: Observe early in the project to have time to fix problems.
2. There is little understanding of the effect of being observed. The study says 71 out of 480 people being observed did the process wrong. When people are being watched, they tend to follow the process differently than normal. In all likelihood, the failure rate is much higher. Key Point: A short term observation often makes things look better than they really are.
3. There is little understanding of statistics. The response of the Census Bureau was that “The OIG only observed 480 interviews out of 480 million households we have visited in our door-to-door operations.” The implication I take from that comment is that the speaker doesn’t believe that the true failure rate is anywhere near the 15% in the study. Key Point: The purpose of a sample is to make an inference about a full population. If the sample was taken correctly, you can’t just dismiss the results.
4. There is no comment about the results of the process failure. While the process failed, the article says nothing about the actual impact on the accuracy of the audit. The pollers were supposed to read the questions aloud. Did their incorrect process actually cause an inaccurate result? Key Point: The article never discusses the impact of the failed process. Prioritizing corrective measures takes an understanding of the results.

This late in the game, it is unlikely that the Census Bureau will take significant action. But if you found that 15% of the time, your team didn’t follow the process, check thing out in this order:

1. Is this a training issue? Do the even know the correct process?
2. Is this a process issue? Why do people skip steps? Does the process make sense?
3. Is this a leadership issue? Do people feel pressured to skip steps to make a quota?
4. Is this a people problem? Have the employees been given ample chances to improve?

Notice that the people step is the last one. Most people want to do a good job. Make sure they have the right opportunities to succeed.

As a side question to the point of this article, how would you poka yoke (mistake proof) this process. Keep in mind that there were 600,000 temporary workers doing this job. Every $1.67 you spend on an individual equals a million bucks added to the budget.