On November 14th 2013 Jack Zenger, co-Founder and CEO of Zenger Folkman, published an article in Forbes entitled“Why Gallup’s 70% Disengagement Data is Wrong.” This disappointing article is misguided on so many levels and contributes little to the pursuit of research-based findings in social science. Indeed it undermines them. Littered with poor reasoning and logical fallacies it deserved a robust, factual response.
Jack,
There is an ongoing and welcome debate about employee engagement – how to best measure it, the extent to which it quantifies company culture, predicts business performance and whether and how it can be improved over time. Your article skirted these issues and failed to prosecute the central claim that Gallup disengagement data is wrong.
- Your whole thesis is built around a category error. Gallup reports employee engagement in three categories – “Engaged”, “Not Engaged” and “Actively Disengaged”. You simply can’t lump together two Gallup categories – “Actively Disengaged” and “Not Engaged” – and collectively call them “Disengaged”. Gallup is very clear in their category descriptions and this misrepresents their findings.
- The Gallup report to which you refer describes US Working population data gained from high survey sample sizes consistently measured over long periods of time. This reveals 28% “Actively Disengaged” and 52% “Not Engaged”. It doesn’t logically follow from this, as you seem to assume, that every company will have 70% of their employees similarly categorized. There is nothing in the Gallup report that suggests this as a working assumption. When you make this claim it is like denying that 5 feet 4.5 inches is the current average height of women in the US based on the fact that the last 10 women you met were all 5 feet 10 inches. This error in reasoning is not trivial.
- Engagement also strongly correlates to performance. More successful companies will have very different engagement ratios from unsuccessful companies. If Joe Folkman and yourself are spending time with more successful companies you will inevitably experience a stronger work culture that is likely the product of higher residual engagement. But this would still be an assumption – only measurement will truly inform the answer. Anecdotal evidence, however sincere and well-intentioned, fails to get close to the definitive claim you make. The fact that you think an organization’s engagement profile is different from Gallup’s nationwide working population polling shouldn’t surprise you. And it really doesn’t matter if you are accurate. Gallup isn’t making the claim that you are making on their behalf.
- Talking to executives in organizations isn’t going to provide you with definitive evidence either. It’s not that their opinion is irrelevant, it’s that they are often wrong. I have lost count of the number of times I have presented executives with engagement data and found them to be surprised and shocked either at overall engagement levels in their companies or at significant pockets of disengagement they never expected to see. It is only through the systematic surveying and measurement of those companies and linkages to business performance that the true cost of this ignorance is revealed. It’s hard to resolve these issues during a cozy chat with an executive in a corner office. Opinions aside, the only way of answering the question about engagement at the company level is to measure it.
- There is variance in data reported by different research organizations and this is why the debate I referenced earlier is so important. Much of this is explained by methodological differences. In the report you cite (AP-NORP) this is hardly an “apples to apples” comparison with Gallup. The AP report measures “satisfaction”, which is a far lower bar to meet than “engagement”. We all know of many instances where employees report being “satisfied” without being “engaged”. This study also groups higher scoring responses (“4’s” and “5’s”) as “positive” whereas Gallup applies strong focus to percent 5’s (top-box scores), and this methodological difference, clearly referenced in both studies, significantly influences the reported outcomes. Gallup also reports on “overall satisfaction” and found 88% of the US working population is “generally satisfied” with their work. Clearly, “engagement” and “satisfaction” are very different things. If you had considered a different reputable study by way of comparison, such as the Towers Watson 2012 study you would have found engagement data that substantially corroborates Gallup data. Is Towers Watson also wrong?
- You raise an interesting question regarding your 360 survey studies. You seem to be making the claim that a subset of employees who are scored by other employees on a set of “employee commitment” questions equates to a measurement of employee engagement, and that this provides evidence that Gallup data on US working population engagement is wrong. There are several problems with this line of reasoning. First, you admit to asking different questions than Gallup in your 360, yet claim your results are a more accurate measure of engagement – this is an interesting opinion, and you might be right, but this doesn’t mean that Gallup is wrong, but that you have come up with an interesting finding. If you claim equivalence between your 360 survey “employee commitment” questions and the Gallup Q12, this flies in the face of overwhelming survey research evidence that suggest subtle shifts in the wording of questions has significant effects on how people respond. Second, the significant methodological difference between 360 assessments and engagement surveys would seem to invalidate the comparison you are inviting. Third, for the claims about 360 assessments to have any validity to your thesis, they would have to be administered to every employee at every level in every position simultaneously and then compared to Gallup engagement measures in those same organizations. Again, methodologically, there is so much difference between the administering of 360 assessments on select employees in single organizations, with national polling of the US working population, that it would be quite remarkable to find any consistency in reported outcomes. Even if this were to be done, and the data broadly correlated, it would have little utility. Regardless, there would be no finding that could support your claim
I am sure we would agree that acting on poor evidence is ill advised in any situation. This is why research is important. It would be highly inappropriate for executives in companies to assume that 70% of their employees are either actively disengaged or not engaged. Gallup isn’t suggesting this, yet it is a claim you are making on their behalf. In late 2012 when Nobel Laureate Danny Kahneman made the plea for social psychologists to clean up their act, his target could have been so much wider. His frustration with poor, unreplicated research and declarative findings based on shoddy or incomplete data on the topic of “social priming” was a salutary message to all who work in the social science arena. It is incumbent on thought leaders to take up his challenge and set the example for all to follow. Misrepresenting Gallup data isn’t the best way to contribute to this process.
Recent Comments