Computing Diversity

Decision Team
Diversity of membership in decision-making teams leads to better outcomes. By Rawpixel.com/Shutterstock

17 July 2019 – It’s come to my attention that a whole lot of people don’t know how to calculate a diversity score, or even that such a thing exists! How can there be so much discussion of diversity and so little understanding of what the word means? In this post I hope to give you a peek behind the curtain, and maybe shed some light on what diversity actually is and how it is measured.

This topic is of particular interest to me at present because momentum is building to make a study of diversity in business-decision making the subject of my doctoral dissertation in Business Administration. Specifically, I’m looking at how decision-making teams (such as boards of directors) can benefit from membership diversity, and what can go wrong.

Estimating Diversity

The dictionary definition of diversity is: “the condition of having or being composed of differing elements.”

So, before we can quantify the diversity of any group, we’ve got to identify what makes different elements different. This, by the way, is all basic set theory. In different contexts what we mean by “different” may vary. When we’re talking about group decision making in a business context, it gets pretty complicated.

A group may be subdivided, or “stratified” along various dimensions. For example, a team of ten people sitting around a table trying to figure out what to do next about, say, a new product could be subdivided in various ways. One way to stratify such a group is by age. You’d have so many individuals in their 20’s, so many might be in their 30’s, and so forth up to the oldest group being aged 50 or more. Another (perhaps more useful) way to subdivide them is by specialty. There may be so many software engineers, so many hardware engineers, so many marketers, and so forth. These days stratifying teams by gender, ethnicity, educational level or political persuasion could be important. What counts as diversity depends on what the team is trying to decide.

The moral of this story is that a team might score high in diversity along one dimension and very poorly along another. I’m not going to say any more about diversity’s multidimensional nature in this essay, however. We have other fish to fry today.

For now, let’s assume a one-dimensional diversity index. What we pick for a dimension makes little difference to the mathematics we use. Let’s just imagine a medium-sized group of, say, ten individuals and stratify them according to the color of tee-shirts they happen to be wearing at the moment.

What the color of their tee-shirts could possibly mean for the group’s decisions about new-product development I can’t imagine, and probably wouldn’t care anyway. It does, however, give us a way to stratify the sample. Let’s say their shirt colors fall out as in Table 1. So, we’ve got five categories of team members stratified by tee-shirt color.Table 1: Tee-Shirt Colors

NOTE: The next bit is mathematically rigorous enough to give most people nosebleeds. You can skip over it if you want to, as I’m going to follow it with a more useful quick-and-dirty estimation method.

The Gini–Simpson diversity index, which I consider to be the most appropriate for evaluating diversity of decision-making teams, has a range of zero to one, with zero being “everybody’s the same” and one being “everybody’s different.” We start by asking: “What is the probability that two members picked at random have the same color tee shirt?”

If you’ve taken my statistical analysis course, you’ll likely loathe remembering that the probability of picking two things from a stratified data set, and having them both fall into the same category is:

Equation 1

Where λ is the probability we want, N is the number of categories (in this case 5), and P is the probability that, given the first pick falling into a certain category (i) the second pick will be in the same category. The superscript 2 just indicates that we’re taking members two at a time. Basically P is the number of members in category i divided by the total number of members in all categories. Thus, if the first pick has a blue tee-shirt, then P is 3/10 = 0.3.

This probability is high when diversity is low, and low when diversity is high. The Gini-Simpson index makes more intuitive sense by simply subtracting that probability from unity (1.0) to get something that is low when diversity is low, and high when diversity is high.

NOTE: Here’s where we stop with the fancy math.

Guesstimating Diversity

Veteran business managers (at least those not suffering from pathological levels of OCD) realize that the vast majority of business decisions – in fact most decisions in general – are not made after extensive detailed mathematical analysis like what I presented in the previous section. In fact, humans have an amazing capacity for making rapid decisions based on what’s called “fuzzy logic.”

Fuzzy logic recognizes that in many situations, precise details may be difficult or impossible to obtain, and may not make a significant difference to the decision outcome, anyway. For example, deciding whether to step out to cross a street could be based on calculations using precise measurements of an oncoming car’s speed, distance, braking capacity, and the probability that the driver will detect your presence in time to apply the brakes to avoid hitting you.

But, it’s usually not.

If we had to make the decision by the detailed mathematical analysis of physical measurements, we’d hardly ever get across the street. We can’t judge speed or distance accurately enough, and have no idea whether the driver is paying attention. We don’t, in general, make these measurements, then apply detailed calculations using Gallilean Transformations to decide if now is a safe time to cross.

No, we have, with experience over time, developed a “gut feel” for whether it’s safe. We use fuzzy categories of “far” and “near,” and “slow” or “fast.” Even the terms “safe” and “unsafe” have imprecise meanings, gradually shifting from one to the other as conditions change. For example “safe to cross” means something quite different on a dry, sunny day in summertime, than when the pavement has a slippery sheen of ice.

Group decision making has a similar fuzzy component. We know that the decision team we’ve got is the decision team we’re going to use. It makes no difference whether it’s diversity score is 4.9 or 5.2, what we’ve got is what we’re going to use. Maybe we could make a half-percent improvement in the odds of making the optimal decision by spending six months recruiting and training a blind Hispanic woman with an MBA to join the team, but are we going to do it? Nope!

We’ll take our chances with the possibly sub-optimal decision made by the team we already have in place.

Hopefully we’re not trying to work out laws affecting 175 million American women with a team consisting of 500 old white guys, but, historically, that’s the team we’ve had. No wonder we’ve got so many sub-optimal laws!

Anyway, we don’t usually need to do the detailed Gini-Simpson Diversity Index calculation to guesstimate how diverse our decision committee is. Let’s look at some examples whose diversity indexes are easy to calculate. That will help us develop a “gut feel” for diversity that’ll be useful in most situations.

So, let’s assume we look around our conference room and see six identical white guys and six identical white women. It’s pretty easy to work out that the team’s diversity index is 0.5. The only way to stratify that group is by gender, and the two strata are the same size. If our first pick happens to be a woman, then there’s a 50:50 chance that the second pick will be a woman, too. One minus that probability (0.5) equals 0.5.

Now, let’s assume we still have twelve team members, but eleven of them are men and there’s only one token woman. If your first pick is the woman, the probability of picking a woman again is 1/12 = 0.8. (The Gini-Simpson formula lets you pick the same member twice.) If, on the other hand, your first pick is a man, the probability that the second pick will also be a man is 11/12 = 0.92. I plugged all this into an online Gini-Simpson-Index calculator (‘cause I’m lazy) and it returned a value of 26%. That’s a whole lot worse.

Let’s see what happens when we maximize diversity by making everyone different. That means we end up stratifying the members into twelve segments. After picking one member, the odds of the second pick being identical are 1/12 = 0.8 for every segment. The online calculator now gives us a diversity index of 91.7%. That’s a whole lot better!

What Could Possibly Go Wrong?

There are two main ways to screw up group diversity: group-think and group-toxicity. These are actually closely related group-dynamic phenomena. Both lower the effective diversity.

Group-think occurs when members are too accommodating. That is, when members strive too hard to reach consensus. They look around to see what other members want to do, and agree to it without trying to come up with their own alternatives. This produces sub-optimal decisions because the group fails to consider all possible alternatives.

Toxic group dynamics occurs when one or more members dominate the conversation either by being more vocal or more numerous. Members with more reticent personalities fail to speak up, thus denying the group their input. Whenever a member fails to speak up, they lower the group’s effective diversity.

A third phenomenon that messes up decision making for  high-diversity teams is that when individual members are too insistent that their ideas are the best, groups often fail to reach consensus at all. At that point more diversity makes reaching consensus harder. That’s the problem facing both houses of the U.S. Congress at the time of this writing.

These phenomena are present to some extent in every group discussion. It’s up to group leadership to suppress them. In the end, creating an effective decision-making team requires two elements: diversity in team membership, and effective team leadership. Membership diversity provides the raw material for effective team decision making. Effective leadership mediates group dynamics to make it possible to maximize the team’s effective diversity.

POTUS and the Peter Principle

Will Rogers & Wiley Post
In 1927, Will Rogers wrote: “I never met a man I didn’t like.” Here he is (on left) posing with aviator Wiley Post before their ill-fated flying exploration of Alaska. Everett Historical/Shutterstock

11 July 2018 – Please bear with me while I, once again, invert the standard news-story pyramid by presenting a great whacking pile of (hopfully entertaining) detail that leads eventually to the point of this column. If you’re too impatient to read it to the end, leave now to check out the latest POTUS rant on Twitter.

Unlike Will Rogers, who famously wrote, “I never met a man I didn’t like,” I’ve run across a whole slew of folks I didn’t like, to the point of being marginally misanthropic.

I’ve made friends with all kinds of people, from murderers to millionaires, but there are a few types that I just can’t abide. Top of that list is people that think they’re smarter than everybody else, and want you to acknowledge it.

I’m telling you this because I’m trying to be honest about why I’ve never been able to abide two recent Presidents: William Jefferson Clinton (#42) and Donald J. Trump (#45). Having been forced to observe their antics over an extended period, I’m pleased to report that they’ve both proved to be among the most corrupt individuals to occupy the Oval Office in recent memory.

I dislike them because they both show that same, smarmy self-satisfied smile when contemplating their own greatness.

Tricky Dick Nixon (#37) was also a world-class scumbag, but he never triggered the same automatic revulsion. That is because, instead of always looking self satisfied, he always looked scared. He was smart enough to recognize that he was walking a tightrope and, if he stayed on it long enough, he eventually would fall off.

And, he did.

I had no reason for disliking #37 until the mid-1960s, when, as a college freshman, I researched a paper for a history class that happened to involve digging into the McCarthy hearings of the early 1950s. Seeing the future #37’s activities in that period helped me form an extremely unflattering picture of his character, which a decade later proved accurate.

During those years in between I had some knock-down, drag-out arguments with my rabid-Nixon-fan grandmother. I hope I had the self control never to have said “I told you so” after Nixon’s fall. She was a nice lady and a wonderful grandma, and wouldn’t have deserved it.

As Abraham Lincoln (#16) famously said: “You can fool all the people some of the time, and some of the people all the time, but you cannot fool all the people all the time.”

Since #45 came on my radar many decades ago, I’ve been trying to figure out what, exactly, is wrong with his brain. At first, when he was a real-estate developer, I just figured he had bad taste and was infantile. That made him easy to dismiss, so I did just that.

Later, he became a reality-TV star. His show, The Apprentice, made it instantly clear that he knew absolutely nothing about running a business.

No wonder his companies went bankrupt. Again, and again, and again….

I’ve known scads of corporate CEOs over the years. During the quarter century I spent covering the testing business as a journalist, I got to spend time with most of the corporate leaders of the world’s major electronics manufacturing companies. Unsurprisingly, the successful ones followed the best practices that I learned in MBA school.

Some of the CEOs I got to know were goofballs. Most, however, were absolutely brilliant. The successful ones all had certain things in common.

Chief among the characteristics of successful corporate executives is that they make the people around them happy to work for them. They make others feel comfortable, empowered, and enthusiastically willing to cooperate to make the CEO’s vision manifest.

Even Commendatore Ferrari, who I’ve heard was Hell to work for and Machiavellian in interpersonal relationships, made underlings glad to have known him. I’ve noticed that ‘most everybody who’s ever worked for Ferrari has become a Ferrari fan for life.

As far as I can determine, nobody ever sued him.

That’s not the impression I got of Donald Trump, the corporate CEO. He seemed to revel in conflict, making those around him feel like dog pooh.

Apparently, everyone who’s ever dealt with him has wanted to sue him.

That worked out fine, however, for Donald Trump, the reality-TV star. So-called “reality” TV shows generally survive by presenting conflict. The more conflict the better. Everybody always seems to be fighting with everybody else, and the winners appear to be those who consistently bully their opponents into feeling like dog pooh.

I see a pattern here.

The inescapable conclusion is that Donald Trump was never a successful corporate executive, but succeeded enormously playing one on TV.

Another characteristic I should mention of reality TV shows is that they’re unscripted. The idea seems to be that nobody knows what’s going to happen next, including the cast.

That leaves off the necessity for reality-TV stars to learn lines. Actual movie stars and stage actors have to learn lines of dialog. Stories are tightly scripted so that they conform to Aristotle’s recommendations for how to write a successful plot.

Having written a handful of traditional motion-picture scripts as well as having produced a few reality-TV episodes, I know the difference. Following Aristotle’s dicta gives you the ability to communicate, and sometimes even teach, something to your audience. The formula reality-TV show, on the other hand, goes nowhere. Everybody (including the audience) ends up exactly where they started, ready to start the same stupid arguments over and over again ad nauseam.

Apparently, reality-TV audiences don’t want to actually learn anything. They’re more focused on ranting and raving.

Later on, following a long tradition among theater, film and TV stars, #45 became a politician.

At first, I listened to what he said. That led me to think he was a Nazi demagogue. Then, I thought maybe he was some kind of petty tyrant, like Mussolini. (I never considered him competent enough to match Hitler.)

Eventually, I realized that it never makes any sense to listen to what #45 says because he lies. That makes anything he says irrelevant.

FIRST PRINCIPAL: If you catch somebody lying to you, stop believing what they say.

So, it’s all bullshit. You can’t draw any conclusion from it. If he says something obviously racist (for example), you can’t conclude that he’s a racist. If he says something that sounds stupid, you can’t conclude he’s stupid, either. It just means he’s said something that sounds stupid.

Piling up this whole load of B.S., then applying Occam’s Razor, leads to the conclusion that #45 is still simply a reality-TV star. His current TV show is titled The Trump Administration. Its supporting characters are U.S. senators and representatives, executive-branch bureaucrats, news-media personalities, and foreign “dignitaries.” Some in that last category (such as Justin Trudeau and Emmanuel Macron) are reluctant conscripts into the cast, and some (such as Vladimir Putin and Kim Jong-un) gleefully play their parts, but all are bit players in #45’s reality TV show.

Oh, yeah. The largest group of bit players in The Trump Administration is every man, woman, child and jackass on the planet. All are, in true reality-TV style, going exactly nowhere as long as the show lasts.

Politicians have always been showmen. Of the Founding Fathers, the one who stands out for never coming close to becoming President was Benjamin Franklin. Franklin was a lot of things, and did a lot of things extremely well. But, he was never really a P.T.-Barnum-like showman.

Really successful politicians, such as Abraham Lincoln, Franklin Roosevelt (#32), Bill Clinton, and Ronald Reagan (#40) were showmen. They could wow the heck out of an audience. They could also remember their lines!

That brings us, as promised, to Donald Trump and the Peter Principle.

Recognizing the close relationship between Presidential success and showmanship gives some idea about why #45 is having so much trouble making a go of being President.

Before I dig into that, however, I need to point out a few things that #45 likes to claim as successes that actually aren’t:

  • The 2016 election was not really a win for Donald Trump. Hillary Clinton was such an unpopular candidate that she decisively lost on her own (de)merits. God knows why she was ever the Democratic Party candidate at all. Anybody could have beaten her. If Donald Trump hadn’t been available, Elmer Fudd could have won!
  • The current economic expansion has absolutely nothing to do with Trump policies. I predicted it back in 2009, long before anybody (with the possible exception of Vladimir Putin, who apparently engineered it) thought Trump had a chance of winning the Presidency. My prediction was based on applying chaos theory to historical data. It was simply time for an economic expansion. The only effect Trump can have on the economy is to screw it up. Being trained as an economist (You did know that, didn’t you?), #45 is unlikely to screw up so badly that he derails the expansion.
  • While #45 likes to claim a win on North Korean denuclearization, the Nobel Peace Prize is on hold while evidence piles up that Kim Jong-un was pulling the wool over Trump’s eyes at the summit.

Finally, we move on to the Peter Principle.

In 1969 Canadian writer Raymond Hull co-wrote a satirical book entitled The Peter Principle with Laurence J. Peter. It was based on research Peter had done on organizational behavior.

Peter was (he died at age 70 in 1990) not a management consultant or a behavioral psychologist. He was an Associate Professor of Education at the University of Southern California. He was also Director of the Evelyn Frieden Centre for Prescriptive Teaching at USC, and Coordinator of Programs for Emotionally Disturbed Children.

The Peter principle states: “In a hierarchy every employee tends to rise to his level of incompetence.”

Horrifying to corporate managers, the book went on to provide real examples and lucid explanations to show the principle’s validity. It works as satire only because it leaves the reader with a choice either to laugh or to cry.

See last week’s discussion of why academic literature is exactly the wrong form with which to explore really tough philosophical questions in an innovative way.

Let’s be clear: I’m convinced that the Peter principle is God’s Own Truth! I’ve seen dozens of examples that confirm it, and no counter examples.

It’s another proof that Mommy Nature has a sense of humor. Anyone who disputes that has, philosophically speaking, a piece of paper taped to the back of his (or her) shirt with the words “Kick Me!” written on it.

A quick perusal of the Wikipedia entry on the Peter Principle elucidates: “An employee is promoted based on their success in previous jobs until they reach a level at which they are no longer competent, as skills in one job do not necessarily translate to another. … If the promoted person lacks the skills required for their new role, then they will be incompetent at their new level, and so they will not be promoted again.”

I leave it as an exercise for the reader (and the media) to find the numerous examples where #45, as a successful reality-TV star, has the skills he needed to be promoted to President, but not those needed to be competent in the job.