Deciding As A Team, Online

Here is a process how to facilitate an team to come to a decision together, online. Included are discussions of each step with examples of emails introducing the use of a common Weighted Evaluation Matrix spreadsheet.

Sound: Gavel pounding 3 times

Vote Links microformat was proposed by Technorati. It uses the "rev" attribute within HTML

Topics this page:

Site Map  
About this site  

The Annoyance

How does your team make a decision?

Usually, one person describes a problem. Another proposes a possible solution. Then others chime in about how that solution would have obstacles and won't solve the problem.

Then another solution is proposed. That person feels slighted when another says "that's not important."

Some don't say anything in fear of being belittled like that.

Suddenly, the team runs out of time when another team has the conference room scheduled.

So the team leader makes a decision.

Later, two at a time, team members privately discuss what a bad decision it was, and how the leader didn't listen.

Interested in an alternative?

What would your colleagues think if they received an email like this:

Best Practices

The top "best practices" in group decision making are:

• Assign roles (agenda facilitator, scribe, timekeeper, etc.).

• Separate "brainstorming" sessions into several phases:
  1. An option generation segment where all wild ideas are presented. No criticisms can be verbalized during this time. (Individuals can write down their thoughts evaluating each option.)

  2. A separate criteria identification segment where options are criticized.

  3. A separate option rating segment where ratings are applied to options.

• Capture options and criteria into a spreadsheet. Using an MS-Excel spreadsheet enable team members to "play" with their decisions, to see what is needed to tip the numbers to a favored option.

Enter the Matrix

The spreadsheet I propose is called a "matrix" because it captures several dimensions of ideas:

Download this sample Excel 2000 spreadsheet file

1. Options (labeled with capital A,B,C, etc. across from the middle of the spreadsheet) are "What if we do this?" proposals for action.

2. Criteria (labeled with lower case a,b,c down the spreadsheet) are from statements like "that won't work because" or "that's great because" which evaluate the options.

Criteria that are absolute "Must have" and "Must not have" (called "screeners") cannot be evaluated as a number, but as the percentage of members who vote "Yes" or "No".

3. weights adjusting for the importance of criteria are expressed so that their total sum up to 100% for all criteria from this Weights spreadsheet.

4. Ratings are assigned from this Ratings spreadsheet.

5. The Numerical winner is determined from the highest sum after each criteria weight is multiplied against each rating.

6. "screener" criteria may change the final decision.

In this example, "B" wins the numerical ratings, but since 0% of members think it "Doable", "A" becomes the final winner with all aspects considered.

Team Weights Spreadsheet

The weights shown on the matrix are obtained from the average of team member's weight assignments on his/her copy of this Weights spreadsheet.



The Minimum (Min.) and Maximum (Max.) values among all members are shown to detect errors and to guage the level of disagreement among members.

The Coefficient of Variation (CoV) is calculated to measure the amount of agreement among team members (the reliability or "dispersion" of the average).

Standard Deviation / Average * 100

Team Ratings Spreadsheet

Caveats

Those who use this need to understand the meaning of percentages and how they are calculated.

This main problem with this approach is its formality. Some people are unwilling to let go of their emotional prejudices even after this exercise. They insist on a certain outcome (winner) even in light of the “numbers•.

• Assign scores to each option/criteria cell in the first matrix. Be aware of the mathematical effect of scoring. Giving a score of 1 to option A and a score of 2 to option B means that option A is - twice as good as option 2. Use scale that is appropriate to the relative difference between the options.
• In a summary matrix, multiply the score in each option/criteria cell in the first matrix by the percentage weight for the corresponding criteria row.
• Add up the results for each option.
• Identify the ordinal ranking (1st, 2nd, 3rd, etc.). The winner is the one with the largest total result. Notice that in the sample the winner won because of a good showing on an important criteria even though it earned nothing on all other criteria.
• Do a Sensitivity Analysis -- make changes to scores slightly so that the winner changes to another option. This is an important step to consider anticipated changes in the future, such as the next release from the vendors. Make another matrix and insert scores for what you expect in the future. This will give you an answer to a possible question of “how close was the race?"

  A further complication of this is when there are several people involved in determining criteria weights and ratings. In such a case, create an additional sub matrix for each number in the evaluation matrix. The average obtained from among the group (or a reference to it) is then applied in the evaluation matrix.

References

A Simple Decision-Making Method for E-mail Groups from the Psychology of Cyberspace website by John Suler, Ph.D.

How To Build An Online Community - an insightful article by Esther Schindler for ZDNet Developer.

Sun's Java Community ProcessSM (JCP) is an inclusive, consensus building approach that produces a specification, a reference implementation (to prove the specification can be implemented), and a technology compatibility kit (a suite of tests, tools, and documentation that is used to test implementations for compliance with the specification). The process involves an ever-widening audience to review and comment on the document:
1. Initiation by a small group
2. Community Draft
3. Public Draft
4. Maintenance

Deciding via Email

How can an entire group collaboratively compose a document as a group ... using email?

Here is a methodology for people to come to agreement using email technology. Example: Agree on a common criteria to evaluate project plans submitted by students in the class.

The problem with email is also its strength -- communications are broadcast from one individual to all others. Emotionally charged emails are called “flames”. Email exchanges consists of offer and response cycles, not interactive negotiation.

A number of software tools are emerging. For over a decade, Bernie Dekoven has been reporting on tools for collaborating online at his coworking.com website

However, many are reluctant to use new technologies.

My strategy for using email technology to come to agreement is adapted from the same strategy used to reach agreement in a face to face meeting.

Step 1 - Willingness to Innovate

First, define stakeholders, their mutual goals, the benefit for achieving it, and the ground rules. Clarifying this is the first and very decisive battle. So many people become frustrated because they never did this to begin with and the lack of clarify undermines everything else they do.

Here is an example of an email to do this:

I would like to see if we can use a common criteria for judging our project plans.

Or should we each use our own criteria?

I think it is worth a try if it saves just one meeting. We'll all have less miles on our cars and more time with the rest of our lives.

What do you guys think?

This trial balloon, in essence, is the first email poll. If enough people respond positively, we go to the next step. A followup phone call or physical visit may be needed for those who don't respond.

Step 2 - Structure

The problem with this email is that two questions were posed. Typically not a good tactic. People might answer only one question and we're left wondering which item they agree with.

But asked two questions is useful to give cover to an innovation that may not be well received.

Anyway, the second step is to identify a pre-existing structure or approach. An experienced team does this automatically.

Here's a sample email:

I thought that we might avoid reinventing the wheel here by identifying a criteria structure that others have already used.

One possibility is the list of Core processes in PMP's Body of Knowledge? It's certainly comprehensive enough. The items are ...

Please let me know if this is OK, or propose a different structure.

A structure consists of deliverables, roles, and steps. If several structures are proposed, it's ususally best to adopt one rather than trying to merge the two.

Step 3 - Rules for Condensing

Rather than, I've found it better to brainstorm on separate lists at the same time.

Example:

Attached is my list of possible options and a separate list of possible criteria.

Please add your items before we identify the top choices.

Poll A: How many items should be on the list of options?

Poll B: How many items should be on the list of criteria?

We'll limit the lists to the average of items we all recommend, OK?

Here we are proposing both content and process at the same time. Again, not the ideal of working out a process then implementing it. But this two-headed approach keeps the emails active at the risk of confusion.

Step 4 - Condense

Example email:
After calculating your responses, it looks like we'll work with the top 6.

Now on to selecting those top options.

Below is are lists of options and criteria. Please allocate 100 points among items in each list.

Now we're on our way.

Step 5 - Collect the Scores

Example email:
Attached is the results of our last poll. Here are the items sorted in order of popularity...

Now we need to assign weights. Please put your numbers next to each criteria. I will turn your weightings into percentages, as shown in the sample.

Step 6, etc.

We're on our way...