What's a feature worth?
You have your project mission. You have your team. You know how you want
to create value for your customers. Selecting a remote service solution is probably a once-in-a-career activity for most people. How do you decide what capabilities are needed to give you the best chance for success?
Agree on a scoring matrix and an evaluation process. Remote service software has a wide variety of capabilities and options. A well designed evaluation process will keep you on track and on schedule.
Think of the last time you chose a place to live or bought a house. You probably had some high level items that were non-negotiable. Things like distance to work, number of bedrooms, price range, etc. For each house you visited, your first thought was how it met these high level criteria. As long as there were no show stoppers, you were interested and wanted to investigate more.
The next level of details include things that are the most difficult to change. I.e., they are the things you have to be willing to live with as long as you stay there. For example, a busy street, a small backyard, or a strange room arrangement. You can't change them, so you create a mental score for each.
Finally you are left with things that you like or not, but that you know you can change. Its nice if the seller has great taste in colors and you know you can move right in and feel very comfortable. If the colors are awful, you have to decide the "cost" you are willing to accept to add a little paint to resolve the problem.
Identify, Divide, Score
You have to approach your remote service selection process in the same way. The cross functional team needs to work backwards from the customer value and the internal business case to clearly identify the high level items that the solution must have. (This generally ends up being the list you use to narrow the vendor list from many down to less than 5.)
Now divide the high level requirements into categories and create working groups of experts for each one. The working group needs to identify the more detailed requirements and create a scoring matrix for each. It is a good idea to pick a consistent scoring scheme for all categories so everyone can get used to working in the same ranges. The working groups should also rank the importance of each requirement and apply an appropriate weighting factor. The idea here is to make sure the truly important things get the most weight, but also allow less important items the chance to break the tie.
Get the team back together and go through the matrix line by line. Yeah, it is tough, but the exercise will help in two ways:
1) The team members will learn why each working group included a requirement and how they want to score them
2) It's a good chance to practice articulating the requirements so that you can be very clear when explaining what you want to vendors. That clarity will really improve the quality of vendor responses and the ability to get proposals that enable apples to apples comparison.
Once you receive the proposals, team members should INDIVIDUALLY score the proposals based on the criteria. Then bring the team back together to discuss the RFP responses and raise any questions. Individual scores may be adjusted at this point based upon discussion.
Remember, decision making is often done at an emotional level and then we spend our intellectual energy building a justification of our feelings or gut decision. Having the cross functional team involved in the process will help prevent this and help keep the decision focused on selecting the best solution that meets the requirements.
Some common sources of bias[1] to watch out for as you collect requirements and scoring results:
- We tend to be willing to gather facts that support certain conclusions, but disregard other facts that support different conclusions.
- We tend to accept the first alternative that looks like it might work. Don't be lazy and just take the first answer.
- We are often reject the unfamiliar or are unwilling to look beyond what was done in the past. Many great ideas are killed because someone says, "We tried that before and it didn't work." Ask why not?
- We tend to attribute our success to our abilities and talents, but we attribute our failures to bad luck and external factors.
- We tend to place more attention on more recent information and either ignore or forget more distant information.
- There is a subtle willingness to believe what we have been told most often and by the greatest number of different of sources.
- We tend to underestimate future uncertainty because we tend to believe we have more control over events than we really do.
- In order to simplify an extremely complex world, we tend to group things and people. These simplifying generalizations can bias decision making processes. Don't assume how something works - ask!
Knowing what features are worth will help you select the right product. It will also help later on in the process when making resource or planning decisions.
Read the related articles for Randy's 10-Step Series:
10 Steps to a Successful Remote Service Evaluation
Step 1. Successful Remote Service Evaluations - Have a Vision of What Success Looks Like
Step 2. Successful Remote Service Evaluations - Work with Cross Functional Teams
Step 3. Successful Remote Service Evaluations - Answer What's in it For Your Customers!
[1] Adapted from http://en.wikipedia.org/wiki/Buyer_decision_processes