Transcript Document
INTRO TO RATING SCALES 2014 v1.0 Define Decision: Build Rating Scales Define Identify Criteria Build Ratings Scales Identify Alternatives Identify Participants 2 Build rating scales to evaluate alternatives Rating scales enable us to assess how well a project, vendor, candidate, etc. fulfills the criteria Defines what constitutes “good” or “bad” Converts judgments into a numeric value In a home remodeling project prioritization, what does a rating for increase the feeling of more space look like? Each lower level criterion requires a rating scale 3 Set Rating Scale Weights Every rating has an assigned weight that converts the rating into a value. The best or highest rating gets a weight of “1” The worst or lowest rating gets a weight of “0” Ask yourself, for this criterion, what is good enough? How much more do I value the next rating up? 4 Three ways to handle quantitative criteria In the Home Improvement model, how would you evaluate a project’s reduction in utility costs (monthly)? Option 1: Enter an absolute number Numeric rating scale Option 2: Select from a range of numbers Verbal rating scale Option 3: Select from multiple descriptive titles Verbal rating scale 5 Option 1: Enter an absolute number Developing a numeric rating scale Numeric rating scales capture numbers Only 2 values are required on your scale, but additional points may be entered Points in between defined values are calculated on a straight line – a value of $100 gets a rating of 0.7 in this example 6 Option 1: Enter an absolute number (cont.) Entering ratings using a numeric rating scale Data can be imported or directly entered by participants Values outside the listed range are allowed. 7 Option 2: Select from a range of numbers Developing a verbal scale for a range of numbers Useful if a specific number value isn’t available or when there is some uncertainty around the actual value 8 Option 3: Select from multiple descriptive titles Developing a verbal scale using descriptive titles 9 Build Rating Scales There are options available for building Rating Scales Option 1: Select from the Blueprints Pane Modify the scale(s) to better fit the criterion Option 2: Copy from existing scales Edit the scales(s) to better fit the criterion 10 Option 1: Select from Blueprints Pane 1 The Decision Blueprints pane provides Sample Scales that may be used. Mouse over the Sample Scales while holding down the Ctrl key to see a quick view and see if they can be applied to rate the criterion 2 Click and drag the chosen scale to the main workspace or click APPLY. After the sample scale is applied to the Decision, it can be modified to better match the criterion 11 Option 2: Copy from previously built scales 1 Select a scale from the drop-down list to copy a previously built scale, click APPLY, then edit accordingly 12 Define Ratings 1 Highlight the criterion for which you want to define a scale 3 2 Click Show Rating Definitions to generate a pop-up window in which to define the scale Define the chosen criteria in the scale and click Hide Rating Definitions to close the windows 13 Best Practices for Building Rating Scales There is no “one size fits all” approach to ratings As with initial criteria development, it works best for a small group to develop a strawman proposal that the larger group can validate A four or five-point scale is preferred to a three-point scale as it allows for a greater level differentiation on large alternative sets Rating values should be considered as ratio numbers, not interval numbers Use Scale Definitions to define what constitutes a specific rating 14 Learn more at DLU.decisionlens.com