BLOGRIGE

The G8 Forum, Rosabeth Moss Kanter, and the Baldrige Criteria

Posted by Harry Hertz, the Baldrige Cheermudgeon

OK, you ask, what could the G8, a distinguished Harvard professor, and the Baldrige Criteria for Performance Excellence possibly have in common? It all started with an HBR blog post by Rosabeth Moss Kanter after the G8 Forum in Paris late in May. Professor Kanter needs no introduction to the business community or readers of the Harvard Business Review. But did you know that she served as a member of the Baldrige Board of Overseers and has written a foreword for Baldrige 20/20, our new executive guide to Baldrige to be published in August?

But back to the topic at hand. The Forum focused on the power and challenges of the internet. Some interesting facts cited by Professor Kanter include the findings of a McKinsey study that internet economic output is bigger than Spain and growing faster than Brazil. Seventeen thousand people are employed by eBay, but 1.3 million people earn a living from it. With these and other facts as background, Professor Kanter's blog post made the point that four things never change (and here is where I bring the Baldrige Criteria to bear):

1. A great customer experience differentiates winners from losers.Winners are made not by their internet technology, but by the quality of their customer experience. That is why the Baldrige Criteria are so focused on customer satisfaction, satisfaction relative to competitors, and engagement to build a longer term relationship.

2. The human side is critical to the use of technology. Points were made that the biggest barrier to the spread of e-technology use in education and health care is the professionals discomfort with their use. That is why the Baldrige Criteria specifically ask how you ensure that hardware and software are user-friendly.

3. Money needs to change hands. The point was for all that internet provides "free," people still need to earn a living for what they produce. That is why Financial and Market Outcomes is now the last item of 17 in the Baldrige Criteria, the bottom line for any enterprise.

4. Government and business still talk past each other. The point is it's still about social, legal, and ethical responsibility and that is the responsibility of business and government collectively, in internet or any other commerce. That is why the Baldrige Criteria start with Senior Leadership and Governance and Societal Responsibilities. It does not get more basic than those primary concepts for conducting ethical business.

So there you have it. The world and business change at a rapid pace, but basic principles still are bedrock. That is why the Baldrige Criteria are about strategy, innovation, and good common business sense. Your thoughts are welcomed!

Criteria Requirement 3.1b.(1) How do you determine customer satisfaction and engagement?

 

Posted by Jeff Lucas

One of the ways that we do it, is by conducting an annual survey of the applicants for the Malcolm Baldrige National Quality Award.  This is a limited but important segment of our overall customer group, since these organizations have made a significant investment in putting together their application, and they are the recipients of one of our core products, the Feedback Report.  We do have other listening posts for our products and services (Bob Hunt recently reported on the evaluations from this year's Examiner training here) including the use of social media. 

We have conducted a survey of our applicants every year for at least a decade, but starting in 2009 we made a change to the format of the survey.  In the past, we had done a fairly traditional satisfaction survey of approximately 35 questions, where we asked for ratings of specific steps in the process, features of the criteria, helpfulness of staff, elements of the feedback report and overall satisfaction.  There were a number of problems with the survey.  First, the amount of effort required to answer all the questions seemed to keep response rates down in what would seem to be a relatively motivated population.  Even with electronic distribution, multiple reminders and the like, we were never able to top a 65% response rate.  Second, while we were getting a lot of data about likes and dislikes, we didn't have any mechanism to find out which of the items were really important to the customers. Third, it generated a lot of suggestions that weren't really actionable (give everyone a site visit) and therefore we didn't act, probably creating a feeling of non-responsiveness on the part of respondents. 

So in 2009 we began using a Net Promoter Score type approach.  We had done some benchmarking of Baldrige applicants and recipients and decided that this method of assessing your overall relationship with customers through the use of a single question was an appropriate way to get at our intended outcomes for the survey.  The question is frequently in the format "How likely is it that you would recommend our X to a friend/colleague."  The question is rated on a 10-point scale from Extremely Likely to Extremely Unlikely and the score is calculated by adding the percentage of respondents giving 9 or 10 and then subtracting the sum of all responses 6 or below. In other words, a pretty high standard. 

There is then a follow-up opportunity to provide information on "what one thing kept you from giving a higher rating?"  We did go ahead and add a couple of specific questions on the Criteria and the Feedback Report, but keeping the same measurement methodology.  

This change almost immediately achieved one of our goals.  The response rate climbed to 79% for the first year.  We also got some interesting feedback and a much clearer picture of what we could do to improve the applicant experience.  You can see the presentation that we made to the Panel of Judges, including both the NPS scores and the aggregated themes for how we could improve here.  For those who don't want to click through a quick summary would be:  

• We have established a strongly positive relationship with these customers (NPS of 65 in 2009 and 70 in 2010; these are really strong scores on this metric with a pretty high bar -- equivalent to those reported for folks like Intuit and Southwest Airlines).
• They love the relevance of the Criteria (71 and 82 respectively)
• Their Feedback Reports?  Not so much.  Those scores were only 6 and 10 for the last two years (Remember the calculation above.  This doesn't mean we had single digit satisfaction, just that the support for that product is much softer than for participation in the program as a whole)

Not surprisingly, on the overall question, the major reason for a lower score was "feedback issues".  When we did the drill down of the Feedback Report itself, the top three responses were:  "difficult to understand", "not Criteria based", and "didn't get our business".  This data was one of the major drivers for the changes we made this year to what comments should look like.  So, we do collect customer data and try to act on it.  Have we made the right choices to improve the customer experience?  Look for the definitive answer in this space six months from now, or let us know what you think. 

Cheers for Baldrige Volunteers!

Posted by Christine Schaefer

No matter where I've worked for pay since college, I've always enjoyed spending some of my off-duty time working for free. Whether that service involved long-term commitments such as mentoring children in challenging situations or simple gigs such as greeting student families at the doors to a school event, I've always felt rewarded for volunteering with a sense of satisfaction in helping others and fulfilling a civic duty. So I think I understand what motivates the many volunteers who individually contribute 100-200 hours each summer to help the Baldrige Program meet its mission to improve the competitiveness and performance of U.S. organizations.

Consider the roles of two groups of volunteers who are key to accomplishing the work of the Baldrige Program. First, now that the 2011 Baldrige Award Process has kicked into high gear, hundreds of highly engaged volunteers known as examiners are using their expertise in every sector of the economy to evaluate the performance of organizations around the country. These examiners have received award applications and read about organizations' processes and results in relation to the Criteria for Performance Excellence. Working on teams that provide feedback to each award applicant, the examiners have been analyzing data and sharing observations with each other via online scorebooks that eventually will become confidential reports for each organization on its strengths and opportunities for improvement.

Second, last month another group of hard-working volunteer examiners put its finishing touches on a key training tool for the Baldrige Program. Members of the 2012 case study writing team created a model award application that depicts a fictitious but credible organization. Each volunteer writer spent countless hours researching relevant industry performance data; envisioning how the invented organization might respond to a particular set of challenges and advantages, among other key factors; and translating the team's creative vision into narrative and results sections that look like an actual Baldrige Award application. These volunteers also each spent at least 15 hours on conference calls with team mates to share ideas and information and ensure that details were aligned in their final document. The case study they created will serve as an invaluable learning resource for future examiners (since actual applications are confidential and can't be used in training) and as a sample for future award applicants.

So why do Baldrige examiners lend so much of their talent and time--usually evenings and weekends--to these highly demanding tasks? Examiners have provided a variety of reasons. They have cited, for example, the opportunities to learn and network with their peers on the Board of Examiners. They have pointed to the professional development they receive in preparation for their service to the Baldrige Program. Perhaps most important, many Baldrige volunteers prize the sense of patriotic duty they experience in helping U.S. organizations improve through the Presidential award program.

Those drivers make perfect sense to me. But I would never take volunteers for granted. I know the Baldrige Program is very fortunate to be able to count on more than 500 highly skilled professionals who volunteer each year to support our mission. If you're an examiner, here's to you--cheers to you! If you're with an organization considering how a large group of volunteers might help you meet your mission, may the service of the Baldrige Program's Board of Examiners inspire you. If you're with an organization that similarly benefits from volunteers' work, please share your thoughts, your best practices, and your kudos for your volunteers!

 

 

 

Baldrige Examiner Training Gets High Marks!

Posted by - Bob Hunt, Workforce Development Team



The evaluations are in!  Overall satisfaction with our examiner classroom training in May at the NIST campus garnered the highest scores in the past decade, and maybe of all time!  On a scale of 1 to 5, with 5 being excellent, the average score for our 2 1/2 day examiner preparation course was 4.9!   Our 1-day new examiner  orientation course scored 4.8, and our senior examiner training course scored 4.7.   These are amazing scores for any training course, and speak to the tremendous efforts and commitment of both Baldrige staff and examiner facilitators!

 The feedback associated with these scores provides some insight on what is driving these numbers. One of the most frequently mentioned positive attributes was the design of the training courses itself.  The cornerstone of the design, using a case study of a fictitious Baldrige award application, continues to be an effective approach for our examiners to learn the Criteria for Performance Excellence and the Baldrige Award application evaluation process.  However, for the past two years the training design has gradually eliminated lecture time in the classroom, moving that material to our online training classes.  This has created the opportunity to provide more skill practice time in the classroom, and significantly increase the time examiners spend learning from each other (an "all teach, all learn" model).  For example, half of the one-day new examiner orientation course is spent with senior level examiners coaching the new examiners as they work their way through the process of evaluating the fictitious award application.  In the two and a half day examiner preparation course, the primary course all examiners attend, examiners at all levels of experience work together in small groups to work through four rounds of applicant evaluations, using the application case study, and are provided feedback on their work by their peers as well as NIST staff facilitators and senior level examiner facilitators.

Thanks to all involved in helping us deliver a successful training.

 

It's that Customer Thing...Again

Posted by Harry Hertz, the Baldrige Cheermudgeon

I had a great trip recently. First, there was the fourth meeting of our Baldrige Executive Fellows. I am still amazed by the great group of people in this first cohort and by the level of shared learning. This time we were graciously hosted by Cargill, with two business units that have received the Baldrige Award, Kitchen Solutions (formerly Sunny Fresh Foods) and Corn Milling. From the Fellows' meeting I went to Dallas for the Texas Award for Performance Excellence. It was a time to see old friends and celebrate state role models. As always, their conference provided many learning opportunities for attendees.

It all was great... until those customer moments at the airport! Two this time. I returned my rental car (well-known national company) and the bill was about 15% higher than the rental confirmation receipt. I questioned the agent at the return lane and he said I had to go to the desk, he only prints receipts. Obviously there was a line at the desk, but fortunately I was early enough to wait. When it was my turn, the agent explained the difference was that I had a car one class higher than the rental confirmation (not a luxury car by any means). I asked how I would know that since I went to the aisle I was told to go to and pick any car. Quiet on the other side of the counter...then, "I'll change it this time but won't in the future." My response, "And will you also learn how to sort cars before then?" Wouldn't life be a lot better if they had read the Customer Focus category of the Baldrige Criteria for Performance Excellence? Then they might have apologized for putting the car in the wrong aisle (BTW there were many of them in that aisle), corrected the bill and given it to me with a smile. The end result in terms of the bill to me is the same. The difference is a customer loyalty victory vs. an unhappy customer.

Next, I went to the food court in the departure area and ordered a pastrami sandwich from a large national vendor. The description read "pastrami on rye with mustard." Perfect (or poifect, as we used to say in New York). When they were wrapping the sandwich, I noticed it came with melted cheese. I said i just want the pastrami, no cheese. The preparer said, "Then why didn't you say so when your ordered it?" I said, "Because the description just says pastrami on rye with mustard, how should I know it comes with melted cheese?" She said, "Remember that for next time." She begrudgingly remade the sandwich. End result, same as with the car rental!

Is the customer concept gone in America or just in airports? And, incidentally, if you want to be a well-served customer, please check out the opportunity to join the second cohort of Baldrige Executive Fellows.

Criteria Requirement 2.1a.(1) How does your organization conduct its strategic planning?

Posted by Jeff Lucas


I am not going to provide details on all of the "items to address" from our Criteria for Performance Excellence(so don't bother posting OFI comments like "unclear how time horizons were set"), but I did want to provide everyone out there in the Baldrige community with an update on our most recent round of strategic planning which has involved the construction of a Strategy Map and an aligned Balanced Scorecard.  

The approach originated from the bottom up.  Acting on our results from administering the Gallup Q12 workforce engagement survey, we had agreed to develop an impact plan.  A team of staff (without senior leadership involvement, mind you) came back with the recommendation that they would like the Program to develop a Balanced Scorecard.  I must admit that my initial reaction was "really?!?" I haven't experienced a lot of examples of staff asking to get involved with the hard work of trying to measure better, but then again, this is the Baldrige staff we are talking about. 

We decided that the use of a scorecard is greatly enhanced by being derived from the direction that is provided by a strategy map.  In constructing ours, we adapted the traditional format used by for-profit organizations to reflect the fact that our customers are largely not the ones paying for our products and services.  Instead, we placed at the top of our map "Funders and Stakeholders," the groups that provide both the financial and social capital necessary for the ongoing success of the program. The remaining perspectives remain pretty true to the traditional approach and include Customers, Internal Processes and Learning & Growth.  

To date we have completed the top two areas and developed the metrics that would support these.  You can view this work, along with the operational definitions of how we propose to measure the concepts listed on the scorecard, here:  Baldrige Strategy Map.  We presented this work to the Baldrige Board of Overseers at their June 15th meeting where it was approved.  

We are working now to complete the map and scorecard and would like to have your feedback.  Please leave us comments here on the blog, join in the discussion on our LinkedIn group, or feel free to email me directly at jeff.lucas@nist.gov