THE DEPARTMENT OF COMMERCE
PERSONNEL MANAGEMENT DEMONSTRATION PROJECT

FACILITATORS’ FOCUS GROUP PROTOCOL - SPRING 2007

Demonstration Group, Non-Supervisors
Demonstration Group, Supervisors
Comparison Group, Non-Supervisors
Comparison Group, Supervisors

DEMONSTRATION GROUP, NON-SUPERVISORS

---

I. INTRODUCTION/BACKGROUND 

Ø Welcome and overview of session

Hello everybody. My name is (facilitator) and this is (co-facilitator). We are from Booz Allen Hamilton, a management consulting firm in McLean, VA. We will be leading today’s discussion about the Department of Commerce's Personnel Management Demonstration Project. 

Ø Demonstration Project Background

We would like to start by briefly talking about the Demonstration Project. As you probably know, Commerce initiated the Demonstration Project in March 1998 as a means of testing whether a series of alternative personnel practices could be more successful than traditional personnel practices in helping Commerce achieve its goals. A series of HR interventions were implemented in a subset of the organization. In 2003, this Demonstration Project was extended another five years to better understand the impact of these HR interventions.

The Demonstration Project includes two groups and our assessment compares human resource practices in these two groups. The first group is made up of those work units within Commerce where a series of human resources practices have been implemented under the Demonstration Project. We call this the Demo Group. The second group consists of Commerce work units similar to those participating in the Demonstration Project but who have maintained the traditional human resources practices. This group is called the Comparison Group. 

Ø Booz Allen’s Role

Commerce contracted with Booz Allen to conduct a comprehensive program evaluation of the Demonstration Project’s performance in meeting its objectives. Booz Allen's role in the Demonstration Project is as an objective, third party evaluator. Booz Allen was not involved in the design of the Demonstration Project's new human resources practices. Commerce has been responsible for implementing the new practices, with oversight provided by the Office of Personnel Management. Booz Allen's job is to collect information and act as an impartial evaluator of how the new human resources practices are working.

Ø Purpose of the Focus Group

The intent of this focus group is to gain an understanding of specific human resource practices within the Department of Commerce from the perspective of employees. We would like to identify how these practices impact employees and their work units. Also we would like to hear from you about any past successes and/or possible future improvements to human resource practices. In particular, we will be talking about:

1. Performance Management
2. Career Paths
3. Hiring/Recruitment 
4. Turnover and Retention
5. Quality of Workforce
6. Organizational Excellence and Workforce Diversity

Information from this focus group will be combined with the information collected from the focus groups taking place in other locations. Booz Allen will then use all the focus group data, plus information from surveys, interviews, and other sources, to evaluate the ninth year of the Demonstration Project.

Ø Participant Selection

I imagine that you are interested in knowing how you were invited to join today's session. We were given a list of employees at this location and from this list we randomly selected names. For this focus group, we selected names of non-supervisors. To be sure that we sampled names correctly, I want to make sure everyone here belongs in this group. Is there anyone here who is a supervisor? (Facilitator's note: If so, thank them for their time and excuse them from the session. Note that we defined supervisors as those who conduct performance appraisals of others.)

[USE THIS TEXT FOR FOCUS GROUPS WITH WAVE 3 PARTICIPANTS ONLY:] Our team is aware that your organization is new to the Demonstration Group. Although your organization has only been in the Demonstration Project for a short amount of time, your insights are still valuable to us. In fact, for a number of the questions, we are particularly interested in gathering perspectives on what employees experience when they are new to the Demonstration Project. However, if we ask any questions that you are truly unable to answer, please do not feel pressured to respond. 

[USE THIS TEXT FOR ALL-MINORITY SESSIONS ONLY:] You may have noticed as we went around the room that our group is rather homogenous. In fact, this is by design. We were given a list of employees at this location and from this list we randomly selected names based on [MINORITY].

The reason that we did this is because we want to assess the impact of the Demonstration Project's human resources interventions on [MINORITY] employees. Our discussion today will be included as part of our overall data collection efforts but will also allow us to compare the perceptions of [MINORITY] employees with the perceptions of employees overall.

We also are conducting focus groups at several other locations across the United States. We are meeting with employees in the Demo Group as well as the Comparison Group. (Facilitator's note: Bring your copy of the list showing where we are conducting focus groups in case anyone asks.)

Ø Focus Group Participant Introductions 

Before we continue, we would like to find out who you are. If you would, please tell us your name, your organization, and department, and how long you’ve been with Commerce.

(FACILITATOR'S NOTE: State the following only at sites at which it applies.) While we are here, we also will be conducting another focus group with supervisors and will be asking them similar questions.

Ø Confidentiality

Any information you share with us today will be held confidential. We will be aggregating results from all focus groups and will not be attributing comments to any particular groups or individuals. 

We also ask that anything said in this group remains in this room. We want everyone to feel comfortable about talking, which means that we have to agree not to discuss what was said here today. Is everyone comfortable with that? 

Ø Timing 

Today’s focus group will last approximately 2 hours. We will take one 15-minute break during this session.

Are there any questions before we get started?

II. DISCUSSION 

A. Performance Management

Performance Appraisal Process

Our first set of questions pertains to the performance appraisal process.

1. Does the current performance appraisal process provide you with an appropriate amount of feedback regarding your job performance? If yes, how so? If not, what is missing?

2. How often do you receive formal or informal feedback from your supervisor? How often does that feedback help you improve your performance?

3. Do you feel that your performance is linked to the organizational mission? If yes, how? If no, why not?

4. Do you believe that the current performance appraisal process provides an accurate assessment of your performance?

5. Is it important for you to know how you are performing in relation to your peers? If so, does the current system provide you with this type of information? [Facilitator’s note: Do not reference “ranking” because ranking is no longer part of the Demo. However, they may be interested in knowing, for example, how their performance score compared to the average score or other methods for comparison.]

6. Does the current performance appraisal process clearly identify the consequences of poor performance to employees, as well as the rewards of good performance?

7. Does the current performance appraisal process help to differentiate between high and low performing employees?

8. Is the current performance appraisal process being implemented fairly by managers?

9. Has your understanding of the performance appraisal process enacted under the Demonstration Project improved, stayed the same, or lessened over the past nine [four] years since the Demonstration Project was implemented? [Wave 3 participants will be unable to respond to this question.]

10. If you could modify aspects of the current performance appraisal process to better meet your needs, what changes would you make?

Pay-for-Performance

Next, I have a series of questions for you about pay and performance.

11. Do you feel that your pay has increased during the nine [four] years that you have been in the Demonstration Project more quickly than it would have under the traditional GS system? If yes, is this due to the Demonstration Project? If no, why not? [Note: This question cannot be asked of Wave 3 focus group participants.]

12. Do you believe that larger pay raises are given to employees who perform better? Do you believe that average or lower performers are treated fairly?

13. Do factors other than performance contribute to an employee's salary? If yes, what are the other factors?

14. Do you think pay-for-performance is motivating employees to perform better on the job? If yes, why? If no, why not?

15. What influence do Rating Officials and Pay Pool Managers have on the pay for performance system? Are they fair in how they do these jobs?

16. What could Commerce do in the future to improve the link between an employee's pay and his/her performance?

Performance Bonuses

Our next set of questions pertains to performance bonuses.

17. Does your work unit distribute performance bonuses?

18. Do performance bonuses motivate employees to perform better?

19. Are performance bonuses distributed fairly (that is, based on performance)? If not, what other factors are considered?

B. Career Paths

As you know, within the Demonstration Project, positions were grouped into one of four career paths—ZP for scientists and engineers, ZT for scientific and engineering technicians, ZA for administration positions, and ZS for support positions. We have a question for you about how these career paths are working.

20. Is there any way you would change these career paths to make them more conducive to employees’ career progression? 

Broadbanding refers to the way that the 15 GS grades have been compressed into 5 broader bands. 

21. What impact, if any, has the broadbanding system had on your career progression?

C. Hiring/Recruitment

Our next set of questions pertains to hiring and recruitment.

22. What does Commerce do to attract and hire high quality candidates?

23. Have you noticed any differences in the quality of new hires in the past nine [four] years, since the Demonstration Project began? If yes, to what do you attribute the difference?

24. What else could Commerce do to attract and hire high quality candidates?

(BREAK: 5-15 MIN)

D. Turnover and Retention

Our next set of questions pertains to turnover and retention.

25. How much of an issue is turnover within your work unit? In what ways, if any, has this impacted morale in your work unit?

26. Why do people leave this organization?

27. In what ways are high performing employees encouraged to stay with DoC?

28. Have any of the following personnel changes made as part of the Demonstration Project helped Commerce retain high performing employees? If so, which ones and how?

• performance-based pay increases
• performance-based bonuses
• potential for pay increases with promotion 

29. What else could Commerce do to retain high performing employees?

E. Quality of Workforce

The next few questions pertain to the quality of the workforce.

30. Thinking about the fact that the Demonstration Project was designed to improve performance, has the quality of work within your work unit improved since the Demonstration Project began? If yes, what impact have new hires had on this improvement?

31. What else could Commerce do to improve employee performance?

F. Organizational Excellence and Workforce Diversity (pass out listing of the nine Merit System Principles and the 12 Prohibited Personnel Practices)

We have several questions about organizational excellence and workforce diversity.

32. Have you noticed any changes in how the nine Merit System Principles are followed under the Demonstration Project versus how they are typically followed elsewhere in Commerce? If yes, how? 

33. Have you noticed any changes in how the twelve Prohibited Personnel Practices are addressed under the Demonstration Project versus how they are typically addressed elsewhere in Commerce? If yes, how?

34. Do you feel that any of the human resource practices implemented as part of the Demonstration Project have had a particularly negative impact on minorities, women, or veterans in the Demonstration Project? If yes, how?

35. Do you feel that any of the human resource practices implemented as part of the Demonstration Project have had a particularly positive impact on minorities, women or veterans in the Demonstration Project? If yes, how?

G. Outstanding Issues

36. Before we end today, do you have any other comments regarding the Demonstration Project that you would like us to know?

III. CLOSING 

In closing, we want to again mention that your comments will be held confidential. We will be combining information that we gather in the focus groups with information gathered from the survey and interviews. We will then prepare our Year Nine evaluation report and will submit it to Commerce and the Office of Personnel Management later this summer.

Thank you for participating in today's session. We appreciate your taking the time and sharing your ideas with us!

DEMONSTRATION GROUP, SUPERVISORS

---

I. INTRODUCTION/BACKGROUND 

Ø Welcome and Overview of Session

Hello everybody. My name is (facilitator) and this is (co-facilitator). We are from Booz Allen Hamilton, a management consulting firm in McLean, VA. We will be leading today’s discussion about the Department of Commerce's Personnel Management Demonstration Project.. 

Ø Demonstration Project Background

We would like to start by briefly talking about the Demonstration Project. As you probably know, Commerce initiated the Demonstration Project in March 1998 as a means of testing whether a series of alternative personnel practices could be more successful than traditional personnel practices in helping Commerce achieve its goals. A series of HR interventions were implemented in a subset of the organization. In 2003, this Demonstration Project was extended another five years to better understand the impact of these HR interventions.

The Demonstration Project includes two groups and our assessment compares human resource practices in these two groups. The first group is made up of those work units within Commerce where a series of human resources practices have been implemented under the Demonstration Project. We call this the Demo Group. The second group consists of Commerce work units similar to those participating in the Demonstration Project but who have maintained the traditional human resources practices. We call this the Comparison Group. All of you here today are part of the first group, the Demo Group.

Ø Booz Allen’s Role

Commerce contracted with Booz Allen to conduct a comprehensive program evaluation of the Demonstration Project’s performance in meeting its objectives. Booz Allen's role in the Demonstration Project is as an objective, third party evaluator. Booz Allen was not involved in the design of the Demonstration Project's new human resources practices. Commerce has been responsible for implementing the new practices, with oversight provided by the Office of Personnel Management. Booz Allen's job is to collect information and act as an impartial evaluator of how the new human resources practices are working.

Ø Purpose of the Focus Group

The intent of this focus group is to gain an understanding of specific human resource practices within the Department of Commerce from the perspective of supervisors. We would like to identify how these practices impact supervisors and their work units. Also we would like to hear from you about any past successes and/or possible future improvements to human resource practices. In particular, we will be talking about:

1. Performance Management
2. Career Paths
3. Classification
4. Hiring/Recruitment 
5. Three-Year Probation for R&D Employees in ZP Career Path
6. Turnover and Retention
7. Quality of Workforce
8. Organizational Excellence and Workforce Diversity

Information from this focus group will be combined with the information collected from the focus groups taking place in other locations. Booz Allen will then use all the focus group data, plus information from surveys, interviews, and other sources, to evaluate the ninth year of the Demonstration Project.

Ø Participant Selection

I imagine that you are interested in knowing how you were invited to join today's session. We were given a list of employees at this location and from this list we randomly selected names. For this focus group, we selected names of supervisors only. To be sure that we sampled names correctly, I want to make sure everyone here belongs in this group. Is there anyone here who is not a supervisor? (Facilitator's note: If so, thank them for their time and excuse them from the session. Note that we defined supervisors as those who conduct performance appraisals of others.)

(Facilitator's note: State the following only at sites in which it applies.) While we are here, we also will be conducting other focus group with non-supervisors and we will be asking them similar questions.

We are also conducting focus groups at several other locations across the United States. We are meeting with employees in the Demo Group as well as the Comparison Group. (Facilitator's note: Bring your copy of the list showing where we are conducting focus groups in case anyone asks.)

[USE THIS TEXT FOR FOCUS GROUPS WITH WAVE 3 PARTICIPANTS ONLY:] Our team is aware that your organization is new to the Demonstration Group. Although your organization has only been in the Demonstration Project for a short amount of time, your insights are still valuable to us. In fact, for a number of the questions, we are particularly interested in gathering perspectives on what employees experience when they are new to the Demonstration Project. However, if we ask any questions that you are truly unable to answer, please do not feel pressured to respond. 

Ø Focus Group Participant Introductions 

Before we continue, we would like to find out who you are. If you would, please tell us your name, your organization, and department, and how long you’ve been with Commerce.

Ø Confidentiality

Any information you share with us today will be held confidential. We will be aggregating results from all focus groups and will not be attributing comments to any particular groups or individuals.

We also ask that anything said in this group remains in this room. We want everyone to feel comfortable about talking, which means that we have to agree not to discuss what was said here today. Is everyone comfortable with that? 

Ø Timing 

Today’s focus group will last approximately 2 hours. We will take one 15-minute break during this session. 

Are there any questions before we get started?

II. DISCUSSION 

A. Performance Management

Performance Appraisal Process

Our first set of questions pertains to the performance appraisal process.

1. Does the current performance appraisal process enable you to accurately evaluate the performance of your employees? If yes, how so? If not, what is missing?

2. Does the current performance appraisal process create an opportunity for you to provide your employees with relevant feedback regarding their performance? If yes, how? If no, how could it be improved?

3. From your perspective as a supervisor, is the current performance appraisal process easier, the same as, or more difficult to use than the traditional process? How so?

4. Does the current performance appraisal process clearly identify the consequences of poor performance to employees, as well as the rewards of good performance?

5. Does the current performance appraisal process help to differentiate between high and low performing employees?

6. Do you believe that average or lower performers are given guidance on how to improve performance? If so, what kind of guidance are they receiving?

7. Has your understanding of the performance appraisal process enacted under the Demonstration Project improved, stayed the same, or lessened over the past nine years [four years] since the Demonstration Project began?

Pay-for-Performance

Next, I have a series of questions for you about pay and performance.

8. Do you believe that larger pay raises are given to employees who perform better? Do you believe that average or lower performances are treated fairly? 

9. Do factors other than performance contribute to an employee's salary? If yes, what are the other factors?

10. Do you think pay-for-performance is motivating employees to perform better on the job? If yes, why? If no, why not?

11. What influence do Rating Officials and Pay Pool Managers have on the pay for performance system? Are they fair in how they do these jobs?

12. What could Commerce do in the future to improve the link between an employee's pay and his/her performance?

Supervisory Pay

We have a couple of questions for you regarding supervisory performance pay. Does your group provide supervisory performance pay? If no, skip section.

13. To what degree does the supervisory performance pay intervention influence your decision to remain at Commerce?

14. In what ways, if any, could the supervisory performance pay intervention be improved?

Performance Bonuses 

Our next set of questions pertains to performance bonuses.

15. Do performance bonuses motivate employees to perform better?

16. Are performance bonuses distributed fairly (that is, based on performance)? If not, what other factors are considered?

17. What role does the first line supervisor have in influencing decisions about performance bonuses? Would you increase or decrease your role in this process?

B. Career Paths

As you know, within the Demonstration Project, positions were grouped into one of four career paths—ZP for scientists and engineers, ZT for scientific and engineering technicians, ZA for administration positions, and ZS for support positions. We have a question for you about how these career paths are working.

18. Is there any way you would change these career paths to make them more conducive to employees’ career progression?

Broadbanding refers to the way that the fifteen GS grades have been compressed into five broader bands. 

19. What impact, if any, has the broadbanding system had on your career progression?

C. Classification

We have a few questions about the job classification process.

20. What role do you have in job classification?

21. Are you using the automated classification system (ACS)? If yes, how efficient is the system?

D. Hiring/Recruitment

Our next set of questions pertains to hiring and recruitment.

22. What does Commerce do to attract and hire high quality candidates? Are any of these strategies unique to the Demonstration Project?

23. Have you noticed any differences in the past nine [four] years in your ability to attract and hire high quality candidates? If yes, to what do you attribute the difference? What techniques have worked well for you in attracting high quality candidates?

24. Have you noticed any differences in the quality of new hires in the past nine [four] years since the Demonstration Project began? If yes, to what do you attribute the difference?

25. Have you lost a qualified candidate because he/she was made a better offer somewhere else? If yes, did this offer come from a federal agency or a private sector organization? Do you know what factors led this candidate to choose the other opportunity?

26. What else could Commerce do to attract and hire high quality candidates?

(BREAK: 5-15 MIN)

E. Three-Year Probation for R&D

Our next set of questions pertains to the three-year probation period for ZP employees performing research and development work. Does your unit use the three-year probation period? [If no, skip to next section.]

27. If you have hired ZP employees performing R&D work, do you feel that you have the flexibility to terminate ZP employees performing research and development work who are covered by the three-year probation period?

28. Do you believe the three-year probation period is fair to employees hired under this initiative? Is it effective for you as a manager?

F. Turnover and Retention

Our next set of questions pertains to turnover and retention.

29. How much of an issue is turnover within your work unit? In what ways, if any, has this impacted morale in your work unit?

30. Why do people leave this organization?

31. Have you lost high performing employees to opportunities outside of the Federal Government?

32. In what ways are high performing employees encouraged to stay with Commerce?

33. Have any of the following personnel changes made as part of the Demonstration Project helped Commerce retain high performing employees? If so, which ones and how?

• performance-based pay increases
• performance-based bonuses
• potential for pay increases with promotion 
• retention bonuses

34. What else could Commerce do to retain high performing employees?

G. Quality of Workforce

The next few questions pertain to the quality of the workforce.

35. Thinking about the fact that the Demonstration Project was designed to improve performance, has the quality of work within your work unit improved since the Demonstration Project began? If yes, what impact have new hires had on this improvement?

36. Considering the employees hired during the Demonstration Project who you expected to be high performers, has their performance met your expectations? If yes, what impact are they having on the work unit? If no, why not?

37. What else could Commerce do to improve employee performance?

H. Organizational Excellence and Workforce Diversity (pass out listing of the nine Merit System Principles and the 12 Prohibited Personnel Practices)

We have several questions about organizational excellence and workforce diversity.

38. Have you noticed any changes in how the nine Merit System Principles are followed under the Demonstration Project versus how they are typically followed elsewhere in Commerce? If yes, how?

39. Have you noticed any changes in how the Prohibited Personnel Practices are addressed under the Demonstration Project versus how they are typically addressed elsewhere in Commerce? If yes, how?

40. Do you feel that any of the human resource practices implemented as part of the Demonstration Project have had a particularly negative impact on minorities, women, or veterans in the Demonstration Project? If yes, how?

41. Do you feel that any of the human resource practices implemented as part of the Demonstration Project have had a particularly positive impact on minorities, women, or veterans in the Demonstration Project? If yes, how?

42. Do you feel restricted by any of the human resources practices introduced as part of the Demonstration Project? If yes, which ones and how?

I. Outstanding Issues

43. Before we end today, do you have any other comments regarding the Demonstration Project that you would like us to know?

III. CLOSING 

In closing, we want to again mention that your comments will be held confidential. We will be combining information that we gather in the focus groups with information gathered from the survey and interviews. We will then prepare our Year Nine evaluation report and will submit it to Commerce and the Office of Personnel Management later this summer.

Thank you for participating in today's session. We appreciate your taking the time and sharing your ideas with us!

COMPARISON GROUP, NON-SUPERVISORS

---

I. INTRODUCTION/BACKGROUND

Ø Welcome And Overview Of Session

Hello everybody. My name is (facilitator) and this is (co-facilitator). We are from Booz Allen Hamilton, a management consulting firm in McLean, VA. We will be leading today’s discussion about the Department of Commerce's Personnel Management Demonstration Project. 

Ø Demonstration Project Background

We would like to start by briefly talking about the Demonstration Project. As you may know, Commerce initiated the Demonstration Project in March 1998 as a means of testing whether a series of alternative personnel practices could be more successful than traditional personnel practices in helping Commerce achieve its goals. A series of HR interventions were implemented in a subset of the organization. In 2003, this Demonstration Project was extended another five years to better understand the impact of these HR interventions.

The Demonstration Project includes two groups and our assessment compares human resource practices in these two groups. The first group is made up of those work units within Commerce where a series of human resources practices have been implemented under the Demonstration Project. We call this the Demo Group. The second group consists of Commerce work units similar to those participating in the Demonstration Project but who have maintained the traditional human resources practices. We call this the Comparison Group. All of you here today are part of the second group, the Comparison Group. So, your work units are still following the traditional HR policies and practices; they have not implemented the interventions that are being tested in other parts of Commerce.

Ø Booz Allen’s Role

Commerce contracted with Booz Allen to conduct a comprehensive program evaluation of the Demonstration Project’s performance in meeting its objectives. Booz Allen's role in the Demonstration Project is as an objective, third party evaluator. Booz Allen was not involved in the design of the Demonstration Project's new human resources practices. Commerce has been responsible for implementing the new practices, with oversight provided by the Office of Personnel Management. Booz Allen's job is to collect information and act as an impartial evaluator of how the new human resources practices are working.

Ø Purpose of the Focus Group

The intent of this focus group is to gain an understanding of specific human resource practices within the Department of Commerce from the perspective of employees. We would like to identify how these practices impact employees and their work units. Also we would like to hear from you about any past successes and/or possible future improvements to human resource practices. In particular, we will be talking about:

1. Performance Management
2. Career Progression
3. Hiring/Recruitment 
4. Turnover and Retention
5. Quality of Workforce
6. Organizational Excellence and Workforce Diversity

Information from this focus group will be combined with the information collected from the focus groups taking place in other locations. Booz Allen will then use all the focus group data, plus information from surveys, interviews, and other sources, to evaluate the ninth year of the Demonstration Project.

Ø Participant Selection

I imagine that you are interested in knowing how you were invited to join today's session. We were given a list of employees at this location and from this list we randomly selected names. For this focus group, we selected names of non-supervisors. To be sure that we sampled names correctly, I want to make sure everyone here belongs in this group. Is there anyone here who is a supervisor? (Facilitator's note: If so, thank them for their time and excuse them from the session. Note that we defined supervisors as those who conduct performance appraisals of others.)

(Facilitator's note: State the following only at sites in which it applies.) While we are here, we also will be conducting another focus group with supervisors and will be asking them similar questions.

We also are conducting focus groups at several other locations across the United States. We are meeting with employees in the Demo Group as well as the Comparison Group. (Facilitator's note: Bring your copy of the list showing where we are conducting focus groups in case anyone asks.)

Ø Focus Group Participant Introductions 

Before we continue, we would like to find out who you are. If you would, please tell us your name, your organization, and department, and how long you’ve been with Commerce.

Ø Confidentiality

Any information you share with us today will be held confidential. We will be aggregating results from all focus groups and will not be attributing comments to any particular groups or individuals.

We also ask that anything said in this group remains in this room. We want everyone to feel comfortable about talking, which means that we have to agree not to discuss what was said here today. Is everyone comfortable with that? 

Ø Timing 

Today’s focus group will last approximately 2 hours. We will take one 15-minute break during this session.

Are there any questions before we get started?

II. DISCUSSION 

A. Performance Management

Performance Appraisal Process

Our first set of questions pertains to the performance appraisal process.

1. Does the current performance appraisal process provide you with an appropriate amount of feedback regarding your job performance? If yes, how so? If not, what is missing?

2. How often do you receive formal or informal feedback from your supervisor? How often does that feedback help you improve your performance?

3. Do you feel that your performance is linked to the organizational mission? If yes, how?

4. Do you believe that the current performance appraisal process provides an accurate assessment of your performance?

5. Is it important for you to know how you are performing in relation to your peers? If so, does the current system provide you with this type of information?

6. Does the current performance appraisal process clearly identify the consequences of poor performance to employees, as well as the rewards of good performance?

7. Does the current performance appraisal process help to differentiate between high and low performing employees? If no, why not? 

8. Is the current performance appraisal process being implemented fairly by managers?

Pay and Performance

Next, I have a series of questions for you about pay and performance.

9. Would a pay-for-performance system – one in which level of performance drives amount of pay – motivate employees to perform better?

10. What could Commerce do in the future to improve the link between an employee's pay and his/her performance?

Awards 

Our next set of questions pertains to awards.

11. Does your work unit distribute awards? If so, what types of accomplishments result in awards?

12. Do awards motivate employees to perform better?

13. Are awards distributed fairly (that is, based on performance)? If not, what other factors are considered?

B. Career Progression

The next question relates to your ability to progress in your career.

14. Is there anything about the way career positions and occupations are structured that affects your opportunity to progress in your career as you would like?

C. Hiring/Recruitment

Our next few questions pertain to hiring and recruitment.

15. What does Commerce do to attract and hire high quality candidates?

16. Have you noticed any differences in the quality of new hires in the past nine [four] years? If yes, to what do you attribute the difference?

17. What else could Commerce do to attract and hire high quality candidates?

(BREAK: 5-15 MIN)

D. Turnover and Retention

Our next set of questions pertains to turnover and retention.

18. How much of an issue is turnover within your organization? In what way, if any, has this impacted morale in your work unit? 

19. Why do people leave this organization?

20. In what ways are high performing employees encouraged to stay with Commerce?

21. What else could Commerce do to retain high performing employees?

E. Quality of Workforce

The next few questions pertain to the quality of the workforce.

22. Has the quality of work within your work unit improved over the past nine [four] years? If yes, what impact have new hires had on this improvement?

23. What else could Commerce do to improve employee performance?

F. Organizational Excellence and Workforce Diversity (pass out listing of the nine Merit System Principles and the 12 Prohibited Personnel Practices)

The next few questions pertain to organizational excellence and workforce diversity.

24. Are the nine Merit System Principles followed in your work unit?

25. Are the twelve Prohibited Personnel Practices addressed in your work unit?

26. Do you feel that any of the human resource practices we have discussed have had a particularly negative impact on minorities, women, or veterans within Commerce? If yes, how?

27. Do you feel that any of the human resource practices we have discussed have had a particularly positive impact on minorities, women, or veterans within Commerce? If yes, how?

G. Outstanding Issues

28. Before we end today, do you have any other comments regarding Commerce human resources practices that you would like us to know?

III. CLOSING 

In closing, we want to again mention that your comments will be held confidential. We will be combining information that we gather in the focus groups with information gathered from the survey and interviews. We will then prepare our Year Nine evaluation report and will submit it to Commerce and the Office of Personnel Management later this summer.

Thank you for participating in today's session. We appreciate your taking the time and sharing your ideas with us!

COMPARISON GROUP, SUPERVISORS

---

I. INTRODUCTION/BACKGROUND 

Ø Welcome And Overview Of Session

Hello everybody. My name is (facilitator) and this is (co-facilitator). We are from Booz Allen Hamilton, a management consulting firm in McLean, VA. We will be leading today’s discussion about the Department of Commerce's Personnel Management Demonstration Project. 

Ø Demonstration Project Background

We would like to start by briefly talking about the Demonstration Project. As you probably know, Commerce initiated the Demonstration Project in March 1998 as a means of testing whether a series of alternative personnel practices could be more successful than traditional personnel practices in helping Commerce achieve its goals. A series of HR interventions were implemented in a subset of the organization. In 2003, this Demonstration Project was extended another five years to better understand the impact of these HR interventions.

The Demonstration Project includes two groups and our assessment compares human resource practices in these two groups. The first group is made up of those work units within Commerce where a series of human resources practices have been implemented under the Demonstration Project. We call this the Demo Group. The second group consists of Commerce work units similar to those participating in the Demonstration Project but who have maintained the traditional human resources practices. We call this the Comparison Group. All of you here today are part of the second group, the Comparison Group. So, your work units are still following the traditional HR policies and practices; they have not implemented the interventions that are being tested in other parts of Commerce.

Ø Booz Allen’s Role

Commerce contracted with Booz Allen to conduct a comprehensive program evaluation of the Demonstration Project’s performance in meeting its objectives. Booz Allen's role in the Demonstration Project is as an objective, third party evaluator. Booz Allen was not involved in the design of the Demonstration Project's new human resources practices. Commerce has been responsible for implementing the new practices, with oversight provided by the Office of Personnel Management. Booz Allen's job is to collect information and act as an impartial evaluator of how the new human resources practices are working.

Ø Purpose of the Focus Group

The intent of this focus group is to gain an understanding of specific human resource practices within the Department of Commerce from the perspective of supervisors. We would like to identify how these practices impact supervisors and their work units. Also we would like to hear from you about any past successes and/or possible future improvements to human resource practices. In particular, we will be talking about:

1. Performance Management
2. Career Progression
3. Classification
4. Hiring/Recruitment 
5. Turnover and Retention
6. Quality of Workforce
7. Organizational Excellence and Workforce Diversity

Information from this focus group will be combined with the information collected from the focus groups taking place in other locations. Booz Allen will then use all the focus group data, plus information from surveys, interviews, and other sources, to evaluate the ninth year of the Demonstration Project.

Ø Participant Selection

I imagine that you are interested in knowing how you were invited to join today's session. We were given a list of employees at this location and from this list we randomly selected names. For this focus group, we selected names of supervisors only. To be sure that we sampled names correctly, I want to make sure everyone here belongs in this group. Is there anyone here who is not a supervisor? (Facilitator's note: If so, thank them for their time and excuse them from the session. Note that we defined supervisors as those who conduct performance appraisals of others.)

(Facilitator's note: State the following only at sites in which it applies.) While we are here, we also will be conducting other focus group with non-supervisors and we will be asking them similar questions.

We are also conducting focus groups at several other locations across the United States. We are meeting with employees in the Demo Group as well as the Comparison Group. (Facilitator's note: Bring your copy of the list showing where we are conducting focus groups in case anyone asks.)

Ø Focus Group Participant Introductions 

Before we continue, we would like to find out who you are. If you would, please tell us your name, your organization, the department you work in, and how long you’ve been with Commerce.

Ø Confidentiality

Any information you share with us today will be held confidential. We will be aggregating results from all focus groups and will not be attributing comments to any particular groups or individuals.

We also ask that anything said in this group remains in this room. We want everyone to feel comfortable about talking, which means that we have to agree not to discuss what was said here today. Is everyone comfortable with that? 

Ø Timing 

Today’s focus group will last approximately 2 hours. We will take one 15-minute break during this session.

Are there any questions before we get started?

II. DISCUSSION 

A. Performance Management

Performance Appraisal Process

Our first set of questions pertains to the performance appraisal process.

1. Does the current performance appraisal process enable you to accurately evaluate the performance of your employees? If yes, how so? If not, what is missing?

2. Does the current performance appraisal process create an opportunity for you to provide your employees with relevant feedback regarding their performance? If yes, how? If no, how could it be improved?

3. From your perspective as a supervisor, is the current performance appraisal process easy or difficult to use? How so?

4. Does the current performance appraisal process clearly identify the consequences of poor performance to employees, as well as the rewards of good performance?

5. Does the current performance appraisal process help to differentiate between high and low performing employees? If no, why not?

6. Do you believe that average or lower performers are given guidance on how to improve performance? If so, what kind of guidance are they receiving?

Pay and Performance

Next, I have a series of questions for you about pay and performance.

7. How do you reward high performers?

8. Do factors other than performance contribute to an employee's salary? If yes, what are the other factors?

9. Would a pay-for-performance system – one in which level of performance drives amount of pay – help you to reward your high performers? If yes, how so? If no, what challenges do you foresee?

10. What could Commerce do in the future to improve the link between an employee's pay and his/her performance?

Supervisory Performance

Our next question pertains to supervisory performance.

Assume that after you've maxed out in pay, you were eligible to receive an additional pay raise. 

11.  To what degree would this supervisory performance pay intervention influence your decision to remain at Commerce? 

Awards

Our next set of questions pertains to awards.

12. Do awards motivate employees to perform better? 

13. Are awards distributed fairly (that is, based on performance)? If not, what other factors are considered?

14. What role does the 1st line supervisor have in influencing decisions about awards? Would you increase or decrease your role in this process?

B. Career Progression

The next question relates to your ability to progress in your career.

15. Is there anything about the way career positions are structured that affects your opportunity to progress in your career as you would like?

C. Classification

Our next two questions pertains to job classification.

16. What role do you have in job classification?

17. Do you think the current classification system works for your work unit? For the agency? In what ways, if any, could it be improved? 

D. Hiring/Recruitment

Our next set of questions pertains to hiring and recruitment.

18. What does Commerce do to attract and retain high quality candidates?

19. Have you noticed any differences in the past nine [four] years in your ability to attract and hire high quality candidates? If yes, to what do you attribute the difference?

20. Have you noticed any differences in the quality of new hires in the past nine [four] years? If yes, to what do you attribute the difference?

21. Have you had any difficulty recruiting high quality candidates as a result of personnel policies? If so, what type of policies have affected you?

22. Have you lost a qualified candidate because he/she was made a better offer somewhere else? If yes, did this offer come from a federal or a private sector organization? Do you know what factors led this candidate to choose the other opportunity?

23. What else could Commerce do to attract and hire high quality candidates?

(BREAK: 5-15 MIN)

E. Turnover and Retention

Our next set of questions pertains to turnover and retention.

24. How much of an issue is turnover within your work unit? In what ways, if any, has this impacted morale in your work unit?

25. Why do people leave this organization?

26. Have you lost high performing employees to opportunities outside of the Federal Government?

27. Have you had any difficulty retaining high quality employees as a result of personnel policies? If so, what type of policies have affected you?

28. In what ways are high performing employees encouraged to stay with Commerce?

29. What else could Commerce do to retain high performing employees?

F. Quality of Workforce

The next few questions pertain to the quality of the workforce.

30. Has the quality of work within your work unit improved over the past nine [four] years? If yes, what impact have new hires had on this improvement?

31. Considering the employees hired in the past nine [four] years who you expected to be high performers, has their performance met your expectations? If yes, what impact are they having on the work unit? If no, why not?

32. What else could Commerce do to improve employee performance?

G. Organizational Excellence and Workforce Diversity (pass out listing of the nine Merit System Principles and the 12 Prohibited Personnel Practices)

We have a few questions about organizational excellence and workforce diversity.

33. Are the nine Merit System Principles followed in your work unit? 

34. Are the twelve Prohibited Personnel Practices addressed in your work unit?

35. Do you feel that any of the human resource practices we have discussed have had a particularly negative impact on minorities, women or veterans within Commerce? If yes, how?

36. Do you feel that any of the human resource practices we have discussed have had a particularly positive impact on minorities, women or veterans within Commerce? If yes, how? If no, why not?

H. Outstanding Issues

37. Before we end today, do you have any other comments regarding Commerce human resources practices that you would like us to know?

III. CLOSING 

In closing, we want to again mention that your comments will be held confidential. We will be combining information that we gather in the focus groups with information gathered from the survey and interviews. We will then prepare our Year Nine evaluation report and will submit it to Commerce and the Office of Personnel Management later this summer.

Thank you for participating in today's session. We appreciate your taking the time and sharing your ideas with us!