Model Practice Title
-
Please provide the name or title of your practice
Measuring Results Related to Coaching Employees to Higher Performance
Practice Categories
-
Model and Promising Practices are stored in an online searchable database. Applications may align with more than one practice category. Please select all the practice areas that apply.
- Access to Care
- Advocacy and Policy Making
- Animal Control
- Coalitions and Partnerships
- Communications/Public Relations
- Community Involvement
- Cultural Competence
- Emergency Preparedness
- Environmental Health
- Food Safety
- Global Climate Change
- Health Equity
- HIV/STI
- Immunization
- Infectious Disease
- Informatics
- Information Technology
- Injury and Violence Prevention
- Marketing and Promotion
- Maternal-Child and Adolescent Health
- Organizational Practices
- Other Infrastructure and Systems
- Organizational Practices
- Primary Care
- Quality Improvement
- Research and Evaluation
- Tobacco
- Vector Control
- Water Quality
- Workforce
- Conference Theme: Bridging Clinical Medicine and Population Health
-
Other:
-
Is this practice evidence based, if so please explain.
Annual employee performance reviews are considered standard practice within human resources and business management disciplines. A systematic review of employee appraisal models within public health has not been done to determine if coaching is an evidence-based practice. However, there is considerable social science evidence within the context of human resources to judge coaching a best practice for employee development.
Winnable Battles
-
To keep pace with emerging public health challenges and to address the leading causes of death and disability, CDC initiated an effort called Winnable Battles to achieve measurable impact quickly.Winnable Battles are public health priorities with large-scale impact on health and known effective strategies to address them. Does this practice address any CDC's seven Winnable Battles? If so, please choose from the following:
- Food Safety
- HIV in the U.S.
- Nutrition, Physical Activity, and Obesity
- Tobacco
- Healthcare-associated Infections
- Motor Vehicle Injuries
- Teen Pregnancy
- None
Overview: Provide a brief summary of the practice in this section (750 Word Maximum)
Your summary must address all the questions below:
- Brief description of LHD- location, demographics of population served in your community
- Describe public health issue
- Goals and objectives of the proposed practice
- How was the practice implemented/activities
- Results/Outcomes (list process milestones and intended/actual outcomes and impacts.
- Were all of the objectives met?
- What specific factors led to the success of this practice?
- Public Health impact of practice
- Website for your program, or LHD.
750 Word Maximum
-
Please use this portion to respond to the questions in the overview section.
OVERVIEW
Description of Local Health Department
The Lawrence and Douglas County community has benefited from a strong public health presence for more than 120 years. Achieving national accreditation through the Public Health Accreditation Board (PHAB) in May 2015, the Lawrence-Douglas County Health Department’s (LDCHD) mission is to advance policies, practices and programs that promote health for all, prevent disease and protect the environment. LDCHD serves a county population of 116,000 residents, the fifth largest in Kansas. Most residents, 91,000, live in Lawrence while the smaller communities of Eudora and Baldwin City have populations of 6,300 and 4,600, respectively. Douglas County is slightly more racially diverse than the state as a whole with 84% of residents identifying as white compared with 87% of residents statewide.
With three universities in the county, Douglas County has the distinction of half of its residents 25 years and older attaining at least a bachelor’s degree compared with just 30% of the Kansas adult population. The university populations also contribute to the county’s age distribution. The median age of Douglas County residents is 28 years whereas the state median age is 36 years. Douglas County has a low unemployment rate with an estimated 3.4% reported for September 2016. Median household income of $51,000 is slightly below the Kansas median household income of $52,000. These factors influence hiring decisions at the LDCHD; our workforce of 45 staff is highly educated with two individuals holding doctorate degrees and another 15 with master’s degrees.
A full description of the Lawrence-Douglas County Health Department can be found on our web site, www.ldchealth.org.
Public Health Issue
Maintaining a competent public health workforce is one of the public health functions addressed in the PHAB Standards and Measures. Effective public health service delivery is facilitated by a skilled workforce able to develop programs, build collaborations and evaluate outcomes in addition to addressing health equity and population health issues. To recruit and retain qualified staff, it is imperative that LDCHD is strategic in aligning workforce development efforts with our agency’s mission, overall goals and needs. One element of a strong workforce development program is an effective staff evaluation and appraisal system.
Although the LDCHD had a good employee appraisal system in place, LDCHD contracted with a local human resources consultant in October 2012 to address a number of needs. In addition to assessing LDCHD’s performance management practices for employees, the consultant reviewed position skills and compensation, audited operational policies, and evaluated salary administration. The LDCHD wanted to enhance the employee appraisal process and encourage ongoing open communication and feedback between supervisors and employees while increasing performance accountability. Because LDCHD values ongoing learning, we were interested in an employee appraisal system that encouraged and rewarded learning new skills as outlined the LDCHD’s Workforce Development Plan. This plan states that LDCHD must have a competent workforce with the skills necessary to perform assigned duties and carry out the agency’s mission. Development of staff is required to ensure adequate level of competence to perform current and future public health work. Furthermore the core competencies identified by the Council on Linkages between Academia and Public Health Practice was the basis of LDCHD’s staff assessment for workforce development needs.
Goals and Objectives
In August 2013 the LDCHD transitioned from an employee review process that mirrored essential functions of staff job descriptions to our current Performance Appraisal System (PAS) designed as a coaching model and focused on future performance. The goals and objectives of the new PAS are to:
Goal 1: Implement system of employee performance review using a coaching model
Objective 1: Improve employee satisfaction with agency performance review practices by increasing consistency of performance ratings
Objective 2: Incentivize employees to learn additional job skills and competencies through employee generated goal-setting
Objective 3: Facilitate improved communication between supervisors and team members through focus on employee strengths
Goal 2: Build stronger connections between agency core values and employee performance
Objective 1: Incorporate demonstration of core values into annual employee performance review
A comprehensive evaluation of the PAS was conducted from August through October 2106. The evaluation showed a high degree of consistency between supervisor feedback and team member creation of development goals. There was also a marked increase in goal complexity and adaptive focus rather than technical focus from 2014 to 2015.
Local health departments are challenged with preparing their workforce to carry out activities related to public health transformation. An effective performance appraisal system can help guide and create action for skill development.
Responsiveness and Innovation
A Model Practice must be responsive to a particular local public health problem or concern. An innovative practice must be (1) new to the field of public health (and not just new to your health department) OR (2) a creative use of an existing tool or practice, including but not limited to use of an Advanced Practice Centers (APC) development tool, The Guide to Community Preventive Services, Healthy People 2020 (HP 2020), Mobilizing for Action through Planning and Partnerships (MAPP), Protocol for Assessing Community Excellence in Environmental Health (PACE EH). Examples of an inventive use of an existing tool or practice are: tailoring to meet the needs of a specific population, adapting from a different discipline, or improving the content.
- Statement of the problem/public health issue
- What target population is affected by problem (please include relevant demographics)
- What is the target population size?
- What percentage did you reach?
- What has been done in the past to address the problem?
- Why is the current/proposed practice better?
- Is current practice innovative? How so/explain?
- Is it new to the field of public health
OR
- Is it a creative use of existing tool or practice:
What tool or practice did you use in an original way to create your practice? (e.g., APC development tool, The Guide to Community Preventive Services, HP 2020, MAPP, PACE EH, a tool from NACCHO’s Toolbox etc.)
- Is the current practice evidence-based? If yes, provide references (Examples of evidence-based guidelines include the Guide to Community Preventive Services, MMWR Recommendations and Reports, National Guideline Clearinghouses, and the USPSTF Recommendations.)
2000 Word Maximum
-
Please state the Responsiveness and Innovation of your practice (2000 Word Maximum)
Statement of the Problem
The Council on Linkages between Academia and Public Health Practice works to further collaboration between these two fields to ensure a well-trained and competent workforce. This collaborative focuses on improving public health education and training, practice, and research. A consensus set of skills for public health practice is defined by eight domains and applied across the country as a foundation for building a strong evidence base for public health practice. The Council’s Strategic Directions, 2016-2020, has as one of its objectives to support the development of a diverse, highly-skilled and motivated public health workforce with the competence and tools to succeed. Strategies for achieving this objective include addressing life-long learning needs to prepare public health workers to meet the leadership challenges of health systems transformation.
A recent study of public health workers, Recruitment and Retention: What’s Influencing the Decisions of Public Health Workers?, found that linking workers to agency mission and vision positively influences employment decisions. Strong organizational leadership and management skills were also found to impact worker satisfaction. More than a quarter of respondents in this 2016 national survey either somewhat or strongly disagreed that they received constructive feedback to help them improve their performance. The same proportion also disagreed that performance evaluations were held in an appropriate manner and were consistent. To ensure an effective workforce, fostering development of public health competencies and skills as outlined by the Council on Linkages could benefit public health agencies. Furthermore, efforts to strengthen leadership and management skills could improve organizational environments and retain employees.
Systems to evaluate employee performance are an opportunity to not only affect employee satisfaction and retention but also identify and build employee skills necessary for carrying out activities related to public health transformation. To be most effective these systems need to grow and evolve to meet the current, ever changing job demands of the public health workforce. In an effort to contribute to evidence of what makes for effective employee performance evaluations and supervisor feedback, the LDCHD evaluated their new system of employee annual reviews based on a coaching model and put in place in 2013. The coaching model of employee development represents a major overhaul in organizational practice and replaced a system in place for more than a decade.
Discussion of Past Employee Review Practice and Value of Innovative Coaching Model
LDCHD’s previous system of employee annual performance review emphasized past performance rather than what the employee could do to improve in the future. The evaluation form itself focused on the tasks outlined in the employee’s job description and created in the minds of employees that they were expected to be perfect, a notion that was reinforced by the rating system where to get an “exceeds expectations” rating an employee needed to perfectly perform 90% of their assigned tasks. In such a system employees rarely hear the valuable improvement messages from their supervisors; rather, they see only the numbers. Employees do not always take responsibility for poor performance and the annual review does not produce the desired outcome of improvement in staff performance.
Additional challenges with this system of annual performance review was the lack of consistency across supervisors where one manager awards all high scores and others rarely give high scores. The problem of consistency was aggravated by the fact that reviews were conducted throughout the year on the employee’s work anniversary date making it even more difficult for supervisors to compare and rank staff on the same team. The agency’s objective to create a sense of fairness and unify staff around agency strategic goals, core values and mission often fell short.
Rather than simply tweaking our system of employee performance reviews, LDCHD decided to embark on a major overhaul and new direction. Starting July 1, 2013 the LDCHD suspended its former system of annual reviews and began using a coaching model as described in Garold Markle’s book, Catalytic Coaching. This system of employee performance review is based on employee strengths, organizational values, and on-going quarterly coaching meetings with supervisors. The focus is on employee future performance and building skills and competencies through employee-directed goal setting and personal development plans. This innovative approach has allowed the LDCHD to better address the gap between current staff skills and those skills and competencies identified by the Council on Linkages such as systems thinking, data analysis, and community partnership formation necessary for effective delivery of public health services.
The entire LDCHD workforce of 45 participates in the performance appraisal system (PAS). There are 11 supervisors who have from one to eight direct reports; these individuals have primary responsibility for carrying out the PAS. Most employees (31) are classified as non-exempt while 14 have exempt status. More than half of the staff (51 percent) has been employed with the department fewer than five years. While several of these individuals have previous work experience in another public health agency, most are new to the field of public health. With a third of the staff reporting 10 or more years of employment, LDCHD enjoys a significant level of institutional knowledge and public health leadership.
Annual employee performance reviews are considered standard practice within human resources and business management disciplines. A systematic review of employee appraisal models within public health has not been done to determine if coaching is an evidence-based practice. However, there is considerable social science evidence within the context of human resources to judge coaching a best practice for employee development. The Catalytic Coaching model of employee development and the performance appraisal system described in this application is about helping people to grow. The goals of coaching are consistent with the values and philosophy of the LDCHD and therefore the practice was deemed an intervention worthy of implementation. The evaluation of the PAS to determine impact on the goals and objectives of the coaching model found that this system of performance appraisal was effective in influencing staff development in ways that are beneficial to the LDCHD.
LHD and Community Collaboration
The LHD should have a role in the practice’s development and/or implementation. Additionally, the practice should demonstrate broad-based involvement and participation of community partners (e.g., government, local residents, business, healthcare, and academia). If the practice is internal to the LHD, it should demonstrate cooperation and participation within the agency (i.e., other LHD staff) and other outside entities, if relevant. An effective implementation strategy includes outlined, actionable steps that are taken to complete the goals and objectives and put the practice into action within the community.
- Goal(s) and objectives of practice
- What did you do to achieve the goals and objectives?
- Steps taken to implement the program
- Any criteria for who was selected to receive the practice (if applicable)?
- What was the timeframe for the practice
- Were other stakeholders involved? What was their role in the planning and implementation process?
- What does the LHD do to foster collaboration with community stakeholders? Describe the relationship(s) and how it furthers the practice goal(s)
- Any start up or in-kind costs and funding services associated with this practice? Please provide actual data, if possible. Otherwise, provide an estimate of start-up costs/ budget breakdown.
5000 words maximum
-
Enter the LHD and Community Collaboration related to your practice (5000 words maximum)
Practice Goals and Objectives
The primary objective of performance reviews is to improve employee performance. The LDCHD’s philosophy is that when employees are clear about what is expected of them in their work role not only does individual staff performance improve, so does the overall performance of the agency. The public health workforce continues to evolve and new skills are needed to be effective in executing Public Health 3.0 activities.
In 2012 the LDCHD contracted with a local human resources company to review several human resources practices including the system for performance reviews. Challenges of the employee appraisal system in place at the time included burden on supervisors due to the amount of time necessary to complete each review as well as lack of consistency in applying rating system. Feedback from management and supervisors indicated that performance evaluations did not always produce the desired results since employees did not tend to accept responsibility for their performance. In general, the appraisal system did not function to incentivize employees to a higher level of performance.
After intensive work with the HR consultant to assess agency needs and desired changes, the LDCHD officially launched a revised Performance Appraisal System (PAS) in August 2013 based on the coaching model of employee development. A coaching model of staff development is future oriented and focuses on building new skills.
The goals and objectives of the new PAS included the following:
Goal 1: Implement system of employee performance review using a coaching model
Objective 1: Improve employee satisfaction with agency performance review practices by increasing consistency of performance ratings
Objective 2: Incentivize employees to learn additional job skills and competencies through employee generated goal-setting
Objective 3: Facilitate improved communication between supervisors and team members through focus on employee strengths
Goal 2: Build stronger connections between agency core values and employee performance
Objective 1: Incorporate demonstration of core values into annual employee performance review
Implementation Steps
The consultant worked with agency management and supervisory staff to review goals of the new PAS and process for transitioning from the former appraisal system. Under the former system, employees received appraisals throughout the year on their work anniversary; with the new PAS all employees receive an appraisal at the same time of year. Forms for the PAS followed the Catalytic Coaching model and were customized to match the needs of LDCHD. The Supervisor Feedback form, in particular, was customized to include LDCHD core values as well as competency areas such as customer service and flexibility that reflect principles important to LDCHD for staff performance. To ready staff for the change, employees received information about the new appraisal system from the agency director through the Director’s Message in the staff newsletter in February 2013 during the early stages of the work and again in August 2013 as the PAS was being rolled out.
Supervisors were updated and participated in discussions early on. At the February 2013 Leadership Team meeting each supervisor was provided a copy of the book Catalytic Coaching to assist with learning and understanding the coaching framework for employee development. Supervisors were being asked to move from grading employees past work to coaching employees to a higher level of future performance. The consultant provided supervisory training on July 23 and staff training on writing goals on August 6 and August 13. The use of PAS forms and processes began in August 2013.
The PAS consists of four components:
Employee Input – employee prepares and presents to supervisor verbally. Supervisor’s job is to listen and ask clarifying questions. The Employee Input form asks employees to respond to questions about strengths, disappointments, career aspirations, and contributions made to the agency lately, such as learning a new skill or building a new collaborative relationship. The focus of this component is to equip supervisors to develop team members through listening, understanding and answering questions. When supervisors do these things, team members will more likely reach their performance capabilities.
Supervisor Feedback – reflection of staff strengths, areas for improvements (are there current skills that could be expanded, are there new skills to add), and development recommendations for the coming year. Also includes assessment of how well the employee demonstrated the LDCHD’s four core values (professionalism, compassion, teamwork, and integrity) and competencies (flexibility, customer service, communication, etc.). Each value or competency was rated as “On Target”, “Above & Beyond”, or “Needs to Improve.” The supervisor must provide justification for rating of “Above & Beyond”, or “Needs to Improve.” The supervisor also assesses level of completion of employee’s personal development goals from the previous year.
Personal Development Plan – based on supervisor feedback on areas for improvement and development recommendations, employee prepares two to four goals with action plans for completion over the next year. The principle utility of the development plan is to identify and build new skills and competencies through goals that employees create and supervisors approve. The idea is that employees are then more accountable for their own performance.
Ongoing Coaching – three times during the year the employee provides written progress on his/her goals and then meets one-on-one with supervisor to report progress to date. Coaching sessions are characterized as a respectful discussion to problem solve next steps in goal implementation and receive suggestions from supervisor about helpful tools or resources. Feedback should be an on-going and continuous activity between supervisors and team members.
Support for agency supervisors during the development and rollout of the PAS was provided through regularly scheduled monthly Leadership Team meetings. Most Leadership Team agendas for 2013 included some element of the HR project on the agenda. Multiple opportunities were provided for supervisors to learn, voice questions and participate in discussions with the Director and their peers. In September supervisors shared experiences with implementing the new PAS including what went well and team member reactions and questions about the new PAS. A follow-up meeting was also held with the consultant on November 6, 2013. The purpose of the meeting was to identify overall positive experiences, challenges encountered and recommendations for enhancements. Supervisors identified as positives the level of staff engagement in the conversation and their willingness to share their thoughts, concerns and perspectives. Because all managers and departments are utilizing the same evaluations forms and process, another positive noted was increased consistency and unity across the agency. Supervisors noted that behavioral goals are more difficult to measure and quantify than process goals. They identified creating meaningful goals as an opportunity for future staff training. Regularly communicating employees’ successes was a future addition mentioned by supervisors.
The LDCHD management team has responsibility for ensuring consistent implementation of the PAS. During the first and second year, the management team reviewed drafts of the Supervisor Feedback forms prior to supervisors sharing results with their team members. This allowed for a review of intra program ratings of employees as well as an assessment of inter organization ratings by comparing ratings across supervisors. This practice brought to light differences among supervisors with some supervisors liberally using the rating “Above & Beyond.” In such instances, supervisors were provided guidance on agency norms for the rating system. Guidance was also provided regarding the need for clarity when providing written feedback for team members. Some comments were found to be too brief to be meaningful or too vague to provide helpful direction for team members as they prepared their Personal Development Plans.
Monitoring Impact on Employee Satisfaction
In July of each year, the LDCHD surveys its staff to gather input on job satisfaction. The proportion of staff rating their supervisor as “Good” or “Very Good” increased from 85% in 2012 to 93% in 2013 and then to 94% in 2014 before falling again to 85% in 2015. The increased level of satisfaction in 2013 cannot be attributed to the PAS since that process was implemented after gathering staff survey responses. However when staff were asked for their level of agreement with the statement “My supervisor provides constructive feedback about my performance”, results showed a significant jump in agreement from 77% in 2013 to 91% 2014 a year after the new coaching model was put in place. That level of staff agreement was not sustained in the 2015 survey however, where only 73% of staff agreed that they received constructive feedback from supervisors on performance. The LDCHD also tracks employees’ overall satisfaction with their job. The percent of staff indicating they are “Satisfied” or “Very Satisfied” with their job ranges from 78% in 2011 to a high of 96% in 2014 before dropping to 88% and 85% in 2015 and 2016, respectively. While all of these ratings show a high level of staff satisfaction, the results are inconclusive as to the impact of the PAS and the shift to a coaching model on levels of satisfaction.
Consultant Selection and Costs
The LDCHD prepared a request for proposals (RFP) that was distributed regionally to 12 human resource consulting firms; eight responses were received. Proposals were reviewed by the Director of Administrative Services and Assistant Director. References were contacted and two top proposals participated in conference calls to review the proposal and timeline. The Health Board received staff recommendations and approved contractor selection. The LDCHD Director signed the contract for services in the amount of $16,800. This amount included work related to PAS development and training as well as a review of position skills and compensation and audit of HR policies and procedures.
Evaluation
Evaluation assesses the value of the practice and the potential worth it has to other LHDs and the populations they serve. It is also an effective means to assess the credibility of the practice. Evaluation helps public health practice maintain standards and improve practice. Two types of evaluation are process and outcome. Process evaluation assesses the effectiveness of the steps taken to achieve the desired practice outcomes. Outcome evaluation summarizes the results of the practice efforts. Results may be long-term, such as an improvement in health status, or short-term, such as an improvement in knowledge/awareness, a policy change, an increase in numbers reached, etc. Results may be quantitative (empirical data such as percentages or numerical counts) and/or qualitative (e.g., focus group results, in-depth interviews, or anecdotal evidence).
- What did you find out? To what extent were your objectives achieved? Please re-state your objectives.
- Did you evaluate your practice?
- List any primary data sources, who collected the data, and how (if applicable)
- List any secondary data sources used (if applicable)
- List performance measures used. Include process and outcome measures as appropriate.
- Describe how results were analyzed
- Were any modifications made to the practice as a result of the data findings?
2000 Words Maximum
-
Please enter the evaluation results of your practice (2000 Words Maximum)
As LDCHD approached the end of its third year using a coaching model for appraising employee performance, the management team perceived increasing levels of frustration by employees regarding the overall process and effectiveness of the PAS. Supervisors commented that some team members were not seriously engaged in the process. Team members created goals that were easily achievable and not complex to implement; this seemed counter to the objective of stretching employees to build new skills and competencies. A common situation reported was a feeling of being ill-prepared for coaching sessions. Staff would report that little to no progress was made on goals from one coaching session to the next due to demands of day-to-day job duties.
To address these concerns and assess if the goals and objectives of the PAS were being met, LDCHD completed an evaluation of the PAS. While the overall stated goals of the PAS have been met – a system of performance review using a coaching model was implemented and agency core values incorporated into the performance review thereby creating a stronger connection between the two – it is less clear that the following objectives have been met:
Objective 1: Improve employee satisfaction with agency performance review practices by increasing consistency of performance ratings
Objective 2: Incentivize employees to learn additional job skills and competencies through employee generated goal-setting
Objective 3: Facilitate improved communication between supervisors and team members through focus on employee strengths
Therefore, a comprehensive evaluation of the PAS was conducted from August through October 2106. The LDCHD collaborated with a graduate student earning a Master’s in Health Services Administration who planned and carried out the evaluation. The primary research question asked was: Given our objectives, is the PAS leading to real and identifiable employee growth?
The evaluation aimed to measure and create a baseline for PAS effectiveness and identify areas for improvement. The three-part analysis included:
1. Separate non-supervisor and supervisor PAS satisfaction surveys
2. Non-supervisor staff focus groups
3. Content analysis of Year 1 and Year 2 PAS documentation
Satisfaction Survey Methodology and Results
To assess general satisfaction with the PAS, surveys were distributed to all staff having sufficient experience with the PAS process, defined as employees employed by LDCHD at least one year and had participated in a PAS cycle. Separate non-supervisor and supervisor surveys were used; however, questions between the two surveys had alignment allowing for comparison between supervisors and non-supervisors. The non-supervisor survey had nine questions; 21 or 26 surveys were returned for an 81% response rate. Among eligible supervisors, six of eight (75%) responded to the eight-question survey.
Survey respondents indicated their level of agreement with a series of statements about the PAS using a five-point Likert scale ranging from 1= Strongly disagree to 5 = strongly agree. Non-supervisor staff responded to statements such as:
1. I am overall satisfied with the LDCHD Performance Appraisal System (PAS).
2. My supervisor has been effective in helping me create my PAS goals
3. The goals I have created through PAS are difficult, but achievable.
4. I am satisfied with the goals I have created through PAS.
5. My supervisor has provided me with helpful resources to stay on track and meet my goals.
While supervisors responded to:
1. I am overall satisfied with the LDCHD Performance Appraisal System (PAS).
2. I make a great effort to ensure that I am helpful in ensuring my employees meet their PAS goals.
3. My employee’s goals they have created through PAS are difficult, but achievable.
4. I am satisfied with the goals my employees have created through PAS.
5. My employees ask for resources or help when needed to stay on track or meet their goals.
Non-supervisor staff indicated slightly higher levels of agreement with statements about the PAS than supervisors with an average score across all statements of 3.4 compared with an overall score among supervisors of 3.2. Average scores to all statements for both supervisors and non-supervisors were either neutral or agree.
When comparing supervisors and non-supervisors, the highest level of consensus was with statement 3 above, with an average agreement score of 3.3 reported by both groups. The largest differences between the two groups were responses to statements 4 and 5. Supervisors’ level of agreement with the statements were 2.8 and 2.5, respectively, while non-supervisors reported higher levels of agreement with the statements as 3.5 and 3.4, respectively.
Focus Group Methodology and Results
The second element of the analysis involved two focus groups with a total of eight individuals. Staff invited to participate were non-supervisors who had who were employed by LDCHD at least one year and had participated in a PAS cycle. To balance perspectives, efforts were made to assign staff from different program areas and different supervisors to each focus group. Focus groups were scheduled for one hour and held at LDCHD during business hours. The student intern implementing the evaluation facilitated the focus groups and summarized results. The goals of the focus groups discussion were to reach consensus, identify common themes and add depth of understanding staff perspective to the other parts of the analysis. Question asked in the focus groups included:
1. What are your thoughts and feelings towards current PAS process?
2. What do you like best about the PAS program?
3. What do you like least about the PAS program?
4. What elements of PAS program need improvement?
5. What would you like to see change about the PAS program?
6. Have we missed anything or is there anything else that you would like to talk about?
Negative themes about the PAS arising from the focus group discussion included the sense that the PAS process is confusing and complex. Staff also voiced that they disliked feeling pressure from the health department; the process was viewed as stressful and staff felt they needed more time to work on implementing goals in their personal development plans. The need for better communication about the PAS was noted along with staff stating they did not understand the connection of the PAS to merit pay.
Focus group participants also identified positive aspects of the PAS. Employees indicated that they liked having the freedom to set their own goals and found the PAS motivating. Focus group participants also said a positive aspect of the PAS was the opportunity to engage in “shared effort” with their supervisor. The PAS was identified as a vehicle to “look forward” and foster passion about work. Finally, the PAS was identified as an opportunity to lead. Respondents expressed varied feelings about the PAS. For example, some enjoyed the opportunity to expand beyond their everyday role while others did not.
Content Analysis Methodology and Results
The content analysis compared Year 1 (2013-2014) and Year 2 (2014-2015) PAS documentation materials. Actual PAS documents completed by LDCHD supervisors comprised the raw data for the analysis. The Assistant Director created a data file by randomly selecting staff from each supervisor who met the criteria of being employed with LDCHD for at least one year and having completed at least one full PAS cycle. As a result PAS documents were collected from 24 employees across nine supervisors; two supervisors did not have direct reports meeting the selection criteria and are not included. To reduce bias, employee names and other identifying information were redacted by the Assistant Director prior to sharing the data file with the student intern who completed the qualitative content analysis. Employee files were identified by number only. To further minimize bias, groupings of employees by supervisor was shared only after the content analysis was completed. The number of PAS employee documents selected per supervisor ranged from two to three. The data file included two years of PAS documentation for 21 employees and one year of documentation for three of the selected employees. The rubric for analyzing the documents was tested on data from three employees; results were reviewed and rubric approved by the Assistant Director prior to continuing with the full analysis.
The content analysis included several segments. The first examined the concept of technical work (defined solution with clear steps) verses adaptive work (undefined solution with unclear steps). Two sections of the Supervisor Feedback form - Areas for improvements and Development recommendations for the coming year – were assessed as to whether the feedback supervisors provided employees was adaptive or technical. Likewise, goals created by team members were assessed as being either technical or adaptive. The basis for this analysis is that adaptive work is likely to create more growth in employee skills and capacity than technical work. Results of the content analysis showed that in both Year 1 and Year 2 supervisors provided feedback for improvement areas related to adaptive work in the vast majority of instances (85% of the time in Year 1 and 92% of the time on Year 2). Employees also were more likely to make adaptive goals than technical goals and increased adaptive goal setting from 50% in Year 1 to 77% in Year 2.
Another segment analyzed was the concept of simple work verses complex work. Complex work is defined as work requiring ongoing activities, involving multiple steps and significant time to complete; simple work is the opposite. The concept premise is that complex work often requires skills in planning, problem-solving and collaboration; these skills are consistent with the type of skill development desired by LDCHD to address population health needs. Feedback and goals were determined to be simple, medium or highly complex. While supervisors rarely recommended areas for improvement that involved simple goals, they were less likely to recommend complex goals in Year 2 than in Year 1. Team members, on the other hand, increased both complex and simple goals in Year 2. These results may reflect the realization that goal implementation is hard work and it takes time. The rate of goal completion improved in Year 2 to 61% compared with 55% goal completion in Year 1.
The concept of congruency was the third segment analyzed. Congruency is defined as the level of agreement between feedback provided by supervisors and goals prepared by team members. Higher levels of congruency are reasoned to represent better communication between supervisors and team members as team members were better able to receive guidance and interpret supervisor feedback. Congruency was rated as low, medium or high. In Year 1 congruency between supervisor feedback and employee goals was rated as low 43% of the time. In Year 2 low congruency fell to 24%. High levels of congruency increased from 49% in Year 1 to 57% in Year 2.
Conclusions and Recommendations
The objectives of LDCHD to implement a performance appraisal system that can create action for skill development have been mostly met through the PAS coaching model. Supervisor and non-supervisor satisfaction with the PAS as evaluated by the satisfaction survey and focus groups indicates a moderate level of satisfaction. The evaluation points to areas that could be improved, such as further communication about the PAS process and goals that will likely lead to higher levels of satisfaction among supervisors and team members alike. Results from all components of the analysis support the idea that additional job skills and competencies were realized through employee generated goal-setting. The content analysis showed a high tendency among supervisors to challenge team members through feedback that leads to adaptive, complex work activities and team members met the challenge by creating like goals. Focus group results highlighted the positive aspects of the PAS including being motivational and future oriented. The PAS also facilitated improved communication between supervisors and team members as shown by the improved level of congruency between supervisor feedback and goal generation. The satisfaction survey and focus groups speak to supervisor helpfulness and positive involvement during coaching sessions. Greater successes in employee development are possible through additional supervisor training related to coaching employees for future performance rather than grading them for past performance. The PAS not only reinforces messaging of agency culture but also builds culture through the active process of coaching and personal goal implementation.
Sustainability
Sustainability is determined by the availability of adequate resources. In addition, the practice should be designed so that the stakeholders are invested in its maintenance and to ensure it is sustained after initial development (NACCHO acknowledges that fiscal challenges may limit the feasibility of a practice's continuation.)
- Lessons learned in relation to practice
- Lessons learned in relation to partner collaboration (if applicable)
- Did you do a cost/benefit analysis? If so, describe.
- Is there sufficient stakeholder commitment to sustain the practice?
- Describe sustainability plans
1500 Words Maximum
-
Please enter the sustainability of your practice (2000 Words Maximum)
Regular, systematic review of employee performance is a solid business practice. Just as other organizational attributes from use of technology to customer needs should be brought up-to-date to align with new challenges and knowledge, so too should an employee appraisal system. A major benefit of switching to a coaching model was that such a model of employee development aligned with LDCHD’s strategic goal to build a workforce ready to participate in activities related to public health transformation. The adaptive work characterizing public health transformation is cultivated by the LDCHD’s PAS coaching model that encourages ongoing open communication and feedback between supervisors and employees while increasing performance accountability through goal setting.
LDCHD’s evaluation of its coaching model for employee development assists in the sustainability of this practice by reinforcing what is working and pointing to areas for improvement. Evaluation results showed the intended connection between supervisor feedback and employee goal creation. Furthermore, employees created complex goals focused on adaptive work processes and employees were generally satisfied with the PAS process. These positive results support continuation of the PAS.
Addressing areas for improvement is part of LDCHD’s plan for sustainability. Improvements will focus on communication about the PAS as well as additional supervisor training aimed at enhancing coaching skills. Communicating the purpose and goals of the PAS and how this process fits with the agency’s vision for employee expectations needs to be carefully executed to reduce misinformation. Tools the LDCHD is developing currently such as an employee intranet web site and enhanced onboarding practices will become important vehicles for communicating to all staff about the PAS.
The LDCHD is committed to sustaining the practice of coaching employees to higher performance. LDCHD describes itself as a learning organization and believes that when employees use their strengths they are happier and more productive. We build commitment to the organization and our mission by challenging staff to grow skills. Effective coaching is one means for supporting staff in those efforts.
Over the last few months graduate students at the University of Kansas School of Medicine, Department of Health Policy & Management collaborated with LDCHD for their class project in Human Resources and Workforce Development. Presented with the concerns LDCHD heard from supervisors, namely, “I don’t know how to be a coach.” and “I don’t know how to recognize and use staff strengths as well as I want to.” the students were asked to develop a project related to improving performance and capacity of supervisors. Results of the PAS evaluation discussed earlier provided essential background information and content for the project. Students were asked to use the evaluation to identify growth recommendations that fit our supervision framework.
To assist with understanding agency philosophy, LDCHD shared tools and resources already put in supervisors’ hands:
• Catalytic Coaching by Garold Markle
• Strengths Finder by Tom Rath
• Motivational Interviewing – a collaborative conversation style for strengthening a person’s own motivation and commitment to change
• Kansas Leadership Center Framework – a set of competencies for civic leadership that when put in practice help address the adaptive challenges facing our communities
These resources have much on common, are supportive of each other and fit with LDCHD’s supervision framework. Because these resources are already in place, the students are creating a training outline that can be incorporated into the monthly Leadership Team meeting as a way of providing training on improving coaching skills that is both effective and within agency budget and time constraints. The training outline would include reading assignments and a set of discussion questions over various topics. The students will present their recommendations for training on Monday, December 12. If accepted, the training outline will be very helpful in sustaining momentum around improving coaching skills.
Additional Information
-
How did you hear about the Model Practices Program:
- I am a previous Model Practices applicant
- At a Conference
- NACCHO Website
- Public Health Dispatch
- Colleague in my LHD
- Model Practices brochure
- NACCHO Exhibit Booth
- NACCHO Connect
- Colleague from another public health agency
- E-Mail from NACCHO
- NACCHO Exchange
