Posts by Hendy Avenue
Designing Systems to Limit the Impact of Bias
In March, we wrote about how coaches and evaluators must identify, then set aside, biases in order to fairly and accurately evaluate and develop teacher practice. This critical work by individuals is powerful, but not the only tool available to leaders and policy makers in reducing bias: Specific design elements within performance management systems can help reduce opportunities for bias to impact results.
Drawing upon findings from the MET Study and our practical experience with school systems across the country, we recommend three design features to limit biases:
- Balance with multiple measures: Evaluations that rely on a single input are not only more vulnerable to year-over-year volatility but they present the greatest opportunity for evaluator bias to impact outcomes. Consider, for example an evaluation system with one input, ratings on a common teaching rubric. As evaluators work to determine ratings, their biases can (and will) creep in. Without a second or third objective measure to balance the overall evaluation, the teacher’s rating is subject to the influence of these biases with no checks or balances.
- Utilize multiple observers/evaluators: As with multiple measures, adding additional observers/evaluators increases statistical reliability (reliability = overall consistency of a measure). MET Study authors write: “When the same administrator observes a second lesson, reliability increases from .51 to .58, but when the second lesson is observed by a different administrator from the same school, reliability increases more than twice as much,from .51 to .67.” While districts must wrestle with the best allocations of limited time and resources, there is great benefit to having a second set of eyes on a teacher’s performance.
- Use outcomes-based tools: Many common evaluation instruments measure inputs–the actions taken toward a goal–rather than assessing the desired outcomes. Unfortunately, measuring inputs can yield false positives: a teacher can try a specific approach and receive high marks, even if the strategy fails to produce appropriate student learning. Instead, strong evaluation systems use outcomes-based tools and rubrics that identify observable and measurable behaviors. For example, we’re big fans of the Core Teaching Rubric, which describes with specific language what students are saying and doing at different levels of performance–rather than teacher actions–so that observers can evaluate performance based on what matters most: student experience and learning. By focusing on student outcomes, there is less risk of observers inserting their biases or preferences for a particular instructional strategy. Instead, an outcome-based tool focuses observers on what and how students are learning, which is what matters the most.
While these structural changes alone will not eliminate bias, they can make a big difference. What have you seen be successful in reducing bias in performance management? Let us know in the comments!
Jeremy Abarno, Welcome To The Hendy Team!
We’re thrilled to announce that Jeremy Abarno, former Chief Talent Officer of DREAM Charter Schools in NYC, joined the Hendy team in October 2019. Jeremy will specialize in school leader coaching and development, network leader coaching (especially talent and academics), curriculum development and training, and comprehensive talent strategy and implementation. Jeremy is the smart, thoughtful person you want to help you solve problems and we are so happy he’s bringing his many talents to support our partner organizations and to continue to make our Hendy team stronger.
Jeremy started as a teacher in East Harlem in 2000 and has dedicated his career to the children of New York City. He served as the Principal at PAVE Academy public charter school. Jeremy was the Managing Director of Mathematics and later Talent at Ascend Public Charter Schools in Brooklyn, a network of nine schools serving more than 4,000 students in central Brooklyn. Before joining Hendy Avenue, Jeremy was the Chief Talent Officer of DREAM in East Harlem. In this role, Jeremy led the development of DREAM’s overall talent strategy including recruitment, professional development, workforce planning and human resources for DREAM’s schools, afterschool and community programs. Jeremy holds a Master’s in Special Education from City College and a School Building Leader license from Baruch. He is a graduate of New York University in elementary education. Jeremy lives in Brooklyn with his wife and three children.
Three Lessons for Meaningful Performance Management Incentives
We’re big fans of performance management. We know that for people to achieve their best, they need clear definitions of excellence, resources and tools to achieve that benchmark, and frequent, action-oriented feedback that drives their development. Too often, however, organizations don’t know how to link performance management efforts with valuable incentives for their talent.
Before we dig in, let’s get one thing out of the way: incentives in the workplace are not inherently good or bad. Done well, incentives can be a powerful tool for motivation, recognition and reward. Done poorly, incentives can lead people to feel controlled or set up for failure.
Incentives come in all shapes and sizes but most fit within one of the five Rs:
- Rewards: salary, incentives, bonuses, benefits
- Responsibilities: variety, autonomy, challenge, promotions, learning
- Relationships: coworkers, supervisors, clients, and customers
- Reputations: image, respect, appreciation, feedback
- Rest & Relation: hours, scheduling, flexible, travel, location
So what now? Maybe you want to give a bonus to high-performers, grant newcomers more autonomy or show appreciation to employees who routinely go above and beyond. In any situation, here are three recommendations to help you build a foundation for offering incentives:
- Set achievable, meaningful, and controllable benchmarks for employees. Ambitious goals are great but they should not be impossible to reach. Similarly, desired outcomes must matter to the person and organization. If mission driven, make clear the connection between the individual’s impact and how it helps the the organization get closer to their vision. It may seem obvious, but people also have to know that what they will be measured against can be reached by the levers within their control. We would never evaluate a bank teller by the number of light bulbs that need to be changed monthly, so test your goals to ensure a direct line back to employee actions and responsibilities.
- Know what people want. A huge problem is that organizations generally don’t know what their people want. For example, leaders may assume everyone is motivated by the same things or rely too heavily on what has historically been motivating. Particularly in large or longstanding organizations, there is too much application of the golden rule (treat others as you want to be treated) by decision-makers and not enough of the platinum rule (treat others as they want to be treated). There’s no secret to fixing this problem; you need to do your research (e.g., ask employees, read research on generational shifts, talk to industry leaders) and pilot before full-scale launch. The best approaches customize incentives for specific groups or individuals.
- Keep support and growth at the center. Earning an incentive should not be the end of the journey for your employees. Milestones are important, but should always be coupled with a “what’s next?” conversation so that people feel you are genuinely invested in their growth and development (which we hope you are!) and can tangibly see the road ahead. If one does not yet exist, take the opportunity to co-create. A common pitfall is focusing too much on monetary or extrinsic rewards, sometimes to the detriment of strong intrinsic motivators. Focusing on continual improvement and long-term development will help mitigate this risk.
How have you effectively implemented incentives? What’s been the most interesting outcome? Sound off in the comments below!
Listen Up: Hendy Avenue on EdPOP Podcast
Curious to learn more about Hendy Avenue Consulting? Our very own Jessica Wilson sat down with the host of EdPOP to talk about our mission, recent projects and how talent strategy can make the difference for kids across the country.
Three Steps to Avoid Common Observation Biases
We all have biases. Whether picking an ice cream flavor or choosing to take the scenic route rather than the highway, we all operate with mental models that place disproportionate weight on certain factors that move our judgment in favor of one option when compared to another.
When observing and evaluating teacher practice, there are numerous opportunities for biases to creep in. Just think of all the factors that go into a lesson: the subject, grade, school, teacher, time of day, lesson structure, materials used and more. An observer may think to themselves, “the students were well-behaved for the first class right after lunch”. A different person observing that same lesson may think, “if I was teaching this class, I would have used a different text.” Both of these sentiments may be true, but they have to be placed aside before conducting a visit so that observers can focus on objective teacher and student actions.
In short, great observers, coaches, and evaluators must identify, then set aside, biases in order to fairly and accurately evaluate and develop teacher practice.
Common biases include:
- Confirmation bias: the tendency to search for or interpret information in a way that confirms one’s preconceptions,
- Halo effect: the tendency for a person’s positive or negative traits to “spill over” from one area of their personality to another in others’ perceptions of them, and
- Mirror bias: the tendency to judge performance as “good” if it is “like I would have done it.
A full table of common observer biases with examples can be found here: Observer Bias Examples
In order to mitigate the impact of these biases, great observers should ask themselves three questions:
On the Air: Jess Talks Rubrics with EdPOP
School leaders are busy people; we know from experience that it can be really tough to keep up on the latest trends and research when you’re in the trenches every day. We also know that being a school leader can be isolating sometimes. It’s rare that a leader gets to step out of his or her building and see what’s happening in other schools. That’s why we are so excited about the new podcast EdPOP: Education Problems of Practice. The podcast, hosted by educators, provides leaders the opportunity to get a glimpse into other schools; to learn about the latest in K-12 education; and to hear from experts – all in an easy-to-digest podcast. Educators can even earn continuing education hours for listening.
We are especially excited about the most recent episode, released today – find the full episode here or the specific segment is S1E3 Chapter 3: Journal Club. Our own Jessica Wilson was interviewed about her experiences with teacher observation rubrics, and recommendations for leaders. Teacher observation is an issue near and dear to our entire team’s hearts, and we are excited to have had the opportunity to share our experiences with others. Take a listen, and let us know what you think!
Round 3: Looking Back, Looking Ahead
In our previous two posts (here and here), Sarah and Grant shared reflections on the past year and projects they are looking forward to in the coming months. To bring us home, Jessica shares lessons learned on working through complexity and opportunities to lead with appreciation.
What I learned: I have spent most of my career in education supporting and working in large bureaucracies, namely large urban districts and state education agencies. Just prior to joining Hendy Avenue I was in senior leadership in one of the largest school districts in Ohio. Each of the organizations I’ve worked with in the past have faced challenges, and I tended to chalk those up to organizational complexity, and the difficulty that comes with arriving at solutions when you must invest a large number of people and perspectives in the strategies. After spending my first year with Hendy working with diverse organizations and districts, I came to appreciate that the challenges I faced in past contexts are not so different from those faced by clients of all sizes. I’ve learned that it’s often not only the scale and bureaucracy that causes the challenges we face in K-12 education, and that we can learn a lot from organizations of different sizes and types in finding solutions. As we partner with our clients this year, we are excited to continue to bring lessons learned from all shapes and sizes of districts, states, schools and networks to arrive at solutions to problems.
What I’m excited about: I am so happy to get to continue to partner with Independence Mission Schools in Philadelphia. Having attended Catholic schools as a child, I have a great appreciation and admiration for the work IMS is doing for some of Philadelphia’s most deserving students. We learned a lot from IMS’ leaders and teachers as we supported them last fall to implement their new instructional framework, and to modify that framework to fit their Catholic culture. Now, I’m excited to continue to support IMS leaders as they deeply invest in teachers through teacher leadership. This project has been a welcome opportunity to explore how others are solving a problem, learn more about the context, strengths and opportunities in IMS schools, and devise a program that makes a difference for teachers, and students, across the network.
Round 2: Looking Back, Looking Ahead
In our last post, Looking Back, Looking Ahead: Lessons Learned and What’s to Come in 2018-19, our founder Sarah shared insights on the difficulty of leading change and the excitement around re-engaging with one of our first partners.
This week, we hear from Grant:
What I learned: Historian and philosopher Will Durant said, “we are what we repeatedly do. Excellence, then, is not an act, but a habit.” Durant’s much-quoted line rings true in most endeavors, especially in efforts to drive change. Over the past year, we’ve seen the power of habitual communication–to teachers, school leaders, regional administrators–in sharing consistent messages, building shared understanding and demonstrating competence. Nothing derails stakeholder buy-in more than mixed messages or a lack of information! Habitual communication requires consistent content, format, and tone through a single channel at a regular, expected frequency. In Houston, we’ve supported KIPP in the development of a weekly message to School Leaders around implementation of Teacher Pathways. Each Friday, leaders know they will receive updates, shout outs, resources, and reminders to guide the week ahead. In Delaware, we’ve launched a monthly newsletter for district leaders on DPAS_II, the state’s teacher evaluation system, with a consistent agenda including deadlines and professional development opportunities. These habitual communications do more than provide information, they demonstrate competence and care for colleagues and trust between stakeholders. As you think about the programs you’re leading, consider how you can habitualize communication as a repeatedly do.
What I’m excited about: In 2014, KIPP Texas – Austin began a comprehensive effort to reshape teacher effectiveness and retention through the development of a Teacher Career Pathway. Knowing that great teachers drive student achievement, Austin’s Teacher Career Pathway develops, recognizes and rewards excellent educators so they will get better and stay longer. This fall, the first cohort of Distinguished Teachers will be announced; a group of accomplished educators who have demonstrated consistent gap-closing results for kids, impeccable teaching practice and exemplary professional contributions to the school community. We cannot wait to celebrate these remarkable educators!
Looking Back, Looking Ahead: Lessons learned and what’s to come in 2018-2019
As consultants, our role is to guide our partners to make informed decisions and to successfully meet their goals. We also prioritize building the knowledge and skills of our partners and they tell us that they learn a lot from working alongside us. In turn, we learn so much every day from the incredibly smart and diverse partners we have the good fortune to work with!
As we reflect on the past and look ahead to the new school year, we are grateful for all that we have learned from our clients. In our next couple of blog posts, each Hendy team member will share something he or she has learned and something we’re excited about it. It’s going to be a great year!
To kick us off, read below from our founder, Sarah Rosskamm:
What I learned: Change is hard. Often times the solution is to prepare for change, engage stakeholders, continuously communicate the “why”, work with influencers, plan for challenges and ultimately to just keep going even when it gets tough. There are times, however, when the solution is to pause, reflect and change course. In working with one of our partner charter networks this year, we learned that sometimes the most courageous and best answer is to stop doing something. In this case, our partner was eager to build a teacher career pathway. They took many important steps to get there, had buy-in from leaders and momentum from teachers believing it was valuable. However, they also had budget changes, shifts in capacity and new demands of their attention. As a result, they smartly decided to pause. They stopped putting their attention into the pathway and instead narrowed the scope of their focus to implementing a highly effective evaluation and development structure that would help their teachers to grow and enable them to target their professional learning activities. They focused on laying a foundation that would immediately benefit teachers through continued growth, and will ultimately allow them to move more quickly toward a pathway if and when they choose to pick it up again. It wasn’t easy (for the network or for the Hendy team) to not complete our original shared goal, but it was the right thing to do for their teachers and students.
What I’m excited about: Hendy Avenue’s very first consulting project was supporting the Delaware Department of Education as they considered revisions to their teacher evaluation rubric. After considering several rubric options based on the best of the available rubrics at the time, the Delaware team, similar to the team described above, decided to pause and learn more before making changes to a statewide tool. So, we shifted course and supported four charter schools in Wilmington to design and implement an alternative evaluation system for their teachers that would utilize this new rubric. I worked closely with the leaders in those schools for several years and together we instituted an alternative system that is now used in a growing number of Delaware schools through their Alternative Evaluation system. I am so excited that five years later, after learning a great deal about the use of the current rubric and about the alternative system, Delaware leadership is eagerly partnering with teachers, leaders and other stakeholders to revise the rubric to ensure the tool is well-aligned to new standards and meets the needs of teachers across the state. I’m also thrilled the state has very wisely decided to prioritize involvement of teachers and leaders in the process and to take the time necessary to ensure it’s a positive and welcomed change for their well deserving teachers. And I’m even more excited that Hendy Avenue will be partners in engaging stakeholders, designing, piloting, revising and ultimately building a rubric that helps teachers and leaders be the very best they can be for their students.
Make Your Pick: How the NFL Draft Applies to Teacher Hiring
While there is a lot of best practice research out there about how to hire a great team, leaders seeking teacher talent can take a cue from how professional sports teams scope out and draft players. In the post below, we bring Cade Massey’s article on 5 lessons we can learn from the NFL draft into the world of teacher hiring.
1. Know what you need. Before you even begin to recruit teachers, be clear on what type of teacher you need for your school. Of course, certification, grade and subject-area matches matter, but identifying a great fit requires more. Assess your current staff to identify where your team has strengths, and where there are gaps. Perhaps you need a teacher with great data skills who can support your team’s efforts to review and act on student outcomes. Or perhaps you need a teacher who can effectively implement writing across the curriculum. Also consider your strengths as a leader; do you have capacity to coach a novice teacher? Or do you need someone with more experience? Being clear about the ideal profile of a candidate can help ensure that you focus your limited resources on a hiring process that will yield the best outcome.
2. Get input from others. While the school leader is often the driver and decision-maker when it comes to hiring, ensuring that teachers, other leaders, and even parents are engaged in selection will help ensure that the best candidate is chosen for the school. Consider a process that allows you to solicit input and ideas from a variety of stakeholders. Allow each stakeholder to have an independent review of finalists, and to form their own perspective about fit. One easy way to engage multiple stakeholders quickly is to use a panel interview, or to have multiple stakeholders act as students in a demo lesson (see item 3).
3. Understand the candidate from multiple angles. Resume reviews and interviews are a great first step in getting to know a teacher candidate. But, often that isn’t enough. As football scouts actually see candidates play, getting a glimpse of your top candidates teaching will help you understand how they may fit into your school culture. Request a video of the candidate teaching, or request that they teach a demo lesson in your school or with your selection committee. Even observing 10 minutes of teaching can help you get a full picture of the candidate’s skills and growth areas.
4. Be consistent in your selection model. Hiring is about assessing people, which can be a messy business. No matter how disciplined we are, our opinions of others are naturally informed by the biases we carry; we’re all human after all. As you design your selection approach, consider a rubric and scoring mechanism that makes considering multiple variables factors more formulaic. Bringing order and data to a process like hiring can help ensure that factors like selection bias do not play a significant role in who is selected for your school.
5. Keep score and reevaluate. The only way to know if your selection process worked is to map it against results. Once you’ve selected your dream candidate(s), keep a record of the selection process and the factors that led to their hire. Then, after their first year, compare the teacher’s results to your selection. How accurate was your assessment of their strengths and growth areas? Did your selection approach yield a candidate that made gains with students? If so, what should you replicate? If not, what might you tweak for future hiring?
Sound off in the comments: What lessons have you learned from teacher hiring? What strategies have been most useful in identifying your best candidates?