Analytics and EBHR
Case Studies: EBHR A New Paradigm
Each of these cases comes from different sectors and uses EBHR in different ways to improve HR decision-making. The first is from Thales – a French multinational which designs and makes electrical systems. The second is from the UK’s Financial Conduct Authority (FCA) – a financial regulator. And the third is from Uber, headquartered in the US.
In each case, the HR function as a whole or individuals within it have explicitly used the EBHR principles and process to help resolve specific business issues (in the first two cases) or to design the way in which their analytics team provides support to internal customers (in the third case).
At the end of each case some commentary is provided.
Thales
Retaining talent on long-term projects
Dave Hodges, Strategic HR Business Partner
Thales manages very large and long-term projects which they describe as mega-programmes. Given the length of these mega-programmes which may span well over a decade, a fundamental challenge is retaining key employees.
The traditional approach had been to offer financial incentives. However, the effectiveness of this particular approach to retention was not clear and, in addition, the effectiveness of other interventions was unknown.
In this case, the problem was already reasonably well-understood and therefore the focus was on using an EBHR approach to identify likely solutions.
A team was assembled consisting of stakeholders, internal customers, individuals from business centres of excellence and others with particular interests in the overall approach and/or interests and expertise in particular areas of evidence. The team met across four sessions – some face to face and some virtual – to work through each of the key stages with agreed deadlines:
- Agreeing the main questions: what was known about the effects of financial incentives on the retention of key staff? Which other interventions might be effective?
- Acquiring the evidence from multiple sources (internal evidence, stakeholder perspectives, professional expertise and scientific evidence).
- Critically appraising the quality and relevance of the evidence.
- Aggregating or pulling together all the evidence.
- Applying it to answer the questions.
The evidence suggested that financial incentives were probably only effective for a small minority. Therefore, their overall effect was probably quite small and likely only short-term, suggesting a poor return on investment. The evidence further indicated that other interventions focusing on intrinsic motivation (using self-determination theory) may be more effective and also more cost-effective.
Rather than offering financial incentives, a tool was developed which allowed programme managers to explore with key employees whether they perceived their work was providing an environment which facilitated intrinsic motivation (autonomy, competence and relatedness). Programme managers used the tool to discuss the extent to which these three aspects of intrinsic motivation were present and, if not, what steps could be taken to enhance them.
Where deficits were discovered, the tool offered methods of improving their work experience in ways that would enhance their intrinsic motivation and increase retention.
Commentary
Some points to note about this case in relation to the EBHR process:
- The EBHR process (see EBHR Toolkit) has two parts. In the first part of the model the nature of the problem or business issue is identified. In the second part, possible solutions are identified. Here, there was less need for Part 1 as the problem was already quite well-understood. However, this is likely quite an unusual scenario. Therefore, when you apply EBHR to your own practice you will likely find that whilst a few issues are already quite well-defined, you will also need to collect data and evidence to establish the precise nature of the issues.
- A diverse team was created to work on the EBHR process which crucially involved stakeholders, internal customers and others with interests and appropriate expertise. In general, the EBHR process is best done with a dedicated group who are assigned clear roles.
- In this case, the EBHR process was organised around a project plan which involved a set number of meetings, each with clear objectives and sufficient time between them for team members to work on their parts of the EBHR project.
- The team gathered evidence across all four sources of evidence (see EBHR Toolkit): internal evidence, stakeholder perspectives, professional expertise and scientific evidence. While it is not always possible to draw on all four sources equally, one of the key principles of EBHR is that it is always more effective to use multiple sources.
- The evidence examined suggested that the existing intervention (financial incentives) was not particularly effective. EBHR can also be used to help understand what isn’t working as well as what might work better.
- The team drew on scientific evidence around theories of motivation and identified a new intervention which seemed likely to be more effective than financial incentives.
Financial Conduct Authority
Enhancing Diversity, Equity and Inclusion data to inform strategy and action
Irene Uwejeyah, Programme Manager and Sara Nursaw, Head of Corporate Responsibility
The FCA aims to be a more effective regulator by ensuring that its staff are representative of broader society. They also believe that ensuring their people fulfil their potential results in better decision-making in the public interest.
As such, as is common with many organisations, the FCA has stretching DEI targets and objectives. Progress had been made over recent years, and much good work was underway, but the FCA was keen to accelerate the pace of change. As a data-led regulator and aligned with the regulatory approach to D&I, the FCA decided to take an evidence-based approach to conduct a major review of its internal DEI work. This review was critical to the FCA identifying and understanding the most important DEI issues and challenges for the organisation and what interventions should be developed to address them.
As part of the research phase, the FCA gathered and analysed the following evidence:
- Internal evidence relevant to DEI from within the organisation, including quantitative data on remuneration, pay gaps, awards, recruitment, performance management and employee survey results.
- Qualitative data from stakeholders was gathered through speaking to their DEI networks, senior leadership across the FCA and colleagues across HR centres of excellence (HRBPs, resourcing, early careers, HR Analytics, HR policy, workforce planning and organisational capability).
- Professional expertise on DEI issues and actions was gathered from engaging with internal professionals, external professional bodies specialising in DEI, other regulators in financial services and companies inside and outside of financial services with well-established DEI programmes. In addition, they attended several sector events focused on DEI.
- Scientific findings on a range of academic and consultancy research reports were also scrutinised for insights into analysing DEI issues and the effectiveness of different interventions.
In addition, the FCA had previously conducted and published a literature review examining the evidence for the effects of diversity and inclusion in the workplace as part of its regulatory work.
Multiple team members rated the quality and relevance of the evidence to provide peer review. Team members’ appraisals of quality and relevance were compared and discussed using a RAG rating scale (Red/Amber/Amber-Green/Green). This ensured focus on the best available evidence.
The team also designed an evidence tracker, which allowed them to record and rate each piece of evidence and thus consider different data points simultaneously to build a more complete picture and develop more nuanced insights.
This process also enabled the team to prioritise those areas and actions where more work was required. The FCA was then able to group evidence into different themes and further categorise it into sub-themes. For example, inclusion was a theme that was identified, and from then, they categorised evidence under the banners of ‘Fairness and belonging’, ‘Accessibility and reasonable adjustments’ and ‘Inclusion and learning culture’.
Decisions about interventions were not based on finding ‘The Right Answer’. Instead, the available evidence was used to design and select interventions most likely to make specific and significant impacts. This also allowed more precise testing of the effectiveness of interventions.
The team presented their evidence-based methodology, the evidence review and the decisions based on the evidence to their Executive Committee (ExCo). As the most senior stakeholders, they wanted ExCo’s approval of the identified priorities and for ExCo to understand the approach the team adopted to identify key DEI issues. This ensured that the new DEI programme was developed with ExCo rather than being an HR strategy simply presented to ExCo.
Commentary
Some points to note about this case in relation to the EBHR process:
- The context of the FCA as a regulator played an important role in facilitating the use of EBHR. Apart from improving the effectiveness of HR, adopting EBHR also helps with transparency, accountability and governance – all crucial for a regulator but also potentially important for any organisation. If any HR function wants the ability to explain its decisions to senior stakeholders, EBHR provides a sound audit trail, setting out exactly which decisions were made, why they were made and the evidence-base used to shape those decisions.
- DEI is a good example of an area of HR activity where there are always multiple rather than single issues or problems. In this case, the evidence was used to identify these multiple issues and, crucially, to prioritise them. EBHR helps differentiate specific issues and identify which are most important and require most attention.
- Like Case 1, this project also used all four sources of evidence (see EBHR Toolkit) used in EBHR: internal evidence, stakeholder perspectives, professional expertise and scientific findings.
- In another action aligned with one of the principles of EBHR, the team used an explicit rating scale to judge the quality of evidence, enabling them to focus on the most trustworthy.
- After diagnosing and prioritising the issues the focus was on finding interventions most likely to make the desired impacts rather than searching for ‘The Right Answer’. The problem with searching for a single ‘right’ answer is that complex issues rarely have a right answer; instead there will be several potential answers each of which comes with its own costs and benefits. Evaluating the costs and benefits of each will help make a more-informed decision about which intervention to choose.
- In this case, ExCo were involved from the beginning of the process and both the evidence and subsequent decisions were presented to them to sense check and ensure ExCo were partners in the process. Engaging senior stakeholders in this way is likely to provide a better EBHR process and outcome.
Uber
Establishing a People Decision Science team to apply evidence-based practice principles
Nicholas Bremner, Head of Organisational Analysis
For many years Uber has conducted various forms of research within its People Science team – using insights from internal data (people analytics) and external scientific studies – to inform HR practice. It was observed that clients both from within HR and leaders and stakeholders in the business more broadly had several issues with the available data and evidence. First, while they did trust the analysis of internal data they were not convinced of the relevance of the external scientific literature. Second, the internal research was trusted but not well-used as it was difficult to translate it into insights to guide action.
Over several decades, evidence-based practice has evolved specifically as a way of incorporating different types of evidence in practitioner decision-making. It was therefore seen as an ideal framework for helping to provide clients with more of what they were asking for and enable them to make rounder and better-informed decisions.
A People Decision Science team of four was established to provide stakeholder-facing consulting roles to supplement and make better use of the research being conducted by the larger People Analytics team. They used an evidence-based management model to guide the way they worked with clients. This focused on key elements of evidence-based practice:
- Incorporating multiple sources of evidence – including but not limited to internal quantitative data.
- Following a structured process which starts with asking a question, gathering evidence, reviewing its quality, pulling it together and then applying it.
In order to develop capability in evidence-based HR, a book club was created to enable members to systematically read and process a key evidence-based management textbook. This enabled the team to collectively understand evidence-based practice, how it applies to HR in general and to their work within Uber in particular.
The team developed several areas of work:
- Measuring for impact: for example, which KPIs really matter and have the most impact on the business?
- Data interpretation and storytelling: going beyond providing data and evidence to help realise the practical insights data and evidence provide.
- Data fluency and enablement: working with clients, sometimes in a coaching style, to help them interpret and better understand data and evidence.
- Creating a repository or library of the research conducted over many years by the People Science team which could be drawn on when consulting with clients.
Data requests from clients are developed and focused through conversations to increase the clarity of the clients’ questions about a perceived problem or possible solutions. This also establishes a shared understanding of the data and evidence that will, in principle, help answer those questions. The idea that the evidence may, or may not, support what the client would prefer to be the case is discussed up-front in order to manage expectations.
The ultimate goal of the team is to help the business deploy HR practices and activities that help the business and also refine certain programmes and activities that are not making a useful contribution in their current form. For example, on the basis of the team’s analyses some types of training and employee development opportunities have been retired or refined in order to improve their impact.
Commentary
Some points to note about this case in relation to the EBHR process:
- In this case, EBHR was not used to deal with a particular business issue; rather it was used as framework to design an internal consultancy service (the People Decision Science Team) to improve their work with clients. EBHR can provide a useful set of processes and content guidelines (see EBHR Toolkit) to shape the support provided by HR to customers.
- The Team taught themselves the principles of evidence-based practice and explicitly adopted its key principles including using multiple sources and following a structured process.
- EBHR helped the People Decision Science Team create products and services including adopting a coaching approach to working with clients.
- As with Case 1, the EBHR process was also used to help identify which current practices were not particularly effective so that a decision could be made about whether to withdraw, improve or replace such practices.
FULL REPORT
Evidence-Based HR:
A New Paradigm
MEMBER LOGIN TO ACCESS ALL CRF CONTENT
