5. Evaluating impact

Assessing the impact of learning and development is important because it makes sure things are working as they should be and there’s a continual process of self-reflection and continuous improvement.

When assessing impact, you should be clear about what good looks like for the ‘whole system’ of care and support. The experiences of people with dementia and their families are the main measure because:

• people with dementia and their families can have contact with multiple services across health and social care at the same time
• if the person with dementia has one negative contact with a service that affects their well-being, it can cancel out many of the benefits of the excellent work done by other services.

Evaluating the impact of dementia learning and development on the well-being of people with dementia is tricky because well-being can be affected by many things.

Don’t strive for perfection. Instead develop evaluation approaches that you and your partners feel are good enough. Gather information from more than one source because having multiple sources makes your evidence more valid.

A good impact assessment:

• measures impact across a regional system of care, as well as in your organisation. Focus on well-being outcomes and people’s experience of care. Developing well-being statements and assessing changes in staff attitude are good places to start
• thinks about the diverse needs of the population of people with dementia
• takes account of other external factors that could affect the well-being outcomes and experience of care under review. For example, funding changes, staffing changes and changes to care packages
• focus on the views of people with dementia, their families and the people who know them best, and reflect on what’s working well and what needs to improve
• get people who are skilled in communicating with people with dementia and their families to ask for their feedback
• measure change over time, gathering people’s attitudes and experiences at different stages of the dementia pathway.

When done well, impact assessments motivate staff to be the best they can be. They:

• focus on what is working and what could be improved, and they celebrate success
• identify safety and well-being problems, and resolve them using openness and honesty rather than blame or creating a feeling of fear of failure.

There are lots of ways to involve people. While questionnaires and surveys have their place, there’s growing evidence about the benefits of using narrative stories in evaluations.

Get information from a range of sources and value the views and stories of everyone involved, including people living with dementia, their families and the health and social care workforce.

The impact measure you develop should link back to Workstream 5b of the dementia standards (measurement).

• The Insight Collective: Evaluation support | Social Care Wales - Research, Data and Innovation.
• Cardiff and Vale of Glamorgan: Training discussion form.

• Cardiff and Vale of Glamorgan: Training evaluation and feedback document.

• Compassionate Practice (2021) Compassionate Circles is a 60-minute guidance sheet for facilitators and hosts. Taking Care Giving Care Rounds in ABUHB is a guide to putting compassionate circles in place. Both can help you introduce the approach locally.

• Experience Based Co-design (Bates and Roberts 2007, Kings Fund). This how to guide sets out how to capture the stories of people in health and social care and the highs and lows of their experiences of care and support. The stories are then shared with staff in group discussions, which can form the basis for service improvements.

• Magic Moments (Andrews et al 2015) is a good practice case study that shows how storytelling work in care homes in Wales has successfully recognised and developed quality care.

• PocketMedic films and an article in the Lancet Neurology about the dementia film series. An evaluation of the PocketMedic films was carried out by the Assistive Technologies Innovation Centre (AWTiC, UWTSD) and is the next best thing to the actual experiential opportunities that have been shown to have the greatest impact.

• Dementia pathways of standards measurement workstream.