The Listening Fund supports youth-focused organisations to develop their practice of listening to young people and responding to what they hear. The Centre for Youth Impact was commissioned to evaluate the Fund and as part of that evaluation, case study research was conducted with six partners, offering the opportunity for an in-depth investigation of their work. The full case study report, including the methodology and all six partners can be downloaded here. This first case study focuses on Youth Access.
Youth Access is a national membership network of 188 Youth Information, Advice and Counselling and Support services (YIACS). Youth Access works to ensure that every young person has access to free and high-quality advice and counselling. It does this through working across its membership to promote good practice via the YIACS model, which is a holistic response to young people’s emotional, social, health and practical needs, and to ensure that links are fostered between the different sectors that have an impact on young people’s lives.
Overview of listening at Youth Access
Prior to the Listening Fund, Youth Access developed the ‘Altogether Better Charter’: a guide for person-centred mental health and wellbeing services. The Charter was created by hundreds of young people all over England, who participated in consultations to inform the content. It sets out seven criteria for services to ensure they are putting service users at the heart of their provision, such as to ‘treat us with respect’ and to ‘have skilled workers that take us seriously’.
The Altogether Better Charter acted as a precursor to Youth Access’s Listening Fund project. The immediate feedback Youth Access received from young people was that, while the Charter is hugely valuable, they wanted services to sign up to it and to be held directly accountable. In other words, young people wanted to ensure it did not go unnoticed or lead to tokenistic action. Youth Access’s Listening Fund project is centred around the creation of a feedback scheme that holds
organisations accountable to the Charter, and therefore, to young people. The two components of the scheme are explored below.
1. Digital Feedback Tool
Youth Access recruited an ethical design agency, Clear Honest Design, to create a digital tool that lies at the heart of the feedback scheme. In essence, this is an online survey that gives young people a say, based on their experiences, as to whether the service is achieving the seven charter points.
The questions in the tool draw directly on the ideas of over 200 young people who originally fed into the Altogether Better Charter, and these were further developed through a series of Advisery workshops with 13 young people and with staff from across the membership. The workshops were structured around the seven criteria in the charter: young people developed and short-listed questions to capture each charter point, and these were then finalised in consultation with staff.
“There are currently a million different types of surveys, forms, and suggestion boxes used between our members for collecting feedback, with varying degrees of reliability and effectiveness. The digital tool has the potential to be attractive, as a free and standardised approach to obtaining feedback, learning from it, and improving.” (CEO)
Youth Access has since worked with three of its member organisations to pilot the digital tool. A key concern was making the tool flexible enough to be applicable in a range of YIACS settings, and so the pilot organisations were chosen based on their diversity. Initially, each of the services was supported to embed the Altogether Better Charter within their service, because “it is important that staff and young people know what the Charter means and why it is important before the
feedback tool is introduced”. This involved giving presentations, putting up posters, and having ongoing discussions about what the Charter involves and why they need to be held accountable. Each of the three services has then undergone a period of data collection, to gather responses from service users over six months.
2. Youth Charter Ambassadors
To supplement the digital feedback tool, a selection of young people within each member organisation in the pilot has been recruited as ‘Youth Charter Ambassadors’. Youth Access acknowledges that there is limited depth and dialogue that can be achieved with young people through an online tool, and the ambassadors re-balance this through collecting qualitative, face-to-face and conversational feedback. The ambassadors have two specific roles:
• To champion the feedback scheme in the service, by offering advice on the feedback tool, giving presentations, and having discussions; and
• To co-facilitate feedback workshops (with an experienced member of staff) using an activity pack provided by Youth Access.
The activity pack for the feedback workshops provides guidance on planning and structuring the sessions, and activities to obtain feedback on each charter point. It is designed to be flexible to ensure it fits the needs of the service and to enable young people to co-design and co-facilitate the process. All feedback from the workshops is recorded on flipcharts, via writing, stickers or drawings, and is then photographed and subsequently analysed by Youth Access’s central team.
What is the key learning and impacts of this work?
Co-production is an organic and multi-stage process
Through the feedback scheme, Youth Access is putting young people’s voices at the heart of measuring quality, as part of a broader, rights-based approach to mental health services. It was essential that young people were involved in the design of the tool since it is about being accountable to them. The process of developing the Charter with young people, prior to the Listening Fund, lasted over a year, and the feedback tool built directly on this process. This was not the original purpose of the Charter, but it has grown organically through the process of taking on board young people’s views. Likewise, the current feedback tool is not in its final incarnation but is still being revised and reviewed in a collaborative way. From this experience, Youth Access appreciates that co-production is a time-consuming process that is not always straightforward:
“To do listening where young people are given lots of power and responsibility is quite longwinded and can be messy. Sometimes you have to go backwards before you can go forwards. Sometimes you have to scrap an idea if it’s not working. In reality, it takes a lot of resources and a lot of time – it’s not just about reading some guidance and getting on with it.” (Programmes Manager)
Having been through this process, Youth Access is proud of the digital tool it has ended up with: the simple appearance of the tool masks the level of time and resources that have gone into building it, and staff feel the conciseness is a sign of high quality. Indeed, it takes a significant amount work “to take all that learning and condense it into something of high value”.
Design and accessibility
Youth Access worked with young people to ensure the tool met their needs and preferences. Youth Access found it was important to young people that it was quick to complete and relatively light touch, and so staff have worked with the developers to ensure it was easy to use, aesthetically pleasing, and mobile-optimised. A large proportion of Youth Access’s Listening Fund grant went to developing the tool, reflecting the high cost often associated with this kind of digital development.
Each service taking part is also given business cards and posters with unique QR codes to enable young people to access the survey quickly via their phones. Accessibility is a key consideration here; some services have communal iPads for those young people who do not have their own devices, and one service required a paper version of the survey to increase response rates.
Making listening useful and actionable
Very often, organisations are required to report feedback to many different funders and commissioners via multiple systems and forms, and they experience delays before they see results, often leading to duplication, reduced value, and frustration. For Youth Access, it was important that its members come to see the tool as a valuable and free resource that is there to benefit them, rather than as an additional burden or as something they ‘have to do’.
Youth Access’s pilot suggested that being able to access data easily and instantaneously motivates organisations to collect it and to act on it. As such, Youth Access staff thought carefully about how the data could be presented in the most useful way. Each participating service has been given a login to the ‘back end’ of the tool, to see a statistical and visual breakdown of its data in real-time, and to see how it is performing with respect to different areas of the Charter.
Creating a culture of low-stakes accountability
Youth Access is keen that services do not ‘cherry pick’ young people to give feedback, in the hope of influencing the responses. As such, staff work carefully to promote a culture of low-stakes accountability, where organisations do not feel pressured or judged based on their scores. This is a challenge with the nature of feedback across the youth sector, and Youth Access primarily tackles it through ongoing communications to remind services of the original purpose of the tool.
“We continually reiterate the message that this is not about getting a good score, it is about learning and improving on the basis of what you find out.” (Programmes Manager)
Additionally, Youth Access has heard from young people on numerous occasions that they would like there to be a ‘charter mark’ where organisations would receive accreditation if they receive a certain feedback score. This would serve as a “stamp of approval” so that the young person can be sure they are entering a service that is friendly and respectful. While there is value in this concept, there are concerns it would lead to a situation where members are less inclined to collect legitimate, representative feedback. In other words, there are concerns that the focus would switch from ‘improving’ to ‘proving’. Youth Access staff are keeping an open mind as to how this dilemma could be overcome but are primarily interested in maintaining an effective way to improve services through acting on what young people say matters to them. A current compromise is to provide services with a standardised logo to demonstrate their commitment to
achieving the Charter via continuous improvement, without adding pressure to get a high score.
What changes have been made as a result of listening?
The tool has proved popular during the pilot, but success ultimately depends on how dedicated each service is to the process, and this will always be variable. As a consequence of taking part, one organisation has already embedded the tool into its regular feedback processes, which is a positive and hoped for outcome. There are reports across all three pilot services that the feedback tool has challenged them to think about how to truly involve young people in decision making,
how they take account of the needs of all service users, and how they act on what they have heard.
For Youth Access, the process of producing the Charter and the feedback scheme has also changed the way it works with young people on other projects, who are being given a more leading role across the organisation. Specifically, it has encouraged thinking around what listening looks like at different levels, beyond traditional models that focus on youth boards and youth governance.
“Now we have an embedded process to ensure young people have power over what we produce. Now what we say is co-produced actually is.” (Programmes Manager)
Ongoing challenges and next steps
Navigating the trade-off between consistency and flexibility
In the next phase of the pilot, the tool will be rolled out across ten services in Youth Access’s membership to understand how it works in a wider range of settings. As a support organisation, Youth Access needs to strike the right balance between consistency, to ensure members are supported to collect data to an equally high-standard, and flexibility, to ensure the approach is adaptable to individual circumstances. There is a trade-off between these, with ongoing
discussions regarding the expectations that will be placed on participating members.
For instance, Youth Access has currently decided not to enforce a minimum number of responses, appreciating that different organisations will have different targets. It has also not defined an optimum ‘testing period’, realising that some organisations may want to embed the tool consistently over a whole year, while others may wish to use it as a ‘snapshot’.
“Our membership is highly diverse in nature of services, and therefore in approaches to evaluation and learning. If directions for adopting the tool become overly prescriptive, there is a danger that organisations will not engage at all.” (CEO)
Longer term, there is potential to ‘white-label’ the tool for use more broadly across the youth, mental health and advice sectors (i.e. to provide the functionality so that organisations outside of Youth Access’s membership can add their own branding and questions). There will, however, be challenges ensuring it is used in the intended way to support continuous learning. As the Programme Manager reflected, “the tool is the nice shiny thing that everyone can see, but it’s not the thing that makes this process effective”.
Ensuring capacity to grow the feedback tool
The feedback scheme generates outputs that require processing by Youth Access’s central team. This includes inputting responses from paper versions of the survey, and analysing outputs from the Youth Ambassador workshops that are captured through flipcharts. As the feedback tool grows, Youth Access will need to consider if and how it can manage this process of listening across its members, as the administrative responsibilities will increase significantly.
Creating a community of practice
Youth Access hope that the feedback scheme will lead to some cultural changes across the sectors it supports. To facilitate this, as the scheme becomes used more widely, Youth Access intends to develop a ‘community of practice’ for organisations to collectively share and reflect.
“We are excited about this process, that is about creating a safe space where people can discuss what they are doing well or not well and what they are learning, rather than just trying to prove they are the best.” (Programmes Manager)