Understanding our website audiences 26 April 2021
Description: Understanding our website audiences 26 April 2021 Private Confidential 1 Research report Contents Introduction Objectives Approach Understanding your audiences 2 5. Executive summary 6. Findings by theme 7. Recommendations 8. Next steps
Related Topics
Download Presentation
"Understanding our website audiences 26 April 2021" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
slide1. Understanding our website audiences 26 April 2021 | Private & Confidential 1 Research report<br>
slide2. Contents Introduction
Objectives
Approach
Understanding your audiences 2 5. Executive summary
6. Findings by theme
7. Recommendations
8. Next steps 30 mins 20 mins 40 mins Q&A<br>
slide3. Introduction<br>
slide4. 1. Introduction 1.1. Background
1.2. Stakeholder engagement 4<br>
slide5. 5 The NAO website is a key channel to reach your audiences. This is an opportunity to invest in developing the technology, design and content of the website so that it better supports your new strategy, enabling the NAO to provide more accessible insight and increase its impact.
NAO commissioned Mace & Menter to run a discovery project to better understand its website audiences and how it might better serve their needs.
This will support and evidence your business case for investment. Background<br>
slide6. 6 Stakeholder engagement A key part of the discovery project is to gain a clear understanding of the expectations from key NAO internal stakeholders.
So far we have conducted in-depth interviews with 12 NAO stakeholders
from across the organisation (knowledge, VfM, ERT and PRT).
We will continue to further engage with stakeholders during the concepting phase of the project.
We welcome any further engagement after this presentation for follow up.<br>
slide7. Objectives<br>
slide8. 2. Objectives 2.1. Project objectives
2.2. Scope of work
2.3. Where we are 8<br>
slide9. 9 Provide evidence to support the business case for redeveloping the website
Understand your key website audiences and their needs from the NAO
Understand how well the website is currently supporting their needs
Uncover the unmet needs of your website users
Validate concepts of how the website could better support unmet needs
Validate the technical feasibility of our recommendations Project objectives<br>
slide10. 10 In scope
Deep website-related focus on Parliamentarians and Civil Servants groups (who we were able speak to)
Snapshot of the website experience of a broad range of audiences
Not in scope
Comprehensive research study into all types of Civil Service user groups
Comprehensive research study into all Parliamentarian user groups Scope of work<br>
slide11. 11 Done
Understood organisational needs and existing research
Conducted qualitative user research to deeply understand needs of key audiences
Surveyed all key NAO audiences to understand broader needs *
Distilled our research into clear insights and recommendations
Next
Work with you to prioritise recommendations as hypotheses that we can test
Work with you to create website concepts to test with our two user groups
Report with user feedback and design recommendations from concept testing
Review the technical implications and feasibility of the recommendations Where we are so far * Survey findings to be presented separately<br>
slide12. Approach<br>
slide13. 3. Approach 13 3.1. Approach
3.2. Methodology
3.3. Recruitment<br>
slide14. 14 Mixed method research approach
A critical part of our work is to gain a deep understanding of your key audiences
We do this in a number of ways combining both qualitative and quantitative research methods to ensure that we balance the need for a deep of understanding of your core audience groups whilst also gaining the breadth to cover broader audiences. Approach<br>
slide15. 15 Quantitative research
Analytics - We reviewed all of your relevant analytics data to help inform our understanding of current website behaviour
Surveys* - We have surveyed a broad spectrum of your audience groups to increase the breadth and depth of our insight
Response - promoted via the website, social media and sent out to all key NAO audience groups mailing lists, we have so far received:
842 responses Methodology * Survey findings to be presented separately<br>
slide16. 16 Qualitative research
We conducted a total of 17 semi-structured 1 hour depth interviews with:
X7 Parliamentarians
X10 Civil servants Methodology<br>
slide17. Understanding our audiences Persona summary<br>
slide18. Persona summary Please note:
We won’t have time to go through the detailed persona groups today, however here’s an overview to help give you some context when we are referring to specific types of users.
We will share the complete presentation in due course. 18<br>
slide19. Civil Servants - Persona summary 19 Collaborators Specialists Collaborators are civil servants who
work with the NAO to:
Support its financial & VfM audits
or
Provide subject matter expertise and advice to help inform reports and investigations. Specialists are civil servants who:
who work in strategic roles driving policy development, project/programme delivery or in operations<br>
slide20. Civil Servants - Persona summary 20 Audit Collaborator Specialist: Policy / Operations / Delivery Report Collaborator Collaborators Specialists Who they might be Director or grade 6 working in finance / audit / risk assurance What they do Manage and audit financial accounts and risk assurance
NAO related goals
Work with NAO to support VFM & financial audits of their departments + sign off Who they might be Directors / specialists in govt depts or from a public body
What they do Lead and manage specialist govt depts or public bodies
NAO related goals
Offer subject matter expertise
Advise on report focus areas Supply knowledge / data Who they might be Senior (grade 6): Head of Policy / Delivery / Operations or public scrutiny body specialists
What they do Devise processes & ways of working to support policy development, delivery planning, business case appraisals often during the inception phase.
NAO related goals
Understand NAOs value proposition Best practice guidance on ways of working Cross-govt learnings & case studies Shared interest Partner Collaborator Who they might be Director level SCS1 (treasury) official What they do Scrutinise public spending & value for money / attend PAC committee meetings
NAO related goals
Understand NAOs remit and how best to work together<br>
slide21. 21 Library Researcher Political Advisor Committee Specialist Commons Staff MPs Staff Clerk Parliamentarians - Persona summary What they do Answer private enquires from Members of Parliament
NAO related goals
Reference NAO reports in their findings Who they might be Economist, statistician, financial analyst, policy expert
What they do Provide specialist help to select committees, writing briefs & reports
NAO related goals
Reference NAO reports in their own reports Who they might be Staff of a Select Committee such as PAC, Treasury, DCMS What they do Support the day-to-day running of the committee and its members
NAO related goals
Direct members to NAO reports and recommendations Who they might be Chief of Staff, Senior Political Advisor What they do Support a MP in their day-to-day tasks, including preparing for briefings, debates and committee meetings
NAO related goals
Looking at particular topics, themes or constituency interests that match those of the MP they support Scrutiny Unit Person* Who they might be Legal specialist, statistician, financial analysts, economist,
policy specialist What they do Providing scrutiny of public expenditure and pre‐legislative draft bills
NAO related goals
Use NAO reports to scrutinise public spending * Did not interview in this study<br>
slide22. Executive summary<br>
slide23. 4. Executive summary Broad unmet need for department agnostic theme and issue based knowledge on the website
Finding reports is the primary motivation to visit the website, however once found, users were offered no compelling reasons to stay on the website and would leave
Finding recent & specific reports was seen as fairly easy, however finding thematic content was difficult
Civil service ‘specialists’ were the group that felt their needs were least supported by the NAO
Clear unmet need from all civil service ‘specialists’ for cross-government best practice guides and case studies to show them how to improve ways of working in their departments
MPs staff are very time poor, having to find and distill information quickly, they did not want more general newsletters, they wanted concise information tailored to be relevant to the interests of their MP & constituents
Civil service ‘auditees’ wanted clearer guidance and transparency about the audit process 23<br>
slide24. Findings<br>
slide25. 5. Findings 5.1. All users
5.2. Civil servants
5.3. Parliamentarians 25 5.4. Personas
5.5. Experience maps<br>
slide26. All users Findings<br>
slide27. Overall website experience Quotes
“Took me a long time to find a report”
Participant 2
“Search function isn't that great”
Participant 10
“Home page looks quite busy... difficult for me to quickly understand if there’s anything relevant to me”
Participant 11
“I find the website very text heavy, text is also quite small, compared to GOV.UK where it is a bit more spaced out and inviting.”
Participant 15 What we heard
Participants across all user groups experienced issues with the website. Those whose need was only to find specific recent reports, were the most satisfied, whereas those looking for broader information were much less satisfied.
The design of the website was seen as dated, ‘busy’ and ‘text heavy’.
Parliamentarian users were more likely to have issues with the findability of content, whereas Civil Servants (specialists in particular) were unsatisfied with the lack of content for them. Some Parliamentarian users felt that the website performed adequately, however this view was to an extent informed by their low expectation from public sector websites as a whole. 27 All users<br>
slide28. Once users find a report they have little reason to stay Quotes
“There is this treasure trove of insight that nobody knows about”
Participant 17
“People think the NAO just does VfM and Audit reports, they do so much more and people should be able to access it”
Participant 16 What we heard
Finding reports was most likely motivation to visit the website, however once found - participants were offered no compelling reasons to stay on the website and would leave. For many participants, the website was merely a destination to find publications and awareness of other types of content was very low.
What this means
There was clear unmet need to surface and join up more related and diverse content (not just reports) giving your users a reason to stay on the website, supporting onward journeys and more engagement into NAOs depth and breadth of knowledge. 28 All users<br>
slide29. Ease of finding reports was dependent on recall and recency Quotes
“Search function isn't that great. Not sure what category it is in... have used filters...hard to know how to filter”
Participant 7
“It is often difficult to know which type of report is which”
Participant 15
“Tend to use Google as it’s much easier”
Participant 2 What we heard
Using the website to find recent and specific reports where the participant remembered the report name was seen as being fairly easy. However, finding reports based on topics or where the report name was not fully remembered, was difficult. Many participants felt that searching via Google was easier than using NAOs site search.
What this means
The website search and filter features does not adequately support users finding topical information and easily being able to narrow down and find what they are looking for. 29 All users<br>
slide30. Need for department agnostic themes & issue based knowledge Quotes
“Being more of a government resource would be helpful... themes from audits and best practice (cross-government) with examples of positive case studies’’
Participant 13
“If it was more agnostic of department or reports and more about a particular problem I would use it more”
Participant 17
“Be good to see a more informal blogs, e.g. BoE’s Bank Underground… good to have themes such as 'procurement'”
Participant 2 What we heard
There was a clear unmet need for department agnostic themes and issue based knowledge on the website from both parliamentarians and civil servants.
What this means
MP’s staff wanted to understand what knowledge the NAO had on specific issues important to their MP and constituents.
Civil servant ‘specialists’ wanted to find thematic information to help support their roles such as guidance on procurement or working from home during the pandemic. 30 All users<br>
slide31. PDF was the favoured report format but there was openness for change Quotes
“I much prefer it to be in PDF, I hate HTML but it wouldn't put me off if it was in HTML as long as I could still download the PDF”
Participant 9
“As long as you have option to print out the report even from HTML format it will be fine”
Participant 6
“HTML should be for engagement … with bullet points of an overview with the ability to use chevrons to go further … with a link to the original [PDF] if you want to go ‘deep” [cites Erskine May website]
Participant 4 What we heard
Participants broadly preferred to read publications (digitally) as PDFs.
Those who (pre-pandemic) would mainly read printed reports were more likely to view PDFs as the closest equivalent and most favoured this format, however there was a willingness from all participants to move content to an HTML format - as long as there still was an option to download as a PDF.
What this means
Switching to an HTML first publishing model won’t put off users from visiting the NAO website and opens a significant opportunity to make your content more accessible, findable, and reusable. 31 All users<br>
slide32. Digital behaviour shifts and workarounds during the pandemic Quotes
“A year ago, it would of been a printed doc… as it’s easier to annotate... but not always the case, if I quickly want to understand something now I would prefer to look at it online”
Participant 14
“Found my own way to classify reports by putting them in OneNote, I also make my own notes in OneNote”
Participant 14
“I like printing stuff off and annotating on paper but ok with PDFs and adding annotations”
Participant 9 What we heard
Many participants missed being able to print reports (driven particularly by a need to make notes and highlight important areas) some changed their view and behaviour (preferring digital for shorter documents or when needing to search) and found digital workarounds for annotating documents such as using OneNote and Office 365.
What this means
One of the key motivations to print reports is the ability to then make notes. There is an opportunity to explore how digital note-taking can help shift more users away from print and PDFs. 32 All users<br>
slide33. Civil servants Findings<br>
slide34. Specialists were the least supported group Quotes
“Nobody knows what it is they are doing so no one will look to them for advice, even on their website, it doesn't tell us what they do.”
Participant 14
“Does not tell me how to solve a problem“ Participant 17
“NAO should be involved with policy development, but wonder if it is their role to do this. Maybe not.”
Participant 16 What we heard
Civil service ‘specialists’ were the user group that felt least supported by the NAO and its website.
There was a strong unmet need for guidance to support common tasks such as writing business cases, conducting appraisals, developing policy and procurement good practice. All participants stated that the NAO could do more to better support their needs. There was also some lack of clarity of NAOs role, remit and offer to them.
What this means
Their unmet needs represent the largest gap and opportunity to improve the website to better support a currently underserved group. 34 Civil servants<br>
slide35. Unmet need for cross-government good practice guidance What we heard
All civil service ‘specialists’ felt that the NAO should do more to support their unmet needs for department-agnostic best practice guidance and case studies to help them solve common cross-government problems.
What this means
There is a clear opportunity to better leverage NAO’s (dept /govt agnostic ) knowledge and learnings to provide this good practice guidance and help them solve problems. 35 Quotes
“So much goes wrong across government and repeat problems that the NAO knows about that they could help civil servants improve... by linking civil servants to guidance and examples of where this has happened before.”
Participant 17
“Should be something on the website that says on ‘are you looking to develop a new policy - here’s some guidance’ with an overview about why this work is important, steps involved and then provide guidance.” Participant 16 Civil servants<br>
slide36. Auditees want more clarity about the audit process Quotes
“We want to know more about the process of audits, so we can make sure we can have the right people helping the NAO”
Participant 5
“Would be good to share best practice across govt for audits… what the NAO expect, what evidence requirements, case studies etc… this would make the process more transparent”
Participant 12 What we heard
Civil service ‘auditees’ (people working for departments being auditing by the NAO) wanted (either for themselves or new/junior staff) clearer guidance and transparency around the audit process, specifically what to expect, what is expected from them and case studies showing them how audits work.
What this means
‘Auditees’ would value practical guidance on the website detailing the audit process to help give them reassurance. 36 Civil servants<br>
slide37. Untapped ‘treasure trove of insight’ Quote
“There is this treasure trove of insight that nobody knows about… because I do collaboration with them outside of audits I know they have this wealth of insights and advice.
Participant 17 What we heard
There was a perception from participants that the NAO had a ‘wealth of insights’ and knowledge, particularly around cross-government learnings and guidance that either existed but they couldn’t find or that the NAO was not making available on the website.
What this means
This is an opportunity to increase awareness of the breadth and depth of NAOs knowledge by doing more to surface and highlight this information on the website. 37 Civil servants<br>
slide38. Information sources - Civil Servants 38 Private sector
Harvard Business Review Thought Leadership and trends in ways of working
Lean.org
Organisational best practice and ways of working
Supreme audit institutions
New Zealand ALB Government and Public sector
IPA Government’s centre of expertise for infrastructure and major projects
The Green Book HM Treasury guidance on how to appraise and evaluate policies, projects and programmes We asked participants during the interviews where they look for information to support their role beyond the NAO.
Civil Servant ‘specialists’ gave the following examples which offered them guidance around good working practice, something which they felt was missing from the NAO. Civil servants<br>
slide39. Findings Parliamentarians<br>
slide40. New and infrequent users faced a steep learning curve What we heard
New and infrequent users faced a steep learning curve to find information they were looking for and often gave up. They would look to colleagues more familiar with the website or get in touch directly with NAO staff if they had an existing contact.
What this means
There is a need to make website more user friendly and intuitive to new and less frequent users.
Most frequent, long time users of the website have become accustomed to its limitations as ‘expert users’, however they also accept there are areas to improve. 40 Quote
“You do have to learn (the website) and spend time with it - there is a learning curve... I mainly use it if I need to refer someone (colleague) to a publication or something coming up. My role is about funnelling information to people. ”
Participant 1
“Took me a long time to find a report [on the website]” Participant 2 Parliamentarians<br>
slide41. Not always clear how the NAO website can support their role What we heard
It wasn’t always clear to Commons Staff how the NAO and its website can help them in their role. Specifically around NAO’s proposition to them. For new staff members coming through this is further exacerbated due to high-levels of staff turnover.
What this means
It may not be feasible for the NAO to actively initiate direct support to new staff. However, this support could be provided by dedicated Commons Staff induction pages or videos on the NAO website. This will enable them to better ‘self-serve’ and clearly understand NAOs proposition for them and the ways it can support their role. 41 Quote
“First time [I went to the NAO website] was when I was being inducted into my current role. My predecessor showed me the pages that would be most relevant for me.”
Participant 1
“I signed up to an induction with NAO to go through what they do, but could not attend. They tend to reach out to me and initiated on their side to say if I need any help or any docs.”
Participant 9
”Didn't even know they had a website!”
Participant 4 Parliamentarians<br>
slide42. Little time or opportunity to read reports in depth Quotes
“I support [committee member in their] Committee role. I try to read the reports before to help shape the questions, but that wasn't sustainable.”
Participant 10
“The chair on the committee goes through the reports and creates bullet points for everyone else. No one else on the committee goes near the NAO report. Better to make it more concise.”
Participant 4 What we heard
There are frequent scenarios where reports need to be distilled on someone else’s behalf so that they understand the key points. This could be ahead of a committee hearing, parliamentary debate or part of a media engagement around a particular event
What this means
There is a strong need to publish a summarised version of the report that will allow someone to quickly understand the context, key findings and any recommendations that were made. This will allow users to quickly determine if a report is relevant to a particular need and have time to form a political angle on the findings 42 Parliamentarians<br>
slide43. MP’s staff key to understanding and distilling reports Quote
“I used to go to the NAO briefings with [MP] in person in our office so we can chat. But now if you don't get an invite you assume you can’t go, it is all on Teams. I do not directly deal with NAO now”
Participant 10 What we heard
Due to MPs being very time poor, there is a dependency on their staff to prepare them for their day-to-day work. They rarely have time to read reports so will often ask their staff to brief them on the key findings and recommendations
What this means
Before the pandemic, some MP’s staff could arrive at an in person report briefing within parliament. Since restrictions are in place, these briefings are often held virtually on Microsoft Teams and consequently staff members are often not invited to attend 43 Parliamentarians<br>
slide44. Keep me updated but only with what's relevant to me Quote
“I would appreciate direct marketing from NAO to be reminded or notified of any work they have done on the specific interests I have”
Participant 6 What we saw
Due to very specific topical interests from advisors and commons staff such as ‘committee specialists’, there was a strong need to be notified of any new reports or developments so that they can remain fully informed.
What this means
Although blanket emails was seen as a potential burden, there is evidence that if the content was targeted to a relevant topical or political interest then it would be of great use. This would mean reduce the dependency on having one-to-one relationships with NAO contacts to be kept updated. 44 Parliamentarians<br>
slide45. Information sources - Parliamentarians 45 Private sector
Chatham House
Royal United Services Institute Government and Public sector
Erskine May on Parliament website
Institute for Government
Office of Budget Responsibility
Office for National Statistics
World Health Organization We asked participants during the interviews where they look for information to support their role beyond the NAO.
They gave the following examples as sources of valuable information.<br>
slide46. Experience Maps<br>
slide47. 47 Experience maps Experience maps are a visual representation of an end-to-end experience that a person goes through when trying to accomplish a goal.
Speaking to a range of users helps us to understand what users do, think and feel over time and may experience throughout their journeys.
Consolidating these into a single map helps us to understand:
How users currently accomplish a goal
How things work (or don’t)
Interdependencies - for example, between different departments or services
Pain points and where things are broken
Opportunities to improve<br>
slide48. Experience Maps - Parliamentary Committee 48 These will be made available during the project<br>
slide49. Experience Maps - Financial audit / VfM report 49 These will be made available during the project<br>
slide50. Recommendations<br>
slide51. Clearer signposting for key user groups to
sections tailored to their needs 51 All users The problem to solve
Many participants looking for role and task specific information were not able to find this on the website (e.g. auditees looking for guidance on the audit process or MP’s staff looking to easily understand what the NAO can offer them and how they can access this).
What we recommend
Clearer and more visible calls-to-action directing specific users to a section of the website with information relevant to their role and links to common tasks.
E.g. ‘Guidance for civil servants’ > ‘Good practice guides’ > ‘When developing policy’
‘Support for MPs and their staff’ > ‘Beginners guide to how the NAO supports you’<br>
slide52. Making it easier to find specific information 52 All users The problem to solve
Whilst users were broadly satisfied with being able to browse or search for specific recent publications, particularly where they recall the report name, however they were less able to find publications where only the broad report theme was searched for due to the lack of relevance from the results and ability to refine the search.
What we recommend
Using a faceted search approach, allows users to see and select multiple (facets) to narrow the number of search results to get to most relevant results quickly.
Add richer content taxonomy via metadata (e.g. tags for ‘themes’ and ‘issues’) to help improve the relevance of results.<br>
slide53. Making it easier to find thematic information 53 All users The problem to solve
Users have an unmet need to find more thematically related information to give them the full breadth and depth of NAO’s knowledge around a specific topic.
What we recommend
Promote the findability and discoverability of thematic content. Users who have specific interests will benefit from having clear visibility of the content (such as reports, articles and data visualisations) related to their themes of interest. Having more related content links will also help the site to drive deeper website engagement.<br>
slide54. Making it easier to discover relevant information 54 All users The problem to solve
The main reason many users currently come to the website is to find a report but then they leave without discovering anything else that could be useful to them.
What we recommend
We need to give website users more compelling reasons to stay on the website (after finding a report) by surfacing more contextually relevant information.
Surfacing all relevant content available (not just publications) including increasing awareness of related support and guidance knowledge and role specific sections of the website.<br>
slide55. Making text easier to read 55 All users The problem to solve
The readability of the text on the website was seen as a pain point, being difficult to read and as result, hard to find what you are looking for and to scan and digest information.
What we recommend
Improve website typography, by increasing text size, line height and spacing across all areas of the website to improve the clarity and legibility of headlines, summaries and links and the readability of content.<br>
slide56. Keep me updated but only when its relevant to me 56 The problem to solve
Participants mentioned how time poor they were (particularly MPs staff) and bemoaned the lack of relevant information of the communications received from the NAO. They wanted more information, but it had to be specifically relevant to their role and interests.
What we recommend
There is an opportunity to offer more tailored ways of keeping users updated based on their specific interests
Receive updates (based on my role, role-specific tasks and specified topics of interest) when:
A new report published
A new report is being worked on (in progress)
Updates to the status of actions/recommendations of a report that I have read
New insight, guidance or case study based on my area of role and role specific tasks All users<br>
slide57. Improve report content findability, reuse & accessibility 57 All users The problem to solve
PDF documents can make your content harder to find, use and maintain and do not work well with assistive technologies like screen readers a lot of the time.
What we recommend
Transition from publishing documents (such as reports) exclusively in PDF format and instead begin publishing documents as accessible ‘HTML gateway pages’.
Gateway pages are html web pages that offer the key points and critical information from a PDF, but still offer users the option to download the full PDF without having to open it in a browser window.<br>
slide58. Make reports more useful 1 of 2 58 The problem to solve
Reports and other PDF publications are currently a digital replication of their printed form.
They offer little (apart from search) to leverage the benefits of digital technology to make them more useful.
Participants reported frustrations with being able to use publications within their work, copying and pasting text and graphics was difficult.
It was also difficult to share or embed NAO content, such as data visualisations in their own online content or reports. All users<br>
slide59. Make reports more useful 2 of 2 59 What we recommend
A key benefit of transitioning to a HTML publishing approach, is being able to make the content within publications more useful in the following ways:
Being ‘bite-sized’ and digestible, with structured HTML content, we can help users to quickly find the most valuable sections and choose to see only what's of interest to them
Easy to find, by more report content being indexed within site search and connected to other type of relevant content
Usable, embeddable and shareable when users are using NAO content in their own work
Dynamic and visually engaging by embedding more real time data and visualisations All users<br>
slide60. Make reports more annotatable 60 The problem to solve
A key user motivation to print NAO publications is being able to then add notes.
During the pandemic, users have found digital workarounds to simulate this, such as annotating documents in digital note taking apps like OneNote.
What we recommend
There is an opportunity to explore how digital technology can help shift more users away from print and PDFs.
This could be a hypothesis to test, e.g. we believe that if we publish a few reports in digital formats that support annotation and ease of reuse (such as G Suite and Office 365) this will result in more users choosing to use digital. We know this is true, if users choose to access these reports in these formats instead of PDFs. All users<br>
slide61. Recommendations Civil servants<br>
slide62. Better communicate your offer to civil servants 62 Civil servants The problem to solve
All groups of civil servants (collaborators and specialists) were not completely clear around NAOs value proposition or support to them.
Collaborators wanted a clearer sense of how the NAO works with them whereas specialists were either completely unaware of NAO’s offer or unclear what their statutory remit covered.
What we recommend
Clear and visible call-to-action on the home page and primary site navigation to guide civil servants to a section dedicated to funneling different user groups to role and task-specific guidance, along with simply and concisely explained NAOs value proposition to them .<br>
slide63. Give ‘specialists’ the guidance to solve common problems 63 The problem to solve
There was a clear unmet need for guidance to support ‘specialists’ with common tasks such as writing business cases, conducting appraisals, developing policy and procurement good practice.
How we might improve
We could use a guided navigation approach, which asks users to ‘self identify’ their role, area of work and what problem they need help with, for example: I am a [Civil Servant] working in [ specialism e.g. policy ] Looking for help with… [ problem to solve e.g. developing new policy ]
Then take them to a ‘resource hub’ with good practice guides, case studies, reusable frameworks along with articles to any relevant NAO insight and thought leadership. Civil servants<br>
slide64. Give auditees clear guidance around the audit process 64 The problem to solve
Civil service ‘auditees’ (people working in departments being audited) wanted (either for themselves or new/junior staff) clearer guidance and transparency around the audit process to give them greater clarity and reassurance to help allay their concerns.
How we might improve
Guide ‘auditees’ via clearer signposting to a ‘collaborator resource hub’, clearly explaining the audit process, the relationship with NAO, what is expected from them and case studies demonstrating good audit practice along with FAQ’s, points of contact and links to relevant resources. Civil servants<br>
slide65. Recommendations Parliamentarians<br>
slide66. Share your cross-gov departmental knowledge 66 What we recommend
The NAO is seen to be in a privileged and unique position of having insights on topics and themes that span multiple departments. Parliamentarians working on a specific topic would benefit from knowing these themes exist and the recommendations that surround them
How we might do this
Curate dedicated theme pages that are common across government departments that communicate the findings, recommendations and case studies. Parliamentarians<br>
slide67. Help me to quickly understand a report 67 What we recommend
It is a common theme that MPs need get up-to-speed quickly on a particular topic or report. For example, a committee hearing has been called or a topical event is being covered in the media that requires a MP’s reaction. This need is currently being facilitated by the MP’s staff briefing them with short lead times.
How we might do this
This could be achieved by providing MPs with an intuitive destination page for topical coverage and shortened versions of reports (with key findings and recommendations) on your website. By having these shortened pages on your website, it will make finding and sharing key report information easier than by using traditional PDF documents. Parliamentarians<br>
slide68. Allow me to attend your report briefings 68 What we recommend
There is a MP’s need to allow their staff members to attend NAO report briefings. This allows the staff members to familiarise themselves with the findings and also help support the MPs preparations. We recommend by allowing details of the schedule of briefings being made available will allow staff members to attend.
How we might do this
By having a calendar of Parliamentary briefings available on your website will allow associated MP staff members to ‘request to attend’ a particular briefing. This would be most beneficial to virtual briefings where currently invitations are sent to a closed list which often doesn’t include the staff members. Parliamentarians<br>
slide2. Contents Introduction
Objectives
Approach
Understanding your audiences 2 5. Executive summary
6. Findings by theme
7. Recommendations
8. Next steps 30 mins 20 mins 40 mins Q&A<br>
slide3. Introduction<br>
slide4. 1. Introduction 1.1. Background
1.2. Stakeholder engagement 4<br>
slide5. 5 The NAO website is a key channel to reach your audiences. This is an opportunity to invest in developing the technology, design and content of the website so that it better supports your new strategy, enabling the NAO to provide more accessible insight and increase its impact.
NAO commissioned Mace & Menter to run a discovery project to better understand its website audiences and how it might better serve their needs.
This will support and evidence your business case for investment. Background<br>
slide6. 6 Stakeholder engagement A key part of the discovery project is to gain a clear understanding of the expectations from key NAO internal stakeholders.
So far we have conducted in-depth interviews with 12 NAO stakeholders
from across the organisation (knowledge, VfM, ERT and PRT).
We will continue to further engage with stakeholders during the concepting phase of the project.
We welcome any further engagement after this presentation for follow up.<br>
slide7. Objectives<br>
slide8. 2. Objectives 2.1. Project objectives
2.2. Scope of work
2.3. Where we are 8<br>
slide9. 9 Provide evidence to support the business case for redeveloping the website
Understand your key website audiences and their needs from the NAO
Understand how well the website is currently supporting their needs
Uncover the unmet needs of your website users
Validate concepts of how the website could better support unmet needs
Validate the technical feasibility of our recommendations Project objectives<br>
slide10. 10 In scope
Deep website-related focus on Parliamentarians and Civil Servants groups (who we were able speak to)
Snapshot of the website experience of a broad range of audiences
Not in scope
Comprehensive research study into all types of Civil Service user groups
Comprehensive research study into all Parliamentarian user groups Scope of work<br>
slide11. 11 Done
Understood organisational needs and existing research
Conducted qualitative user research to deeply understand needs of key audiences
Surveyed all key NAO audiences to understand broader needs *
Distilled our research into clear insights and recommendations
Next
Work with you to prioritise recommendations as hypotheses that we can test
Work with you to create website concepts to test with our two user groups
Report with user feedback and design recommendations from concept testing
Review the technical implications and feasibility of the recommendations Where we are so far * Survey findings to be presented separately<br>
slide12. Approach<br>
slide13. 3. Approach 13 3.1. Approach
3.2. Methodology
3.3. Recruitment<br>
slide14. 14 Mixed method research approach
A critical part of our work is to gain a deep understanding of your key audiences
We do this in a number of ways combining both qualitative and quantitative research methods to ensure that we balance the need for a deep of understanding of your core audience groups whilst also gaining the breadth to cover broader audiences. Approach<br>
slide15. 15 Quantitative research
Analytics - We reviewed all of your relevant analytics data to help inform our understanding of current website behaviour
Surveys* - We have surveyed a broad spectrum of your audience groups to increase the breadth and depth of our insight
Response - promoted via the website, social media and sent out to all key NAO audience groups mailing lists, we have so far received:
842 responses Methodology * Survey findings to be presented separately<br>
slide16. 16 Qualitative research
We conducted a total of 17 semi-structured 1 hour depth interviews with:
X7 Parliamentarians
X10 Civil servants Methodology<br>
slide17. Understanding our audiences Persona summary<br>
slide18. Persona summary Please note:
We won’t have time to go through the detailed persona groups today, however here’s an overview to help give you some context when we are referring to specific types of users.
We will share the complete presentation in due course. 18<br>
slide19. Civil Servants - Persona summary 19 Collaborators Specialists Collaborators are civil servants who
work with the NAO to:
Support its financial & VfM audits
or
Provide subject matter expertise and advice to help inform reports and investigations. Specialists are civil servants who:
who work in strategic roles driving policy development, project/programme delivery or in operations<br>
slide20. Civil Servants - Persona summary 20 Audit Collaborator Specialist: Policy / Operations / Delivery Report Collaborator Collaborators Specialists Who they might be Director or grade 6 working in finance / audit / risk assurance What they do Manage and audit financial accounts and risk assurance
NAO related goals
Work with NAO to support VFM & financial audits of their departments + sign off Who they might be Directors / specialists in govt depts or from a public body
What they do Lead and manage specialist govt depts or public bodies
NAO related goals
Offer subject matter expertise
Advise on report focus areas Supply knowledge / data Who they might be Senior (grade 6): Head of Policy / Delivery / Operations or public scrutiny body specialists
What they do Devise processes & ways of working to support policy development, delivery planning, business case appraisals often during the inception phase.
NAO related goals
Understand NAOs value proposition Best practice guidance on ways of working Cross-govt learnings & case studies Shared interest Partner Collaborator Who they might be Director level SCS1 (treasury) official What they do Scrutinise public spending & value for money / attend PAC committee meetings
NAO related goals
Understand NAOs remit and how best to work together<br>
slide21. 21 Library Researcher Political Advisor Committee Specialist Commons Staff MPs Staff Clerk Parliamentarians - Persona summary What they do Answer private enquires from Members of Parliament
NAO related goals
Reference NAO reports in their findings Who they might be Economist, statistician, financial analyst, policy expert
What they do Provide specialist help to select committees, writing briefs & reports
NAO related goals
Reference NAO reports in their own reports Who they might be Staff of a Select Committee such as PAC, Treasury, DCMS What they do Support the day-to-day running of the committee and its members
NAO related goals
Direct members to NAO reports and recommendations Who they might be Chief of Staff, Senior Political Advisor What they do Support a MP in their day-to-day tasks, including preparing for briefings, debates and committee meetings
NAO related goals
Looking at particular topics, themes or constituency interests that match those of the MP they support Scrutiny Unit Person* Who they might be Legal specialist, statistician, financial analysts, economist,
policy specialist What they do Providing scrutiny of public expenditure and pre‐legislative draft bills
NAO related goals
Use NAO reports to scrutinise public spending * Did not interview in this study<br>
slide22. Executive summary<br>
slide23. 4. Executive summary Broad unmet need for department agnostic theme and issue based knowledge on the website
Finding reports is the primary motivation to visit the website, however once found, users were offered no compelling reasons to stay on the website and would leave
Finding recent & specific reports was seen as fairly easy, however finding thematic content was difficult
Civil service ‘specialists’ were the group that felt their needs were least supported by the NAO
Clear unmet need from all civil service ‘specialists’ for cross-government best practice guides and case studies to show them how to improve ways of working in their departments
MPs staff are very time poor, having to find and distill information quickly, they did not want more general newsletters, they wanted concise information tailored to be relevant to the interests of their MP & constituents
Civil service ‘auditees’ wanted clearer guidance and transparency about the audit process 23<br>
slide24. Findings<br>
slide25. 5. Findings 5.1. All users
5.2. Civil servants
5.3. Parliamentarians 25 5.4. Personas
5.5. Experience maps<br>
slide26. All users Findings<br>
slide27. Overall website experience Quotes
“Took me a long time to find a report”
Participant 2
“Search function isn't that great”
Participant 10
“Home page looks quite busy... difficult for me to quickly understand if there’s anything relevant to me”
Participant 11
“I find the website very text heavy, text is also quite small, compared to GOV.UK where it is a bit more spaced out and inviting.”
Participant 15 What we heard
Participants across all user groups experienced issues with the website. Those whose need was only to find specific recent reports, were the most satisfied, whereas those looking for broader information were much less satisfied.
The design of the website was seen as dated, ‘busy’ and ‘text heavy’.
Parliamentarian users were more likely to have issues with the findability of content, whereas Civil Servants (specialists in particular) were unsatisfied with the lack of content for them. Some Parliamentarian users felt that the website performed adequately, however this view was to an extent informed by their low expectation from public sector websites as a whole. 27 All users<br>
slide28. Once users find a report they have little reason to stay Quotes
“There is this treasure trove of insight that nobody knows about”
Participant 17
“People think the NAO just does VfM and Audit reports, they do so much more and people should be able to access it”
Participant 16 What we heard
Finding reports was most likely motivation to visit the website, however once found - participants were offered no compelling reasons to stay on the website and would leave. For many participants, the website was merely a destination to find publications and awareness of other types of content was very low.
What this means
There was clear unmet need to surface and join up more related and diverse content (not just reports) giving your users a reason to stay on the website, supporting onward journeys and more engagement into NAOs depth and breadth of knowledge. 28 All users<br>
slide29. Ease of finding reports was dependent on recall and recency Quotes
“Search function isn't that great. Not sure what category it is in... have used filters...hard to know how to filter”
Participant 7
“It is often difficult to know which type of report is which”
Participant 15
“Tend to use Google as it’s much easier”
Participant 2 What we heard
Using the website to find recent and specific reports where the participant remembered the report name was seen as being fairly easy. However, finding reports based on topics or where the report name was not fully remembered, was difficult. Many participants felt that searching via Google was easier than using NAOs site search.
What this means
The website search and filter features does not adequately support users finding topical information and easily being able to narrow down and find what they are looking for. 29 All users<br>
slide30. Need for department agnostic themes & issue based knowledge Quotes
“Being more of a government resource would be helpful... themes from audits and best practice (cross-government) with examples of positive case studies’’
Participant 13
“If it was more agnostic of department or reports and more about a particular problem I would use it more”
Participant 17
“Be good to see a more informal blogs, e.g. BoE’s Bank Underground… good to have themes such as 'procurement'”
Participant 2 What we heard
There was a clear unmet need for department agnostic themes and issue based knowledge on the website from both parliamentarians and civil servants.
What this means
MP’s staff wanted to understand what knowledge the NAO had on specific issues important to their MP and constituents.
Civil servant ‘specialists’ wanted to find thematic information to help support their roles such as guidance on procurement or working from home during the pandemic. 30 All users<br>
slide31. PDF was the favoured report format but there was openness for change Quotes
“I much prefer it to be in PDF, I hate HTML but it wouldn't put me off if it was in HTML as long as I could still download the PDF”
Participant 9
“As long as you have option to print out the report even from HTML format it will be fine”
Participant 6
“HTML should be for engagement … with bullet points of an overview with the ability to use chevrons to go further … with a link to the original [PDF] if you want to go ‘deep” [cites Erskine May website]
Participant 4 What we heard
Participants broadly preferred to read publications (digitally) as PDFs.
Those who (pre-pandemic) would mainly read printed reports were more likely to view PDFs as the closest equivalent and most favoured this format, however there was a willingness from all participants to move content to an HTML format - as long as there still was an option to download as a PDF.
What this means
Switching to an HTML first publishing model won’t put off users from visiting the NAO website and opens a significant opportunity to make your content more accessible, findable, and reusable. 31 All users<br>
slide32. Digital behaviour shifts and workarounds during the pandemic Quotes
“A year ago, it would of been a printed doc… as it’s easier to annotate... but not always the case, if I quickly want to understand something now I would prefer to look at it online”
Participant 14
“Found my own way to classify reports by putting them in OneNote, I also make my own notes in OneNote”
Participant 14
“I like printing stuff off and annotating on paper but ok with PDFs and adding annotations”
Participant 9 What we heard
Many participants missed being able to print reports (driven particularly by a need to make notes and highlight important areas) some changed their view and behaviour (preferring digital for shorter documents or when needing to search) and found digital workarounds for annotating documents such as using OneNote and Office 365.
What this means
One of the key motivations to print reports is the ability to then make notes. There is an opportunity to explore how digital note-taking can help shift more users away from print and PDFs. 32 All users<br>
slide33. Civil servants Findings<br>
slide34. Specialists were the least supported group Quotes
“Nobody knows what it is they are doing so no one will look to them for advice, even on their website, it doesn't tell us what they do.”
Participant 14
“Does not tell me how to solve a problem“ Participant 17
“NAO should be involved with policy development, but wonder if it is their role to do this. Maybe not.”
Participant 16 What we heard
Civil service ‘specialists’ were the user group that felt least supported by the NAO and its website.
There was a strong unmet need for guidance to support common tasks such as writing business cases, conducting appraisals, developing policy and procurement good practice. All participants stated that the NAO could do more to better support their needs. There was also some lack of clarity of NAOs role, remit and offer to them.
What this means
Their unmet needs represent the largest gap and opportunity to improve the website to better support a currently underserved group. 34 Civil servants<br>
slide35. Unmet need for cross-government good practice guidance What we heard
All civil service ‘specialists’ felt that the NAO should do more to support their unmet needs for department-agnostic best practice guidance and case studies to help them solve common cross-government problems.
What this means
There is a clear opportunity to better leverage NAO’s (dept /govt agnostic ) knowledge and learnings to provide this good practice guidance and help them solve problems. 35 Quotes
“So much goes wrong across government and repeat problems that the NAO knows about that they could help civil servants improve... by linking civil servants to guidance and examples of where this has happened before.”
Participant 17
“Should be something on the website that says on ‘are you looking to develop a new policy - here’s some guidance’ with an overview about why this work is important, steps involved and then provide guidance.” Participant 16 Civil servants<br>
slide36. Auditees want more clarity about the audit process Quotes
“We want to know more about the process of audits, so we can make sure we can have the right people helping the NAO”
Participant 5
“Would be good to share best practice across govt for audits… what the NAO expect, what evidence requirements, case studies etc… this would make the process more transparent”
Participant 12 What we heard
Civil service ‘auditees’ (people working for departments being auditing by the NAO) wanted (either for themselves or new/junior staff) clearer guidance and transparency around the audit process, specifically what to expect, what is expected from them and case studies showing them how audits work.
What this means
‘Auditees’ would value practical guidance on the website detailing the audit process to help give them reassurance. 36 Civil servants<br>
slide37. Untapped ‘treasure trove of insight’ Quote
“There is this treasure trove of insight that nobody knows about… because I do collaboration with them outside of audits I know they have this wealth of insights and advice.
Participant 17 What we heard
There was a perception from participants that the NAO had a ‘wealth of insights’ and knowledge, particularly around cross-government learnings and guidance that either existed but they couldn’t find or that the NAO was not making available on the website.
What this means
This is an opportunity to increase awareness of the breadth and depth of NAOs knowledge by doing more to surface and highlight this information on the website. 37 Civil servants<br>
slide38. Information sources - Civil Servants 38 Private sector
Harvard Business Review Thought Leadership and trends in ways of working
Lean.org
Organisational best practice and ways of working
Supreme audit institutions
New Zealand ALB Government and Public sector
IPA Government’s centre of expertise for infrastructure and major projects
The Green Book HM Treasury guidance on how to appraise and evaluate policies, projects and programmes We asked participants during the interviews where they look for information to support their role beyond the NAO.
Civil Servant ‘specialists’ gave the following examples which offered them guidance around good working practice, something which they felt was missing from the NAO. Civil servants<br>
slide39. Findings Parliamentarians<br>
slide40. New and infrequent users faced a steep learning curve What we heard
New and infrequent users faced a steep learning curve to find information they were looking for and often gave up. They would look to colleagues more familiar with the website or get in touch directly with NAO staff if they had an existing contact.
What this means
There is a need to make website more user friendly and intuitive to new and less frequent users.
Most frequent, long time users of the website have become accustomed to its limitations as ‘expert users’, however they also accept there are areas to improve. 40 Quote
“You do have to learn (the website) and spend time with it - there is a learning curve... I mainly use it if I need to refer someone (colleague) to a publication or something coming up. My role is about funnelling information to people. ”
Participant 1
“Took me a long time to find a report [on the website]” Participant 2 Parliamentarians<br>
slide41. Not always clear how the NAO website can support their role What we heard
It wasn’t always clear to Commons Staff how the NAO and its website can help them in their role. Specifically around NAO’s proposition to them. For new staff members coming through this is further exacerbated due to high-levels of staff turnover.
What this means
It may not be feasible for the NAO to actively initiate direct support to new staff. However, this support could be provided by dedicated Commons Staff induction pages or videos on the NAO website. This will enable them to better ‘self-serve’ and clearly understand NAOs proposition for them and the ways it can support their role. 41 Quote
“First time [I went to the NAO website] was when I was being inducted into my current role. My predecessor showed me the pages that would be most relevant for me.”
Participant 1
“I signed up to an induction with NAO to go through what they do, but could not attend. They tend to reach out to me and initiated on their side to say if I need any help or any docs.”
Participant 9
”Didn't even know they had a website!”
Participant 4 Parliamentarians<br>
slide42. Little time or opportunity to read reports in depth Quotes
“I support [committee member in their] Committee role. I try to read the reports before to help shape the questions, but that wasn't sustainable.”
Participant 10
“The chair on the committee goes through the reports and creates bullet points for everyone else. No one else on the committee goes near the NAO report. Better to make it more concise.”
Participant 4 What we heard
There are frequent scenarios where reports need to be distilled on someone else’s behalf so that they understand the key points. This could be ahead of a committee hearing, parliamentary debate or part of a media engagement around a particular event
What this means
There is a strong need to publish a summarised version of the report that will allow someone to quickly understand the context, key findings and any recommendations that were made. This will allow users to quickly determine if a report is relevant to a particular need and have time to form a political angle on the findings 42 Parliamentarians<br>
slide43. MP’s staff key to understanding and distilling reports Quote
“I used to go to the NAO briefings with [MP] in person in our office so we can chat. But now if you don't get an invite you assume you can’t go, it is all on Teams. I do not directly deal with NAO now”
Participant 10 What we heard
Due to MPs being very time poor, there is a dependency on their staff to prepare them for their day-to-day work. They rarely have time to read reports so will often ask their staff to brief them on the key findings and recommendations
What this means
Before the pandemic, some MP’s staff could arrive at an in person report briefing within parliament. Since restrictions are in place, these briefings are often held virtually on Microsoft Teams and consequently staff members are often not invited to attend 43 Parliamentarians<br>
slide44. Keep me updated but only with what's relevant to me Quote
“I would appreciate direct marketing from NAO to be reminded or notified of any work they have done on the specific interests I have”
Participant 6 What we saw
Due to very specific topical interests from advisors and commons staff such as ‘committee specialists’, there was a strong need to be notified of any new reports or developments so that they can remain fully informed.
What this means
Although blanket emails was seen as a potential burden, there is evidence that if the content was targeted to a relevant topical or political interest then it would be of great use. This would mean reduce the dependency on having one-to-one relationships with NAO contacts to be kept updated. 44 Parliamentarians<br>
slide45. Information sources - Parliamentarians 45 Private sector
Chatham House
Royal United Services Institute Government and Public sector
Erskine May on Parliament website
Institute for Government
Office of Budget Responsibility
Office for National Statistics
World Health Organization We asked participants during the interviews where they look for information to support their role beyond the NAO.
They gave the following examples as sources of valuable information.<br>
slide46. Experience Maps<br>
slide47. 47 Experience maps Experience maps are a visual representation of an end-to-end experience that a person goes through when trying to accomplish a goal.
Speaking to a range of users helps us to understand what users do, think and feel over time and may experience throughout their journeys.
Consolidating these into a single map helps us to understand:
How users currently accomplish a goal
How things work (or don’t)
Interdependencies - for example, between different departments or services
Pain points and where things are broken
Opportunities to improve<br>
slide48. Experience Maps - Parliamentary Committee 48 These will be made available during the project<br>
slide49. Experience Maps - Financial audit / VfM report 49 These will be made available during the project<br>
slide50. Recommendations<br>
slide51. Clearer signposting for key user groups to
sections tailored to their needs 51 All users The problem to solve
Many participants looking for role and task specific information were not able to find this on the website (e.g. auditees looking for guidance on the audit process or MP’s staff looking to easily understand what the NAO can offer them and how they can access this).
What we recommend
Clearer and more visible calls-to-action directing specific users to a section of the website with information relevant to their role and links to common tasks.
E.g. ‘Guidance for civil servants’ > ‘Good practice guides’ > ‘When developing policy’
‘Support for MPs and their staff’ > ‘Beginners guide to how the NAO supports you’<br>
slide52. Making it easier to find specific information 52 All users The problem to solve
Whilst users were broadly satisfied with being able to browse or search for specific recent publications, particularly where they recall the report name, however they were less able to find publications where only the broad report theme was searched for due to the lack of relevance from the results and ability to refine the search.
What we recommend
Using a faceted search approach, allows users to see and select multiple (facets) to narrow the number of search results to get to most relevant results quickly.
Add richer content taxonomy via metadata (e.g. tags for ‘themes’ and ‘issues’) to help improve the relevance of results.<br>
slide53. Making it easier to find thematic information 53 All users The problem to solve
Users have an unmet need to find more thematically related information to give them the full breadth and depth of NAO’s knowledge around a specific topic.
What we recommend
Promote the findability and discoverability of thematic content. Users who have specific interests will benefit from having clear visibility of the content (such as reports, articles and data visualisations) related to their themes of interest. Having more related content links will also help the site to drive deeper website engagement.<br>
slide54. Making it easier to discover relevant information 54 All users The problem to solve
The main reason many users currently come to the website is to find a report but then they leave without discovering anything else that could be useful to them.
What we recommend
We need to give website users more compelling reasons to stay on the website (after finding a report) by surfacing more contextually relevant information.
Surfacing all relevant content available (not just publications) including increasing awareness of related support and guidance knowledge and role specific sections of the website.<br>
slide55. Making text easier to read 55 All users The problem to solve
The readability of the text on the website was seen as a pain point, being difficult to read and as result, hard to find what you are looking for and to scan and digest information.
What we recommend
Improve website typography, by increasing text size, line height and spacing across all areas of the website to improve the clarity and legibility of headlines, summaries and links and the readability of content.<br>
slide56. Keep me updated but only when its relevant to me 56 The problem to solve
Participants mentioned how time poor they were (particularly MPs staff) and bemoaned the lack of relevant information of the communications received from the NAO. They wanted more information, but it had to be specifically relevant to their role and interests.
What we recommend
There is an opportunity to offer more tailored ways of keeping users updated based on their specific interests
Receive updates (based on my role, role-specific tasks and specified topics of interest) when:
A new report published
A new report is being worked on (in progress)
Updates to the status of actions/recommendations of a report that I have read
New insight, guidance or case study based on my area of role and role specific tasks All users<br>
slide57. Improve report content findability, reuse & accessibility 57 All users The problem to solve
PDF documents can make your content harder to find, use and maintain and do not work well with assistive technologies like screen readers a lot of the time.
What we recommend
Transition from publishing documents (such as reports) exclusively in PDF format and instead begin publishing documents as accessible ‘HTML gateway pages’.
Gateway pages are html web pages that offer the key points and critical information from a PDF, but still offer users the option to download the full PDF without having to open it in a browser window.<br>
slide58. Make reports more useful 1 of 2 58 The problem to solve
Reports and other PDF publications are currently a digital replication of their printed form.
They offer little (apart from search) to leverage the benefits of digital technology to make them more useful.
Participants reported frustrations with being able to use publications within their work, copying and pasting text and graphics was difficult.
It was also difficult to share or embed NAO content, such as data visualisations in their own online content or reports. All users<br>
slide59. Make reports more useful 2 of 2 59 What we recommend
A key benefit of transitioning to a HTML publishing approach, is being able to make the content within publications more useful in the following ways:
Being ‘bite-sized’ and digestible, with structured HTML content, we can help users to quickly find the most valuable sections and choose to see only what's of interest to them
Easy to find, by more report content being indexed within site search and connected to other type of relevant content
Usable, embeddable and shareable when users are using NAO content in their own work
Dynamic and visually engaging by embedding more real time data and visualisations All users<br>
slide60. Make reports more annotatable 60 The problem to solve
A key user motivation to print NAO publications is being able to then add notes.
During the pandemic, users have found digital workarounds to simulate this, such as annotating documents in digital note taking apps like OneNote.
What we recommend
There is an opportunity to explore how digital technology can help shift more users away from print and PDFs.
This could be a hypothesis to test, e.g. we believe that if we publish a few reports in digital formats that support annotation and ease of reuse (such as G Suite and Office 365) this will result in more users choosing to use digital. We know this is true, if users choose to access these reports in these formats instead of PDFs. All users<br>
slide61. Recommendations Civil servants<br>
slide62. Better communicate your offer to civil servants 62 Civil servants The problem to solve
All groups of civil servants (collaborators and specialists) were not completely clear around NAOs value proposition or support to them.
Collaborators wanted a clearer sense of how the NAO works with them whereas specialists were either completely unaware of NAO’s offer or unclear what their statutory remit covered.
What we recommend
Clear and visible call-to-action on the home page and primary site navigation to guide civil servants to a section dedicated to funneling different user groups to role and task-specific guidance, along with simply and concisely explained NAOs value proposition to them .<br>
slide63. Give ‘specialists’ the guidance to solve common problems 63 The problem to solve
There was a clear unmet need for guidance to support ‘specialists’ with common tasks such as writing business cases, conducting appraisals, developing policy and procurement good practice.
How we might improve
We could use a guided navigation approach, which asks users to ‘self identify’ their role, area of work and what problem they need help with, for example: I am a [Civil Servant] working in [ specialism e.g. policy ] Looking for help with… [ problem to solve e.g. developing new policy ]
Then take them to a ‘resource hub’ with good practice guides, case studies, reusable frameworks along with articles to any relevant NAO insight and thought leadership. Civil servants<br>
slide64. Give auditees clear guidance around the audit process 64 The problem to solve
Civil service ‘auditees’ (people working in departments being audited) wanted (either for themselves or new/junior staff) clearer guidance and transparency around the audit process to give them greater clarity and reassurance to help allay their concerns.
How we might improve
Guide ‘auditees’ via clearer signposting to a ‘collaborator resource hub’, clearly explaining the audit process, the relationship with NAO, what is expected from them and case studies demonstrating good audit practice along with FAQ’s, points of contact and links to relevant resources. Civil servants<br>
slide65. Recommendations Parliamentarians<br>
slide66. Share your cross-gov departmental knowledge 66 What we recommend
The NAO is seen to be in a privileged and unique position of having insights on topics and themes that span multiple departments. Parliamentarians working on a specific topic would benefit from knowing these themes exist and the recommendations that surround them
How we might do this
Curate dedicated theme pages that are common across government departments that communicate the findings, recommendations and case studies. Parliamentarians<br>
slide67. Help me to quickly understand a report 67 What we recommend
It is a common theme that MPs need get up-to-speed quickly on a particular topic or report. For example, a committee hearing has been called or a topical event is being covered in the media that requires a MP’s reaction. This need is currently being facilitated by the MP’s staff briefing them with short lead times.
How we might do this
This could be achieved by providing MPs with an intuitive destination page for topical coverage and shortened versions of reports (with key findings and recommendations) on your website. By having these shortened pages on your website, it will make finding and sharing key report information easier than by using traditional PDF documents. Parliamentarians<br>
slide68. Allow me to attend your report briefings 68 What we recommend
There is a MP’s need to allow their staff members to attend NAO report briefings. This allows the staff members to familiarise themselves with the findings and also help support the MPs preparations. We recommend by allowing details of the schedule of briefings being made available will allow staff members to attend.
How we might do this
By having a calendar of Parliamentary briefings available on your website will allow associated MP staff members to ‘request to attend’ a particular briefing. This would be most beneficial to virtual briefings where currently invitations are sent to a closed list which often doesn’t include the staff members. Parliamentarians<br>