Introduction
Devon County Council (DCC) is a top tier local authority which follows a strategic plan[1]
focused on working together to improve life across Devon for everyone. It wants to create a place that people enjoy living in, as well as a place where it is enjoyable to work. Devon has a population of over 773,000 from diverse backgrounds, covering 2,500 Sq. miles, with a budget of over £1.1billion.
This article is based on:
Culture – understanding the current culture and being honest about where we currently are with Data Ethics.
Behaviours – two behaviours we want to focus on.
Generosity: Create the conditions for yourself and others to learn, develop and grow. Recognise the contribution each person makes; and give others effective and supportive feedback that promotes learning and development.
Inspiring: Encourage others and give them support and confidence to try new things; challenge and ask questions; and make good decisions. Commit to doing the right thing, not the easy thing.
Smarter Devon is the Council’s data and intelligence change programme. One of the recommendations from the Smarter Devon Strategy 2019 – 2024 is to introduce data ethics to DCC. We do not have any process or guidance currently linked to this topic and this could be a potential risk for the organisation. There are legal risks and also the risk of negative impacts on the people of Devon.
What is the problem?
The Smarter Devon programme started in 2018 with a small team of officers working in partnership with colleagues from the ICT team. The team have been working with Peak Indicators[5] (PI) to deliver a set of data projects across the organisation. This partnership has helped gain a better understanding of the future of data science and why data ethics are important in this context. PI, our delivery partner, has also recorded the following video ‘The Next 10 Years of Data Science in Local Government’. This video explains how everyone can be data literate and this will have benefits for DCC, bringing more confidence and ownership of the data to improve quality. PI have been working with Smarter Devon to support DCC in building future sustainability and our future and our internal staff’s skills. Paul Clough from PI mentions the Governments National Data Strategy (NDS) that recognises the power of data has and how this links to the data revolution.
Over the past two years, the Smarter Devon Programme has delivered increased capability in data teams across the Council together with a centralised cloud-based data repository in Microsoft Azure populated with a series of Data Lakes. As the project with PI draws to a close, we are developing an in-house support team to continue on this work.
Why do we want to introduce Data Ethics to DCC?
Data ethics will help to build trust in our data, it’s the insight that counts. Different scenarios need different approaches and being transparent with our approach will mitigate bias.
The lead for Smarter Devon, Sofie Francis and Technical Architect, Gary Cregan explains why we need data ethics:
“Having a set of data Ethic principles to use in our data projects is an essential pre-requisite before we begin applying more advanced data science techniques. We want to make sure we treat all our citizens equally and fairly and use their data wisely to make good decisions. Sofie Francis – Smarter Devon Programme Lead (March 2022)
“As a local authority, we have an obligation to be open, transparent, and fair in our interactions with the public and internal staff, and this includes our use of data relating to those individuals and groups. Responsibility for the ethical capture, storage and sharing of data relating to individuals and groups sits with us, as does the fair and unbiased use of analytics and intelligence tools to inform decision making. It also falls upon the Authority to address any corruption, misuse, or abuse of data, and to ensure a level of quality and accuracy of data, with an accurate representation of the real world, is used to train artificial intelligence models”. –Gary Cregan – Technical Architect (March 2022)
Demonstrable ethical treatment of data will gain the trust of all stakeholders involved, including those to whom the data relates, leading to further trust the Authority’s intelligence-driven decision making”.
Oliver Dowden CBE MP recognises Data Ethics also in his statement in the National Data Strategy [6]
Part of the UK National Data Strategy is the Government Digital Service – Data Ethics Framework. This guidance is for public sector organisations to develop a change in culture on how to use data appropriately and responsibly. It helps public servants understand ethical considerations, address these within their projects, and encourages responsible innovation. It’s not about being right or wrong it’s about changing the approach of staff and processes by introducing new principles to support a culture change on how to use data appropriately.
Objectives the Smarter Devon programme want to deliver:
- Introduce a set of people focused Data Ethic principles to the organisation
- Raise awareness of what data ethics is and how to apply
- Raise awareness of the different frameworks available for us to use including Government Digital Service – Data Ethics Framework and the Open Data institutes Data Ethics Canvas
- Ensure our data projects are aligned to these principles so we are delivering ethically correct data.
Actions – impacts
The pandemic has shown just how flexible and resilient our Council staff are in keeping things moving forward. Following this a ‘How we work with data’ survey has been completed by 316 officers to help to seek to understand the current culture and create an honest picture about our current data practices .
The results showed good data literacy skills. 79% of colleagues believed they have a good level of data literacy. 76% of the respondents said we use data regularly for reporting. 57% believe we use data to influence policy and decision makers. 80% believe we collect the right data.
An important factor to consider alongside these results is cognitive bias. Self-assessment results were much higher than predicted by preparatory and exploratory work completed by Smarter Devon. This may suggest the presence of the Dunning-Kruger effect[7]? Previous investigation indicated that small pockets of data experts exist, which is at odds with the high level of confidence shown in the self assessment.
Another important factor to consider is sample bias. Those who are interested in data may have been more likely to engage with the assessment, while others may not have felt it a relevant use of time or may be uncomfortable highlighting skill gaps. Sample size may also be important, as 316 people amounts to approximately 6 % of DCC’s total employee count.
In the context of using our data, 38% say we use data to challenge assumptions about the communities we serve. This is reflects a relatively low use of evaluating our services
Bias and exploratory data analysis.
DCC is moving towards a more mature, data-literate organisation. In the approach to machine-learning, we need to consider the risks before moving forward in the data revolution. Understanding the risks involved in each data collection could potentially highlight any widespread intrinsic bias, helping to acknowledge and mitigate in a humanistic manner (general humanism philosophical standpoint)
Caroline Criado Perez[9] at the 2019 Open Data Institute (ODI) Summit quoted that transparency would help towards mitigating Bias by doing the following:
- Understanding what is your data
- Where it has come from
- How you are using it
- What tools are you using?
She wrote ‘Invisible Women’[10] which highlights women are living in a world largely built for men. It exposes the gender data gap – a gap in our knowledge that is at the root of perpetual, systemic discrimination against women, and that has created a pervasive but invisible bias with a profound effect on women’s lives.
An example at DCC is the Race Equality Audit findings[11].The audit followed a clear brief and ensured exploratory data analysis was implemented. The project team was a diverse team of subject experts with an in-depth and broad understanding of data literacy. They applied a system of principles that critically changed past considerations that may not have highlighted any racism. The outcomes were clear and, by applying a clear methodology, a clean data set was published that included many ethical considerations
These were:
- Every interviewee was asked to give content with clear transparency about the purpose of the data collection and an understanding for what the data was to be used for now and in the future.
- As a government organisation ,we had a big emphasis on social responsibility
- Clear set of objectives
- Data Protection Impact assessment completed
- A data literate project team able to think about data in different contexts
- A project team able to apply different approaches when collecting the data (Group and individual research, completed digitally and face-to-face)
We would like to see further exploratory data analysis (EDA) practices applied so further exploration of the data collected matches the original key themes in the recommended pillars.[12]
- EDA would support the findings further and I have suggested to the EDI lead about creating a Power BI dashboard with the data collected to help discover more patterns and anomalies. This may lead onto outliers being highlighted with the data and more interesting relations with the variables collected.
- Using EDA tools and techniques can help identify correlations of data and therefore any demographics missing from the data collection process – including mapping techniques that will help to create graphical visualisations form the results collected across Devon. The council have staff from many locations and using bivariate visualisations from the data collected could help us to understand the relationship between variables. For example, could low poverty high employment in Devon link to higher racism or vice versa?
- This may lead onto further predictive models using a coding tool such as Python to review data sets to predict risk areas of employment in the council with higher risks of racism. This would be a highlight services that may need more interventions applied to raise awareness and understanding of race equality.
When AI technology is introduced to DCC, it is vital that we follow / introduce and follow best practice guidelines, principles, and governance rulings. AI technology can be used to help replace task-based processes and for process automation, However, it should never be used to replace the thought process. The thought process is where Data Ethics comes in. It’s not an algorithm[13] that can be simply applied; it requires careful consideration before proceeding to the AI technology. For thought workers, AI technology must only augment the decision-making process, empowering enhanced informed data decisions.
Tools that can help to evaluate ethical impacts
ODI Data Ethics Canvas[14] tool for anyone who collects, shares, and uses data. It helps organisations to manage ethical issues from the beginning of a data project.
Marie Maloney-Lucey from Public Health Intelligence has used the canvas and reported back (April 2022):
“The data canvas has acted as a useful and thought provoking framework to use when considering data ethics in the work we undertake. It is evident in the Public Health Intelligence team that although there are aspects of the data canvas which are implemented and considered in our approach to work, there are certainly fundamental aspects that are not. Some of t he biggest gaps identified relate to the lack of engagement with relevant stakeholders; openness and transparency; rigor relating to the biggest gaps identified relate to limitations and rights around data sets; and the consideration of negative impacts and possible mitigation measures. Trust in what we do as a Public Health Intelligence team and as a local authority is an essential component in maintaining public confidence around services we deliver and in organisations who could collaborate with us to improve the populations health and wellbeing.”
Government Digital Service – Data Ethics Framework[15] gives guidance for public sector organisations on how to use data appropriately and responsibly when planning, implementing, and evaluating a new policy or service. It is for anyone working directly or indirectly with data projects. It should be used throughout an entire project, revisited throughout and as a living, breathing document, especially when any changes to data collection, storage, analysis happens.
Outcomes
Introducing Data Ethics principles for Devon County Council.
This is only the beginning of our data revolution journey. Supported by the frameworks from ODI and UK Government, Smarter Devon we will conduct data ethic consultations at DCC so everyone’s voice can be included in the design of a set of data ethics principles. We will be using a practice called active learning, involving Action Learning sets[16] to bring the principles to life that are aligned to our corporate principles and behaviours. We are hoping to align the data ethic principles with our corporate principles and behaviours because they are timely and relevant to the organisation here and now and will continue the golden thread through all work streams.
Data Ethics Principles
The Data Ethics Principles may include the following suggestions outlined below.
We will apply different methodologies when collecting the data using consultations, one-to-one interviews and use a digital capture to ensure we have fair and consistent data collections.
Intent
- What is your purpose of collecting this data?
- What society measures will it make better?
- What positive effects will it have for Devon citizens?
- What negative effects could this data project effect?
- What could you do to minimise a negative impact?
- What benefits will these bring?
- How will personal data be anonymised outside of legal obligations?
- How will data be safeguarded from appearing in the public domain?
- How will we record and decide a clear and evidenced purpose behind sharing, linking and analysis of person-level data?
- How we will assess our actions and confirm that they are ethically correct?
- How will we seek, process, and act on feedback?
- How will we ensure that we reach the appropriate audience?
- How will we ensure we do not unintentionally exclude or discriminate?
Culture
- Do you have a diverse project team to understand the current culture?
- How are you going to work outside of your silo?
- How will you communicate your purpose and capture your methodology, metadata, datasets, code, or impact measure to the wider audience?
Learning
- How will we create a space where everyone can feel safe to learn by doing, testing, and reflecting?
- How will we capture what worked and what didn’t?
Clarity
- How will we create understanding and trust in our data?
- How will we demonstrate openness and transparency?
- Where has the data come from?
- How is the data being used?
- What decisions will the data inform and is it sufficient?
- Is the data accurate?
- Is the data being interpreted correctly?
Honesty (Honest and productive conversations)
- Will this data be shared with other organisations?
- Who has the potential for innovation outside of the council that would benefit Devon?
- Have you captured this on the project plan?
- Will this become an open data set that will be published and, if so, under what conditions?
[7] Dunning-Kruger effect | Definition, Examples, & Facts | Britannica
[10] Invisible Women | Caroline Criado Perez
[11] Race Equality Audit 2021 – Equality and Diversity (devon.gov.uk)
[12] Race Equality Audit 202112 Infographic.pdf
[13] What is an algorithm? – BBC Bitesize
[14] The Data Ethics Canvas – The ODI