Use case 1: Credit scoring
Credit reference agencies (CRAs) provide information services to lenders in order to help them make decisions on credit applications from individuals. When a credit application is rejected, a lender often informs the applicant that the rejection was made using data from the CRA. The applicant could contact the CRA to ask for a review of their existing credit report. Each lender also has their own ‘score card’, which decides how various information/data are combined when an application is considered. A CRA has a duty to offer meaningful information to the individual.
Use case 2: School admission
The School Admissions team at Southampton allocates school places to children in Southampton every year across the primary, junior, and secondary levels, according to parental preference and oversubscription criteria. Parents who are dissatisfied with the offered places can make enquires to the admission team or appeal the decisions.
1
Scoping Explainable
Scenarios
What
Select specific use case(s) and document stakeholders, processing purposes, current explanation & audit practices
Why
To correctly scope the boundaries of the produced explanations
By who
Business analysts
When
Beginning of design phase
Input
Information about the processing activities that may require/benefit from an explanation
Output
Explainable scenario(s)
Credit card application scenario
Purpose
Approval or rejection of an application for a credit card product using automated decision making and CRA data
Stakeholders
- Banko
- CRA
- Applicant
Current practice
- Privacy policies
- CRAIN
- Call centre
School places admissions scenario
Purpose
Computer-assisted allocation of school places according to parental preference and school policy
Stakeholders
- Local authority (& admission authorities)
- Maintained schools
- Prospective students & guardians
- ICO
Current practice
- Decision letters
- Privacy policies
- Call centres
2
Analyse Policy
Requirements
What
Analyse hard and soft regulation applicable to the scenario
- Legislation
- Codes of conduct
- Best practices
- Business needs / public image
Why
To identify mandatory explainability and accountability requirements
To determine opportunities where discretionary explanations can be used as an additional tool
By who
Legal experts
When
Design phase
Input
Applicable legal, policy and business needs
Output
Sets of applicable rules that contain mandatory/discretionary explainable requirements
Credit card application scenario
- SSFA 1998
- School Admissions Regulations 2014
- School Admissions Appeals Arrangements Regulations 2012
- Equality Act 2010
- Human Rights Act 1998
- DPA 2018
- School Admissions Code 2014
- School Admissions Appeals Code 2012
School places admissions scenario
- CCA 1974
- FCA Regulations
- GDPR/DPA 2018
- ICO guidance
- SCOR guidance
3
Specifying
Explanations
What
- Classify requirements against 9 qualifiers
- Sort by type (mandatory/discretionary)
- Source of requirement
- Audience
- Goal
Why
To identify common building blocks in expected explanations (recurring phrases, common data points)
By who
Legal experts
When
Design phase
Input
Explainable requirements
Output
Explanation specifications
3.1 Determine minimum content
What
Identify the information necessary for your explanations based on your requirement analysis
Why
Explanations must provide all information designated as minimum content
Explanations can contain further information beyond the minimum content
By who
Legal experts
When
Design phase
Input
Explanation specifications
Output
Minimum content of explanation(s)
4
Provenance
Modelling
What
Construct provenance patterns
Why
To support tracking of the elements influencing a decision
By who
Data engineers
When
Design phase
Input
Explanation Requirements; Data Flows
Output
Provenance Patterns; Provenance Traces
For each explanation requirement:
- 4.1 Inspect the application's Data Flows and identify where the minimum required content for all the specified explanations (provided by the Explanation Requirements) can be found
- 4.2 Identify how the data entities identified in Step 4.1 lead to the decision, the possible intermediary data entities and/or activities that connect them, and who/what are responsible for all those (i.e., the agents)
- 4.3 Capture all the above provenance elements and how one influences another in a set of Provenance Patterns.
- 4.4 Instantiate the Provenance Patterns with sample logged data from the application to ensure that the modelled provenance satisfies the provenance requirements, producing a set of sample Provenance Traces
5
Build Provenance
Queries
What
Construct provenance queries for each explanation
Why
To extract the data that must be included in the explanation's narrative
By who
Data engineers
When
Design phase
Input
Explanation Requirements; Provenance Traces
Output
Provenance Queries
For each explanation requirement:
- 5.1 Identify where the data elements required by an explanation's exemplar narratives can be found in the sample provenance traces.
- 5.2 Describe a generic graph pattern that link a decision to all the identified data elements using a graph query language.
6
Build Explanation
Plans
What
Construct the syntax tree for each explanation
Why
To define the explanation's narrative to be generated for each explanation
By who
Data engineers; Provenance experts
When
Design phase
Input
Explanation Requirements; Provenance Queries
Output
Explanation Plans; Explanation Dictionary
For each explanation requirement:
- 6.1 Construct the NLG syntax tree from the exemplar narrative of the explanation.
- 6.2 Replace the data elements of the narrative with the corresponding variables available from the provenance query for the explanation.
- 6.3 Specify the linguistic elements specific to each supported profiles in the Explanation Dictionary.
7
Deploy Explanation
Assistant
What
Configure the Explanation Assistant service
Why
To deploy an software service that generate explanation from application data
By who
Data engineers
When
Design phase
Input
Provenance Patterns; Provenance Queries; Explanation Plans, Explanation Dictionary
Output
Explanation Assistant Service; Sample Explanations
For each explanation requirement:
- 7.1 Configure the Explanation Assistant with the Provenance Patterns, Provenance Queries, Explanation Plans, and Explanation Dictionary produced in the previous tasks.
- 7.2 Test the service with sample data from the Data Flows to generate Sample Explanations.
8 Evaluate
What
Run validation studies with relevant stakeholders
Why
To validate that the produced explanations meet their intended goals for the intended recipient(s)
By who
- Legal experts
- Data engineers
- Business analysts
- Clients
- Data subjects
When
Evaluation phase
Input
Sample Explanations produced by the Explanation Assistant
Output
Feedback for improvement
