Company logo

Department for Business & Trade

Data Quality Lead – Active SC Clearance

London·On-site·6d ago

Salary

£550 – £

Work type

Onsite

Level

lead

Category

Data Governance

About the role

Your new company

One of the largest Public Sector Organisations in the UK managing trade

Your new role

Data Quality Lead – Active SC Clearance

What you'll need to succeed

Role Overview

We are seeking a strategic and technically proficient Data Quality Lead to oversee the ingestion, evaluation, and quality assurance of critical datasets supporting the Global Supply Chains Intelligence Programme (GSCIP). This role will be pivotal in ensuring data integrity, enhancing operational efficiency, and supporting high-impact analysis across government and commercial domains.

This role can only be offered to candidates with Active and Existing SC or DV Clearance.

Key Responsibilities

Data Ingestion & Source Management

  • Oversee ingestion pipelines for key data sources including Sayari, S&P, and Altana.
  • Liaise with internal and external stakeholders (Microdata team, HMG equivalents and external suppliers) to ensure procurement tracking and data quality.
  • Monitor and resolve backlog issues in data quality logs.

Dataset Evaluation & Maintenance

  • Lead on the assessment of data quality of all GSCIP data. This includes exploratory data analysis of new datasets and pushing suppliers to rectify issues.
  • Tracking and logging existing data quality issues by coordinating with GSCIP data scientists and licence users. Prioritise issues based on feedback.
  • Raising issues with suppliers, suggesting solutions (based on DS feedback), communicating updates to GSCIP users, understanding contractual agreements to keep suppliers to account.
  • Write pipelines to ensure that ingested data meets expected format/contractual criteria
  • Proactively conduct EDA and anomaly detection on data sources to identify unknown issues.
  • Produce dashboards and reports to disseminate features of interest of the graph to GSCIP data scientists and users, e.g. where the largest outliers are, where coverage is best/worst etc.
  • Actively work on enhancements to data quality, e.g. developing reproducible pipeline functions that handle data quality features in a standardised way for gscip_utils/dashboards such as transaction deduplication, imputation etc.
  • Investigate the relative value of each data source to feed into future commercial procurement strategy.
  • Track the strengths and weaknesses of each data supplier.
  • Scope out new data sources to fill existing data gaps.
  • Lead annual evaluations of datasets to determine retention and relevance.
  • Conduct IRAP assessments for new datasets.
  • Maintain comprehensive documentation including data dictionaries, caveats, and usage guides.

Cyber Monitoring & Pipeline Development

  • Collaborate with engineers and data scientists to support engineers to build robust, high-quality data pipelines.
  • Explore new data sources, particularly those related to risk and company intelligence.
  • Proactively investigate data anomalies and ensure datasets meet commercial requirements.

Governance of Unowned or Informally Managed Datasets

  • Formalise ownership and tracking for datasets such as HMRC, ComTrade, TDM, and Shipping Instructions.
  • Coordinate with microdata teams to align procurement and sharing protocols.

Learning & Development

  • Design and deliver learning modules to support GSCIP data literacy.
  • Encourage new team members to engage in QA activities to build familiarity with datasets.

Essential Skills & Experience

  • Proven experience in quality assurance of data and working with data engineers.
  • Strong understanding of commercial and government data sources.
  • Ability to manage cross-functional relationships and coordinate across departments.
  • Experience with documentation standards and data governance frameworks.

Desirable Attributes

  • Strategic thinker with a proactive approach to data quality and risk mitigation.
  • Excellent communicator with the ability to translate technical concepts for non-technical audiences.
  • Comfortable working in a fast-paced, multi-stakeholder environment.
  • Familiarity with systems including databricks, Amazon Neptune and AWS

What you'll get in return

This is a fantastic opportunity to join a team making a difference through data here and now and with an excellent reputation in the Public Sector.

Tech stack

DatabricksAWSAmazon Neptune