User Guide

Welcome to the synthpop-py User Guide. This guide provides a detailed introduction to the concepts, algorithms, and workflows used throughout the package. It explains how synthetic data are generated, how the available synthesis methods and preprocessing components work, and how to evaluate the utility and privacy of synthetic datasets.

If you are new to synthpop-py, we recommended starting with the Getting Started Guide. It introduces the package, explains how to install it and walks through the generation of your first synthetic dataset.

The guides below provide more detailed explanations of specific topics:

• 1. Introduction introduces synthetic data and its role in statistical disclosure control. It discusses the motivation for generating synthetic data, common use cases, and the terminology used throughout the package.

• 2. Synthetic data generation describes the synthetic data generation process in synthpop-py. It introduces the synthesis workflow, the Synthesiser class, sequential synthesis, and creating synthetic data.

• 3. Synthesis methods provides an overview of the available synthesis methods, explains when they are appropriate, and describes how they can be customised.

• 4. Data preparation during synthesis explains how data are prepared during synthesis, including preprocessing, encoding categorical variables, and handling missing values.

• 5. Evaluating and improving utility discusses how to assess whether synthetic data preserve the statistical properties of the original dataset. It introduces the utility measures currently implemented in synthpop-py and provides guidance on interpreting the results.

• 6. Evaluating and improving privacy introduces the main types of disclosure risk associated with synthetic data and explains how privacy should be considered when releasing synthetic datasets.

• 7. Visualisations describes the visualisation tools included in synthpop-py for comparing original and synthetic datasets and interpreting utility metrics.

• 8. Custom synthesis methods explains how to extend synthpop-py by implementing custom synthesis methods, preprocessing components, and other extensible parts of the package.

Guides

• 1. Introduction
  • 1.1. Synthetic data
  • 1.2. Why synthpop-py?
  • 1.3. When should you use this package?
  • 1.4. Terminology

• 2. Synthetic data generation
  • 2.1. Overview of the synthesis workflow
  • 2.2. The Synthesiser class
  • 2.3. Preprocessing
  • 2.4. Fitting the synthesiser
  • 2.5. Generating synthetic data
  • 2.6. Reproducibility

• 3. Synthesis methods
  • 3.1. CART synthesis (default method)
  • 3.2. Sample synthesis method
  • 3.3. Copy synthesis method
  • 3.4. Method comparison
  • 3.5. Practical guidance

• 4. Data preparation during synthesis
  • 4.1 Encoding categorical predictors
    • 4.1.1. PCA encoding
    • 4.1.2. Mean encoding
  • 4.2. Handling missing values
    • 4.2.1. Treating missing as category
    • 4.2.2. Predicting missing values
  • 4.3. Customising preprocessing components
    • 4.3.1. Choosing the number of principal components
    • 4.3.2. Changing the missing value handling strategy

• 5. Evaluating and improving utility
  • 5.1. Workflow
  • 5.2. Univariate distributions
  • 5.3. Bivariate distributions and relationships
    • 5.3.1. Standardised propensity Mean Squared Error (S_pMSE)
  • 5.4. Improving utility

• 6. Evaluating and improving privacy
  • 6.1. Privacy risks
    • 6.1.1. Identity disclosure
    • 6.1.2. Attribute disclosure
    • 6.1.3. Membership disclosure

• 7. Visualisations
  • 7.1. Univariate distribution visualisation
  • 7.2. S_pMSE heatmap

• 8. Custom synthesis methods
  • 8.1. Synthesis method workflow
  • 8.2. Required interface
  • 8.3. Implementing a custom synthesis method
  • 8.4. Adding custom processing components