Anonymization

Classes of ecgdatakit.anonymize. Usage, options and the catalog are described in Anonymization.

Import: from ecgdatakit.anonymize import Anonymizer, Report, Catalog, DatasetLocked

from ecgdatakit.anonymize import Anonymizer

reports = Anonymizer("/path/to/source", datasets=True, patients_dir_name="RAW").run()
for report in reports:
    print(report.dataset, report.anonymized, report.failed, report.errors)

class ecgdatakit.anonymize.Anonymizer

Bases: object

Anonymize the ECG files under a source path.

Raw files are only read, never modified.

Parameters:

• source (str | Path) – Folder to anonymize, or a single file. A file is anonymized on its own: its folder holds the output folder and the catalog.

• datasets (bool) – False (default): source is one dataset. True: every sub-folder of source is a separate dataset, with its own output folder and catalog.

• patients_dir_name (str | None) – Name of the folder that holds the patient folders, the same in every dataset (for example "RAW"). Only files under this folder are read. When None, every file of every folder is read (except the output folder).

• group_by_patient_folders (bool) – False (default): files are not grouped; each file gets its own patient pseudonym, and the patient ID field is not replaced (it may hold something else than a patient ID). True: each patient folder is one patient and its files share one patient pseudonym. The patient folders are the sub-folders of the patients_dir_name folders, or without patients_dir_name the folders directly inside the dataset. The patient ID field of a file in a patient folder is replaced by the patient pseudonym (a date is kept); a value that does not match the folder name is reported as risk.

• anonymize_patient_folders (bool) – False (default): folder names are copied as they are. True (needs group_by_patient_folders): each patient folder is named after the patient pseudonym in the output, and its name is replaced wherever it appears in the files. Every other folder, sub-folders of the patient folder included, keeps its name.

• out_dir (str) – Output folder created in each dataset, with the same sub-folders as the dataset.

• catalog_name (str) – CSV catalog file created in each dataset.

• threads (int) – Parallel workers.

• settle_seconds (float) – Files modified less than this many seconds ago are left for a later run (they may still be copied).

• lock_stale_seconds (float) – Age after which a lock left by a stopped run is ignored.

• dry_run (bool) – Report what would be done, write nothing.

__init__(source, datasets=False, patients_dir_name=None, group_by_patient_folders=False, anonymize_patient_folders=False, out_dir='ANONYMIZED', catalog_name='anonymization_catalog.csv', threads=8, settle_seconds=0.0, lock_stale_seconds=43200, dry_run=False)

Parameters:

• source (str | Path)

• datasets (bool)

• patients_dir_name (str | None)

• group_by_patient_folders (bool)

• anonymize_patient_folders (bool)

• out_dir (str)

• catalog_name (str)

• threads (int)

• settle_seconds (float)

• lock_stale_seconds (float)

• dry_run (bool)

Return type:

None

datasets()

The source itself, or its sub-folders with datasets=True.

Return type:

list[Path]

dataset_of(path)

Return type:

Path

Parameters:

path (Path)

catalog_path(dataset)

Return type:

Path

Parameters:

dataset (Path)

run(paths=None, recursive=True, datasets=None)

Anonymize every dataset, the named datasets, or only paths.

Return type:

list[Report]

Parameters:

• paths (list[str | Path] | None)

• recursive (bool)

• datasets (list[str] | None)

run_dataset(dataset, scope=None, recursive=True)

Return type:

Report

Parameters:

• dataset (Path)

• scope (list[Path] | None)

• recursive (bool)

stats(dataset)

Counts of the raw files and of the anonymized files, each read from disk (see ecgdatakit.anonymize._stats).

Return type:

list[Stat]

Parameters:

dataset (Path)

stats_and_empty_folders(dataset)

stats(), and the raw patient folders holding no ECG file (nothing is copied for them), relative to the dataset.

Return type:

tuple[list[Stat], list[str]]

Parameters:

dataset (Path)

check(dataset, catalog, entries=None)

Collision and count checks between raw and anonymized files.

Return type:

list[str]

Parameters:

• dataset (Path)

• catalog (Catalog)

• entries (dict[str, Entry] | None)

class ecgdatakit.anonymize.Report

Bases: object

Report(dataset: ‘str’, found: ‘int’ = 0, anonymized: ‘int’ = 0, copied: ‘int’ = 0, changed: ‘int’ = 0, unchanged: ‘int’ = 0, deleted: ‘int’ = 0, failed: ‘int’ = 0, risks: ‘int’ = 0, skipped_unsupported: ‘int’ = 0, waiting: ‘int’ = 0, raw_files: ‘int’ = 0, anonymized_files: ‘int’ = 0, ecg_codes: ‘int’ = 0, patients: ‘int’ = 0, patient_codes: ‘int’ = 0, errors: ‘list[str]’ = <factory>, stats: ‘list’ = <factory>, empty_patient_folders: ‘list[str]’ = <factory>)

dataset: str

found: int = 0

anonymized: int = 0

copied: int = 0

changed: int = 0

unchanged: int = 0

deleted: int = 0

failed: int = 0

risks: int = 0

skipped_unsupported: int = 0

waiting: int = 0

raw_files: int = 0

Raw files found by this run (the same as found).

anonymized_files: int = 0

Of these, files with an anonymized copy on disk.

ecg_codes: int = 0

Distinct ECG pseudonyms of these files.

patients: int = 0

Distinct patients of these files (patient folders, plus files outside a patient folder, each counted as one patient).

patient_codes: int = 0

Distinct patient pseudonyms of these files.

errors: list[str]

stats: list

Counts of the raw and anonymized files of the whole dataset, read from disk (see Anonymizer.stats()), also written to anonymization_stats.csv.

empty_patient_folders: list[str]

nothing is copied for them.

Type:

Raw patient folders holding no ECG file

property ok: bool

__init__(dataset, found=0, anonymized=0, copied=0, changed=0, unchanged=0, deleted=0, failed=0, risks=0, skipped_unsupported=0, waiting=0, raw_files=0, anonymized_files=0, ecg_codes=0, patients=0, patient_codes=0, errors=<factory>, stats=<factory>, empty_patient_folders=<factory>)

Parameters:

• dataset (str)

• found (int)

• anonymized (int)

• copied (int)

• changed (int)

• unchanged (int)

• deleted (int)

• failed (int)

• risks (int)

• skipped_unsupported (int)

• waiting (int)

• raw_files (int)

• anonymized_files (int)

• ecg_codes (int)

• patients (int)

• patient_codes (int)

• errors (list[str])

• stats (list)

• empty_patient_folders (list[str])

Return type:

None

class ecgdatakit.anonymize.Catalog

Bases: object

Rows keyed by raw path (relative to the dataset folder).

Writes go to a temporary file renamed over the catalog, so the CSV is never left half written.

__init__(path)

Parameters:

path (Path)

Return type:

None

get(raw_path)

Return type:

dict[str, str] | None

Parameters:

raw_path (str)

put(row)

Return type:

None

Parameters:

row (dict[str, str])

active()

Return type:

list[dict[str, str]]

used_codes()

Return type:

set[str]

patient_codes()

Patient code by patient folder (every row ever written keeps its code).

Return type:

dict[str, str]

save()

Return type:

None

exception ecgdatakit.anonymize.DatasetLocked

Bases: RuntimeError

Another process is anonymizing this dataset.

Edit this page Report an issue