robustness.defaults module

This module is used to set up arguments and defaults. For information on how to use it, see Step 2 of the Using robustness as a general training library (Part 1: Getting started) walkthrough.

robustness.defaults.TRAINING_DEFAULTS = {<class 'robustness.datasets.CIFAR'>: {'epochs': 150, 'batch_size': 128, 'weight_decay': 0.0005, 'step_lr': 50}, <class 'robustness.datasets.CINIC'>: {'epochs': 150, 'batch_size': 128, 'weight_decay': 0.0005, 'step_lr': 50}, <class 'robustness.datasets.ImageNet'>: {'epochs': 200, 'batch_size': 256, 'weight_decay': 0.0001, 'step_lr': 50}, <class 'robustness.datasets.Places365'>: {'epochs': 200, 'batch_size': 256, 'weight_decay': 0.0001, 'step_lr': 50}, <class 'robustness.datasets.RestrictedImageNet'>: {'epochs': 150, 'batch_size': 256, 'weight_decay': 0.0001, 'step_lr': 50}, <class 'robustness.datasets.CustomImageNet'>: {'epochs': 200, 'batch_size': 256, 'weight_decay': 0.0001, 'step_lr': 50}, <class 'robustness.datasets.A2B'>: {'epochs': 150, 'batch_size': 64, 'weight_decay': 0.0005, 'step_lr': 50}, <class 'robustness.datasets.OpenImages'>: {'epochs': 200, 'batch_size': 256, 'weight_decay': 0.0001, 'step_lr': 50}}

Default hyperparameters for training by dataset (tested for resnet50). Parameters can be accessed as TRAINING_DEFAULTS[dataset_class][param_name]

robustness.defaults.TRAINING_ARGS = [['out-dir', <class 'str'>, 'where to save training logs and checkpoints', 'REQUIRED'], ['epochs', <class 'int'>, 'number of epochs to train for', 'varies by dataset'], ['lr', <class 'float'>, 'initial learning rate for training', 0.1], ['weight-decay', <class 'float'>, 'SGD weight decay parameter', 'varies by dataset'], ['momentum', <class 'float'>, 'SGD momentum parameter', 0.9], ['step-lr', <class 'int'>, 'number of steps between step-lr-gamma x LR drops', 'varies by dataset'], ['step-lr-gamma', <class 'float'>, 'multiplier by which LR drops in step scheduler', 0.1], ['custom-lr-multiplier', <class 'str'>, 'LR multiplier sched (format: [(epoch, LR),...])', None], ['lr-interpolation', ['linear', 'step'], 'Drop LR as step function or linearly', 'step'], ['adv-train', [0, 1], 'whether to train adversarially', 'REQUIRED'], ['adv-eval', [0, 1], 'whether to adversarially evaluate', None], ['log-iters', <class 'int'>, 'how frequently (in epochs) to log', 5], ['save-ckpt-iters', <class 'int'>, 'how frequently (epochs) to save (-1 for none, only saves best and last)', -1]]

Arguments essential for the train_model function.

Format: [NAME, TYPE/CHOICES, HELP STRING, DEFAULT (REQ=required, BY_DATASET=looked up in TRAINING_DEFAULTS at runtime)]

robustness.defaults.PGD_ARGS = [['attack-steps', <class 'int'>, 'number of steps for PGD attack', 7], ['constraint', ['inf', '2', 'unconstrained', 'fourier', 'random_smooth'], 'adv constraint', 'REQUIRED'], ['eps', <class 'str'>, 'adversarial perturbation budget', 'REQUIRED'], ['attack-lr', <class 'str'>, 'step size for PGD', 'REQUIRED'], ['use-best', [0, 1], 'if 1 (0) use best (final) PGD step as example', 1], ['random-restarts', <class 'int'>, 'number of random PGD restarts for eval', 0], ['random-start', [0, 1], 'start with random noise instead of pgd step', 0], ['custom-eps-multiplier', <class 'str'>, 'eps mult. sched (same format as LR)', None]]

Arguments essential for the robustness.train.train_model() function if adversarially training or evaluating.

Format: [NAME, TYPE/CHOICES, HELP STRING, DEFAULT (REQ=required, BY_DATASET=looked up in TRAINING_DEFAULTS at runtime)]

robustness.defaults.MODEL_LOADER_ARGS = [['dataset', ['imagenet', 'restricted_imagenet', 'custom_imagenet', 'cifar', 'cinic', 'a2b', 'places365', 'openimages'], '', 'REQUIRED'], ['data', <class 'str'>, 'path to the dataset', '/tmp/'], ['arch', <class 'str'>, 'architecture (see {cifar,imagenet}_models/', 'REQUIRED'], ['batch-size', <class 'int'>, 'batch size for data loading', 'varies by dataset'], ['workers', <class 'int'>, '# data loading workers', 30], ['resume', <class 'str'>, 'path to checkpoint to resume from', None], ['resume-optimizer', [0, 1], 'whether to also resume optimizers', 0], ['data-aug', [0, 1], 'whether to use data augmentation', 1], ['mixed-precision', [0, 1], 'whether to use MP training (faster)', 0]]

Arguments essential for constructing the model and dataloaders that will be fed into robustness.train.train_model() or robustness.train.eval_model()

Format: [NAME, TYPE/CHOICES, HELP STRING, DEFAULT (REQ=required, BY_DATASET=looked up in TRAINING_DEFAULTS at runtime)]

robustness.defaults.CONFIG_ARGS = [['config-path', <class 'str'>, 'config path for loading in parameters', None], ['eval-only', [0, 1], 'just run evaluation (no training)', 0], ['exp-name', <class 'str'>, 'where to save in (inside out_dir)', None]]

Arguments for main.py specifically

Format: [NAME, TYPE/CHOICES, HELP STRING, DEFAULT (REQ=required, BY_DATASET=looked up in TRAINING_DEFAULTS at runtime)]

robustness.defaults.add_args_to_parser(arg_list, parser)

Adds arguments from one of the argument lists above to a passed-in arparse.ArgumentParser object. Formats helpstrings according to the defaults, but does NOT set the actual argparse defaults (important).

Parameters:
  • arg_list (list) – A list of the same format as the lists above, i.e. containing entries of the form [NAME, TYPE/CHOICES, HELP, DEFAULT]
  • parser (argparse.ArgumentParser) – An ArgumentParser object to which the arguments will be added
Returns:

The original parser, now with the arguments added in.
robustness.defaults.check_and_fill_args(args, arg_list, ds_class)

Fills in defaults based on an arguments list (e.g., TRAINING_ARGS) and a dataset class (e.g., datasets.CIFAR).

Parameters:
  • args (object) – Any object subclass exposing setattr and getattr (e.g. cox.utils.Parameters)
  • arg_list (list) – A list of the same format as the lists above, i.e. containing entries of the form [NAME, TYPE/CHOICES, HELP, DEFAULT]
  • ds_class (type) – A dataset class name (i.e. a robustness.datasets.DataSet subclass name)
Returns:

The args object with all the defaults filled in according to arg_list defaults.
Return type:

args (object)