flip.nvflare.controllers.fed_evaluation

Classes

ModelEval

Model Evaluation workflow.

Module Contents

class flip.nvflare.controllers.fed_evaluation.ModelEval(task_check_period=0.5, submit_model_timeout=600, validation_timeout: int = 6000, model_locator_id='', formatter_id='', submit_model_task_name=AppConstants.TASK_SUBMIT_MODEL, evaluation_task_name=PTConstants.EvalTaskName, cleanup_models=False, participating_clients=None, wait_for_clients_timeout=300, cleanup_timeout=600, fatal_error_delay=5, model_id='')[source]

Bases: nvflare.apis.impl.controller.Controller

Model Evaluation workflow.

Parameters:

  • task_check_period (float, optional) – How often to check for new tasks or tasks being finished. Defaults to 0.5.

  • submit_model_timeout (int, optional) – Timeout of submit_model_task. Defaults to 600 secs.

  • validation_timeout (int, optional) – Timeout for validate_model task. Defaults to 6000 secs.

  • model_locator_id (str, optional) – ID for model_locator component. Defaults to “”.

  • formatter_id (str, optional) – ID for formatter component. Defaults to “”.

  • submit_model_task_name (str, optional) – Name of submit_model task. Defaults to “”.

  • validation_task_name (str, optional) – Name of validate_model task. Defaults to “validate”.

  • cleanup_models (bool, optional) – Whether models should be deleted after run. Defaults to False.

  • participating_clients (list, optional) – List of participating client names. If not provided, defaults to all clients connected at start of controller.

  • wait_for_clients_timeout (int, optional) – Timeout for clients to appear. Defaults to 300 secs

  • fatal_error_delay (int, optional) – Time in seconds to delay before calling ‘system_panic’ if a task returns an error result and ignore_result_error is set to false

  • model_id (str, required) – ID of the model that the training is being performed under.

flip[source]
start_controller(fl_ctx: nvflare.apis.fl_context.FLContext)[source]
control_flow(abort_signal: nvflare.apis.signal.Signal, fl_ctx: nvflare.apis.fl_context.FLContext)[source]
stop_controller(fl_ctx: nvflare.apis.fl_context.FLContext)[source]
handle_event(event_type: str, fl_ctx: nvflare.apis.fl_context.FLContext)[source]
process_result_of_unknown_task(client: nvflare.apis.client.Client, task_name: str, client_task_id: str, result: nvflare.apis.shareable.Shareable, fl_ctx: nvflare.apis.fl_context.FLContext)[source]