flip.nvflare.components ======================= .. py:module:: flip.nvflare.components .. autoapi-nested-parse:: FLIP Components module containing reusable FL components. Components include event handlers, model locators, JSON generators, and persistence utilities. Exports: - ClientEventHandler: Client-side event handler - ServerEventHandler: Server-side event handler - PTModelLocator: PyTorch model locator - InitialPTModelLocator: PyTorch model locator for initial models with safehouse fallback - EvaluationPTModelLocator: PyTorch model locator for evaluation workflows - ValidationJsonGenerator: Validation results JSON generator - EvaluationJsonGenerator: Evaluation results JSON generator - PersistToS3AndCleanup: S3 persistence and cleanup component - PercentilePrivacy: Percentile-based privacy filter - StagePercentilePrivacy: Stage-aware percentile-based privacy filter - CleanupImages: Image cleanup executor Submodules ---------- .. toctree:: :maxdepth: 1 /reference/api/flip/nvflare/components/cleanup/index /reference/api/flip/nvflare/components/custom_percentile_privacy/index /reference/api/flip/nvflare/components/evaluation_json_generator/index /reference/api/flip/nvflare/components/flip_client_event_handler/index /reference/api/flip/nvflare/components/flip_server_event_handler/index /reference/api/flip/nvflare/components/persist_and_cleanup/index /reference/api/flip/nvflare/components/pt_model_locator/index /reference/api/flip/nvflare/components/stage_percentile_privacy/index /reference/api/flip/nvflare/components/validation_json_generator/index Classes ------- .. autoapisummary:: flip.nvflare.components.CleanupImages flip.nvflare.components.PercentilePrivacy flip.nvflare.components.EvaluationJsonGenerator flip.nvflare.components.ClientEventHandler flip.nvflare.components.ServerEventHandler flip.nvflare.components.PersistToS3AndCleanup flip.nvflare.components.EvaluationPTModelLocator flip.nvflare.components.InitialPTModelLocator flip.nvflare.components.PTModelLocator flip.nvflare.components.StagePercentilePrivacy flip.nvflare.components.ValidationJsonGenerator Package Contents ---------------- .. py:class:: CleanupImages Bases: :py:obj:`nvflare.apis.executor.Executor` CleanupImages takes place at the start and end of the run. All the images used for the training are deleted to prevent the build-up of unnecessary files on the storage space. Executing at the start of a run ensures that any training code is executed with a clean slate. Args: Raises: .. py:method:: execute(task_name: str, shareable: nvflare.apis.shareable.Shareable, fl_ctx: nvflare.apis.fl_context.FLContext, abort_signal: nvflare.apis.signal.Signal) .. py:class:: PercentilePrivacy(percentile=10, gamma=0.01, data_kinds: List[str] | None = None, off: bool = False) Bases: :py:obj:`nvflare.apis.dxo_filter.DXOFilter` Implementation of "largest percentile to share" privacy preserving policy. Shokri and Shmatikov, Privacy-preserving deep learning, CCS '15 :param percentile: Only abs diff greater than this percentile is updated. Allowed range 0..100. Defaults to 10. :type percentile: int, optional :param gamma: The upper limit to truncate abs values of weight diff. Defaults to 0.01. :type gamma: float, optional :param Any weight diff with abs Union[None, nvflare.apis.dxo.DXO] Compute the percentile on the abs delta_W. Only share the params where absolute delta_W greater than the percentile value :param dxo: information from client :param shareable: that the dxo belongs to :param fl_ctx: context provided by workflow Returns: filtered dxo .. py:class:: EvaluationJsonGenerator(results_dir=PTConstants.EvalDir, json_file_name=PTConstants.EvalResultsFilename) Bases: :py:obj:`nvflare.apis.fl_component.FLComponent` Handles EVALUATION_RESULT_RECEIVED event and generates a results.json containing accuracy of each validated model. :param results_dir: Name of the results directory. Defaults to cross_site_eval :type results_dir: str, optional :param json_file_name: Name of the json file. Defaults to evaluation_results.json :type json_file_name: str, optional .. py:method:: handle_evaluation_events(event_type: str, fl_ctx: nvflare.apis.fl_context.FLContext) .. py:class:: ClientEventHandler Bases: :py:obj:`nvflare.apis.fl_component.FLComponent` ClientEventHandler is a generic component that handles system events triggered by nvflare or custom flip events. It executes logic inside its own event handler but may also call other component's event handlers directly to help overcome the non-deterministic order in which nvflare handles events. Args: Raises: .. py:method:: handle_event(event_type: str, fl_ctx: nvflare.apis.fl_context.FLContext) .. py:class:: ServerEventHandler(model_id: str = '', validation_json_generator_id: str = 'json_generator', persist_and_cleanup_id: str = 'persist_and_cleanup', flip: flip.FLIP = FLIP()) Bases: :py:obj:`nvflare.apis.fl_component.FLComponent` ServerEventHandler is a generic component that handles system events triggered by nvflare or custom flip events. It executes logic inside its own event handler but may also call other component's event handlers directly to overcome the non-deterministic order in which nvflare handles events i.e handling ValidationJsonGenerator component events. :param model_id: :type model_id: string, not required :param validation_json_generator_id: :type validation_json_generator_id: string, not required :param persist_and_cleanup_id: :type persist_and_cleanup_id: string, not required :param flip: :type flip: object, not required :raises ValueError: when model ID is not a valid UUID .. py:attribute:: model_id :value: '' .. py:attribute:: validation_json_generator_id :value: 'json_generator' .. py:attribute:: validation_json_generator :value: None .. py:attribute:: persist_and_cleanup_id :value: 'persist_and_cleanup' .. py:attribute:: persist_and_cleanup :value: None .. py:attribute:: flip .. py:attribute:: fatal_error :value: False .. py:attribute:: final_status :value: None .. py:method:: handle_event(event_type: str, fl_ctx: nvflare.apis.fl_context.FLContext) .. py:class:: PersistToS3AndCleanup(model_id: str, persistor_id: str = AppConstants.DEFAULT_PERSISTOR_ID, flip: flip.FLIP = FLIP()) Bases: :py:obj:`nvflare.apis.fl_component.FLComponent` The component that is executed post training and is a part of the FLIP training model The PersistToS3AndCleanup workflow saves the aggregated model (once training has finished) to an S3 bucket, and then deletes files created as part of the run :param model_id: ID of the model that the training is being performed under. :type model_id: str :param persistor_id: ID of the persistor component. Defaults to "persistor". :type persistor_id: str, optional :raises ValueError: - when the model ID is not a valid UUID. FileNotFoundError: boto3 error for when the zip file does not exist. .. py:attribute:: model_id .. py:attribute:: persistor_id :type: str .. py:attribute:: model_persistor :type: nvflare.app_opt.pt.file_model_persistor.PTFileModelPersistor | None :value: None .. py:attribute:: model_inventory :type: dict .. py:attribute:: model_dir :type: str :value: '' .. py:attribute:: bucket_name :type: str :value: '' .. py:attribute:: flip .. py:method:: execute(fl_ctx: nvflare.apis.fl_context.FLContext) .. py:method:: upload_results_to_s3_bucket(fl_ctx: nvflare.apis.fl_context.FLContext) Uploads the final aggregated model and reports to an S3 bucket as a zip file. .. py:method:: cleanup(fl_ctx: nvflare.apis.fl_context.FLContext) Cleans up the workspace by deleting the transfer and save directories for the model ID. .. py:class:: EvaluationPTModelLocator(exclude_vars=None) Bases: :py:obj:`nvflare.app_common.abstract.model_locator.ModelLocator` .. py:attribute:: models :value: None .. py:attribute:: exclude_vars :value: None .. py:method:: locate_model(fl_ctx: nvflare.apis.fl_context.FLContext) -> Union[nvflare.apis.dxo.DXO, None] .. py:class:: InitialPTModelLocator(exclude_vars=None, model=None) Bases: :py:obj:`nvflare.app_common.abstract.model_locator.ModelLocator` .. py:attribute:: model :value: None .. py:attribute:: exclude_vars :value: None .. py:method:: get_model_names(fl_ctx: nvflare.apis.fl_context.FLContext) -> List[str] .. py:method:: locate_model(model_name, fl_ctx: nvflare.apis.fl_context.FLContext) -> Union[nvflare.apis.dxo.DXO, None] .. py:class:: PTModelLocator(exclude_vars=None, model=None) Bases: :py:obj:`nvflare.app_common.abstract.model_locator.ModelLocator` .. py:attribute:: model :value: None .. py:attribute:: exclude_vars :value: None .. py:method:: get_model_names(fl_ctx: nvflare.apis.fl_context.FLContext) -> List[str] .. py:method:: locate_model(model_name, fl_ctx: nvflare.apis.fl_context.FLContext) -> Union[nvflare.apis.dxo.DXO, None] .. py:class:: StagePercentilePrivacy(percentile=10, gamma=0.01, data_kinds: List[str] = None, off: bool = False) Bases: :py:obj:`nvflare.apis.dxo_filter.DXOFilter` Implementation of "largest percentile to share" privacy preserving policy. Shokri and Shmatikov, Privacy-preserving deep learning, CCS '15 :param percentile: Only abs diff greater than this percentile is updated. Allowed range 0..100. Defaults to 10. :type percentile: int, optional :param gamma: The upper limit to truncate abs values of weight diff. Defaults to 0.01. :type gamma: float, optional :param Any weight diff with abs Union[None, nvflare.apis.dxo.DXO] Compute the percentile on the abs delta_W. Only share the params where absolute delta_W greater than the percentile value :param dxo: information from client :param shareable: that the dxo belongs to :param fl_ctx: context provided by workflow Returns: filtered dxo .. py:class:: ValidationJsonGenerator(results_dir=AppConstants.CROSS_VAL_DIR, json_file_name=PTConstants.CrossValResultsJsonFilename) Bases: :py:obj:`nvflare.apis.fl_component.FLComponent` Handles VALIDATION_RESULT_RECEIVED event and generates a results.json containing accuracy of each validated model. :param results_dir: Name of the results directory. Defaults to cross_site_val :type results_dir: str, optional :param json_file_name: Name of the json file. Defaults to cross_val_results.json :type json_file_name: str, optional .. py:method:: handle_evaluation_events(event_type: str, fl_ctx: nvflare.apis.fl_context.FLContext)