xbtorch.deployment.committee
Committee machine (ensemble) evaluation module.
Provides functions to evaluate ensembles of models by aggregating their predictions. The main function test_committee computes classification accuracy by averaging outputs from all models in the committee.
This approach improves robustness and generalization, particularly for networks deployed on noisy or non-ideal hardware.
Functions
|
Evaluate a committee (ensemble) of models on a dataset. |
- xbtorch.deployment.committee.test_committee(dataloader, all_models, device)[source]
Evaluate a committee (ensemble) of models on a dataset.
This method implements the committee machines approach, where predictions from multiple models are aggregated by averaging their outputs before classification. It is commonly used to improve robustness and generalization, particularly in the presence of hardware noise or device variability.
- Parameters:
dataloader (torch.utils.data.DataLoader) – DataLoader providing input samples and labels for evaluation.
all_models (list of torch.nn.Module) – A list of models to be ensembled. Each model must be compatible with the given input shape and return class logits.
device (str or torch.device) – Device on which evaluation will be performed (e.g., “cpu” or “cuda”).
- Returns:
Classification accuracy of the committee (in percentage).
- Return type:
float
Notes
Each model in the committee processes the same input batch.
Predictions are averaged across models before applying
torch.max()to determine the predicted class.Returns only the accuracy; predicted labels and ground truth are accumulated internally but not returned.
References
Joksas, “Committee machines—a universal method to deal with non-idealities in memristor-based neural networks”, Nature Communications, 2020.
Opitz & Maclin, “Popular Ensemble Methods: An Empirical Study”, Journal of Artificial Intelligence Research, 1999.