RealFL: A Realistic Platform for Federated Learning

Federated Learning (FL) enabled creating models that are competitive to centralized Machine Learning models while preserving privacy by allowing clients to train data locally. Despite FL research growth, most of the work assessment and existing open-source FL testbeds/frameworks have drawbacks that prohibit convenient deployment over a large spectrum of heterogeneous clients in realistic environments. These drawbacks include simulations, unrealistic datasets, not supporting heterogeneity, and not having a realistic environment control in terms of network and client churn, for example. In this paper, we introduce (RealFL) a novel, realistic, open-source, and extendable platform for FL that supports a large scale of heterogeneous clients. It enables realistic assessment of FL solutions by controlling various environmental parameters; e.g. network, client churn, data distribution, training complexity, and client heterogeneity. Using these parameters, we assess RealFL performance through an extensive evaluation. The results show a performance gap of up to 77.5% in training time and 23.9% in accuracy between FL unrealistic environments and RealFL.