Airport image analysis using machine learning is an efficient way to diagnose security. However, the sharing of in-formation across airports is usually constrained due to national security policies. The menace of cyber-attacks on airport secu-rity and surveillance systems continues to pose major threats to national security. Also, the protection and privacy preservation of information and technology assets are growing in complexity and importance. Airport image analysis (e.g., remote sensing, UAV screening, etc.) using deep learning emerges as the new methodology to perform surveillance tasks. Nevertheless, the major concern continues to be safety regulations and privacy issues for data sharing. Thus, causing insufficiency in dataset availability for training any computer vision application model. To improve data availability, security, performance, and communication efficiency, we propose a novel fusion-based feder-ated learning approach for airport surveillance through object detections using remote sensing images. The proposed archi-tecture serves as a guide for the Federated learning system design that preserves privacy by generating an independent global model based on local model updates from different clients without revealing the data itself. Secondly, a real-time fusion method is proposed to decide the participating clients according to their local model performance. Based on critical parameters like training time, dataset size, client system configuration, etc., at the participating clients, we schedule the model dynamically. A custom dataset annotated for 6 object categories, character-izing the real-world scenario with airports across the world is generated at different spatial resolutions. We evaluate the model for the object detection tasks using two benchmark algorithms (YOLOV3 and Faster R-CNN). Our experiments indicated that this methodology is robust to unbalanced and non-IID (Indepen-dent and Identically Distributed) datasets. Besides, this method can be used to reduce the round of communication needed to train a deep network on decentralized data. © 2022 IEEE.