Unmanned Aerial Vehicle (UAV) based Hyperspectral imaging (HSI) technology has proven potential in remote sensing (RS) and monitoring applications due to their large field of coverages, high spectral, spatial, and temporal resolutions. This paper presents the use of UAV-based hyperspectral RS and Convolutional neural networks (CNN) for vegetation categorization. The HSI data of vegetation was acquired using a push-broom HSI sensor (400 nm-1000 nm) mounted on a UAV by conducting flights from different altitudes. The acquired HSI data was classified for different vegetation types using state-of-the-art CNN models. The impact of UAV flight altitude and spatial contextual range on the HSI data analysis was investigated. Vegetation classification accuracies of 96.77%, 97.32%, 95.45%, and 93.96% were achieved on HS images acquired from 30m, 40m, 50m, and 60m flight altitudes respectively, which demonstrates the effectiveness of UAV-based HS remote sensing for vegetation categorization and mapping. © 2022 IEEE.