Modern Deep Neural Network (DNN) architectures produce high accuracy across applications, however incur high computational complexity and memory requirements, making it challenging for execution on resource constrained mobile platforms. Driven by application requirements, there has been a shift in execution paradigm of Deep Nets from cloud based computation to sensor/mobile platforms. The limited memory available onboard a mobile platform, necessitates an effective mechanism for storage of network parameters (viz. weights) generated offline post-training. Hence, we propose a modified Huffman encoding-decoding technique, with dynamic usage of net layers, executed on-the-fly in parallel, which can be applied on a memory constrained multicore environment. To the best of our knowledge, this is the first study on applying compression based on multiple bit pattern sequences, to achieve a maximum compression rate of 64 percent and a single module decompression time of about 0.33 seconds without trading-off accuracy. © 2018 IEEE.