Twitter provides access to latest information. Whenever a major event happens, people try to search for event related information in social media platforms like Twitter. So, it is essential to develop methods to get good quality of event related tweets. People share different opinions, feelings, feedback, etc. about events happening around the world in Twitter in the form of tweets. These tweets are often short and contain noise. So, it is very difficult to get the most relevant data for a given event from Twitter. We propose a two-phase approach to retrieve the tweets related to planned events. In the first phase, initial retrieval is done by using BM25 algorithm. In the second phase, reranking is done by combining three scoring mechanisms namely BM25 score, top hashtags score related to an event, and top TF-IDF terms score related to an event. A learning to rank algorithm SVM_Rank is applied to give weights to these three methods and combine them to get the final score of the tweet. We performed experiments on two benchmark datasets CLEF and TREC. Experimental results show that our method outperforms baseline and literature methods for both the datasets according to multiple evaluation metrics. © 2021, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.