Recently, the migration of refugees has been drastically increasing and became a global issue which indirectly affects the shipmasters, authorities and shipowners during rescue operations. The objective of this paper is to study the challenges or difficulties in rescuing refugees at sea from the perspectives of shipmasters, authorities and shipowners in Malaysia. In this study, the key challenges faced by these three groups of individuals are collected and classified based on the primary data reviewed. The interview sessions were conducted with selected expertise and the data collected are analyzed based on the Analytical Hierarchy Process (AHP) or precisely the pair-wise comparison methods. Through the calculations, the challenges are ranked from the most important challenges to the least important challenges. Furthermore, the data is analyzed by using expert judgments focusing on the relatively most important challenges faced by the three groups of individuals. Finally, the most important challenge faced by shipmasters is the safety of ships and crews, while for authorities, the most important challenge faced is delays in the initiation of rescue operations. For ship-owners, the most important challenge faced is time sensitive cargo loss. Ultimately, two suggestions have been proposed to encounter the most important challenge faced by these groups.