Modular buildings are constructed by assembling modular units which are prefabricated in a factory and delivered to the site. However, due to a problem of noise between floors, concrete slab is usually poured at the top or bottom level of a modular unit in Korea. This greatly increases the weight of buildings, but designing vertical members of modular units to resist overall gravity loads is very inefficient. In this study, considering domestic building construction practices, feasible structural systems for tall modular buildings are proposed in which separate steel frames and reinforced concrete core walls are designed to resist gravity and lateral loads. To verify performance, a three-dimensional structural analysis has been performed with two types of prototype buildings, i.e., a residential building and a hotel. From the results, wind-induced lateral displacements and seismic story drifts are examined and compared with their limit values. Between the two kinds of buildings, the efficiency of the proposed system is also evaluated through a comparison of the weight of structural components. Finally, the effect of a floor diaphragm on the overall behavior is analyzed and discussed.