This paper proposes a convolutional neural network model for distinguishing areas occupied by obstacles from a LiDAR image converted from a 3D point cloud. The channels of a LiDAR image used as input consist of the distances to 3D points, the reflectivities of 3D points, and the heights of 3D points from the ground. The proposed model uses a LiDAR image as an input and outputs a result of a segmented LiDAR image. The proposed model adopts refinement modules with skip connections to segment a LiDAR image. The refinement modules with skip connections in the proposed model make it possible to construct a complex structure with a small number of parameters than a convolutional neural network model with a linear structure. Using the proposed model, it is possible to distinguish areas in a LiDAR image occupied by obstacles such as vehicles, pedestrians, and bicyclists. The proposed model can be applied to recognize surrounding obstacles and to search for safe paths.