I review the current status of understanding when, how long, and how giant elliptical galaxies formed, focusing on the globular clusters. Several observational evidences show that massive elliptical galaxies formed at z > 2 (> 10 Gyr ago). Giant elliptical galaxies show mostly a bimodal color distribution of globular clusters, indicating a factor of ≈ 20 metallicity difference between the two peaks. The red globular clusters (RGCs) are closely related with the stellar halo in color and spatial distribution, while the blue globular clusters (BGCs) are not. The ratio of the number of the RGCs and that of the BGCs varies depending on galaxies. It is concluded that the BGCs might have formed 12-13 Gyr ago, while the RGCs and giant elliptical galaxies might have formed similarly 10-11 Gyr ago. It remains now to explain the existence of a gap between the RGC formation epoch and the BGC formation epoch, and the rapid metallicity increase during the gap (Δt ≈ 2 Gyr). If hierarchical merging can form a significant number of giant elliptical galaxies > 10 Gyr ago, several observational constraints from stars and globular clusters in elliptical galaxies can be explained.