Many contemporary architectural avant gardes tend to use painting as a medium to create architecture which goes beyond the rationalized spatial conception of modem architecture represented by perspectivism. They produce non perspective drawings to represent spatial Ideas, and expand it through poetic imagination to create an unexpected architectural form and space. This paper attempts to analyze the historical origin and background of dominance of drawing in the production of architecture. It was with the invention of perspective that architectural representation became important tool for architectural production. Thereafter, drawing was considered prior to actual building and architecture was considered a three dimensional realization of two dimensional drawing. Modernist avant gardes such as Cubism shattered the rationalized pictorial space of perspective and found a new pictorial space. They tried to extend it to three dimensional space through parallel projection largely based on the Hildebrand's theory of pure visibility. However, due to the ambiguity of the position of the viewing subject, their attempts could not succeed in creating a new architecture. The new architectural avant garde of the 70's rediscovered the early 20th century avant gardes in their attempt to create a new architecture which can register the fragmented spatial condition of contemporary society, and used painting as a medium to create architecture. Their difference from the early avant gardes was that they used poetic imagination rather than parallel projection in the process of projecting three dimensional space and form from the painting. However, their architecture cannot escape the scopic field of perspectivism in that they rely on the picture plane and the distance between object and viewing subject. Therefore, I conclude that in order to create architecture which goes beyond the rationalized space of modern architecture, it is necessary to resort to other tradition of modern architecture than visual one.