Infrared visual surveillance has become an important mean to secure military camps, reassure soldier security, and detect suspected terror activities in the battle fields. An intelligent infrared surveillance system is aimed to provide real-time intelligent analysis of the perceived scene and find out human targets instantly to assist the soldier/commanders to make the right decision in a just-in-time mode to save our soldiers from life risks. To attain this, automatic detection of moving human objects from the scene is an essential step. In this paper, we present an FPGAs-based architecture to perform on-chip human silhouette extraction using a parallel architecture with systolic arrays. The architecture is designed in VHDL and simulated with real FLIR videos. The experiment shows that the designed processor on FPGAs can efficiently extract the human contour instantly from infrared videos, which exhibits great potential to facilitate the further analysis of the battlefield scenes for military-purpose surveillance systems.