Convolution filters play important roles in improving the abstraction ability of segmentation networks due to its method of fusing features from all input channels. Such a method pays little attention to protecting the channel-wise features which is important in preserving the detail features in the final result. Though a few works has discussed this problem, the function of channel-wise features is still not sufficiently exploited. For investigating the beneficial effect of channel-wise feature on improving details representation ability, we design a Hybrid Pyramid Pooling (HPP) structure to explicitly exploit the channel-wise features. Through utilizing different scales of depthwise dilated convolution, segmentation networks can fully utilize the superiority of channel-wise features to improve the details representation ability. For deeply developing the advantages of the channel-wisely calculated features by HPP, we design another Stacked Pyramid Structure (SPS), which contains stacked pyramid pooling structures. The SPS distributes sufficient sampling points in each finely divided sampling area, and fuses features from a large range of receptive fields to produce rich feature representation. Finally, the combination of HPP and SPS can not only maintains both the advantages of features calculated from a particular channel and all input channels, but also provides sufficiently enriched receptive fields for depicting details in the final result.