Recently, web services security has shown a significant gesture as several specifications have been developed and implemented to meet the security challenges of web services. However, the performance of the security mechanisms is fraught with concerns due to additional security, contents in SOAP messages, the higher number of message exchanges to establish trust as well as extra CPU time to process these additions. In this paper, we consider and compare the performance of various security mechanisms applied on a simple web service tested with different initial message sizes. The test results show that transport layer security mechanisms are considerably faster than message level security mechanisms. Moreover, the effect of adding SAMIL-tokens is negligible and the performance of SAML-based web services depends mostly on the underlying security mechanisms. Finally, the performance penalty of applying STS security mechanisms is significantly high comparing to non-STS mechanisms.