VDS Box and SRCDS

I'm running a 12 Core Xeon E5-2690 @ 3 GHZ, With 12GB Of memory, Using Windows Server 2012.


I'm running a garry's mod server, and when the user-cap reaches around 20-30
Players something like that my SRCDS is only reaching 10-17 FPS.



Any ideas what I should do?