Antimalware service executable using memory

The Antimalware service executable, msmpeng.exe, is using memory on my computer. During idle with all programs closed, other processes use like 30MB-40MB each. But msmpeng.exe uses like 100MB. I tried tuing off Windows Defender but the process is still there. Seems I can't go into task manager and end the process. Are there any other solutions to keeping windows defender off forever, and to have the antimalware service executable stop using so much memory?

I fixed this issue in 8.1 by tuing off defender, but the problem has come back when i upgraded.