I run an Exile server and the "3 FPS issue" became a chronic problem for me ramping up the more I added to the server (mods/assets/scripts). Ultimately I was experiencing the issue within 5-15 min of server startup and after weeks of experiencing this bug regularly, I stumbled upon the setting that was creating most my havoc (loot spawning). I realized that I had loot spawns set to 30 minutes and maximum radius set (300m). So as I traveled around the world, I left behind tons of loot which stayed in the world far beyond my presence. The more I traveled, the worse it got.
I realize that this isn't likely the cause for others experiencing this issue but this would be a good way to attempt to reproduce the memory leak. Once I changed these settings 15 minutes for loot spawns and 75m range, my crashing went from every 5-15 min to once a server restart if it crashed at all.