Server FPS - The drama on FacePunch Street

[GamingHQ]

Let me start at the beginning. 

I'm hosting servers for like, 6, 7 years now with my trustworthy partners. I know quite a lot when it comes on taking actions when a server starts hogging. However, for the past two months... i dont know what's going on, but its killing my mood bigtime as i see slowly the server bleed due to terrible server FPS. 

All plugins responding perfectly within less then 2 seconds time respond, but since the past 3 days, it started to go haywire, yet my server its FPS is so bad, that the server even crashes time to time ( and takes my whole node with it including dayz, night of the dead, valheim etc. ). Firstly, i thought this was an issue with raid-able bases, but the owner his response started me to look even further. 

my conclusion... i'm out of idea's and options. I updated all plugins to latest versions, No big files, just a 4k custom map with aproxx 50k ents on it. I opened a separate server, placed a copy of my server there and the same poopshow happens. Removing plugins one by one is possible, but there are several crucial plugins we need to keep in order to have the server running. 

Does anyone know anything by any chance? Is there something that i should avoid for now? Knowing there was a issue last month with boomboxes? 

 

01 "Admin Radar" (5.1.8) by nivex (9.58s) - AdminRadar.cs
02 "Advert Messages" (3.0.2) by LaserHydra (0.03s) - AdvertMessages.cs
03 "Anti Ladder and Twig" (1.2.1) by kaucsenta (0.00s) - AntiLadderandTwig.cs
04 "Anti Spam Names" (1.3.1) by MON@H (0.02s) - AntiSpamNames.cs
05 "Auto Lock" (2.4.4) by birthdates (0.01s) - AutoLock.cs
06 "Automatic Authorization" (1.3.0) by k1lly0u/Arainrr (0.25s) - AutomaticAuthorization.cs
07 "Backpacks" (3.7.2) by LaserHydra (0.10s) - Backpacks.cs
08 "Bank Heist" (1.4.3) by Bazz3l (1.13s) - BankHeist.cs
09 "Better Chat" (5.2.8) by LaserHydra (0.01s) - BetterChat.cs
10 "Blueprint Share" (1.2.10) by c_creep (0.00s) - BlueprintShare.cs
11 "Building Workbench" (1.3.2) by MJSU (0.18s) - BuildingWorkbench.cs
12 "CarVendor" (1.1.1) by senyaa (0.45s) - CarVendor.cs
13 "Cases" (1.0.11) by Mevent (0.40s) - Cases.cs
14 "Chat Prefix" (1.2.1) by Gonzi (0.01s) - ChatPrefix.cs
15 "CircularNetworkDistance" (1.0.0) by Vice (0.00s) - CircularNetworkDistance.cs
16 "Clans" (1.0.22) by Mevent (8.47s) - Clans.cs
17 "Copy Paste" (4.1.32) by misticos (0.03s) - CopyPaste.cs
18 "Corpse Location" (2.3.5) by shinnova (0.49s) - CorpseLocation.cs
19 "Countries" (2.0.1) by Wulf (0.00s) - Countries.cs
20 "Crafts" (2.7.4) by Mevent (1.11s) - Crafts.cs
21 "Dangerous Treasures" (2.2.5) by nivex (12.98s) - DangerousTreasures.cs
22 "Death Notes" (6.3.7) by LaserHydra (0.62s) - DeathNotes.cs
23 "Discord Logger" (2.0.9) by MON@H (0.10s) - DiscordLogger.cs
24 "Economics" (3.9.1) by Wulf (0.07s) - Economics.cs
25 "EventHelper" (1.0.5) by imthenewguy (0.06s) - EventHelper.cs
26 "Everlight" (3.4.16) by Wulf/lukespragg/Arainrr (13.11s) - Everlight.cs
27 "ExcavatorLock" (0.4.7) by Lorenzo (0.08s) - ExcavatorLock.cs
28 "FancyDrop" (3.2.2) by FastBurst (1.89s) - FancyDrop.cs
29 "Furnace Splitter" (2.3.3) by FastBurst (2.39s) - FurnaceSplitter.cs
30 "Gathering Manager" (2.2.75) by Mughisi (0.21s) - GatherManager.cs
31 "GUIAnnouncements" (2.0.3) by JoeSheep (2.26s) - GUIAnnouncements.cs
32 "Image Library" (2.0.60) by Absolut & K1lly0u (0.35s) - ImageLibrary.cs
33 "JunkpileNPC" (1.0.0) by Krungh Crow (7.06s) - JunkpileNPC.cs
34 "Kits" (4.0.15) by k1lly0u (0.08s) - Kits.cs
35 "Limit Entities" (1.4.4) by MON@H (0.67s) - LimitEntities.cs
36 "Lock Master" (1.1.7) by FastBurst (2.00s) - LockMaster.cs
37 "Marker API" (1.0.3) by The_Kiiiing (0.07s) - MarkerAPI.cs
38 "Nav Mesh Error Fix" (1.1.1) by Ryz0r (0.00s) - NavMeshErrorFix.cs
39 "NightMessage" (1.0.1) by Mordeus (0.15s) - NightMessage.cs
40 "No Give Notices" (0.3.0) by Wulf (0.00s) - NoGiveNotices.cs
41 "NoRaid" (1.3.21) by Ryan (1.92s) - NoRaid.cs
42 "Notify" (1.0.4) by Mevent (0.00s) - Notify.cs
43 "NpcSpawn" (2.2.0) by KpucTaJl (10.10s) - NpcSpawn.cs
44 "NTeleportation" (1.7.0) by nivex (0.50s) - NTeleportation.cs
45 "Panel System" (1.0.1) by Mevent (1.00s) - PanelSystem.cs
46 "PermissionsManager" (2.0.6) by Steenamaroo (0.12s) - PermissionsManager.cs
47 "Player Loot Logs" (1.1.0) by zenmarev (0.15s) - PlayerLootLogs.cs
48 "PlayerRanks" (2.2.0) by Steenamaroo (5.52s) - PlayerRanks.cs
49 "Playtime Tracker" (0.2.2) by k1lly0u (0.91s) - PlaytimeTracker.cs
50 "Plugin Update Notifications" (1.0.9) by Whispers88 (0.01s) - PluginUpdateNotifications.cs
51 "PoliceVehicles" (1.3.1) by HexOptimal (0.06s) - PoliceVehicles.cs
52 "PreventLooting" (1.12.1) by CaseMan (0.02s) - PreventLooting.cs
53 "Remover Tool" (4.3.31) by Reneb/Fuji/Arainrr (0.01s) - RemoverTool.cs
54 "Rustcord" (3.2.5) by Kirollos & OuTSMoKE (0.06s) - Rustcord.cs
55 "RustRewards" (3.0.9) by Steenamaroo (3.41s) - RustRewards.cs
56 "Save Announcer" (1.0.4) by Ryan (0.03s) - SaveAnnouncer.cs
57 "Shop" (1.0.30) by Mevent (0.19s) - Shop.cs
58 "Signal Cooldown" (1.0.31) by Vliek (0.00s) - SignalCooldown.cs
59 "Sign Artist" (1.4.0) by Whispers88 (0.00s) - SignArtist.cs
60 "Skills" (1.30.3) by Mevent (0.75s) - Skills.cs
61 "SkinBox" (2.0.14) by k1lly0u (0.02s) - SkinBox.cs
62 "Smart Chat Bot" (2.0.13) by Iv Misticos (0.01s) - SmartChatBot.cs
63 "Sort Button" (1.2.4) by MON@H (0.00s) - SortButton.cs
64 "Spawn Modular Car" (5.0.4) by WhiteThunder (0.05s) - SpawnModularCar.cs
65 "Spawns" (2.0.36) by Reneb / k1lly0u (0.01s) - Spawns.cs
66 "Stack Size Controller" (4.1.1) by AnExiledDev (0.62s) - StackSizeController.cs
67 "Tebex Donate" (1.7.0) by Tebex (0.88s) - TebexDonate.cs
68 "Timed Permissions" (1.6.0) by LaserHydra (0.01s) - TimedPermissions.cs
>>> Plugin being haywire time to time for no reason 69 "TruePVE" (2.0.6) by nivex (72.99s) - TruePVE.cs
70 "UpdatesChecker" (1.1.0) by Steenamaroo (0.03s) - UpdatesChecker.cs
71 "Vanish" (1.5.6) by Whispers88 (0.00s) - Vanish.cs
72 "Vehicle Licence" (1.7.29) by Arainrr (0.90s) - VehicleLicence.cs
73 "WelcomePanel" (3.0.7) by David (0.18s) - WelcomePanel.cs
74 "Wipe Info Api" (1.1.0) by MJSU (0.00s) - WipeInfoApi.cs
75 "WPKits" (1.1.2) by David (0.02s) - WPKits.cs
76 "WPSocialLinks" (1.1.2) by David (0.01s) - WPSocialLinks.cs
77 "WPVipRanks" (1.1.2) by David (0.00s) - WPVipRanks.cs
78 "WPWipeCycle" (1.1.2) by David (0.00s) - WPWipeCycle.cs
79 "XP System" (1.5.8) by ZTL/FastBurst (0.06s) - XPSystem.cs
80 "Zone Domes" (0.1.5) by k1lly0u (0.01s) - ZoneDomes.cs
81 "Zone Manager" (3.0.23) by k1lly0u (15.15s) - ZoneManager.cs

( Raidable bases disabled, Convoy disabled, Storm event disabled )

 

[GamingHQ]

On 5/21/2022 at 6:26 AM, Jeff.Thibault1 said:

You are not alone that face this problem. I run servers since 3 years now and since april month, both my 2 servers crash every 2 or 3 hours, even when there are no one playing.

They did something, for sure.

I started to drop the map size of my 2 servers.
from 4500 to 4000, still crash.
from 4000 to 3500, still crash.

Both servers are modded, my PVE server have around 131 plugins and my PVP server 59 plugins. They inialy had more until i sorted and removed out few to lower the use.
I have tested live on my own computer over a private local server and it did not crash... maybe its the server provider the problem? Is it Facepunch?, is it Oxide?
I used to make custom map over these size, and when to Procedural to see if any difference... none.

I am clueless, something is wrong somehow.

heya, i have a few tips for you as i have been able to resolve my issue as it was dayz being a D to me with his mem leak of 220 GB useage on boot. 

 

1. Ask your provider for your server specs. If its xeon, request a new server with a I7 or I9 as rust is single core threaded. 
2. Check if there is nothing ongoing on your other servers. I have 7 servers running and dayz was the bad boii in mine situation.
3. If you check your logs, do you see plugins having a hard time? TIP: use oxide.plugins to check their response time. If there is a plugin being heavy, it might be the cause of your problems.
4. Copy paste plugin can sometimes leave traces with raid-able bases. Like the saddle test prefab for example. 
5. Go ingame to 0,0,0 If you see items there, it might because some plugin is not removing the items properly. Everyone getting near 0,0,0 will have a rubber banding time of their life.
6. There was something ongoing with invisible items being saved somehow. As you mentioning rustedit, i assume you are on their discord. See this post for more detail https://discord.com/channels/475486514768510999/475486515292667904/976965665296617522

 

 

[Jeff.Thibault1]

You are not alone that face this problem. I run servers since 3 years now and since april month, both my 2 servers crash every 2 or 3 hours, even when there are no one playing.

They did something, for sure.

I started to drop the map size of my 2 servers.
from 4500 to 4000, still crash.
from 4000 to 3500, still crash.

Both servers are modded, my PVE server have around 131 plugins and my PVP server 59 plugins. They inialy had more until i sorted and removed out few to lower the use.
I have tested live on my own computer over a private local server and it did not crash... maybe its the server provider the problem? Is it Facepunch?, is it Oxide?
I used to make custom map over these size, and when to Procedural to see if any difference... none.

I am clueless, something is wrong somehow.