PlayARK Australia has moved to GamingAlliance.Net,
GamingAlliance.Net covers a larger number of games.
This forum is now locked for historical purpose/reference.
Please visit GamingAlliance.Net or join their discord at https://discord.gg/asFpJ7A

Devs Turret Changes - BE PREPARED

This is where you can discuss anything, doesn’t have to be related to ARK.
Please no Spamming/Advertising or Abuse.
User avatar
DrunkMunki
Server Admin
Posts: 1094
Tribe: Admin
Contact:
Status: Offline

Devs Turret Changes - BE PREPARED

Post#1 » Sat Nov 18, 2017 8:53 pm

source: https://survivetheark.com/index.php?/ar ... news-r602/

Upcoming Turret Change

Over the past few weeks, we have been carrying out analysis on our Official Servers to see where we can improve server performance and what the biggest difference is currently versus Early Access. Since ARK went retail, we were expecting to see our servers have better performance due to optimisations, better allocation and usage of hardware, and improved DDoS mitigation. What we have learnt is that there has been a dramatic change in how turrets are being used on Official Servers. The fundamental issue is that tribes are not only significantly increasing the number of turrets they use, but reducing the space they occupy, so the server is having to do more calculations in denser areas. In some cases, we've seen an increase of 7x versus the previous server-profiling data we had. It is clear that we need to make a change in regards to how turrets are placed, but we understand that they are an important part meta-game on Official PvP servers and therefore want to give you as much notice as possible.

With the following change, we are expecting a performance increase of 50% on average for our Official Servers (extreme cases up to 300%) and this will have an impact on all maps.

Starting from today on PC (and next week on console), you'll receive an update which indicates the number of turrets you have within 10k units. When you exceed 100 turrets within 10k units, there will be a warning placed on the turret to inform you, so you can make adjustments as necessary. On December the 5th, we will flick a switch on our Official Servers which will disable all turrets that exceed the allocated limit. You have the period from now until then to make any changes you want to avoid your turrets from being turned off. Once the turrets are within the permitted amount, they will function again. This patch will also allow the mounted minigun turret to shoot non-mounted tames, but reduce their range to be the equivalent of the auto turret.

In accordance with this change, we will also be increasing the damage of autoturrets and TEK turrets by 20%, increasing the item slot count of autoturrets by 50% on the 5th of December.

The turret limit will effect: Plant Species X, Autoturrets, and TEK Turrets.
For emergencies such as server going down email: drunkmunki[at]gamingalliance.net

Silversage
Dino Tamer
Posts: 47
Tribe: House Stark
Status: Offline

Re: Devs Turret Changes - BE PREPARED

Post#2 » Sun Nov 19, 2017 8:19 am

Will you be applying the official limit of 100 per 10k units or making your own?

User avatar
King_Kiff
Ultimate Survivor
Posts: 255
Tribe: All Aussie Adventures
Status: Offline

Re: Devs Turret Changes - BE PREPARED

Post#3 » Sun Nov 19, 2017 7:25 pm

Just goes to show once again the devs do not play this game at all.

I've never had a base with over 100 turrets due to mostly playing on my own but I've seen many large bases with well over that amount.

This will literally destroy official PVP worse than a band of 70 DDOSing Chinese hackers.

User avatar
DrunkMunki
Server Admin
Posts: 1094
Tribe: Admin
Contact:
Status: Offline

Re: Devs Turret Changes - BE PREPARED

Post#4 » Sun Nov 19, 2017 8:43 pm

Source: https://survivetheark.com/index.php?/fo ... about-why/

Hey guys, sure has been a crazy day hasn't it?

So, I know we have that megathread over there, and guess what my weekend gets to be (It's reading that and every other thread.) but I wanted to make a new post here just so that information was in a good, easy place to see.

Let's just get right into it:

This is some profiling output from a fun little tool we made, which analyzes the server and finds each thread that is running and times how long it takes for the server to tick. It does a lot more than that, but that's what you need to know to understand this:

Total Duration 765.60 MS 1.31 FPS

StructureTurretBaseBP_C : 7977 234.98 MS
StructureTurretPlant_C : 9722 119.4 MS

This is turrets. BaseBP is normal metal turrets, Plant is SpeciesX.

Now, with this information, you need to have some other information:

Since about October or so, we saw a MASSIVE spike in the size of stalls and poor performance on our dedicated servers, but only some of them, most of them Ragnarok Servers.

So we investigated, and we found that an interesting change in the meta for base building had occured: People were making new, super-dense bases with insane turret counts, far beyond anything we'd ever seen before. The bases of these types were on all of our servers, but primarily belonged to Alpha Tribes who had set up bases on the Ragnarok servers. Similar setups caused equally bad performance on Center and Island servers in the rare cases we found them.

I want you to understand what all of this means to us, so I'll try to be brief.

Turrets, were very suddenly being used in a configuration that was atypical, and up to 7x the number of turrets we would see on any other server, even if it was a highly populated, well trafficked server. There were definitely other cases of densely packed bases, but this was higher by many magnitudes.

Let me emphasize now, that for the last two years, turrets have been a pain in our butt, but we have not encountered people making monstrosities like this. Every alpha tribe in the game has existed just fine, and gotten along well enough without these insanely dense crazy bases.

So, we looked at what we could do about it. Let me wrap this all up in context, these things I'm going to put here are facts:

Our turrets are highly optimized for what they do with respect to gameplay. They use a very fast bitmasked octree overlap to return all of specific types of actors (excluding irrelivant actors) in the aoe that they defend, and then act according to the settings you configure, early-outing from any extra calculations or wasted instructions if their shot is invalid. We could make that area they search smaller, which would be faster, but then they would need correspondingly smaller ranges.

We could make them search for less things in their overlap check, but then you'd have turrets that didn't fire on some things.

We could make them acquire targets less frequently, but then they'd be much less accurate and in many cases would not pick up things like rocket projectiles that entered their range between scans.

We could make them scan at different ranges for different things, but then their functionality would be weirdly ambiguous and what things would we reduce scan range for anyways? Anything that a turret should be shooting at is important.

Additionally, even if we did one or all of these things, the per-frame cost of each turret would not be greatly reduced (fractions of milliseconds) and 15k+ turrets would still cripple servers.

Another truth is that in the worst case, servers were ticking at as low as 1 frame per second. That means that no matter what you're seeing in game, the server can only update the whole game state one frame per one second. That is the worst our servers have ever run, and is completely unplayable, in every way.

No matter who you are, I think you can agree that it's not enjoyable to play the game like that.

As such, we are left with a dillema:

We can reduce the number of turrets that people can place, and attempt to recover some of the performance loss that happens because of them, or we can not, or we can try and find some medium where we save as much performance as we can, while still allowing players to defend themselves.

On a server with 70 people playing on it, after reducing turret density, this is the new framerate on the server:

Total Duration 222.86 MS 4.5 FPS

Now, this is a worst case scenario, but the difference is absolutely clear to us, and one thing that we cannot under any circumstances forget, is the impact that servers running at 1 fps has on all of the players on the server. 4-5 fps is actually playable-- it can be a little bit choppy, but it's not infuriatingly awful.

Okay, so, if you're still with me, you understand what the raw numbers are that went into this, and I'll provide a to-now TLDR:

A certain method of building bases with turrets caused server fps to plummet to historic lows. The culprit is undeniably turrets: in some cases costing between 400 and 500ms per frame on the server tick. Our turrets are already extremely well optimized, but players are placing tens of thousands of them. That's too many. Our changes showed a change of 1.1 -> 4.5 fps. Reducing their numbers was now not a question of "if" but of "how much".

So what do we actually do?

Well, we try to solve it. And we try to talk about how best to solve it. Are there going to be some edge cases we have to deal with? Sure. Will we maybe have to adjust the balance of some things? Probably. We're letting you know weeks in advance as part of a method of opening up that communication channel. So that you can make decisions about what to do well ahead of time. Crunch numbers. Adjust strategies. Yell at us, and let us know what some of the things that are important to you guys are that we might not be thinking of-- hundreds of thousands of people play this game in all sorts of different ways every single day, we simply can't know what all of those ways are.

Now you know the technical issues we face. There is no other place to "save" 500ms of server tick time and let you just keep turrets how they are. There's no way to make turrets faster in any meaningful way, without fundamentally changing turrets and how they work.

As an important note, this change will also make all of the turrets you DO have much more accurate and reliable, because the low server fps made them inaccurate and miss often. Kind of a compounding problem.

https://gyazo.com/e307c4f0324ee5fc239862ace0365c38

This is a graph of the tick times of our worst servers, right off our backend. A delta time of 500 simply means that the server is locked at 2.5fps (it can actually run slower than that, but that's as low as our recording goes.)

PC servers are far, far worse off than console servers, due to their increased caps on everything, and each and every one of these servers suffers enormously from turret-induced slowdown. PvE doesn't even come close. Out of the top 150 slowest servers we have, only 15 of them in fact are PvE servers. This is a pvp issue, and it's really bad. There is no truth to the claim that PvE runs worse. It is a myth.

So let's talk about it. I am going to post this, and then I'm going to sit down and try to have dinner, and when I come back from dinner, I'm going to read more of what people have had to say. We want you to still be able to defend your bases and all of the stuff you've spent enormous amounts of time to make. We also want you to be able to play the game at higher than 1 fps.

- The Right Hand
For emergencies such as server going down email: drunkmunki[at]gamingalliance.net

User avatar
DrunkMunki
Server Admin
Posts: 1094
Tribe: Admin
Contact:
Status: Offline

Re: Devs Turret Changes - BE PREPARED

Post#5 » Mon Nov 20, 2017 10:15 pm

Source: https://www.reddit.com/r/playark/commen ... at_are_we/

Hey everyone, it's me again, that guy who says stuff, and then tries to make sure other stuff gets done. It's a crazy time!
So, here's what's going down for sure:
Advanced Auto Turret - BobCorp has provided me with authorization to distribute the BobCorp Automated Laterally Attenuated Nano-Cell Electronic Defense system.
This is a new turret we'll be introducing along with these changes that will essentially be a bigger, beefier turret, allowing for people who still need that damage density to populate their turret slots with turrets that are much more powerful. We're aiming for about a 4x increase in overall effectivness and cost for these things, along with disabling their use on rafts/boats. Construction costs and ammunition costs would be similar.
We opted for a new turret because of a few things:
It conveys the power of the new turret clearly without needing to add any new functionality to anything else.
It doesn't create a sudden need to change the balance/functionality of existing turrets in any way, and thus rebalance all sorts of other stuff that could potentially break. We're aware we have already done a lot of that, let's not push it farther.
Existing turrets don't get a sudden, massive buff.
We can scale the cost/maintenance/upkeep of the new structure without inconveniencing everyone who isn't pushing these upper limits.
If we need to introduce any specific kinds of modifications to this new turret, we can do it without impacting anything else.
So yeah, that's why that.
In addition, this stuff:
We're coalating feedback that we're getting along the way to develop a better sense of what exploits may exist or severely disproportionately effective tactics might need additional adjustment. For example:
Stego Armor Plates are going to have to block less damage from either bullets, or turrets specifically.
Veggie Cakes may need a more prohibitive delay on their use in combat.
Adding knockback to either the new turret, or both new and existing turret to offset specific movement speed + shield + rocket/c4 configurations may be in order.
These are just examples of things that we'll be seeing how they play out and what kinds of impacts they might have outside of just the general pvp game (for instance, veggie cakes and Therizinos are a good way to fight the dragon right now.) and whatever we decide to act on, I'll make a post about it sometime before December 5th.
I also want to take this opportunity to address a couple of things I've seen in comment threads:
With the exception of some really really large dinos (dinos like the Bronto) we disable idle animations on dinos on the server. Idle dinos also only tick once every 4 seconds or so instead of every frame. Idle dinos, while expensive, are only expensive in a general sense. Things like their animations have next to no cost on the server, and we use paralellized animations, which means most animations aren't even run on the main game thread (They're much faster.)
Dino head tracking is client-side, not server side. It has no cost on the server.
The vast majority of the servers that people play on are very expensive, custom-built servers with top of the line 8 core/16 thread cpus running at 4ghz, 64 gb of ram. We pay out the nose to make sure your servers are powerful.
The ARK Server also only uses 2 threads on the CPU. This is because the version of Unreal that we built the game on did not support multi-threading/paralellization. We have integrated Paralellized animation, and networking, so our networking overhead and cost of animation work are done on a 2nd thread. We're doing more work to try and thread more elements of the server, but it is incredibly complex and difficult work to do. Running 3 instances per server only takes up 6 cores at most. Having less instances per box would have no impact on the performance of each individual server.
We do a lot to make the game run as smoothly as possible, while still enabling the kind of freeform, open-world experience that the game was designed to be.
Anyways, I'll do a follow-up sometime later in the upcoming week about any things that we've decided we're going to do for sure. For now, TLDR:
Adding a new, more powerful turret that'll be able to replace about 4-5 of our current turrets.
Making Stego Armor Plates take more damage from turrets. Still collecting feedback and looking into additional measures.
Thanks for your time, and your patience.
The Right Hand
For emergencies such as server going down email: drunkmunki[at]gamingalliance.net

Who is online

Users browsing this forum: No registered users and 14 guests