64-bit executable

[white 1]

NO, he wrote that 99+% usage is yet to be seen (and he's right - if your game uses 100% cpu/gpu, something is wrong, not good)

And? Since i am aparently so stupid, maybe you could explain what is wrong with this statement, please.

its not just apparent. stop posting.

[5133p39 14]

its not just apparent. stop posting.

You only prove my point - you have no arguments.

[jiltedjock 10]

Myke;2343168']Just out of curiosity' date=' may i ask about the programming/developing background the people in this thread have?[/quote']

14 years in Java based software dev. At the moment I am the infrastrucuture manager for a group of websites serving 14 million unique visitors p.a., using a 64 bit Solaris 10/Oracle 10G/Java 5/Apache environment which we built and host.

Nobody in this thread who keeps referring to "streaming from disk" has indicated where they think that data is being streamed to. Presumably because they know it is being streamed into RAM, and therefore the logic behind the change to a bigger usable address space being a positive step for the engine is inarguable.

Edited March 17, 2013 by jiltedjock

[white 1]

You only prove my point - you have no arguments.

yes, that must be it.

[k3lt 3]

NO, he wrote that 99+% usage is yet to be seen (and he's right - if your game uses 100% cpu/gpu, something is wrong, not good)

I'm sorry but this caught me, what are you saying game which is properly utilizing available hardware to the maximum is wrong / not good?

You do realize this can be done in pretty much every game with proper multithreading / utilization?

Not that long time ago i was also posting screenshots in http://forums.bistudio.com/showthread.php?147533-Low-CPU-utilization-amp-Low-FPS/ from project cars (80-95% cpu on all cores / 99% gpu utilization) as reply to Dwarden which was claiming it's impossible to see that kind of usage in games.

So can you elaborate how is that wrong exactly?

Edit:

Found it http://i.imgur.com/K0sJc1i.jpg , http://i.imgur.com/NIAoa5i.jpg

And CPU usage stays that high no matter if you're CPU bottlenecked or not, it's just properly unloading all workload on all availabile cores.

Edited March 17, 2013 by k3lt

[[frl]myke 14]

an argumentum ad hominem is prone to happen

I wasn't arguing, i was asking a simple question, no need to be defensive about.

Not the most professional background, which I'm sure will be used against me.

No, it wont. It is far better than expected to see in this thread. Experience showed me that most user discussin in such threads mostly just know how to include a script in a mission. I've been proven wrong but i was really just curious.

Now this doesn't mean i do follow your argumentation, though. ;)

[Moderator mode]Please try to keep this discussion on a non-inflammatory level, else i will have to close it. Thank you.[/moderator mode]

[white 1]

Myke;2343257']I wasn't arguing' date=' i was asking a simple question, no need to be defensive about.[/moderator mode']

sorry, you are right. this is the "internets" afterall, im just used to it.

Edited March 17, 2013 by white

[Insanatrix 0]

Myke;2343257']I wasn't arguing' date=' i was asking a simple question, no need to be defensive about.

No, it wont. It is far better than expected to see in this thread. Experience showed me that most user discussin in such threads mostly just know how to include a script in a mission. I've been proven wrong but i was really just curious.

Now this doesn't mean i do follow your argumentation, though. ;)

[Moderator mode']Please try to keep this discussion on a non-inflammatory level, else i will have to close it. Thank you.[/moderator mode]

No amount of optimization of the engine will ever cause you to exceed the physical limitation of a bus bandwidth. You will never see Physical memory bandwidth or access time's from a HDD or an SSD. The fact that they had to program a way around the 32bit limitation shows that they have exceeded it. They are now using a HDD or an SSD as a replacement for physical memory. If HDD's and SSD's were meant to be used as a replacement for physical ram, why aren't they?

I asked one of my friends I mentioned, his opinion on it. He basically said the same thing. He basically said, Why would you create a system that can bottleneck so easily when you have an alternative, 64bit addressing, especially if you know about it and allow it to continue in future iterations of software. RAM exists for a reason. It's faster, has much quicker access to the main system bus therefor it can keep feeding the CPU and GPU data to process.

Why does CPU and GPU usage go down as pagefile usage and I/O buffer usage go up? Why do things like increasing draw distance and increasing or decreasing texture size or using low terrain texture's to get rid of the grass layer, which in turn causes a significant boost in fps and significant drop in pagefile usage, not effect GPU or CPU usage?

He even said that the fact that CPU and GPU usage go down as pagefile usage goes up, and FPS goes down while CPU and GPU usage goes down is about the only proof you need that it's a bandwidth bottleneck. The processing and rendering threads are stalling because the SATA bus or the physical hard drive speed can't keep up with the engine demands, therefor your CPU and GPU have less to do because they are constantly waiting for data to be sent to them. The slower the data stream from the source, the lower your GPU and CPU usage gets and the lower your fps gets because both threads are tied to one another.

Also it's kind of hard to prove that 64bit would have no advantage when you have nothing to test it against. You have no way of testing the engine in a 64bit environment with 64 bit pointers, so how can you say there would be no improvement?

So I still say it's a fallacy that 64bit addressing would not have any benefit on the RV engine. You can sit and misconstrue it and try to fit how the RV engine works now versus how it should work optimally to fit your argument that it would not benefit all you want. At least I can say I'm trying to push for a course that might fix it rather than looking at the situation through rose colored glasses and simply saying, no.

[calin_banc 19]

With all the comparison between BF3 and ArmA 2/3, here's one for GPU and CPU (scaling).

http://gamegpu.ru/action-/-fps-/-tps/arma-iii-alpha-test-gpu/testovaya-chast.html

http://gamegpu.ru/action-/-fps-/-tps/battlefield-3-ehd-game-test-gpu.html

http://gamegpu.ru/images/stories/Test_GPU/Action/Battlefield%203%20End%20Game/test/bf3%20EG%20PROZ.jpg (126 kB)

http://gamegpu.ru/images/stories/Test_GPU/Action/ARMA%20III%20Alpha/test/arma%203%20proz%20u.jpg (124 kB)

[5133p39 14]

I'm sorry but this caught me, what are you saying game which is properly utilizing available hardware to the maximum is wrong / not good?

You do realize this can be done in pretty much every game with proper multithreading / utilization?

Not that long time ago i was also posting screenshots in http://forums.bistudio.com/showthread.php?147533-Low-CPU-utilization-amp-Low-FPS/ from project cars (80-95% cpu on all cores / 99% gpu utilization) as reply to Dwarden which was claiming it's impossible to see that kind of usage in games.

So can you elaborate how is that wrong exactly?

Edit:

Found it http://i.imgur.com/K0sJc1i.jpg , http://i.imgur.com/NIAoa5i.jpg

And CPU usage stays that high no matter if you're CPU bottlenecked or not, it's just properly unloading all workload on all availabile cores.

Anybody can write few lines of code that can utilize CPU up to 100% - it doesnt tell you anything about the efficiency of such code, but normally you can't get a 100% utilization from a complex program (unless you write the program specially to do that), there will allways be some blocking, process switching, cache/memory wait times, etc.

And actually, with multithreading, it may be even harder to get near those 100% on CPU due to issues brought by threading (depending on complexity and implementation).

[Leon86 13]

stuff

It'll have benefits, sure, but how big would the benefits be? Arma might stream stuff in and out of ram, but it's running from what's in ram. If it actually needs stuff from a harddrive or starts paging you'll know, it'll stutter like mad, while the real problem seems to be low (but consistent) fps.

[5133p39 14]

So I still say it's a fallacy that 64bit addressing would not have any benefit on the RV engine.

I do not think anybody said there wouldn't be any benefit, only that the potential benefit wouldn't be worth the effort.

Also, perhaps you didn't realized that going 64bit wouldn't getrid of streaming - we would still need it, as 99% of players haven't got the needed amount of RAM to accomodate the whole game (+addons) without streaming.

So, for those few with 32+GB of RAM it might bring some benefit, but even with all the magic optimizations, it won't make the game run in 60 FPS minimum, if they currently have only 30 (as most people seems to be thinking - not saying you too).

Maybe a little code redesign here and there could gain more benefits than this whole 64bit fantasy.

[Insanatrix 0]

It'll have benefits, sure, but how big would the benefits be? Arma might stream stuff in and out of ram, but it's running from what's in ram. If it actually needs stuff from a harddrive or starts paging you'll know, it'll stutter like mad, while the real problem seems to be low (but consistent) fps.

For example to render a scene in front of you, if you kept the camera static, assuming a 2,000 meter view distance, with nothing but static objects within draw distance, if the total sum of memory needed is larger than 3gb, that means you are constantly swapping from HDD to memory any change or variation that happens.

So for instance taking the prior example, you're rendering that scene and everything is preloaded into RAM, and another object comes on screen. You have to swap something out of RAM, load up that new object into RAM, assign it a memory address, if it's a texture it's then sent to the GPU which shares the same memory limitations as your physical system memory because in terms of addressing, memory is memory.

What happens to whatever it was that you swapped out of ram to the pagefile? What if it's still needed? What if an LOD changes because you moved? What if a function calls for that piece of data that you swapped out? It has to be loaded back into RAM. Imagine this taking place for every object. Every Player, every AI soldier, every car, every truck, every house, every fence, every terrain texture, every blade of grass, every single thing in the game world. Then you factor in things like precomputed functions, scripts, all the code that's in use, AI FSM's, config files, audio files... Literally everything that is in use in the world, even the things behind the scene's that you don't see, reside in RAM to be called upon by the CPU or GPU. If it's not in ram it has to be paged out of virtual memory and into ram and if your RAM is full because of the 32 bit limitation then something has to be paged back to virtual memory to make room, It's a constant cycle and it happens very very fast. Your FPS is limited by this because the CPU or GPU cannot process something until it has it in memory to process. It could take 100 nanoseconds or it could take 10,000 nanoseconds, all depending on load and saturation of bandwidth across the interface. And while it is transferring, your CPU or GPU is in a halt state until it gets that information.

Now imagine if all that data, all those texture's resided in RAM, to be called upon by your CPU or GPU when they needed to be. Imagine if very little had to be paged to virtual memory, only non essential texture's and files or code. Imagine if instead of trying to swap 5gb worth of data across a 150-200mb/sec interface ( that's being generous too ) you could store it in a place that could be directly accessed by the CPU or GPU, across a interface with a bandwidth of over 10,000mb/sec. Your CPU wouldn't constantly be going in and out of a halt state, waiting for data to be streamed from the HDD to memory and then to L2 or L3 cache. Doesn't that sound much more optimal, as well as a faster and more efficient way of doing it?

[rogue trdr 10]

Myke;2306175']Simply wont happen and making a 64bit exe wouldn't help either. The RV engine isn't designed to load and keep huge amounts into ram but stream required data from harddisk/SSD. Now one might ask why they don't redesign the engine to preload data into Ram. Simply put' date=' business decision. It would raise the required/recommended amount of Ram much higher than it actually is. Which, in return, reduces the possible customers as they wont have those required/recommended amount of Ram. Also not everyone uses a 64bit OS these days.

At last, they could have decided to design some kind of hybrid: load data into Ram if enough ram is located, else stream from disk. Could have been done. Would have required a major redesign of the engine. Finally not everyone could have profited from this, so, what should BI do? Assign resources for a redesign of the engine which can't be used by everone anyway? Or assign those resources for engine improvements everyone can profit of? Development cost time and money, both resources that are limited.

So unless not absolutely everyone has a 64bit OS and at least 16GB ram, a redesign wont happen.

:EDITH:

Oh, you have access to BI's bank account i see....

:facepalm:[/quote']

Nice to see the moderators are mature folk.

Ram is cheap, you can pick up 16gb for around $100 in Australia where we typically pay two or three times what the US and UK do for anything PC related. Your argument is also invalid in that any PC that is just running 2gb ram these days is going to struggle even running minecraft if they are running Windows Vista or later, and I'm pretty sure I haven't even seen a laptop with that little ram on board from stock in a few years.

[Insanatrix 0]

Did a quick test with a draw distance of 10,000m. CPU usage dropped to around 40% average, GPU usage dropped to 10-20%, Memory usage stayed roughly the same was only slightly higher. Logical Disk read/write's were through the roof, while memory transfers were capped by the Logical Disk transfers.

So pretty much the processing threads are stalling because of the bottleneck on the I/O operations.

[Leon86 13]

For example to render a scene in front of you, if you kept the camera static, assuming a 2,000 meter view distance, with nothing but static objects within draw distance, if the total sum of memory needed is larger than 3gb, that means you are constantly swapping from HDD to memory any change or variation that happens.

So for instance taking the prior example, you're rendering that scene and everything is preloaded into RAM, and another object comes on screen. You have to swap something out of RAM, load up that new object into RAM, assign it a memory address, if it's a texture it's then sent to the GPU which shares the same memory limitations as your physical system memory because in terms of addressing, memory is memory.

What happens to whatever it was that you swapped out of ram to the pagefile? What if it's still needed? What if an LOD changes because you moved? What if a function calls for that piece of data that you swapped out? It has to be loaded back into RAM. Imagine this taking place for every object. Every Player, every AI soldier, every car, every truck, every house, every fence, every terrain texture, every blade of grass, every single thing in the game world. Then you factor in things like precomputed functions, scripts, all the code that's in use, AI FSM's, config files, audio files... Literally everything that is in use in the world, even the things behind the scene's that you don't see, reside in RAM to be called upon by the CPU or GPU. If it's not in ram it has to be paged out of virtual memory and into ram and if your RAM is full because of the 32 bit limitation then something has to be paged back to virtual memory to make room, It's a constant cycle and it happens very very fast. Your FPS is limited by this because the CPU or GPU cannot process something until it has it in memory to process. It could take 100 nanoseconds or it could take 10,000 nanoseconds, all depending on load and saturation of bandwidth across the interface. And while it is transferring, your CPU or GPU is in a halt state until it gets that information.

Now imagine if all that data, all those texture's resided in RAM, to be called upon by your CPU or GPU when they needed to be. Imagine if very little had to be paged to virtual memory, only non essential texture's and files or code. Imagine if instead of trying to swap 5gb worth of data across a 150-200mb/sec interface ( that's being generous too ) you could store it in a place that could be directly accessed by the CPU or GPU, across a interface with a bandwidth of over 10,000mb/sec. Your CPU wouldn't constantly be going in and out of a halt state, waiting for data to be streamed from the HDD to memory and then to L2 or L3 cache. Doesn't that sound much more optimal, as well as a faster and more efficient way of doing it?

Yeah, but if it doesn't run out of space the benefits are fairly limited.

[jiltedjock 10]

I do not think anybody said there wouldn't be any benefit, only that the potential benefit wouldn't be worth the effort.

Also, perhaps you didn't realized that going 64bit wouldn't getrid of streaming - we would still need it, as 99% of players haven't got the needed amount of RAM to accomodate the whole game (+addons) without streaming.

So, for those few with 32+GB of RAM it might bring some benefit, but even with all the magic optimizations, it won't make the game run in 60 FPS minimum, if they currently have only 30 (as most people seems to be thinking - not saying you too).

Maybe a little code redesign here and there could gain more benefits than this whole 64bit fantasy.

Why would you need to accommodate the whole game? In a given mission, only one island is streamed in, for example. Try watching which pbos are constantly accessed during a typical mission, you can see that 4GB of address space would be more than enough for most missions.

For many of us, 4GB on a graphics card is not unusual, never mind system RAM.

[ric 1]

It's not worse then what i see when BF3 is on. The difference here is that BF3 is not using any sort of AI though

That said, do you actually believe that 64 binaries would change that?

but the frostbite engine is capable of utilizing the AVAILABLE hardware, hence why my game became smoother when i went from a tri core to a quad core.

And the engine will not be re-written mid development no matter how much you voice your opinion around those forums.

nore should they have to, it should have been ongoing...if your in the business of creating games for PC then you have tobe aware of there evolution or you wind up with the situation we have now....

It obviously wasn't a priority. It has been said long time ago that BI will not take the 64 binary route.

Maybe because some moderators, as well as some of users, have been working in the gaming industry, and they have a clue about what they are talking about?

well you and the MOD's better get over to sony and stopping them from making a horrific mistake by releasing the new PS4 with 8 cores and 8 gigs of ddr5 ram.

fair enough, but then you would also need to rename some accounts around here to "windmill01", "windmill02" etc ;)

while your renaming change some to spindoctor01-?

[simjedi 10]

The processing and rendering threads are stalling because the SATA bus or the physical hard drive speed can't keep up with the engine demands, therefor your CPU and GPU have less to do because they are constantly waiting for data to be sent to them. The slower the data stream from the source, the lower your GPU and CPU usage gets and the lower your fps gets because both threads are tied to one another.

This is not the case with ArmA. I use a 4gb RAM-Drive with nearly all the big PBO's sitting inside of it and there is no fps difference to be made. The only thing that speeds up is loading/texture streaming times. Something else is going on. I even tried a combo with spanning some of the PBO's across multiple SSD's and still no fps difference.

Edited March 17, 2013 by SIMJEDI

[Insanatrix 0]

Yeah, but if it doesn't run out of space the benefits are fairly limited.

Yet it's running out of space, it runs out of space upon load hence why they had to create this whole data streaming architecture in the first place. Why do you think their post about it was entitled breaking the 32 bit barrier.

This is not the case with ArmA. I use a 4gb RAM-Drive with nearly all the big PBO's sitting inside of it and there is no fps difference to be made. The only thing that speeds up is loading/texture streaming times. Something else is going on. I even tried a combo with spanning some of the PBO's across multiple SSD's and still no fps difference.

Because your windows swapfile is where it is storing the data. Put your swapfile on a RAM-Drive, see how that works for ya. It's not something I would run with normally, but if you want to test it out go ahead. I may honestly try it myself.

Just tried it and it's impossible to get your only swapfile on a RAM-Drive. It will always create another swapfile on an actual physical disk.

Edited March 17, 2013 by Insanatrix

[Leon86 13]

Yet it's running out of space, it runs out of space upon load hence why they had to create this whole data streaming architecture in the first place.

Well, they ran out of space 10 years ago. And it's nice to have tech like that.

anyway, it's perfectly possible to completely wreck performance with virtually no read from disk. 64bit would be nice, but the performance problem wont be solved with it.

[simjedi 10]

Yet it's running out of space, it runs out of space upon load hence why they had to create this whole data streaming architecture in the first place. Why do you think their post about it was entitled breaking the 32 bit barrier.

Because your windows swapfile is where it is storing the data. Put your swapfile on a RAM-Drive, see how that works for ya. It's not something I would run with normally, but if you want to test it out go ahead. I may honestly try it myself.

I only have a small 100mb page file on my C drive. Since I have 12gb of RAM that's all I set it at.

[Insanatrix 0]

I only have a small 100mb page file on my C drive. Since I have 12gb of RAM that's all I set it at.

And windows will re-size it if it needs to even if you tell it not to.

[ric 1]

And windows will re-size it if it needs to even if you tell it not to.

and that is why mine is turned off :)

[simjedi 10]

And windows will re-size it if it needs to even if you tell it not to.

It does no such thing as I do not run any programs that require that much RAM. Dosen't it only write to the page file once you run out of RAM? Isn't that what it's designed for?