A plea for scientific proof

[walker 0]

Hi all

There are a number of threads asking for changes to this and changes to that in ArmA II

I do not yet have the game as I have said I will wait patiently for the 505 release I will, I will. Any way back to the subject of the thread

I hear all this hoo haa and it is nothing but hot air, to which I have contributed as a die hard realism and difficulty fan. Hoo Haa means nothing including mine, it is opinion and little more. What is needed is science and verifiable proof.

If people want to argue for changes to the core abilities of ArmA II or that there is a "Bug" they have to do so with EXPERIMENTAL PROOF.

All the witness statements and videos are great recordings of a scientific experiment but they mean nothing without context. And I am sorry but all the witness reports and videos people make is not experimental proof.

The very least you need is an example experimental mission. Then others can repeat and examine your argument for its merits.

Scientific Method

We should all be using scientific method to support our arguments. Most of us leaned this in science classes at school for darwin's sake, scuse the expletive.

If you have not learned it or your like me damn lazy and like shortcuts use this as a Template

----------------------------------------------------

For the attention of:

Who needs to take notice, who is the intended audience

Name of the Experiment

Instructions to reproduce the "..."

Required Resources:

A list of what is needed to repeat the experiment.

At the very least a link to the experimental mission as a download

Machine Specs Used:

Use Belarc Advisor or some such

Player Settings Used:

Screen Shot of settings etc.

Summary

A description of what to expect

Method

Describe how to do the experiment

Observations

What you have observed to happen

Results

The effects of the experiment

Further:

Any additional thoughts or advice

Cause

Your hypothesis of what is causing the effect

Solutions

Suggestions on how to solve it if it is a problem

Known Workarounds

Problems

Problems with each workaround

Additional Material

Materials that provide additional support for the results

------------------------------------------------------------------------

You do not need to include each field and you can add you own; it is guideline not a bible.

Kind Regards walker

Edited June 8, 2009 by walker
Added machine specs and settings as requested by Killerwat

[okarr 10]

edit: ah nevermind. got it. i m slow this morning :)

[odjob 0]

Sounds pretty much like the community issue tracker:

http://forums.bistudio.com/showthread.php?t=73478

[walker 0]

Hi Odjob

I know about it.

The thread is about the need for the same rigour to be shown in the forums when people argue that something that is not necessarily a "bug" needs changing.

Eg the argument about AI reactions to being shot at, AI use of cover, accuracy of AI shots etc.

These are matters of debate rather than a missing texture or definable bug. That they may graduate to the class of bug and thus be entered in to a bug tracking system is a matter that comes through open but ideally scientific debate.

It is the purpose of the forum to debate stuff not to fix stuff.

As you point out the bug tracker is what is there for fixing stuff.

Kind Regards walker

Edited April 19, 2010 by walker

[odjob 0]

As you point out the forums are for discussions and we have the tracker for bug reporting so were does the scientific reports fit in?

[andersson 285]

In order to give a valid reply to some of the "bugs" people talk about there is need for a way to reproduce that. F.ex there is talk about AI that can see through buildings and objects, then you find people that dont experience that and have proof of the other..

If anyone experience what can seem wrong (bug, badly tweaked whatever) its easier for others to talk about it if they can reproduce what that person experienced. With that template Walker made its easier for me to setup the same situation that person had and look for the same things. Then its easier for me to test it more and give some valid input into the discussions instead of just writing about my experiences (that most often are very biased depending on the outcome of the last game..).

Its easier for BIS and us if people research what they experience more before they call it a bug and put it up on the bugtracker.

[walker 0]

Hi Odjob

If you are making an argument for a point of view such as:

The AI is too {hard/easy/just right} I have seen it, have video etc. its value as an argument is limited as you could just have an axe to grind, or be a troll, or a fanboy, your argument could be falacious and evidense fake.

Even if you make such an argument on honnest grounds and evidense; your assumptions can be wrong.

It is only by the use of scientific method and a repeatable experiment that such things can be verified.

My purpose is to remove the subjective and replace it with the objective while still giving room for forum debate for the passion that speaks to the qualities of the argument.

Kind regards walker

Edited June 8, 2009 by walker

[andersson 285]

One potential problem though can be all the different settings and tweaks that can be done + the different hardware. A better faster CPU gives a better AI as one example. Another is do you play cadet or veteran and if so what AI settings..

But I guess if it truly is a bug it should sow up regardless?

[odjob 0]

Oh, i completely missunderstood your purpose, i apologize for the intrusion :)

[walker 0]

Oh, i completely missunderstood your purpose, i apologize for the intrusion :)

No Odjob

You contributed to the quality of debate and scepticly asked for clarrification.

By doing so you made it easier for others including me to understand.

Kind Regards walker

[zoog 18]

May I add one of the most important thing of the scientific method: Falsify. Do not search for support of your hypothesis but search for observations that prove you wrong.

The school class example being:

Hypothesis: all swans are white.

You can look very long searching for white swans and find them every day. But as soon as you see a brown or black swan, dis confirming your hypothesis, you are a step further. Seeing white swans all your life doesn't prove that all swans are white. Seeing a brown or black swan proves that your hypothesis is in fact wrong. Which brings you hopefully to a new hypothesis.

[max power 21]

If we are going that deep into it, then we should touch on the null hypothesis. That is, when you are attempting to prove something, try to prove the opposite. The null hypothesis for 'All swans are white' is 'There is at least one swan that is not white'. If you prove the null hypothesis to be true, you experimental hypothesis is false.

So, if you believe the AI is reacting too quickly, you would try to set up situations where it should be easy for the AI to react appropriately ie. not setting up a turkey shoot where you are the turkey.

[walker 0]

Hi all

I think the full panoply of scientific tools should be used when testing ArmA.

Anything that is not an experiment or supported by experimental data has little value other than as an usupported plea to improve the product.

BUT making expansive all encompasing statments and pronouncments without repeatable experimental proof are of little or no value.

Kind Regards walker

[walker 0]

Hi all

fabrizioT has done an experiment that alows us to focus on improving the ArmA II AI I think it proves why it is important to follow scientific methods when arguing for changes to ArmA II.

Kind Regards walker

[De_willy 10]

OP, you hit the nail on the head. :)

[An Fiach 10]

One potential problem though can be all the different settings and tweaks that can be done + the different hardware. A better faster CPU gives a better AI as one example. Another is do you play cadet or veteran and if so what AI settings..

But I guess if it truly is a bug it should sow up regardless?

This is very true. Even identical systems which may have different software installed can have varying results and even identical hardware can vary in performance.

[householddog 10]

I am all up for scientific proof. How much does a scientist earn these days?

Just shoot me your address, Walker, so I know where to send the bill. ;) :)

Really though, does any other product require you to present a thesis, when it has issues?

The German release can be considered a beta. If there are a lot of people having the same issue, then it comes down to empirical evidence.

To conduct a scientific study, the engine would have to be opened up for testing. Something I doubt BI wants to do (nor should they).

Edited June 13, 2009 by householddog

[walker 0]

Hi householddog

1) The problem is that people present issues in such a way as to make it impossible to track down a bug.

2) And some less nice people even present non existent issues because they are trolling or because they find the game too hard and their pride prevents them from turning down the difficulty or the graphics quality, the I have the latest Graphic card but a crap CPU brigade or variations of the above.

The latter group 2) are just a nuisance but they waste valuable developer and community time. Forcing them to present real data makes them obvious.

The former group 1) are an important group so presenting them with a cut and paste method allows both the developers and the community to help them. It is a long tradition on this forum to present and examine bugs in a scientific way so that they get fixed and BIS have a reputation for fixing bugs once they are identified in fact their support is better than any other developer out there.

fabrizioT's work has discovered a bug in the knowsabout function of AI this will allow bis to address it in a later patch and allow people to alter settings in the mean time as a work around.

http://forums.bistudio.com/showthread.php?t=73139&page=29

Whining does nothing. What we are interested in this community is dealing in solutions.

The Real Virtuality Engine is the most moddable game engine there has ever been and in fact has been and is used in research and has a very scientifically minded and adult community who work to produce amazing things that no one not even the developers have thought of.

The community appreciation proper scientific debate. We all realise that all the other developers and publishers treat their customers like ego sheep to be run through corridor hoops and locked up in shoebox prisons that is why we have all come to ArmA as a refuge from the COD/CS/stock BF environments.

ArmA is the most open development environment there is and it has attracted an adult audience that wants to experience the freedom that ArmA offers.

The reason why you see scientific debate over poonzor leet ego sheep whining in ArmA forums is because ArmA is not a prison for your mind.

Kind Regards walker

Edited June 14, 2009 by walker

[Espectro (DayZ) 0]

What to do, when the bug happens seemingly randomly, then?

[walker 0]

What to do, when the bug happens seemingly randomly, then?

Hi Espectro

Seemingly random bugs are historicly the hardest for developers to spot but as the wording of your post implies they are only seemingly random. And it is by applying rational, systematic and scientific methods that they can be spotted identified, characterised and hence fixed.

Kind Regards walker

[Yoma 0]

I understand your point, but most of the times for a developer a bug simply needs 4 things:

-description

-affected version

-steps to reproduce

-hardware/OS/drivers used

Of course this leaves out a lot of the harder bugs, but not every bug needs to be submitted by someone in a white labcoat with a master in *some* science.

Science can be dreadfully slow if every minor issue would need the full "blabla" your proposing and in software development speed is of the essence.