Voting - Ticket System

[UNN 0]

If a voting system is in place for bug fixes, does this mean we should promote the benefits of fixing specific bugs on the forums? In order to try and gather enough popular support, to get a fix applied by BI?

Describing a bug for developers is not the same as describing the benefits a bug fix may bring.

Cheers

[suma 8]

Voting for meta issues, like http://bugs.armed-assault.net/view.php?id=2202 , makes no sense to me. We are unable to fix meta issues, only real individual issues.

[boecko 0]

Voting for meta issues, like http://bugs.armed-assault.net/view.php?id=2202 , makes no sense to me. We are unable to fix meta issues, only real individual issues.

I thought about that, too.

But on the other hand, why not?

I describes the wish for improvement in that area.

The normal player would say:

"The AI Pathfinding could be better"

He doesn't say:

"The AI should be able to cross the bridge in sector XY"

[suma 8]

Why not? Because the feedback is not specific enough to allow for useful decision making, instead they make the decision making even harder. The "AI pathfinding" is a parent of 20 issues, some of them seem to be important, other not. Such usage seems ill-defined and not well formalized to me. What should I do with them? Solve random one of them, or the most voted one, or the most priority one? And if I do this, what should happen to the voting score of the metabug? In reality, it will stay the same until all sub-issues are fixed, which is probably not what the voters intended.

I hereby declare I will ignore the voted metabugs, as I do not know what could I do with them, and I perceive them as a noise only.

Quote[/b] ]The normal player would say:

"The AI Pathfinding could be better"

He doesn't say:

"The AI should be able to cross the bridge in sector XY"

This very example shows what is broken in such usage. When user votes "The AI Pathfinding could be better", while having "AI on bridges" in mind, it will little help him if we will fix "AI fails to find a path between some objects and/or vehicles".

[bravo 6 0]

shouldn't this link: http://www.mantisbt.org/bugs/view.php?id=668 be on 1st post?

would make easyer to vote if forbiden links were removed, updating 1st post

ARF!

[suma 8]

To continue discussion where is it relevant:

Quote[/b] ]The votes for meta-bugs just indicate that the whole topic is important for the community and all related child should be looked into.

Actually most users vote on metabugs not because they think all related child should be looked into, but because some of them seem important to them (or sometimes even them mean some issues which they think are covered by this, but not specified in the BTS at all).

While not that obvius with anticheat metabug, which has only a few childern, thus is very obvious with AI metabug - http://bugs.armed-assault.net/view.php?id=2202. I think most people will agree some issues here are important, and other more a nit picking.

If the priority system should allow us to decide what to fix, it must allow us to identify a few top issues. If 30 issues are considered to be of same importance, it does not help us to make any decision at all.

I repeat: I ignore votes on metabugs, as they do not fit into my decision making process at all.

Quote[/b] ]What would be the point in meta-bugs otherwise?

I do not think they serve any real purpose, I think they support bad reporting habits. Bugs need to be reported as individual issues with concrete particular repro steps, not as "Everything is broken and needs to be reworked".

[sbsmac 0]

Now I've been reminded of this thread - and it does clearly state you would ignore meta-bug votes! .....

Quote[/b] ]I do not think they serve any real purpose, I think they support bad reporting habits. Bugs need to be reported as individual issues with concrete particular repro steps, not as "Everything is broken and needs to be reworked".

Can I respectfully suggest that this is not necessarily the best way to think of things ? (I don't at all wish to tell you how BIS should run it's develoment activities but I do have extensive experience in gathering customer requirements and bug reports so this is not just an academic post )

Classic bug-reports do come with clear repro steps - "do this and you will see that incorrect thing happen". In real-life any software without an explicit acceptance spec tends to suffer from less clear-cut complaints - "response is too slow" (what would be fast enough?), "GUI is confusing" (what would be clearer?), "can't transfer data whilst device is off" (doesn't user realise this is physically impossible?). The complaints are still useful feedback for developers in focussing on broad areas where effort might be spent. The danger of requiring very clear descriptions of features is that you run the risk of users trying to state the bug in the form of a required implementation. Eg, "server doesn't automatically disconnect users with 10 digit ids" when actually the problem statement is that the server does not detect fake users - 10 digit ids are just one possible symptom and obviously the implementor (BIS) is in a much better position to state the possible solution to the problem than the user is. The AI meta-bug is another good example. Many specific examples of potential AI improvements have been suggested but collectively they provide information to BIS that the AI behaviour is felt to be less than convincing. Some of the specific suggestions may be impossible to implement but you might be able to persuade users that AI had been improved by implementing a completely unrelated feature. (I'm reminded of the story of the Halo AI where MS found that user's perception of the AI was vastly improved when the aliens were made to verbally describe what they were doing, even though the AI routines themselves were untouched.)

Anyway, it is not my place to tell you how to prioritise your development but as someone who enjoys your products and would like to see them succeed I can only suggest that 'meta-bugs' are a useful source of market information even if they are difficult to use for specific actions.

[suma 8]

OK. I guess I should make a little bit more cleared what I mean:

With votes on normal bugs, it is quite easy to have a formalised process how to decide which to fix, which might be something like:

* browse bugs with most votes

* evaluate the workload needed to fix them

* check their report quality (most of all reproducibility)

* re-asses their importance, and put them into internal queues for fixing

Such formalized process is very hard to be applied to metabugs, and as a results, they are skipped in this process.

Frankly, I think given the number of different feedback routes we have (including this forum, reviews, contacts with players) I do not think coarse feedback provided by metabugs provides any useful information not seen in the other channels.

Feel free to vote for any metabugs, it will do no harm, but in the end it is the same as voicing your opinion here in the forum. I would like to avoid those voting having impressing BTS voting is some magical wand which will immediately make sure what was voted for is fixed - in the end it ends in the hand of the very same people who are reading this forum.

On the other hand, if someone really wants to help us, he can do it by post a good bug report, or a good feature design - such ticket on BTS has much better chance of being accepted into our internal queues.

[sbsmac 0]

Quote[/b] ]I would like to avoid those voting having impressing BTS voting is some magical wand which will immediately make sure what was voted for is fixed

Absolutely agree. As I said elsewhere, I understand the difference between customer interest and developer priority ! BTS does at least give the reassurance that it is clear that BIS review it, even if requests are rejected. In that sense its value is actually as a feedback mechanism for users. (In my own business we found customers were much happier when support tickets were visibly tracked - even a rejected ticket caused less stress than an untracked ticket.)

I thought it was also quite interesting what you posted in the multiplayer forum...

Quote[/b] ]? Everyone knows cheating is an important problem. There is no BTS voting needed for this. Forum poll would be probably more appropriate for such subject.

One of the reasons the meta-bug and 'call for votes' happened was that I don't think it _was_ obvious what BIS's attitude to MP cheating was. Again, no disrespect meant - I'm sure that BIS believes this is an important issue - but in the absence of feedback it is easy for users to assume that an issue is being ignored or not noticed. Straying off-topic a bit here but maybe one way of giving feedback to the community would be an irregular 'BIS-blog' from various members of the development team ? Even an off-hand comment like "we've noticed the concern over on-line cheating on the forums and are considering ways to stop some of the more serious cheats" would go a long way to placating the users (and would probably take less time than posting to the forums as much as you have today )

[-DirTyDeeDs--Ziggy- 0]

from what I understand, the BTS is not a tool for communicating to BIS about the game hacking. ok.

We must be content to voice our concerns about the hacking on the forum alone, and hope, without any official statements from BIS, that they are working on a solution.

heres to hoping

[.kju 3240]

Warning - very long post! Hopefully quality and useful though

It took me more than two hours to write that one.. and even

more time to think this through..

However sometimes it's hard to describe something with less

word, at least it's been very hard for me here.

At first, Suma sorry about the creating a meta for the public

cheating and promoting people to vote it!

I've been out of the ArmA community for 6 months just before this.

So i wasn't aware of it. Yet I agree mostly with you - more

about this later in this post.

Let's get to actual topic in here.

Preface

It is all about communication here.

You are right Suma that you have different channels for this.

Most are very time consuming and have a limited use.

PM and mail's are among the more useful ones I'd say.

However the CBTS has the potential to be a very efficient and

effective one!

The reasons are the technical framework can lead to high quality

and the technical gateway hurdle leads to more competent

people using this tool.

To get to this level, i'd like to propose a few changes.

There are three parties involved:

The community / reporters, the CBTS moderators and you (BI).

As preface - we can work on the first two,

yet we need your thoughts, comments and if you like the idea

and actual process, we need your commitment to this project.

From a realistic economic and updated over time perspective of

course - nothing is made forever!

Overall background:

Certain parts are moddable in ArmA, engine problems and

additions aren't.

However it even makes sense to get moddable parts into the

core as the core unmodified product is used by many people

(especially MP).

This could become a project to regain much lost credit and

sympathy from certain individuals.

Vision

It is all about time and money.

This means as long as the CBTS isn't useful to BI, it has no

reason to exist.

So the CBTS needs to become an efficient and effective tool for

bug and solution( ! ) reporting.

To some point it should be a tool to get improvements into ArmA(1).

Via various ways it should tell BI what issues need most and/or

the first attention.

It might serve as a platform to gather ideas for ArmA(2) as well.

Even more it could help improve ArmA(2) due to being based on

the same but improved engine.

In return to the hard work of the reporters and CBTS

moderators, BI is committing itself once the effort is appropriate

to integrate fixes and improvements into core product via game

patches.

Goal

~ Overall high quality in the reports. This includes:

Precise issue summary line.

Short and precise issue description (ie. max 10 short or 5 long

sentences).

Steps to reproduction required for bugs.

Big plus: actual solutions (ie for configs/missions or maybe

suggestions for script topics).

The additional information field can contain additional thoughts

and ideas. Everything which didn't fit in the description field.

~ Screenshots and small videos to visualize the problem / suggestion.

~ Demo missions once useful with short description and

steps to reproduce.

~ Active and strict filtering of issues before presented to you /

BI to lessen the effort on your side significantly. This is about a

recommendation system to tell you once issues are worth to

be checked from your side, as well as the general process

beforehand to get an issue to a quality level to take the

quality( ! ) barrier.

~ Active feedback about a ticket's status. No hundreds of dead

tickets. Either work in progress or resolved / closed. This

includes "Won't Fix / Never", "ArmA(2) / Future", "Not in this

patch, maybe in the following one(s)" (pushing tickets). People

prefer even a repulse / negative response than no response at all!

~ Community voted issues once the effort is appropriate might

get some priority.

~ However the actual criterion has to be quality. If we have a

very well presented problem / suggestion which just takes little

or appropriate effort, this one should have highest priority.

Especially if an appropriate and plug-in solution is presented

(best on the config/mission level).

~ _Goal_ No active tickets for the current work-in-progress

patch. Either the issue is work-in-progress (assigned), won't

be handled in ArmA(1) ("Won't fix or "future products") or is

assigned to a future patch version (somewhat the milestone

principle). That is the vision and the goal to be aimed for. The

practical part is only the process to get as close as possible to

the vision/goal!

Strategy

I) CBTS itself or report problems / suggestions

~ First and foremost is quality. This means people have to

provide quality and the CBTS moderators have to support and

even enforce this. Setting an issue to feedback means that the

ticket creator has to supply additional information and improve

the ticket quality. The requirements are outlined by the CBTS

staff. Of course other users can provide the requirements too.

A ticket in feedback status is no meant to get a review by BI.

The CBTS moderators have to change the status first after

giving the ticket a review for themselves.

~ Tickets with great report quality and especially with solutions

provided get a recommendation by attested "high quality" report.

~ Quality gateway: BI only looks at issues in the confirmed

status. None else!

The process works like this. The CBTS moderators review

issues and once the quality of a report seems sufficient, they

will set an issue to confirmed.

Now it's the the turn of you (BI). Once you (BI) need additional

information or cannot reproduce the problem, you will

document this with a few words and set the ticket back to

feedback. This triggers the process anew. As long as the

request isn't fulfilled the ticket won't get the confirmed status

by the CBTS moderators and therefore continue the process.

Otherwise you assign the ticket to yourself / BI members and

hopefully solve it as soon as possible.

Very important: We want you to be fair to yourself and the

reporter. If you think the issues is out of the scope of

ArmA(1) resources, please set it to "Won't fix" or "Future

products" - if you intend to review this for ArmA(2). Think

back to the opening, people prefer to know the truth and they

will accept it.

Another option in the process is to assign an issue to a patch

version after the one work-in-progress to highlight that this

one won't be resolved in the next patch. Via filtering you can

remove these tickets from your attention and also get them

back once the patch version is reached. This might be a bit of

an overkill though.

Yet the overall topic is here time spent. If you already take the

time to review a ticket and you decide not to work on it, you

have already spent quite some time for the review. The time

needed to use the technical options to close a ticket, change

the target version, request feedback just needs a few clicks

and a few words. Yet your part of documentation, your

thoughts about the issue are very crucial to the overall

process. Without it, no proper work flow is possible!

~ Priority could be used to have the CBTS moderators indicate a

possible priority from the community view (very soft and spare

use assumed though - NOT the same as number of votes. A bit

more unbiased hopefully).

~ Report quality "high", severity as well as number votes should

serve as the primary's indicator what should have priority of

attention on your side (BI).

II) SVN or provide solutions

Suma this is a topic which you have discussed a bit with IandC already.

The idea is to get solutions more into the focus of what we are

aiming for.

The simple part is just to report solutions in a CBTS ticket.

In addition to the CBTS we will provide an SVN server.

Everyone will have access to the SVN server with read access.

Everyone with a CBTS account will be able to commit changes

to the repository (write access).

Commit comments will be required and can be interlinked to

CBTS tickets (pre and post commit hook).

The repository will contain a basic ArmA installation - yet only

the standard ArmA configs in human readable format, scripts and

description.ext files.

NO textures, p3d and sound files! I hope that won't infringe your

copyright etc.

People will be able to checkout the repos and change the

standard ArmA files and commit them. By this you will be able to

get the diff for each commit, the diff itself. As long as commits

are thematic atomic (related to a specific issue solely), have

good commit comments and are linked to the correct ticket, this

option could serve as a good addition to the current CBTS

system.

Each commit will require a ticket to be interlinked with. So as

long as people won't abuse the system, this could generate

additional solutions on the code level.

This will just outline solutions, it will be still up to the quality of a

report and up to your decision whether to integrate a commit

into the core product or not. If not, a short comment in the

CBTS ticket from your side (after the ticket went through the

basic work flow described above) about the reason is to be made.

These commit and tickets can serve as concrete base for

discussion on the community side whether this should be

implemented or not.

Two practical examples:

~ AI infantry uses AT rounds on infantry - this is definitely not

in the interest of the community.

~ Standard AI values, ie skillXXX and precisionXXX, are not very

well set for the normal player. Different standard values are

in the interest of the normal player.

Severity tweak could be used to indicate that the issue "merely"

needs a config/mission/script tweak to be solved.

The concept of metabugs needs review. I think we should

disable votes in metabugs (close them) and metabugs should

only serve a stickies to pinpoint people to existing tickets of one

specific area.

Closing words

I hope you guys are still with me!

Hopefully Suma you like the overall idea of the suggestion or at

least parts of it.

After all it's down to your decision how to treat your product

and how much time and effort you still like to invest.

We can only try to lessen your effort and help you with making

this and maybe future products better.

Again i think this project could serve as strong positive basis to

heal wounds and to regain the lost respect amongst certain

individuals.

Your recent blog post is indicating that you plan to support

ArmA(1) in the future. So we have good faith.

Every feedback, thoughts and comments are very welcome -

especially how to improve the concept from the POV of you Suma (BI)!

best regards

Q

Credits and supporters

IandC

Boecko

PS: Now i have to hurry .. 3 hours late for work already ..

I'll try to polish to post tonight and fix spelling errors.