9
\$\begingroup\$

I realize this question is complex and subjective, but bear with me for a moment.

I firmly believe that video-game software is essentially different from, for example spreadsheet software, as from a user point of view, some games have more similarities with film art than with regular software. However, when it comes to requirements prioritization methods that help developers sort out which feature or idea is most important, there is none that facilitate the creative aspects of video games. Most available techniques only aid regular software development where focus is on the risk and cost of requirements, and do not take into account the creative aspect of video-game requirements.

To illustrate the problem, think of a requirement (i.e. feature idea) such as the one seen in Super Paper Mario, where Mario can exit the 3D world and enter a 2D world, and vice versa in order to go around obstacles. This is a fairly challenging requirement to implement and probably costly, but definitely crucial to the success of the game. Available prioritization methods overlook the creative/entertainment value of this requirement and mainly just estimate how costly and (technologically) risky implementation is. Of course, this is a very simplistic example, but you can imagine a case where there is a pool of great game ideas (functional and non-functional requirements) and not enough resources to realize them all. When it comes to value calculation, academic research doesn't provide game developers with an adequate way to estimate the value of game requirements.

UPDATE / Clarification: In my research, I study available software product management solutions (more specifically requirements prioritization algorithms) and try to find out why are they not suitable for game development. It seems that it is the creative or entertainment nature of the software itself that introduces this incompatibility. It is the (available) techniques inadequacy, to recognize the core value of the software and its conceptually different purpose.

With regular software, it is the user or his/her needs that most often help estimate the value of requirements. Moreover, it is the user that produces most requirements. In video-game software, it is the creative vision that drives the requirements. With regards to requirements origin, it is the outside in vs. the inside out paradigm. Having established this fundamental difference, we can deduce that if requirements prioritization (RP) algorithms focus on user ideas/needs in order to estimate the value of requirements for regular software, then in the case of video-game software, the RP algorithm should satisfy the creative vision. By creative/entertainment value of a requirement, I refer to the degree to which the core vision relies on this particular requirement.

What I try to do is find a way to prioritize the requirements according to their relevance/importance to this core creative vision. This will ultimately provide a creative value, but it is relative to the central idea and the stakeholders’ ability to subjectively assess the requirements. This is just one side of the RP algorithm, as other factors such as risk and cost need to be taken into account as well, but the available RP solutions already offer adequate ways to do that, and they are compatible with the needs of game development.

The reason I’m writing here is because I am trying to see how developers deal with these problems (PR and focusing on the core idea) when dealing with more complex projects.

I try to refine part of the preproduction process by developing a requirements prioritization method tailored to the needs of game development industry. A pivotal element in such a method is the ability to identify and estimate the creative/entertainment value of requirements. However, in order to do that, I need to understand how game developers perceive this creative/entertainment value of requirements. In a nutshell, I am seeking answers to the following questions:

Question 1: How would you define the creative/entertainment value of video-game requirements?

Question 2: How would you measure it?

Question 3: Who should measure it?

I would love to see how these issues are perceived by game developers and I would appreciate your take on them here, but if you want to contribute to this research – receiving my eternal gratitude and a proper credit/citation in the research and all the publications that will follow, please fill up my extremely short survey (just 7 questions, 3 of which you already see above):

The extremely short survey which will cement you as my personal hero.

Research information and trigger

This research is being conducted at Utrecht University, The Netherlands, as part of an Information Science master’s thesis.

Game development is in many ways similar to product software development as developers follow certain software development processes. Following a poor development method (or none at all) could result in longer development times, going over budget and/or delivering buggy products (Bethke, 2003). What makes video games different is the creative game vision that must be shared by the entire team to ensure that the end product is consistent and of good quality. This is especially true for full-blown game titles where from a user point of view, there are more similarities with film art than with any other software. Unfortunately, this curious creative aspect renders many software product management techniques unadoptable by the gaming industry.

I am quite interested in working towards improving the game development process in the preproduction stage, by creating a requirements prioritization method tailored to the specific needs of the game industry and I need your help! The easiest way to contribute to the research is by filling up my short survey (link above). If you find this research interesting, please contact me at [email protected]

Researcher information

My name is Alex Chervenkoff, an avid gamer and quite excited about this research! You can contact me at: [email protected]

Degree in BSc Computer Science, The University of Sheffield, UK.

Currently doing MSc Information Science at Utrecht University, The Netherlands.

Thank you!

Improve this question
\$\endgroup\$
3
  • 8
    \$\begingroup\$ Scan the source, make sure #pragma fun is set \$\endgroup\$
    – Doug-W
    Aug 13, 2011 at 14:07
  • 2
    \$\begingroup\$ @Doug - Not all fun can be determined at compile time, hence the addition of an assert(isFun()) gives us the complete picture \$\endgroup\$
    – Maik Semder
    Aug 13, 2011 at 21:50
  • 4
    \$\begingroup\$ Thankfully, it's compiled out in release builds, so we can still ship something... \$\endgroup\$
    – user744
    Aug 13, 2011 at 23:37

3 Answers 3

Reset to default
3
\$\begingroup\$

The only real way we have to test such things is with, well, testing. Professional game developers will tell you all the time that the single most effective way to know if certain gameplay is working and fun is to give it to players and observe.

Indeed, one of the reasons that achievements are everywhere these days in games is because they give invaluable information to the developer. If you want to know where players stopped playing your game, make an achievement for completing each level. If you want to know how many players aren't playing a certain race in Civilization, look at how many people got the achievement for playing that race. And so on.

Improve this answer
\$\endgroup\$
1
\$\begingroup\$

Question 1: How would you define the creative/entertainment value of video-game requirements?

I don't know. I'm not convinced the science is ready, yet.

Question 2: How would you measure it?

We can't.

Question 3: Who should measure it?

Us. :(

Right now, the only useful metrics we have are sales figures and ratings (eg. MetaCritic). We don't understand the domain well enough to do much more than that, which is why you see a lot of copycat games - taking a game that we know to have worked and tweaking the formula a bit gives a good chance of having a second game that works. 'The acorn doesn't fall far from the tree', and all that.

Some designers have attempted to come up with broad checklists or systems for judging the quality of a game or its mechanics. eg. In Jesse Schell's 'The Art of Game Design' he describes 100 'lenses' which are each a way of critically evaluating a game. This provides a system of measurement, but the measurements themselves are still fairly subjective. Similarly the MDA system (Mechanics/Dynamics/Aesthetics) broadly implies that your mechanics should combine to form dynamics, and your dynamics should support the aesthetic, but again this provides a tool for measurement but no units of measurement.

Improve this answer
\$\endgroup\$
6
  • \$\begingroup\$ Right now, the only useful metrics we have are sales figures and ratings (eg. MetaCritic). I would suggest that neither of those are "useful metrics" for anything other than sales and how much people rated something. \$\endgroup\$
    – Nicol Bolas
    Aug 19, 2011 at 2:18
  • \$\begingroup\$ I would suggest both sales and MetaCritic ratings correlate strongly with quality. \$\endgroup\$
    – Kylotan
    Aug 19, 2011 at 14:02
  • \$\begingroup\$ Why? If there was such a thing as journalistic integrity in the videogame industry, that might be the case. But there are too many bought-and-paid-for reviewers for MetaCritic to have any legitimacy. And popularity (ie: sales) has never been a legitimate measure of quality. \$\endgroup\$
    – Nicol Bolas
    Aug 19, 2011 at 17:15
  • \$\begingroup\$ There are plenty of journalists with integrity, and the idea of aggregating scores is that the average is more resilient to outlying data, just as with any study or observation. As for sales, simply saying it's not a legitimate measure doesn't make it so. Players vote with their wallets. Quality is only one factor in what they decide to buy, but it -is- a factor. \$\endgroup\$
    – Kylotan
    Aug 19, 2011 at 19:14
  • \$\begingroup\$ You must not look very hard at videogame journalists if you think that there are plenty with journalistic integrity. Most game reviews from the big sites for "AAA" games are bought and paid for. It doesn't have to be all of them, but the big publishers buy enough big reviews to skew MetaCritic (which is why they buy them). As for sales, yes, quality is one metric that factors into what people buy. But it is not the only metric. And if you're talking about any form of scientific accuracy, you cannot accurately deduce anything if multiple factors are involved. \$\endgroup\$
    – Nicol Bolas
    Aug 19, 2011 at 20:16
0
\$\begingroup\$

As mentioned in other answer(s), the definitive answer is testing.

The usual "silent observer" method of user testing I suspect is sufficient for a lot of metrics with follow up questions to explore any areas of confusion about what the testers might be feeling.

Sidenote: I would suggest that your core premise that games are fundamentally different to applications due to the entertainment factor is incorrect. Sneak a peek at something like Danc's "Princess rescuing application" presentation ( http://blog.iainlobb.com/2008/10/princess-rescuing-application.html ) and see what you think.

Improve this answer
\$\endgroup\$
1
  • \$\begingroup\$ Showing that you can incorporate lessons from game development into normal application development doesn't show that the reverse holds true however. I would say they are fundamentally different for precisely the reason this question is asking - there are no clear requirements nor metrics to test them against. \$\endgroup\$
    – Kylotan
    Aug 18, 2011 at 11:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .