Advance multiplayer statistics

[Lewin]

Krom and I want to make the master server record more statistics about multiplayer games. Currently the only thing that is sent to the master server is the map name and the number of players, which is sent after 30 minutes for building maps and 3 minutes for battle maps (so false starts aren't counted).

We'd like to be able to gather a lot more information like army compositions, kill statistics, number of citizens/houses. Maybe even sending the data every 15 minutes so you can see behaviour over time.

The idea is we would keep all of this information in a database so we can query it. We'd also like to make it publicly accessible so anybody can use it (there would be no private data in there, only game statistics). If it's in a database you could do complicated queries like: "Average number of serfs used on location 3 of the map Back in the Desert", as well as simpler things like comparing crossbowmen usage to bowmen usage.

Note that we would not be collecting or storing any personal information like nicknames, just anonymous game statistics.

We would like your suggestions for what data should be stored and how we could use it. What kind of queries would you like to be able to do? Remember that this could be helpful with balance discussions, or helping map authors see how users are playing on their maps. Please discuss your ideas

[T*AnTi-V!RuZz]

First thing that pops into my mind: server load. Can you say something useful about this, like an educated guess or something? Or will we only know by doing it?

[sado1]

Note that we would not be collecting or storing any personal information like nicknames, just anonymous game statistics.

People that want player accounts+stats coming in 3, 2, 1...

Well. From a point of being a mapmaker, it'd be fun to see some general average loc stats on various things like: how easy a loc is to defend on average, compared to other locs; how easy to attack; which locs are good for eg. wood production and which not...

[Lewin]

First thing that pops into my mind: server load. Can you say something useful about this, like an educated guess or something? Or will we only know by doing it?

It should be tiny. It would be done with PHP scripts and MySQL like the rest of the master server. For each game played the host would send the information to the master server (as it already does, but a lot more data) when the game ends, this would then be stored in a MySQL database. So that's one HTTP request per game played. Querying could be a bit slower once we have a lot of records but we won't be doing that too often and database are designed to process queries efficiently.

[T*AnTi-V!RuZz]

First thing that pops into my mind: server load. Can you say something useful about this, like an educated guess or something? Or will we only know by doing it?

It should be tiny. It would be done with PHP scripts and MySQL like the rest of the master server. For each game played the host would send the information to the master server (as it already does, but a lot more data) when the game ends, this would then be stored in a MySQL database. So that's one HTTP request per game played. Querying could be a bit slower once we have a lot of records but we won't be doing that too often and database are designed to process queries efficiently.

Ah, I see. That should be doable

[Krom]

@Sado: Part of the proposal is to identify how to do that. For example, how easy loc is to attack, what kind of stats would allow us to see that? We can have stats of how many times the loc was defeated, would that be enough approximation?

[Nissarin]

As tempting as it seems, in my opinion it would be impossible to gather useful information from such statistics, at least for most of the team games. They are very chaotic and tend to be unbalanced (at least "public" games), accompanied by small player base and not so small map pool you could probably build random generator from it but not the kind of tool you want to create.
If you want to gather some information about specific location it would be faster to ask the players, maybe implement some questionnaire with several common questions and optional "leave comment" section on the (in game) statistics screen and then send them to server.

As for any actual area where this proposal might be useful (IMHO) - comparing build order, which means gathering information about timings of specific buildings, accompanied by "graphs" (resource collection, production, number of citizens, stamina level), it maybe still too generic (compared to watching the replay) but could be useful.

[Krom]

@Nissarin: I doubt we can repeatedly ask representative pool of players tens of different questions about each map. Also note that opinions might be very biased and have different weight when said by different players. We already have that with the forum (see all discussions that went into flame ).

Comparing build order is an interesting idea, so we can categorize it into some typical sequences, but I'm not sure what use we could learn some valuable info from that. Can you explain that please?

My suggestions:
- game ID, so we can uniquely see Hands that played in same match
- PT length
- alliance info of Hands, so we compare alliance to each other in games outcomes
- collect market stats to see which trades are the most used and either tweak the prices a little bit. That would require 27x2 columns (counts of each ware traded From and To)
- collect towers placed/lost count. That way we could see how many towers kept player safe or with how few destroyed he was defeated, see if player went defensive or not when compared to army counts.
- army counts by type, trained and lost. Obviously to see balance changes and see if some army type is too rarely used and can be buffed
- serf, workers, recruits count. To see how economy-oriented player played
- houses count? Not sure if we should split that down to houses.. maybe overall count is enough?
- percentage of map explored?
- percentage of raw materials mined? (stone, gold, iron, coal)

All this except PT length could be in one big table

[Nissarin]

@Nissarin: I doubt we can repeatedly ask representative pool of players tens of different questions about each map. Also note that opinions might be very biased and have different weight when said by different players. We already have that with the forum (see all discussions that went into flame ).

First of all it should be optional, something like "Leave (map) feedback", with basic yes/no (maybe 3 step rating) questions. Things like, was it easy to gather wood/stone/gold/iron, is it easy to defend, is there enough space, is it balanced compared to other locs, etc. (maybe 15-20 in total). Granted, there will be a lot of idiots leaving useless information but you still can get some useful information from it.

Comparing build order is an interesting idea, so we can categorize it into some typical sequences, but I'm not sure what use we could learn some valuable info from that. Can you explain that please?

Normally you would watch your/other player replay and compare results based on what was build and when (how many woodcutters, stone masons, one or two gold mines, etc.). Not very useful to you (developers) but it could be useful for (some) players. Just select some games you're interested in (from top to bottom), display build order (left to right), add interesting graph to it (resource collection/spending rate), compare end result (number of soldiers/weapons whatever). Probably a lot of work to implement (the UI side at least).

- game ID, so we can uniquely see Hands that played in same match

Server id (ip/port or hash), game id (timestamp or hidden - non repeating - lobby number), globally unique game id would be hard to pull off.

- alliance info of Hands, so we compare alliance to each other in games outcomes

Like I said, with public games you'll get very random data also keep in mind there is no option to surrender in KaM, how will you know who won if players simply left (which happens most of the time) ?

- collect market stats to see which trades are the most used and either tweak the prices a little bit. That would require 27x2 columns (counts of each ware traded From and To)

First column - (From << 4 | To), seconds - amount, you can easily query what was "popular" and keep it small.

- collect towers placed/lost count. That way we could see how many towers kept player safe or with how few destroyed he was defeated, see if player went defensive or not when compared to army counts.

You must account for empty towers, e.g. player made 10 towers but failed to equip them in time, other time he made 4 but was ready on time, does that mean that 4 is better ?

- army counts by type, trained and lost. Obviously to see balance changes and see if some army type is too rarely used and can be buffed

Few knights can obliterate tens of bowman, you must take formations, positioning, towers into consideration, raw number won't tell you this, long story short it's not that simple.

[Ben]

At first, I was really excited, then I saw this:

Note that we would not be collecting or storing any personal information like nicknames, just anonymous game statistics.

and I'm all like, "Who cares?"

Also, coming from someone who plays and watches replays (my brother's) of many games with "randoms," most of your information gathered will be entirely useless for improving the Remake. Randoms don't even know what they are doing, especially when it comes to what type of units they are training. So why use this information for determining what type of units are underused? If anything, swords+bows are underused, and they sure as heck don't need a buff!

[sado1]

At first, I was really excited, then I saw this:

Note that we would not be collecting or storing any personal information like nicknames, just anonymous game statistics.

and I'm all like, "Who cares?"

People that want player accounts+stats coming in 3, 2, 1...

[pawel95]

Didn't read all, but what about a leaving quote? XD so you can see in which speed/mode/map most players are leaving in pt for example.

[Bence791]

@DevTeam:

Also, coming from someone who plays and watches replays (my brother's) of many games with "randoms," most of your information gathered will be entirely useless for improving the Remake. Randoms don't even know what they are doing, especially when it comes to what type of units they are training. So why use this information for determining what type of units are underused? If anything, swords+bows are underused, and they sure as heck don't need a buff!

This is the point.

Though I like the idea, but it really won't provide you useful information. There can always be leavers (destroying the "average troops, citizens etc" stats) and generally people who don't know how to play. For example there is location 5 on Back in the Desert. In our games, it has the biggest disadvantage over the others (very very complicated to create a late game base) and therefore is rather a support (with kind of many swordfighters) to fast crushes at loc 2 or 4. Then imagine a "random game", where no-one (except loc5) has army in 60 minutes. Location 5 can win even alone, right? 40 swordfighters + many militia wins over nothing, I guess.

[Krom]

If we collect the said data we can detect Pro players (winners with big armies and few units lost) and we can filter stats only for them
Advanced stats are not meant to replace valuable players feedback in no way, but they allow us to have more solid data that will help us to identify problems scale.

Remember Market discussions, even after prices were tweaked we got many "this trade is unfair" responses, where in fact now trading is quite fair and that fairness is based not private opinions but on hard data (see my devblog article about that). If we did not had data to stand against we would be tweaking prices endlessly by trial and error.

[T*AnTi-V!RuZz]

If we collect the said data we can detect Pro players

That does not determine if a player is 'pro' or not.. What if he plays versus another 'pro' player and one of them wins by just outnumbering him slightly in the end?