HoME General Discussion Thread

[White Knight]

We had over 50 playtests on Gandalf and couldn't come to a consensus on points and put up a final vote. On HoSS, I think we are happy with 4 or 5 playtests per figure.

In addition, Merry and Pippin were close, but we could not agree on final abilities.

When the group cannot compromise, there is no hope for it to go anywhere.

[caps]

50 is a lot of playtests. 4 or 5 may not be enough.

Not being on the same page, and the consequent inability to compromisem, was the heart of our problem.

[caps]

Looking at gandalf again it seems we were hemming and hawing. If one of us had forced the issue with a vote proposal I think we would have finalized him.

[White Knight]

Originally Posted by capsocrates

Looking at gandalf again it seems we were hemming and hawing. If one of us had forced the issue with a vote proposal I think we would have finalized him.

In the beginning of the project we voted on an ERB of three people to lead the group. SoA and Xorlof stopped contributing. That left you alone to call the shots.


As Gandalf's lead designer, I tried to listen to the playtesters and take their comments into account. They felt Gandalf needed to be stronger. I compiled a detailed summary on all 57 playtests which agreed that Gandalf was not really competitive. I made suggestions to make him stronger. As the only contributing ERB member, you resisted those changes and we got nowhere.


When the lead designer and the ERB and the playtesters disagree, it's hard to get anywhere.

[caps]

I think we have different approaches to playtesting. I believe playtests and playtesting tell you if the unit is playing the way you want and worth the amount of points you are spending. The design team (in conjunction with their lead playtesters and sometimes the ERB) takes the playtesting feedback and incorporates it into the design to make the design how they envision it. The playtesters are not in the driver-seat; the designers are. So if the playtesters say "it should be stronger" or "it would be cool if it had power X" the designers consider that in the context of what they're envisioning. What I get out of feedback like "it should be stronger" is "it does not feel like it is performing according the point value we assigned; we can make it stronger or we can change the points." It seemed to me that making Gandalf even stronger would be a departure from our goals for the design, so I pushed for us to lower the points and leave it as-is. You were on board with that, as were several others.

EDIT: In C3V, we have several times sent a design all the way back to Design when it was underperforming to make it stronger to fit the point value and feel we were shooting for. There have been other times we have just adjusted the points and-or tweaked some stats or a d20 threshold. It's all about the goals of the designer(s).


EDIT #2: to make this more blunt, I don't care if the playtesters think the design should be stronger. I care if the *Designers* think the design should be stronger.

[caps]

If I were to start over on process, I would do something similar to what C3V and AotV do now. Each design is owned by a *team* of people who together set the vision. The ERB (or equivalent) keeps them within certain boundaries, but otherwise respects the collective will of the team. It is a lot easier for the designers to manage these conflicts when they have a team of people supporting each other than a single designer against however-many ERB members.


EDIT: To make this more concrete, let's suppose your picture of the situation is right. I would have had a much harder time pushing back on your desire to power-up Gandalf if the other two members of the design team were in 100% agreement with you on doing it.

[OEAO]

IMO, you’d do well to establish at least a general sense of competitiveness you want units to be at. Is it B? B+? IIRC, I left primarily because I felt like the figures were much too weak. Admittedly, I think I was hoping to aim at around A- (not a good idea, looking back), but I think I got very frustrated with that Gandalf being at about a C or C+. Even looking at him now, I think he’s really overcosted. I think having a general target goal for power level, rather than leaving that up to one or two people in specific roles, would make a lot of sense.

[caps]

I think we were looking to follow the precedent set by VC, which is to shoot for the B range. Some figures we intentionally shoot for B+, some we intentionally shoot for B-.

There's a lot to reflect on when it comes to Gandalf specifically. We made him into a figure that you build your whole army around--I wonder if that was a mistake. It seems like it made him a lot harder to settle on.

[White Knight]

Yes, we definitely have different approaches to playtesting.

For me, I want to design a figure that is fun for people to play. If my playtesters consistently tell it's NOT fun to play because it's too weak, well then, it needs to get stronger. Because honestly, if people don't enjoy to play a figure than I don't care how balanced he is, or whether he's in the B range you are shooting for.

And this wasn't some relatively unknown soldier in the 4th row. This was GANDALF. And an A-ranked Gandalf needing certain very specific army builds utilizing all A-ranked support figures to meet that rank is so much better to me than a C-ranked Gandalf.

I've never had this kind of issue with HoSS or other groups I've designed with. HoME had a mostly absent ERB that turned into a one person ERB who had a different agenda than many of the members. Why stall and eventually kill a whole project over a fear of producing A-ranked figures?

[Porkins]

57 playtests!!!

[caps]

Originally Posted by White Knight

Yes, we definitely have different approaches to playtesting.

For me, I want to design a figure that is fun for people to play. If my playtesters consistently tell it's NOT fun to play because it's too weak, well then, it needs to get stronger. Because honestly, if people don't enjoy to play a figure than I don't care how balanced he is, or whether he's in the B range you are shooting for.

Reading playtest reports takes discernment. "This figure is not fun" is important feedback. "it should be stronger" is advice from the playtesters on one way to fix the "not fun" issue. These are two separate things.


In my observation and experience I think it's really valuable to have three distinct parties:
lead team - they are in the driver seat for the design
playtesting department (aka lead playtesters) - they do initial playtests and help summarize/parse playtest feedback. Theoretically they are more impartial at doing this than the lead team.
playtesters - people who play with the figure and submit reports on how it went


There frequently is overlap (members of the PT department or the ERB are frequently also members of a lead team) but there should never be a group that is composed completely of members of another group in the list.

Originally Posted by White Knight

And this wasn't some relatively unknown soldier in the 4th row. This was GANDALF. And an A-ranked Gandalf needing certain very specific army builds utilizing all A-ranked support figures to meet that rank is so much better to me than a C-ranked Gandalf.

An A-ranked Gandalf is concerning because of how much it limits future design space. And because of how much we have to crank up the power level of Gandalf to get there. A B-ranked or even B+ Gandalf that is weaker (but cheaper) seems cleaner to me.

Originally Posted by White Knight

I've never had this kind of issue with HoSS or other groups I've designed with. HoME had a mostly absent ERB that turned into a one person ERB who had a different agenda than many of the members. Why stall and eventually kill a whole project over a fear of producing A-ranked figures?

I would agree that I was one of the problems that led to us stalling out. There was some definite dysfunction in how we were handling design + playtest feedback. Some of that was process (a group of mostly two active people that don't know how to compromise with each other is not really a process) and some of that was me. More on that below.



I think the point I'm making, now, about how playtesting feedback should be handled is important. Below are some example of a non-dysfunctional playtest department that functions how I described.



--


playtest department (PtD), playtester (Pt), and lead team (LT) interactions in C3V, with details scrubbed:


Unit1

Spoiler Alert!

Pt: (misc. feedback) and I would recommend a <powername> cap of -2
LT: My sense is that Unit1 was a welcome addition to the game and you guys had fun. The power text seems pretty clear to me. (comment about how Unit1 was particularly effective due to the opponent's army composition)
PtD: It's disappointing that (describes constraint). That's just one test so we'll see. I wonder if (conjecture). It's early but it came to mind so I threw it out there.
LT: That's a clever tweak to increase the effectiveness.
PtD: Good Change to (described above) I do wonder if he should have (more) defense, but of course the <powername> could be very potent at the endgame... It was definitely a neat trick, but in this game the single attack of (a number) and (effect of a power) wasn't really close to making up for the <point value>.
I'd probably be willing to put him out to public playtesting at <lower point value> with (change described above). Or <point value> if we bump up the defense a point. Willing to hear some other thoughts here.
LT: From a theme perspective, I like the idea of bumping the defense and increasing his value. It's an imposing mini

Unit2

Spoiler Alert!

PtD: What's the goal here? Noone really refuted that 1x or 2x <unitname> in a massed <armyname> build is best (or one of the best) <unitname> builds, which I had initially thought was the goal.
The changes will only make them better than the original design (not as good as the (other) version) in that build, and they will have more utility in an endgame now.
Do we care about them working better in a <different armyname> build? I know the <powername> change was more of a thematic request, but how much do we care about them doing poorly there?
LT: I want the attack power to be useful. In the current version, it's not.


This was followed by some back and forth. The PtD proposed and even tried a different version of <powername> but the LT of three people unanimously decided it was a theme break for them and sent it back to design to start over, since the powers they had weren't delivering on the theme and gameplay that they wanted.

Unit3

Spoiler Alert!

PtD1: 2 games from Public.
He is definitely not putting his points worth in. I think in 7 games he is averaging <number> points, although these 2 game were probably his best.
Maybe it's just a matter of dropping him to <pointvalue>. Otherwise would an acceptable adjustment be to be able to <proposed power change>?
Or maybe <other proposed power change>?
LT1: Now that's an interesting thought. @LT2, @LT3, thoughts?
Should we turn dials (points/stats) or do you think a slight power change could make him a little better? i.e. either <power change one> or <power change two>.
Personally I really like that suggestion as it seems more thematic and exciting.
LT2: I’d like to try <power change two>.
PtD2: My first suggestion would be to drop the roll for <powername> significantly. <proposed number> or something. He has too many turns where he fails to (do the thing).
LT2: Gotta' say I like this too.


(discussion continues mildly)


PtD2: Okay, in need of some Lead Team decisions here. Three options to move forward:

1. Lower the <powername> roll to <number>
2. <proposed change one from above>
3. <other proposed change>

--



Lead team collectively agrees on 1 as a starting point and testing resumes.


Note that the playtesting department initially made several proposals for changes, but the lead team made the call on what was the best way to deliver the gameplay they wanted at the level of complexity they wanted.

There are more great examples but I won't take the time to edit them enough to get them in here.


The important thing is that there are teams of people (no role is held by an individual), there are not too many people, there is brainstorming and back-and-forth, ideas and suggestions are heard out, and finally the lead team is what drives decisions about the design, not playtesters.


There is a time and a place (it has happened before at literally every stage of a design in C3V) for the ERB to say "no, not that way" but it should be rare. And it has to be a majority of the ERB. A single ERB member having this power is effectively just a tyrant, and that's no good. These projects must trundle on, usually over the objections of a minority on this or that matter, or they grind to a halt and die.

[caps]

Originally Posted by Porkins

57 playtests!!!

One of the problems was someone (probably me) wanting to always see more playtests to make sure it was balanced before we had settled on whether it actually felt the way we wanted it to. The first goal of playtesting should be to see if the design feels right on the table. Once that is done you can worry more about making sure it's not broken in various edge cases.


The fact that Gandalf had synergy with more than half the other army cards in the game didn't help though, haha.