Methods of Testing for Point Cost

[GreyOwl]

I've been thinking about different methods to test for point cost. I'm not sure which is best, or if any of them are even accurate.

• Points Killed - this is keeping track of how many points the figure kills, including partials by counting individual wounds, before it is destroyed. The theory is that this should match its own point cost. Is this valid? Should a 240 point figure be able to destroy 240 points worth before being destroyed (on average)?
• Points Remaining (One-on-One)Killed - face the figure off one-on-one against other figures to see how often it wins vs. loses, and try to match its point cost to the average of what it "ties" against.
• Points Remaining (Multiple figures) - play with a whole army of official figures, and the figure being tested included on one side. The idea is that the game should end as close to possible (ideally, one side loses and one side has 1 life left on 1 figure).

The concern is, are any of these actually valid? Has anyone tried these methods with officials figures to confirm whether they arrive at the cost on the official figure? I've brought this example up before, but if I take Isamu, I don't think I would arrive at 10 points for any of the above methods. I know that figure is somewhat of an extreme case, but we should still have a method to create and cost "extreme cases".

I'm worried that if our testing method isn't accurate, and we use TNT'd figures to test new ones, the errors will keep propagating.

[IAmBatman]

I think a composite of all three of those things you mentioned above, combined with constant comparison to official cards and what they offer is really the best we can do. At the core, this is an educated guess. Which, I believe, is what the official designers are forced to do as well. You can't tell me all 50 point or 100 point or 150 point figures in the official game are created equally. All ours aren't going to be either. The hope is that we can mitigate under or overcosted issues as much as possible. But I don't think there's one, true, inarguable method to do that.

[GreyOwl]

Originally Posted by IAmBatman

I think it's fine. No one significant is really complaining. I knocked a couple little used threads down. As long as we keep the pretrials and trials moving, it shouldn't be an issue. Besides, those get a lot more attention than anything that would pop up to the second half of the first page if I knocked them down. Besides, it's really not that hard to click on the next page icon.
Suffice to say, I'd be more concerned if people I'm convinced actually use this forum get upset in droves with the number of stickies. But I've heard no such complaints.

Did you....maybe....post this in the wrong thread?

[Nooblar]

I don't do much with supers, but I think this is a great thread that can be applied to all custom units. I'm not saying I'm all that smart, but I think I see an error in using any of these methods.

Support units are very important to a solid victory. I never see Raelin on the front lines (at lest not for very long, if you know what I mean). I suppose you could extend this argument to Jexik's entire article on the use of order markers, too. Anyway, my point is just that not every figure has been designed like a bazooka, and with some of the wild designs the fans create I don't think points/figures killed is an accurate measure for many customs.

[IAmBatman]

Originally Posted by GreyOwl

Did you....maybe....post this in the wrong thread?

I'm sure I have no idea what you're talking about.

And, Nooblar, do you have a proposed alternate method beyond randomly assigning point value with a D20? I don't necessarily disagree with what you're saying - there are certainly units designed more for support than combat, but beyond comparing units to official ones, and playtesting in individual and group combat, how can we really determine a unit's proper cost?

[GreyOwl]

Whatever method we decide to use, I think it should be tested out on official units to see how it fares. I understand that not all 150 point units are equal, but if one of our methods rates a 150 point official figure as 20 points or 340 points, then something would clearly be wrong with that method.

[hi1hi1hi1hi1]

How exactly I play-test depends on the mood I'm in , but what I strive to do with my customs is what I feel gives a very accurate point description. It consists of several battles, and the method, described below, is for testing heroes not squads:

• Custom vs. Hero: Begin with 1v1 battles versus as many similarly priced figures as are available. This gives us a good starting point to the figures value. Granted some are better for support, but we'll cover that later on. You should try to get in at least 10 quick tests.
• Custom vs. Squad: Once you have a good idea of his 1v1 cost, play-test the figure versus squads of similar combined total. For this part it should be expected that the figure will preform worse than 1v1 due to the nature of more attacks focused on 1 target as opposed to 1 attack that needs to hit a lot of targets. Unless the figure is designed to be a squad killer, a 20-30 point drop is appropriate. These can take longer and begin to get tedious, so 2/3 tests of this nature is ample.
• Custom (w/team) vs. Team: Now this becomes very important, because by now you should have a pretty good feel for what point level your figure should be at, somewhere between the first two bullets. Now there are three types of team vs. team battles that I make sure to get in:
. . . . . . . . . . . . . -Ideal: This means give the hero its perfect situation, teammates that benifit from him and enemies that he is perfect to beat. This is to see
. . . . . . . . . . . . . the upper point limit of how much the hero should cost. One or two is plenty.
. . . . . . . . . . . . . -Worst Nightmare: Just the opposite of Ideal, pull out the stops so that the entire enemy team was drafted specifically to kill your custom.
. . . . . . . . . . . . . This is to find the lower bound. Again only 1 or 2 works.
. . . . . . . . . . . . . -Regular: Then you have those in the middle that counters are drafted, teammates benefit each other, but generally how you would play a
. . . . . . . . . . . . . friendly game. For regular battles it comes down to as many as you can get in, I try for a minimum of 5 if I'm doing serious testing.
• Costing: The part we all have been waiting for... you should have an upper bound and lower bound by now in addition to an average for hero v. hero, hero. v. squad, and team v. team. Hopefully you upper and lower bounds will average out to be somewhere close to these other averages. Generally speaking I take the team v. team as most accurate and go with a little higher or lower (5-15 points) depending on the other results.

Again this is all I strive for, but hardly ever acheive. However, I don't see how you can get the price wrong if you go through all this.

[GreyOwl]

hi1hi1hi1hi1, your method sounds very reasonable. Have you ever tried it with an official figure to see how it turns out?

Although tedious, I think doing all these steps would be very useful for TNTs. With several people, it could probably be split up and not take too long. Instead of everyone doing their own random playtest, we could have something more organized to cover all the major situations. Sort of a "test plan" for playtesters to follow.

[hi1hi1hi1hi1]

Originally Posted by GreyOwl

hi1hi1hi1hi1, your method sounds very reasonable. Have you ever tried it with an official figure to see how it turns out?

Although tedious, I think doing all these steps would be very useful for TNTs. With several people, it could probably be split up and not take too long. Instead of everyone doing their own random playtest, we could have something more organized to cover all the major situations. Sort of a "test plan" for playtesters to follow.

Never with an official unit, but I tested it for a lot of my old HQ customs and it works well if done right and changes aren't made in the middle of things.

As for implanting a test plan, it could work well with others helping out where holes needed to be filled. I started the process with Boomerang, but with the changed range, I may have to begin again.

[Nooblar]

Originally Posted by IAmBatman

And, Nooblar, do you have a proposed alternate method beyond randomly assigning point value with a D20? I don't necessarily disagree with what you're saying - there are certainly units designed more for support than combat, but beyond comparing units to official ones, and playtesting in individual and group combat, how can we really determine a unit's proper cost?

Not really. That's the problem with units which provide indirect benefit. Raelin helping the Krav Maga or Warriors of Ashra adds a lot more value than Raelin helping Krug. I'm not saying these methods are bad, but they should only be used for units meant for direct attack on the enemy. The synergies (both intentional and un-) make it hard to cost support units without extensive playtesting.

In case I wasn't clear, I was talking specifically about the three suggestions in the OP, not just playtesting in general. If I were to design a costing method, I'd rely more on comparison than number-crunching. Send two balanced armies against each other, pick an official (or well-tested) figure which is similar to your custom, swap out the two and play some more while taking notes. Still is playtesting, but I think it's a *leetle* more scientific than playing only the custom a whole lot.

[IAmBatman]

Some good ideas, Hi1 and Nooblar. If you all want to start a committee and come up with a proposal for an "official TNT testing process" I would be interested in seeing it. And if y'all want to try it out to see how it fares with official figures in place of customs on trial, I say go for it. I'm an enabler with all of this TNT stuff, but I'm hardly the gatekeeper.

[hi1hi1hi1hi1]

I'll be up for starting a proposal here. I don't want it to be that this becomes a bother to completing TNT's however, it should be a guideline and generally used practice. I'm sure there are things to add to my post above, but should it be the basic outline to build off of? GreyOwl you are good at these things.