Boring dice testing

A collection of tips and tutorials for all your miniature gaming hobby needs from painting to sculpting and more!

Re: Boring dice testing

Postby Anderas » Monday August 28th, 2017 12:25pm

whitebeard wrote:What is the spec you are looking for with these dice? You are currently testing against a perfect random process, but how good is "good enough"?
I ask this because the more samples you collect, the greater the precision to which you are able to show that ANY die is not perfect. If you roll a million times, you could show that a die which is great for all practical purposes is outside of 5 sigma of a perfect die. In short, perfect should not be your objective.

I don't require perfection. If i invent monsters, if i make equipment, if i make a quest, i work under the assumption that in a game of Heroquest, you will not see differences smaller than 10%. That's because you do not roll often enough to "feel" the difference. That's different for games like 40k, but we're at the Heroquest forum here.
For example, if i make new equipment; making the next weapon a little bit more than 10% stronger than the one before, is for me kind of a prerequisite before i even think about producing and printing a new card.

To avoid hitting the 10% border accidentally with a die just because I don't want to test eternally, I would wish him to undercut that "feeling" border by half - so up to 5% away from the perfect die would be acceptable for me.
I know now, that those normal mass-produced dice are likely to cross this border more often than not. |_P
You could avoid the entire problem if everybody round the table uses one common set of dice, so that every time you roll, you choose randomly among the dice set and it's failures. Only, that mitigation doesn't work at my home and in my experience because people take a "lucky" die and keep it for the rest of the evening. "That's my die now!" :) At least my daughter does. And who am i to intervene there? That would greatly overstretch the competences that myself being the daddy brings with it. :D :lol:

How bad are these dice really?

Bad. I see 3x more failures than expected. They are bad. Not a little bit, but really really bad. But i see light: The failure seems to be connected to the color. So sorting out and throwing away the failing sets should be easy.

Also, you have omitted the constraint in one of your assertions. It's not terribly important, but it does change the expected value for the number of results outside of a sigma value. Once you have one side which registers an extreme value (because of randomness or because of defect), the liklihood that the other sides will be wrong is not random. So there is not a 7.9% chance that a perfect die shows outside of 2.2 sigma, it is even lower.

I took only the most extreme % and calculated: How probable is it that at least one side shows extremes like that? So that's 100% - ((100% - extreme side) pow 6).

Now, to reduce the dice rolling work, i would like to calculate "how probable is it that this die shows more (or less) than the mean 4 out of 5 times?"
Worst case, i have to go with 50% 4 out of 5 times, best case i can somehow incorporate the Z-value/sigma that i have there. :)
Last edited by Anderas on Monday August 28th, 2017 12:39pm, edited 1 time in total.


Rewards:
Hosted a Play-by-Post game. Played a turn in a Play-by-Post game. Created a Hot Topic. Participated in three (3) Miniature Exchanges. Zealot Miniatures: Twisting Catacombs Kickstarter Backer Encountered a menacing Chaos Warlock!
User avatar
Anderas
NOT Andreas!

Swordsman
Swordsman
 
Posts: 1911
Images: 37
Joined: Saturday September 20th, 2014 7:02am
Forum Language: British English
Evil Sorcerer: Morcar
Hero:
Usergroups:
Adventurers' Guild Group Member Artists Group Member
Champion Group Member

Advertisement

Re: Boring dice testing

Postby whitebeard » Monday August 28th, 2017 12:33pm

What you want is a 95% confidence interval that any side is off by 10%.

What you have is a 95% confidence interval that the die is not absolutely perfect.

They are very different things.


Rewards:
Participated in two (2) Miniature Exchanges. Zealot Miniatures: Twisting Catacombs Kickstarter Backer Grin's Stone Map Destroyed a Zombie! Smashed a massive Gargoyle!
whitebeard

Giant Wolf
Giant Wolf
 
Posts: 786
Images: 0
Joined: Friday September 12th, 2014 7:15pm
Forum Language: English (United States)
Evil Sorcerer: Zargon
Usergroups:
Artists Group Member Champion Group Member

Re: Boring dice testing

Postby Anderas » Monday August 28th, 2017 12:42pm

I didn't use confidence intervals at all as i have the theoretical basis available instead of a sample.

Well, i will check out what you say. Thanks for the feedback! :)


Rewards:
Hosted a Play-by-Post game. Played a turn in a Play-by-Post game. Created a Hot Topic. Participated in three (3) Miniature Exchanges. Zealot Miniatures: Twisting Catacombs Kickstarter Backer Encountered a menacing Chaos Warlock!
User avatar
Anderas
NOT Andreas!

Swordsman
Swordsman
 
Posts: 1911
Images: 37
Joined: Saturday September 20th, 2014 7:02am
Forum Language: British English
Evil Sorcerer: Morcar
Hero:
Usergroups:
Adventurers' Guild Group Member Artists Group Member
Champion Group Member

Re: Boring dice testing

Postby benvoliothefirst » Monday August 28th, 2017 1:15pm

THIS IS FASCINATING!!!

Thanks for all the hard work, Anderas!


Rewards:
Participated in a Miniature Exchange.
Encountered a menacing Chaos Warlock!
User avatar
benvoliothefirst

Gargoyle
Gargoyle
 
Posts: 126
Joined: Thursday October 6th, 2016 1:54pm
Location: Manchester, NH, USA
Forum Language: English (United States)
Evil Sorcerer: Morcar
Hero:
Usergroups:
Champion Group Member

Re: Boring dice testing

Postby cornixt » Tuesday August 29th, 2017 9:40am

So now that you have this data, what are you going to do with it?


Rewards:
Destroyed a Zombie!
cornixt

Orc Shaman
Orc Shaman
 
Posts: 396
Joined: Tuesday November 4th, 2014 12:56pm
Forum Language: British English
Evil Sorcerer: Morcar
Hero:
Usergroups:
Champion Group Member

Re: Boring dice testing

Postby Anderas » Tuesday August 29th, 2017 12:07pm

The plan has two parts.
The first part is called the "realistic" part, the other one the "unrealistic" part.

The realistic part:
First, play around with it until i have sufficient material for my study's lesson write-up (done)
Second, play around with it until i am convinced that i found a method that's able to test dice without so much work.
Third, throw away the white, blue, clear and red dice; keep only the turqouise. I could do that right now, but i want to keep the opportunity for more data on bad dice. :lol:

The unrealistic part:
Wrap my Zombicide dice tower in some transparent material.
Drill a hole and fix it to a motor.
Use my Rasperry or my Laptop or any part from my box full of microcontrollers to turn it 360 degree and stop each time when it is upright.
Mount a PC Webcam or a mobile phone so that it has a good view on the dice tower.
Use my newly achieved python coding abilities to read the pips on the dice after each turn of 360 degree and write it to a csv file.
Drop the plan because it costs too much time. :mrgreen:

:lol:


About the second part of my first plan:
I guess i have to update my "operational definition" of a good die, and the calculation of my test results.

I want only dice that are showing more than 15.8% and less than 17.5% of each single number. That's 1/6 plus or minus 5% of 1/6.
I want to identify the dice that are off that threshold with 90% accuracy. That means, if i throw away a die, i have a 10% chance of throwing away a good die.
Now my head is steaming to find out exactly what can i do with my numbers to calculate this from my data.

Before, i was calculating the probability of a result being possible if the die was theoretically perfect, so my method currently is not the right one.


Rewards:
Hosted a Play-by-Post game. Played a turn in a Play-by-Post game. Created a Hot Topic. Participated in three (3) Miniature Exchanges. Zealot Miniatures: Twisting Catacombs Kickstarter Backer Encountered a menacing Chaos Warlock!
User avatar
Anderas
NOT Andreas!

Swordsman
Swordsman
 
Posts: 1911
Images: 37
Joined: Saturday September 20th, 2014 7:02am
Forum Language: British English
Evil Sorcerer: Morcar
Hero:
Usergroups:
Adventurers' Guild Group Member Artists Group Member
Champion Group Member

Re: Boring dice testing

Postby whitebeard » Tuesday August 29th, 2017 7:29pm

Anderas wrote:I want only dice that are showing more than 15.8% and less than 17.5% of each single number. That's 1/6 plus or minus 5% of 1/6.
I want to identify the dice that are off that threshold with 90% accuracy. That means, if i throw away a die, i have a 10% chance of throwing away a good die.
Now my head is steaming to find out exactly what can i do with my numbers to calculate this from my data.

Before, i was calculating the probability of a result being possible if the die was theoretically perfect, so my method currently is not the right one.


Glad to see you coming around. :D

You had said that you would accept a die if it was off by 10%... so if the mean was supposed to be 10, you would accept a side where the mean was actually 9 or 11, but not any further away. This would be 10% error in either direction.

I am very curious to find out "how bad" these dice actually are. And once you can calculate one result, you should be able to move the thresholds and make claims like.

With an 80% CI, 20 of 40 dice are outside of a 10% probability threshold.
With a 95% CI, 5 of 40 dice are outside of a 10% probability threshold.
With a 95% CI, 18 of 40 dice are outside of a 20% probability threshold.

Don't throw them away just yet! But if the results say you need to toss one red one, do know which one it is? Did you lable them?

Good luck!


Rewards:
Participated in two (2) Miniature Exchanges. Zealot Miniatures: Twisting Catacombs Kickstarter Backer Grin's Stone Map Destroyed a Zombie! Smashed a massive Gargoyle!
whitebeard

Giant Wolf
Giant Wolf
 
Posts: 786
Images: 0
Joined: Friday September 12th, 2014 7:15pm
Forum Language: English (United States)
Evil Sorcerer: Zargon
Usergroups:
Artists Group Member Champion Group Member

Re: Boring dice testing

Postby Anderas » Wednesday August 30th, 2017 12:21am

Yes, i put them in a box with a label, just in case. I mean, what's the use of such a work if you don't keep the results? :D
Maybe i sell them labelled instead of throwing them away. "Buy a die that prefers the six, unobtrusively" ... Well, but that would be immoral, wouldn't it be?

Image


Do you think that i can use those subsamples with the central limit theorem to prove that the mean is off? Because, what i did see today is that the general mean is far more off than just one. So i don't really know if i can show a plusminus 5% or plusminus 10% border, to be honest.

I rolled 150 times to have a mean of exactly 25 at each number. That's nice and round as a number. Results were more, like,

1 2825,3%
2 342,3%
3 2550,0%
4 2332,9%
5 130,39%
6 2732,9%


The probabilities are the answer to the question "how likely is the result if this die was perfect?"
If you translate into confidence intervals, you can say it's 99.6% sure the die is not perfect.
If i just substract 5% from the positive extremes and add 5% to the negative extremes it doesn't really get better. Instead i am 99.1% confident that he's broken.
Is a die a "he", "she" or "it"? I think of them as persons. :D

So this die prefers the 2 and rolls less 5.


Rewards:
Hosted a Play-by-Post game. Played a turn in a Play-by-Post game. Created a Hot Topic. Participated in three (3) Miniature Exchanges. Zealot Miniatures: Twisting Catacombs Kickstarter Backer Encountered a menacing Chaos Warlock!
User avatar
Anderas
NOT Andreas!

Swordsman
Swordsman
 
Posts: 1911
Images: 37
Joined: Saturday September 20th, 2014 7:02am
Forum Language: British English
Evil Sorcerer: Morcar
Hero:
Usergroups:
Adventurers' Guild Group Member Artists Group Member
Champion Group Member

Re: Boring dice testing

Postby cornixt » Wednesday August 30th, 2017 9:27am

Have you done a float test to see which sides are dominant?


Rewards:
Destroyed a Zombie!
cornixt

Orc Shaman
Orc Shaman
 
Posts: 396
Joined: Tuesday November 4th, 2014 12:56pm
Forum Language: British English
Evil Sorcerer: Morcar
Hero:
Usergroups:
Champion Group Member

Re: Boring dice testing

Postby whitebeard » Wednesday August 30th, 2017 9:48am

What you need to do is compute the standard deviation for a side which is off by +10% and a side which is off by -10%. If you rolled three times you need the probabilities of 0, 1, 2, and 3, occurences. Then the standard deviation is readily computed. You rolled 150 times, so you will need to appeal to some clever solution... probably in a textbook and/or coded in excel already. I don't work with probabilities much, so my vocabulary is off. You already have the standard deviation for an unbiased die, however you computed that, just recompute for these boundarys.

If the result lies outside X standard deviations north of the higher probability mean, or X standard deviations south of the low... then you are certain it is out of spec (and not just barely in spec) to whatever confidence the end of the standard deviation curve implies.


Rewards:
Participated in two (2) Miniature Exchanges. Zealot Miniatures: Twisting Catacombs Kickstarter Backer Grin's Stone Map Destroyed a Zombie! Smashed a massive Gargoyle!
whitebeard

Giant Wolf
Giant Wolf
 
Posts: 786
Images: 0
Joined: Friday September 12th, 2014 7:15pm
Forum Language: English (United States)
Evil Sorcerer: Zargon
Usergroups:
Artists Group Member Champion Group Member

PreviousNext

Return to Hobby Tips and Tricks

Who is online

Users browsing this forum: CommonCrawl [Bot] and 0 guests