https://pastebin.com/H8EJvmQ6
Here's a bare bones windower version. Edit in Player TH on line 13.
No visible UI or screenshot stuff included, but it'll make the csv. Same rules apply for naming, copy the file to look at it while the program is running so excel doesn't lock it.
Edit: Interesting note while testing, if you fight the same mob ID twice in a row, it'll think it's still at TH14 (or whatever you left the last one at). 99% I imagine you'd change mobs rather than waiting for a single repop, but keep it in mind.
Treasure Hunter Proc Rate Testing -TH 8 Versus 14
|
Treasure hunter proc rate testing -TH 8 versus 14
Bismarck.Radec said: » Edit: Interesting note while testing, if you fight the same mob ID twice in a row, it'll think it's still at TH14 (or whatever you left the last one at). 99% I imagine you'd change mobs rather than waiting for a single repop, but keep it in mind. Mine has a check in render for that, if current mob goes idle or it's hp goes to 0 it gets reset to base TH level. Definitely room for some other improvements if trying to make it suitable for general gameplay, but shouldn't really matter for dedicated testing. FWIW, a ~6 hour session with TH15 ended at: 17/593 TH8->9 (2.87%) 17/835 TH9->10 (2.04%) 17/2265 TH10->11 (0.75%) 17/2741 TH11->12 (0.62%) 16/8959 TH12->13 (0.18%) 16/4370 TH13->14 (0.37%) Which really goes to underscore just how many hits you would need to be confident on percents. It doesn't seem likely that TH14 would actually have a higher rate than TH13. Will do a similar session with TH8 tomorrow, then try one each terminating early (TH11?) on friday/sat. Make sure to keep your CSVs and note target if doing any testing, and we can combine them to start getting a better idea on precise numbers.
[+]
Carbuncle.Nynja said: » https://www.bg-wiki.com/ffxi/Treasure_Hunter Quote: Upgrading TH levels from normal attack rounds is inversely proportional to the difference between your current TH level in trait/gear and the monster's current TH level[2]. Example 1: If you plant TH6 on the monster by pulling with your TH gear on and then continue to use your TH gear while you kill the monster, there is a 0-level gap between your current TH level and the monster's current TH level. This makes it "easy" to proc (approximately a 6% chance per melee round). Example 2: If you plant TH6 on the monster by pulling with your TH gear on and then swap to TH3 to kill the monster, there is a 3-level gap between your current TH level and the monster's current TH level. This makes it comparatively more difficult to proc (approximately a 2% chance per melee round). links to data is there IIRC specifically I was (at the time) referencing TH8 in gear, and the point (if any) of going higher or not. But I see where I errored the other night. Shiva.Thorny said: » 17/593 TH8->9 (2.87%) 17/835 TH9->10 (2.04%) 17/2265 TH10->11 (0.75%) 17/2741 TH11->12 (0.62%) 16/8959 TH12->13 (0.18%) 16/4370 TH13->14 (0.37%) Sample size is definitely not adequate to say anything meaningful really, they're all pretty clear below 1% but beyond that it's anyone's guess. So far, TH8 is not looking very much different, but differentiating the direct rates in this circumstance would require days of data.
I think the most conclusive test is going to be reducing to TH5 or so, and seeing if the same ramp (5->6 at ~3% 6->7 at ~2% 7-8 at <1%) exists. If it does, then it's pretty clearly operating off of a delta and not respecting TH over 8 as meaning anything. Ofc, it's definitely a bad sign if the original concept was that the delta would count outright all the way up. Very clearly a sharp falloff even with TH well in excess of the current value. Shiva.Thorny said: » 17/593 TH8->9 (2.87%) 17/835 TH9->10 (2.04%) 17/2265 TH10->11 (0.75%) 17/2741 TH11->12 (0.62%) 16/8959 TH12->13 (0.18%) 16/4370 TH13->14 (0.37%) Do you have the individual proc rounds stats? like for the first 8->9 you got it in 100 attack rounds and so on. I wanted to help show how many samples we would need to actually have an answer and with a given confidence, and also show how wide the error is on the 2.87 in the number currently indicated. but would need the number of attack rounds to each proc for the 17 procs rather than just the total procs vs total attacks. https://www.dropbox.com/scl/fi/kos8j24uik0vydttzss78/thtest.zip?rlkey=8xal00fr99ijcejq8pg9q9bys&dl=0
this has raw data, i didn't measure specifically for that but i'm sure there's a way to do it since data is organized in chronological order (even if it came down to subtracting row counts)  Not looking great for the extra TH squad, though. If we're expecting a delta-based jump (TH13->14 being the same with TH13+ as TH8->9 is with TH8+), it's more than enough to disprove that. If you're considering a more marginal benefit, it's still up in the air.
[+]
dumped it with an addon..
TH8: Quote: Upgrade 8->9 12,18,51,64,48,5,11,16,3,78,10,6,30,16,42,65,23 Upgrade 9->10 36,32,65,18,90,2,87,155,43,22,41,8,40,8,18,99,72 Upgrade 10->11 168,165,232,58,151,69,97,123,84,389,27,88,8,251,5,158 Upgrade 11->12 38,489,108,81,389,22,762,501,396,69,92,68,34,112,270,7 Upgrade 12->13 415,10,125,301,691,163,23,571,28,516,313,42,373,597,398,1815 Upgrade 13->14 181,730,14,195,760,434,643,202,334,284,7,198,210,447 TH15: Quote: Upgrade 8->9 74,4,3,24,3,97,19,16,5,79,10,6,45,1,59,4,9 Upgrade 9->10 32,27,32,15,38,34,124,153,48,22,41,8,12,16,59,25,55 Upgrade 10->11 85,8,375,179,108,115,44,128,89,402,26,87,87,4,107,257,96 Upgrade 11->12 12,108,37,169,35,211,150,550,402,72,92,68,127,370,133,55,98 Upgrade 12->13 16,305,58,812,249,546,908,565,28,514,316,42,426,54,442,767 Upgrade 13->14 100,24,315,286,118,14,431,652,203,338,288,7,504,419,479,26
[+]
Quote: Upgrade 12->13 812…908…767  Shiva.Thorny said: » https://www.dropbox.com/scl/fi/kos8j24uik0vydttzss78/thtest.zip?rlkey=8xal00fr99ijcejq8pg9q9bys&dl=0 this has raw data, i didn't measure specifically for that but i'm sure there's a way to do it since data is organized in chronological order (even if it came down to subtracting row counts) Not looking great for the extra TH squad, though. If we're expecting a delta-based jump (TH13->14 being the same with TH13+ as TH8->9 is with TH8+), it's more than enough to disprove that. If you're considering a more marginal benefit, it's still up in the air. I mean, there could be a difference in how often a SA or TA procs a TH upgrade if you wear more TH+ but good luck with running tests to get data for that. Shiva.Thorny said: » TH8: Quote: Upgrade 8->9 12,18,51,64,48,5,11,16,3,78,10,6,30,16,42,65,23 Quote: Upgrade 8->9 74,4,3,24,3,97,19,16,5,79,10,6,45,1,59,4,9 just a bit of napkin math, with only a total of 24 samples(7 more then you currently had) we could say that the TH15 upgrade rate is different from TH8, based on the numbers you had and assuming the mean does not get closer to the rate for TH8 or the standard deviation doesn't widen. but the means are pretty far apart as it is, but the high standard deviation in the TH15 data set makes it a bit shaky making it hard to draw a conclusion with low samples like you have been saying. I bolded different because we might not be able to say what the exact rates are but that they are different proving the increase in TH gear increases proc rate. Its also possible the proc rate is lower in general the higher it goes.
K123 said: » This basically proves there's no point in carrying or equipping anything more than +5. As I expected because I knew SE wouldn't bother to code anything harder than the bare minimum. I mean, there could be a difference in how often a SA or TA procs a TH upgrade if you wear more TH+ but good luck with running tests to get data for that. Like Thorny has said dont draw conclusions from small samples sizes. If you look at the proc data as a histogram it tells a very different story.  here the x axis is the number of attack rounds to get the proc binned in decades Fenrir.Niflheim said: » the means are pretty far apart as it is You specifically quoted TH8-9, which would have the lowest range (as the highest proc rate), so I assume you ran numbers on that in particular. That said, the means are 34.29% and 34.13%.. unless I'm misunderstanding what you did it seems like you may be applying the confidence interval to the wrong means. Regardless, this data only serves to illustrate that it is more complicated than proc rate being a function of TH delta, which may have been the simplest way it could be coded to help but is certainly not the only way it could. Shiva.Thorny said: » Fenrir.Niflheim said: » the means are pretty far apart as it is You specifically quoted TH8-9, which would have the lowest range (as the highest proc rate), so I assume you ran numbers on that in particular. That said, the means are 34.29% and 34.13%.. unless I'm misunderstanding what you did it seems like you may be applying the confidence interval to the wrong means. maybe i misunderstood the data Shiva.Thorny said: » TH8: Quote: Upgrade 8->9 12,18,51,64,48,5,11,16,3,78,10,6,30,16,42,65,23 for these does like the first 12 mean in that it proc'ed on the 12th attack round? my math showed a 7.88% proc rate for TH8 and a 17.5% rate for TH15 Yea, that is correct. Each comma-seperated entry is the number of attack rounds(where the first hit did not miss) that it took to proc.
Shiva.Thorny said: » Yea, that is correct. Each comma-seperated entry is the number of attack rounds(where the first hit did not miss) that it took to proc. 17 / 498 = 3.41% proc rate for TH8
17 / 468 = 3.63% proc rate for TH15 (the file i pulled those numbers from was missing 2 samples the full excel sheet has) Ah, treating them as independant samples and modeling as hits to proc instead of chance to proc. But, [we assume] it's coded as an independant chance, and with this number of samples even a single outliar causes huge variance. Plan is to continue gathering data anyway, I still want a sample of each that cuts off early (TH10-11) to give a greater idea on those numbers since it's more attainable, and a sample around TH5 to see if the rating is only using a delta or is also influenced by the level of TH itself.
[+]
Shiva.Thorny said: » Ah, treating them as independant samples and modeling as hits to proc instead of chance to proc. But, [we assume] it's coded as an independant chance, and with this number of samples even a single outliar causes huge variance. If you wouldn't mind, you should collect data on a TH8 tag and then TH3 (no gear) to see if there is an actual difference in upgrade rate below 8.
[+]
Fenrir.Niflheim said: » K123 said: » This basically proves there's no point in carrying or equipping anything more than +5. As I expected because I knew SE wouldn't bother to code anything harder than the bare minimum. I mean, there could be a difference in how often a SA or TA procs a TH upgrade if you wear more TH+ but good luck with running tests to get data for that. Like Thorny has said dont draw conclusions from small samples sizes. If you look at the proc data as a histogram it tells a very different story. here the x axis is the number of attack rounds to get the proc binned in decades I just want to point out, it's a good idea to look at histograms for trends, but don't get too caught up on the patterns. SE doesn't have to be continuous with thier values. A good example, is Ultra Rare TH13 -> TH14, actually produces the highest relative increase across all values TH0-TH13, in the Ultra Rare category. FFXI TH8-14 proc rates and Relative gain, as compared to TH4:  Same idea, relative to TH0.  Basically, expect occasional outliers in your "trending" data. The outlier I'm trying to illustrate here is that Ultra Rare TH13->TH14 is the highest relative increase in TH proc in that category, (0.2% increase relative to 0.1-0.15% increase elsewhere in the category). So, please temper your expectations and resist the urge to apply "Diminishing returns" to everything. Weird point to bring up, since there are 6 other categories of items where a TH13 -> TH14 proc makes only a marginal increase to the drop rates and in some cases it's one of/the worst increases you can get.
Going from 24% -> 48% with TH0 -> TH1 is much more impactful than 78% -> 80% with TH13 -> TH14. Guess it depends which cherries you're interested in picking though. Felgarr said: » Same idea, relative to TH0. A histogram shows the distribution of a data set, it is a simpler visual than something like a control chart. first paragraph on Wikipedia for "histogram" Wikipedia said: A histogram is an approximate representation of the distribution of numerical data. The term was first introduced by Karl Pearson.[1] To construct a histogram, the first step is to "bin" (or "bucket") the range of values— divide the entire range of values into a series of intervals—and then count how many values fall into each interval. The bins are usually specified as consecutive, non-overlapping intervals of a variable. The bins (intervals) must be adjacent and are often (but not required to be) of equal size.[2] Carbuncle.Maletaru said: » Weird point to bring up, since there are 6 other categories of items where a TH13 -> TH14 proc makes only a marginal increase to the drop rates and in some cases it's one of/the worst increases you can get. Going from 24% -> 48% with TH0 -> TH1 is much more impactful than 78% -> 80% with TH13 -> TH14. Guess it depends which cherries you're interested in picking though. Yes, I specifically mentioned Ultra-Rare for this reason. Fenrir.Niflheim said: » Felgarr said: » Same idea, relative to TH0. A histogram shows the distribution of a data set, it is a simpler visual than something like a control chart. first paragraph on Wikipedia for "histogram" Wikipedia said: A histogram is an approximate representation of the distribution of numerical data. The term was first introduced by Karl Pearson.[1] To construct a histogram, the first step is to "bin" (or "bucket") the range of values— divide the entire range of values into a series of intervals—and then count how many values fall into each interval. The bins are usually specified as consecutive, non-overlapping intervals of a variable. The bins (intervals) must be adjacent and are often (but not required to be) of equal size.[2] I know, my guy. It could be a pie chart for all we care. That's why I included the raw relative-to-TH4 data in the chart above it. It's NOT a perfect, direct comparison to compare TH14-relative-to-TH4 and TH14-relative-to-TH0...but the point still stands. TLDR: TH13 to TH14 proc in the Ultra-Rare category is one example of an increase in TH proc rate that is NOT a diminishing return. That is all I was trying to say. If I wasn't clear enough or you don't understand my point, DM me and I'll be happy to explain further. Please carry on. For those of us less inclined to comprehend all of the paragraphs of data the brave souls out there put forward, is there a conclusion or rule of thumb that can be inferred from the data yet, or is the sample size still too small?
I *** hate that "diminishing return" has become a buzzword.
Also this has nothing to do whether TH14 is worth it. No one is discussing drop rates. Asura.Dexprozius said: » For those of us less inclined to comprehend all of the paragraphs of data the brave souls out there put forward, is there a conclusion or rule of thumb that can be inferred from the data yet, or is the sample size still too small? And even if you aren't sure if TH affects that drop, bring it anyway. 20 dps vs any amount of higher drop rate. You don't need 6 pages of testing to know the answer.
[+]
Asura.Eiryl said: » Asura.Dexprozius said: » For those of us less inclined to comprehend all of the paragraphs of data the brave souls out there put forward, is there a conclusion or rule of thumb that can be inferred from the data yet, or is the sample size still too small? And even if you aren't sure if TH affects that drop, bring it anyway. 20 dps vs any amount of higher drop rate. You don't need 6 pages of testing to know the answer. I mean that kind of goes without saying. I meant moreso, is there any evidence as to what the 'best' or most efficient way to achieve this is? There was discussion of "Sit in TH 9 gear until you hit TH 14 VS overcapping TH up to 14 and comparing rates" I thought? Was that question ever really solved? When you need a drop, do you experience upgrades in TH faster if you equip the TH value 1 above what the mob you're tagging is currently at (i.e. the mob is at TH9, equip TH10 in gear to push to TH10) or does equipping TH20 see any significant increase in the proc rate. OR... sit in TH9 gear all day and once it's reached TH9, the 'overcapping' via JP etc is a flat rate that decreases the higher you go and can't be influenced? Obviously these are very abstract and, albeit, unnecessary questions. But it could influence whether one full times Max TH when aiming for procs Vs just subbing it in for actions such as SaTa etc. Here is what I would say the experiment thorny did showed.
 I would TH8 could almost fit a theory of each TH level = +1% to your proc rate and after each proc that is cut in half. so ideally it would be 8%, 4%, 2%, 1%, 0.5%, 0.25%. Doing a null hypothesis for each value based on the data they are possible, with such high standard deviations and low sample sizes a lot of things are possible. But TH15 really doesn't support that theory much, TH9->10 is pretty far from the 7.5% that is would be under the theory and the standard deviation is only 3.02%. so we can be 99% certain that the real value is 3.96% +/- 1.887%, so between 2.073% and 5.847% Anyway can't wait for more data :) |
All FFXIV and FFXI content and images © 2002-2024 SQUARE ENIX CO., LTD.
FINAL FANTASY is a registered trademark of Square Enix Co., Ltd.