Ranger Firing Range - Testing

Eorzea Time
 
 
 
言語: JP EN FR DE
日本語版のFFXIVPRO利用したい場合は、上記の"JP"を設定して、又はjp.ffxivpro.comを直接に利用してもいいです
users online
フォーラム » FFXI » Jobs » Ranger » Ranger Firing Range - Testing
Ranger Firing Range - Testing
 Leviathan.Comeatmebro
Offline
サーバ: Leviathan
Game: FFXI
user: Rairin
Posts: 6052
By Leviathan.Comeatmebro 2017-02-13 15:58:41  
Outgoing packets are sent roughly every 400ms.

The network delay is (relatively) constant and varies in an unpredictable manner.

The server will decide when your shot ends(and subsequently, the time the next shot can be fired assuming the current shot is allowed to resolve) when it queues up your shot.

Thus, the time between shots when using a plugin of this nature is never more than 400ms greater than the time the server allows, as the server never goes 400ms without receiving a packet telling it to queue up a shot. It's also never greater than the minimum you can sustain using any method of input we have available to us.

While you are right that there may be delays in the server sending packets, they would again be constant over a large enough amount of shots and all evidence indicates that if they exist they are infinitesimal. By measuring the amount of time between shots, you're getting the only number that matters: How many shots per unit of time can I get in using this weapon? Anything else is an argument of semantics and unlikely to be adequately resolved without SE outright answering it.
[+]
 Leviathan.Comeatmebro
Offline
サーバ: Leviathan
Game: FFXI
user: Rairin
Posts: 6052
By Leviathan.Comeatmebro 2017-02-13 16:02:33  
As to lag not mattering, look at it this way. If you have a 150ms one-way trip, your packets get received at:

0(+150) - 150ms
400(+150) - 550ms
800(+150) - 950ms
1200(+150) - 1350ms
and so on. It doesn't matter what you change that (+150) to, the server is still going to receive a packet to start the next shot less than 400ms after it allows it.

As far as WSing, my plugin uses packets for that too. But that's an argument of implementation, as far as testing the actual delay this is the best you're going to get. It removes far more variables than your tests.
Offline
By Verda 2017-02-13 16:18:37  
Ok, that's all good thanks.

My standing question then is, for the sake of the dps spreadsheet, do we all concur that adding a static average value is the best model?

If so, can we agree on what to put that static average value as? I proposed .562 seconds but wondering what you guys think.
 Leviathan.Comeatmebro
Offline
サーバ: Leviathan
Game: FFXI
user: Rairin
Posts: 6052
By Leviathan.Comeatmebro 2017-02-13 16:21:33  
Spreadsheet is complicated, because it's going to depend on your plugin heavily. A static wait plugin that's potentially too high will lose 1-2 packet windows(400-800ms). One that's too low might fail to shoot and lose a full second. I would say modeling it against the best case scenario is probably most practical for me, but that might not be most practical for everyone.

Definitely make it easily viewable and changable, imo. Maybe some example values or a few options(manually shooting, delay based plugin, packet based plugin).
[+]
 Lakshmi.Byrth
VIP
Offline
サーバ: Lakshmi
Game: FFXI
user: Byrthnoth
Posts: 6137
By Lakshmi.Byrth 2017-02-13 19:16:41  
Anyway, point is that 1.8s is about one packet cycle higher than I'd have expected and the line's slope is low.
[+]
 Fenrir.Snaps
Offline
サーバ: Fenrir
Game: FFXI
user: Mojopojo
Posts: 1139
By Fenrir.Snaps 2017-02-13 20:33:10  
Lakshmi.Byrth said: »
Anyway, point is that 1.8s is about one packet cycle higher than I'd have expected and the line's slope is low.

How are you forming your slope expectation? We don't have a verified model of Rapid Shot or know the actual values for Rapid Shot I/II. The LLSR yielded a slope of 0.001257002. Assuming the model on JP wiki and assuming RS2 = 30

ss = Snapshot
vs = Velocity Shot
rs = Rapid Shot Rate
rsm = Rapid Shot Multiplier = (sum(n/16) for n = 1 to 8)/8 = .28125

m = (1/114) * (1 - ss/100)*(1 - vs/100)*(1 - rsm*rs/100) = (1/114)*(.3)*(.66)*(.8425) = 0.0014632894736842103.

So it is lower, but again we don't actually know how Rapid Shot works or what RSI/RS2 values are. If you assume the JP wiki model and 100% RS activation rate, you get a slope of 0.0012483552631578948 (less than 1% of what was observed.) Either way the linear fit with respect to weapon delay is very good so I don't think there is some static component of the weapon delay at play.

Also in regards to your comment about not wanting to have the ranged attack start/finish in the same packet - How would this affect things? The slope is very smooth with changes in delta that lower than .4s.

I have some more comments I'll add later (getting nagged atm) I think the best next step is to perform the same test using Ashita and see if the results are different.
 Fenrir.Snaps
Offline
サーバ: Fenrir
Game: FFXI
user: Mojopojo
Posts: 1139
By Fenrir.Snaps 2017-02-13 21:13:50  
I just understood what Byrth meant by a static component of ranged delay. I'll run my tests again and measure the time between ranged attack start and ranged attack finish packets. If the static delay component is larger than .4 seconds than we should expect to see a linear fit with an intercept equal to the static component. That said, for modeling purposes, there should be no difference between a .4 static delay component/1.4 cooldown or a 0 static delay component/1.8 cooldown.
 Fenrir.Tarowyn
Offline
サーバ: Fenrir
Game: FFXI
user: Tarowyn
Posts: 580
By Fenrir.Tarowyn 2017-02-13 21:23:55  
Do you think you could try running the tests on a job without RS? It seems like adding a random effect to over 50% of all shots just seems like a good way to get fudged results. And I don't think ~200 shots would be enough to even out the randomness of it.

You'd have to live without VS but I imagine you should be able to use most of the snapshot gear on say thief.
[+]
 Leviathan.Comeatmebro
Offline
サーバ: Leviathan
Game: FFXI
user: Rairin
Posts: 6052
By Leviathan.Comeatmebro 2017-02-13 21:42:46  
Going to do the test using my plugin through ashita. Got THF with 50% in taeon, can throw a few saps at a cape for 60 and keep flurry1 up for what should be cap.
[+]
 Leviathan.Comeatmebro
Offline
サーバ: Leviathan
Game: FFXI
user: Rairin
Posts: 6052
By Leviathan.Comeatmebro 2017-02-13 23:00:28  
Using pirate's gun and bronze bullet, 10% snapshot taeonx4, 10% snapshot toutatis's cape, and adhemar legs+1(path C, so no rapid shot). Flurry cast before test start and never allowed to drop.

Total shots: 259
Fastest shot: 3059ms
Slowest shot: 3435ms
Mean shot: 3265ms

Full log:
https://www.dropbox.com/s/cjiq1gylb7p8qnz/20170213_234445.log?dl=0
Raw delays for GREP or further data analysis:
https://www.dropbox.com/s/xud7c1fa9epa0hj/20170213_234445.raw?dl=0

Getting late here, but I already have all my logging set up so I will run additional tests tomorrow or wednesday, if any specific requests let me know.
[+]
 Leviathan.Comeatmebro
Offline
サーバ: Leviathan
Game: FFXI
user: Rairin
Posts: 6052
By Leviathan.Comeatmebro 2017-02-13 23:28:46  
so far list is:
shortbow
self bow
lamian kaman-1(same delay as longbow)
velocity bow+1(thf cant use atalanta, fastest easily attainable piece)
aureole(tiny bit faster and want to see if throwing is the same)
[+]
 Fenrir.Snaps
Offline
サーバ: Fenrir
Game: FFXI
user: Mojopojo
Posts: 1139
By Fenrir.Snaps 2017-02-14 00:37:43  
Alright, I have more data, but first some follow up.

I somehow managed to use the wrong delay for 3 of the bows in the LSR I performed on the last data set. The revised results are

m = 0.001248714
b = 1.807944125
R^2 = 0.984716636

The times below are the mean deltas between ranged attack start and ranged attack finish packets for the same bows.

Code
Weapon      Delay Delta
Atalanta    130   0.407858
Shortbow    360   0.691971
Self Bow    450   0.772906
Longbow     540   0.954130
Shigeto Bow 600   1.080449
I only collected data for just over 200 shots of each. The UPD packet period being so close in magnitude to the deltas end up making the data a little rougher. The LSR for this is

m = 0.001389268
b = 0.203527219
R^2 = 0.978668582

So still very linear and implies a static delay of around .2 seconds. This odd part is the slope is different than before. This might be to weird artifacting from Atalanta or Rapid Shot functioning in an unexpected way. Suppose I should get some cleaner data. I also assumed that a ranged attack start chunk would always come before a ranged attack finish chunk even if they were in the same UDP packet but that may be a false assumption.
[+]
Offline
By Verda 2017-02-14 07:23:41  
Yay data :D

Leviathan.Comeatmebro said: »
Total shots: 259
Fastest shot: 3059ms
Slowest shot: 3435ms
Mean shot: 3265ms

I formatted it as a comma list here if you like: http://pastebin.com/8E7vXQrF
Can also plug your data into here for more stuff:
http://www.calculator.net/standard-deviation-calculator.html

This gives me
Sample Standard Deviation 56.784026786794

That's a pretty smooth result all things considered.

It's really interesting to me that the Fastest and Slowest shot are almost exactly .4 seconds apart looking forward to more!

Fenrir.Snaps said: »
So still very linear and implies a static delay of around .2 seconds.
That would be the average you'd expect for packet scheduling so that's interesting to me. Also Atalanta delay is pretty close to .4 seconds. Thanks for the tests :)

I would actually expect the slope to be different for total ranged shot to ranged shot time and only aiming delay if for no other reason than reload is a static 1 second but there very well may be more going on too. I think the danger in setting delay=0 to get what's left as delay added to every shot is if the slope is off even a bit, then you can get exaggerated numbers and be off by quite a bit. Also, that wait 1 after getting the ranged_end packet to start the next shot doesn't always work bothers me a bit so extra delay there too.
 Leviathan.Comeatmebro
Offline
サーバ: Leviathan
Game: FFXI
user: Rairin
Posts: 6052
By Leviathan.Comeatmebro 2017-02-14 11:09:44  
Finished my test. Results in next post.

Job: THF99/WHM49
Gear:
Pitchfork/x/test/test
Taeon Chapeau/Loricate Torque+1/Etiolation Earring/Sanare Earring
Taeon Tabard/Taeon Gloves/Defending Ring/Shneddick Ring
Toutatis's Cape/Grunfeld Rope/Adhemar Kecks+1/Taeon Boots
10% Snapshot augment on cape, all 4 taeon. Kecks Path C rank 15.
Buffs: Flurry1, Indi-Haste, Geo-Refresh

Methodology:
Used packet based plugin to inject a ranged start packet on every outgoing UDP transmission. Used seperate plugin to log all ranged attack start packets(incoming id 0x28 where bytes 5-8 match character id and unpacking bits 82 to 85 has a value of 12). Used approximately 220 ammo(or 15 minute sample in aureole's case) before flushing data to file. Variations in amount of shots are due to lack of precision in stopping at 220 ammo used and recycle from kecks. First shot ignored and not represented in data file as no base delay is available to compare to it. While the more detailed log says 'shot fired', it is actually the time when the previously described packet was received, initiating the shot animation.

Raw Data: https://www.dropbox.com/s/x0t3ruatvbijiw5/RangeTest.zip?dl=0
The .log files have weapon specified at the start of them. The .raw file with matching name is the millisecond delays between each shot start seperated by \r\n.

I will leave it to people more familiar with the currently known data to draw conclusions, but one thing is pretty obvious: throwing weapon delays are calculated differently.
[+]
 Leviathan.Comeatmebro
Offline
サーバ: Leviathan
Game: FFXI
user: Rairin
Posts: 6052
By Leviathan.Comeatmebro 2017-02-14 11:09:51  
Aureole(192)
Total shots: 252
Fastest shot: 3603ms
Slowest shot: 3978ms
Mean shot: 3678ms

Velocity Bow +1(210) + Sleep Bolt(288)
Total shots: 263
Fastest shot: 1977ms
Slowest shot: 2550ms
Mean shot: 2299ms

Shortbow(360) + Stone Arrow(120)
Total shots: 234
Fastest shot: 2413ms
Slowest shot: 2979ms
Mean shot: 2794ms

Self Bow(450) + Stone Arrow(120)
Total shots: 243
Fastest shot: 2720ms
Slowest shot: 2993ms
Mean shot: 2859ms

Lamian Kaman -1(540) + Stone Arrow(120)
Total shots: 259
Fastest shot: 2890ms
Slowest shot: 3335ms
Mean shot: 3251ms

Pirate's Gun(600) + Bronze Bullet(240)
Total shots: 259
Fastest shot: 3059ms
Slowest shot: 3435ms
Mean shot: 3265ms
[+]
 Lakshmi.Byrth
VIP
Offline
サーバ: Lakshmi
Game: FFXI
user: Byrthnoth
Posts: 6137
By Lakshmi.Byrth 2017-02-14 11:12:38  
poor Aureole

Plotting the time between shots (y, ms) vs. delay (x) and running linear regression gives:
y = 2.53x+1800.6

So we have the same 1.8s offset. This time, without Velocity Shot or Rapid Shot, we see a slope that's approximately what we'd expect.

(1-0.70 Snapshot)*(1000 ms/s)/(120 Delay/s) = 2.5

This time we saw 2.53, which is easily within the wiggle room of that 120 Delay/s number.
 Leviathan.Comeatmebro
Offline
サーバ: Leviathan
Game: FFXI
user: Rairin
Posts: 6052
By Leviathan.Comeatmebro 2017-02-14 11:16:08  
No velocity shot, THF/WHM with 60% gear snapshot and flurry.
 Fenrir.Snaps
Offline
サーバ: Fenrir
Game: FFXI
user: Mojopojo
Posts: 1139
By Fenrir.Snaps 2017-02-14 11:19:31  
I can think of a few reasons why the slope would be different but in any case I need more data to remark on it. Also Comeatmebro's initial data is in the ballpark with the data I collected before. A 1.8 cooldown (or 1.6 cooldown + .2 static delay) would result in the following expected delay between shots with capped Snapshot.

600 * .3 + 1.8 = 3.378947368421053

This is close to 3.265 (what he observed.) We'll know for sure when we can regress his data for the other weapons. It is definitely a big kick to the balls to the perceived usefulness of lower delay weapons and ranger in general :(
 Fenrir.Snaps
Offline
サーバ: Fenrir
Game: FFXI
user: Mojopojo
Posts: 1139
By Fenrir.Snaps 2017-02-14 11:25:42  
LSR on his data (omitting aureole)

m = .002530038023
b = 1.800623574
R^2 = .9614750727

Note: Fixed cause I used the wrong delays (again!)
[+]
 Lakshmi.Byrth
VIP
Offline
サーバ: Lakshmi
Game: FFXI
user: Byrthnoth
Posts: 6137
By Lakshmi.Byrth 2017-02-14 11:32:21  
Lakshmi.Byrth said: »
poor Aureole

Plotting the time between shots (y, ms) vs. delay (x) and running linear regression gives:
y = 2.53x+1800.6

So we have the same 1.8s offset. This time, without Velocity Shot or Rapid Shot, we see a slope that's approximately what we'd expect.

(1-0.70 Snapshot)*(1000 ms/s)/(120 Delay/s) = 2.5

This time we saw 2.53, which is easily within the wiggle room of that 120 Delay/s number.

Sorry, I edited.

Anyway, it's interesting to me that the slope changes when fitting the lower and upper bounds. I would have expected the line to just be translated up a packet interval or two.
 Leviathan.Comeatmebro
Offline
サーバ: Leviathan
Game: FFXI
user: Rairin
Posts: 6052
By Leviathan.Comeatmebro 2017-02-14 11:33:51  
I'd assume lower bound is a small lag spike on the previous shot and upper bound is a small lag spike on the current shot. I'm not really positive, the variation isn't much higher than a packet interval.
 Lakshmi.Byrth
VIP
Offline
サーバ: Lakshmi
Game: FFXI
user: Byrthnoth
Posts: 6137
By Lakshmi.Byrth 2017-02-14 11:34:41  
Self Bow is particularly interesting. I'd be unsurprised if it was the same number of packet intervals between those shots every time, because the difference between the slowest and fastest shot is only 273ms.

Indeed, 171 of the 243 shots fired come in between 2855 and 2858 ms.
 Fenrir.Snaps
Offline
サーバ: Fenrir
Game: FFXI
user: Mojopojo
Posts: 1139
By Fenrir.Snaps 2017-02-14 12:10:55  
It might be something that goes away with higher sample sizes (seems unlikely) or perhaps some strange chronotropic effect caused by client/server UDP packet exchange.
 Leviathan.Comeatmebro
Offline
サーバ: Leviathan
Game: FFXI
user: Rairin
Posts: 6052
By Leviathan.Comeatmebro 2017-02-14 12:11:00  
Is it possible to cap rapid shot on RNG without any snapshot? Think that's the next test I'd like to do.
 Leviathan.Comeatmebro
Offline
サーバ: Leviathan
Game: FFXI
user: Rairin
Posts: 6052
By Leviathan.Comeatmebro 2017-02-14 12:16:10  
2 kustawi - 14
mrig gloves - 15
orion beret+1 - 14
arcadian jerkin+1 - 12
pursuer's pants - 9
arcadian socks+1 - 6
yemaya belt - 5

With merits that should be 80% using only blatantly specified items. Only char I have gloves on doesn't have RNG though(usually toss them because obviously carmine+1 are better, one slipped through the cracks). Guess I can burn it up and make some AF today. Relatively confident that will hit 100 with trait, since the assumed value is 30%. Probably easier on COR, but every COR i have that's remotely close to that gear already has 100+ JP.
 Fenrir.Snaps
Offline
サーバ: Fenrir
Game: FFXI
user: Mojopojo
Posts: 1139
By Fenrir.Snaps 2017-02-14 12:17:46  
It should be.

ItemSet 349523

You can augment pursuer's for 10 each

Edit: Also just noticed Atalanta has 5 Rapid Shot. m'data
 Leviathan.Comeatmebro
Offline
サーバ: Leviathan
Game: FFXI
user: Rairin
Posts: 6052
By Leviathan.Comeatmebro 2017-02-14 12:22:15  
Oh, that makes life easier. Same char already has everything there besides body and head(will just use pursuer's). Just need to throw an arcadian body onto the NPC while I burn rng to 99 and change my pursuer's paths.

Your set with pursuer's head should still be 85 + traits/merits and no real hassle to get.
 Fenrir.Snaps
Offline
サーバ: Fenrir
Game: FFXI
user: Mojopojo
Posts: 1139
By Fenrir.Snaps 2017-02-14 12:26:18  
It would probably be good to test the effects at least two other levels of Rapid Shot to examine whether or not there is a cap on it.
 Leviathan.Comeatmebro
Offline
サーバ: Leviathan
Game: FFXI
user: Rairin
Posts: 6052
By Leviathan.Comeatmebro 2017-02-14 12:28:09  
I figured on using relic gun and doing a baseline 200 shots with no rapid shot or snap. As long as it always provides some benefit, should be pretty obvious whether I'm getting 100% or not.
 Fenrir.Snaps
Offline
サーバ: Fenrir
Game: FFXI
user: Mojopojo
Posts: 1139
By Fenrir.Snaps 2017-02-14 12:29:45  
Well I mean, we don't actually have data indicating how it actually works. Also I think you should probably stick with a 600 delay weapon since we have good data for that delay already.