Now announcing, PacketFlow for both Ashita and Windower!
Players have been having persistent issues with "lag" causing messages to drop in instances for years. In reality, this "lag" can be decomposed into two separate problems:
1. Chunk prioritization is messed up in instances
2. Instances generate a lot of chunks
PacketFlow attempts to address the second problem.
For a brief overview of FFXI network dynamics and terms, I recommend reading the spoiler:
FFXI communicates between the server and client using UDP packets that have a max size around 2.5KB. Those packets are composed of multiple "chunks" of FFXI data, carrying information to either tell the server what you're doing (client -> server) or for the server to update you on the state of the game (server -> client). People frequently use the term "packet" to refer to chunks, but I'm going to keep the terms separate here because it is mechanistically important.
Examples:
When my FFXI client displays a message like "Byrth gains the effects of Phalanx" in the chat log, it knows to do so because it received a chunk telling it to display a gain buff message for my character within a larger UDP packet that may also contain a lot of other information. When you see my character run around, it is because my FFXI client sent the server a message telling it my new position (one chunk in a client->server UDP packet), and the server relayed my new position to your FFXI client (another single chunk in a server->client UDP packet).
In non-instanced zones, SE appears to use a prioritization system to make sure that your client receives more important chunks. If you pull all of the monsters in Korroloka Tunnel at once, for instance, and cast Phalanx on yourself, you will always see the "gain the effect of Phalanx" message. More generally, chunks containing information involving your character are prioritized over more contextual chunks (like monsters moving around). Most of the time, this prioritization system does not matter because FFXI is sending you less than 2.5KB of data and so everything makes it into the UDP packet. However, sometimes you're in a very active environment and there's more than 2.5KB of data to send you.
You can experience this when by pull multiple monsters. If 5 monsters are chasing you, they typically move pretty reasonably. If 20 are chasing you, some monster start to jump between locations because the chunk that updates those monsters' positions have low priority and do not fit in the packet. However, even with 20 chasing you the messages about their attacks are generally included (and fill up your chat log). This prioritization system lets FFXI get away with a pretty low bandwidth requirement, and the fact that most of us have never really thought about it indicates that it works pretty well.
However, in instances the prioritization system is at best nonfunctional or worse is functioning in the opposite way from normal zones. You will frequently miss messages about yourself in a Dynamis - D instance (buff gain, action completed, etc.), that you would never miss in a vanilla Dynamis zone. This is all server-side, so we can only guess at the specifics of the packet prioritization system and how it's messed up in instances, but it's clearly not working as it was designed to 2 decades ago.
What are solutions here?
1. Minimize the number of chunks you are sent: fight one monster at a time, turn off party buff information, don't use GearSwap, etc. Anything that decreases the number of chunks you are sent for FFXI is going to help you avoid missing chunks, because prioritization doesn't matter unless the packet is full.
2. Increase the server -> client bandwidth.
Option #1 sucks and is very limiting, so option #2 is the interesting one.
Bandwidth is more or less max packet size * frequency. For the FFXI server -> client connection, this is (2.5 kb / 1 packet) * ( 1 packet / 0.4s) = 6.25 kb/s most of the time. We cannot change either of those parameters directly because they are part of the server software.
However, years ago I noticed that the server responds to any UDP packet it is sent with another UDP packet. Thus, we can indirectly control the incoming UDP packet frequency by adjusting the outgoing packet frequency. This idea came to nothing for the better part of a decade mostly because I had no idea how to trigger the FFXI client to send UDP packets more frequently.
I was reminded of it earlier this year when KnugenJulian started posting in the Windower Discord asking if anyone had a fix for Odyssey lag. I brought up my purely theoretical fix, and Thorny made it real by connecting it with atom0s' UDP packet handler reverse engineering.
Thus, there are now plugins (both named Packetflow) for Ashita and Windower that decrease that time-per-packet from 400ms to 250ms, effectively increasing bandwidth by 60%. They do not completely solve the problem of dropped chunks in instances, but they do make it better. Additionally, they make loading zones quite a bit faster.
I trained 20 Orcs in Dynamis D - Sandy on my RUN, running them 20' back and forth so they'd keep moving with two WHMs to keep me alive. The only purpose of this was extra packet generation (many movement and attack chunks from the monsters).
I also made a party of 6 (brd x1, rdm x5). Each RDM cast 6 barspells in rotation 20x (120 casts total). They precast naked and midcast in full 16 slots to generate equipset chunks. I set up a Lua to log cast completions and loaded it on two of the RDMs, one with PacketFlow and one without.
600 casts actually occurred, and:
* the character with packetflow saw 588, and all 12 missed were their own. (10% missed of their own, 0% of others)
* the character without packetflow saw 517, and 36 of those missing 83 were their own. (30% missed of their own, 10% of others)
As with all 3P tools, but perhaps even more here, you risk your account when you use this plugin. Packetflow is unambiguously detectable and increases the outgoing bandwidth costs for FFXI. Thus, for Windower it is in the Konami code section of the launcher.
As far as due diligence, we actually more or less finished these plugins in mid-April and I have been using the Windower one on my main and mule for the last month ~24/7. I'm not banned yet. We don't have a way to test it better than that and that's all of the testing that has been done. Caveat Emptor!
Attempts have been made to reach out to SE about these problems and to get a temperature read on the proposed solution:
*
2016 AMA - Bandwidth - basically "it would be too hard to fix"
*
2022 AMA - Prioritization
*
2022 AMA - Bandwidth
We waited a week for SE to post their hypothetical 2022 AMA forum follow-up where they answer those questions, but no dice.
Credit to:
* Thorny for the implementation and empirical testing
* atom0s for the reverse engineering that made it possible
* KnugenJulian for starting the conversation that revived the concept
