If I take out the waterfalls / mists its a stable 56 fps out of 57
if they remain I hit 54-56/57
There is also the lighting glows but I don't want to strip out the soul of the place but I also want to be able to roam around home base without annoyances.
I'm just happy that we're collectively learning and sharing info about how to make these changes. I imagine that folk will want to have these tweaks applied to HD dats once the methods have been hammered out.
Yes, theoretically. They are the same format in hex side by side I would assume, but just replaced texture values but this also means the actual hex is different.
It will be easier if the pattern lines up but will still require much testing.
Since you're discussing editing the hex values of the .DAT files directly, have you considered using one of the available tools like ChangeTex or Tex Hammer? Just a thought, but they may be able to make the task easier if you haven't already tried.
Also, for those using these .DATs (and aren't crashing), are you actually noticing performance improvements? I know there's been some doubt in the past as to whether these sorts of mods actually improve performance as much as some would hope. I've heard mixed opinions from others who have tried previous similar mods.
For zone particle emitters, another option is to zero the auto-run flag instead of deleting the entire resource (section_body_start + 0x69, 0b0001 0000). Alternatively, reducing the emission frequency & amount can help.
More generally, for DAT resource structure, it would probably be useful to reference one of the open-source implementations (Xim, XiClient, etc). Your LLM is making some really strange claims about how the retail client & D3D8's fixed-function-pipeline work.
Example for clearing the auto-run flag - setting this 1 bit to 0 will turn off one of the chimneys in Selbina (though, I'm skeptical that these specific particles will make much difference wrt FPS):
Just crashed twice trying to HP from East Ado 1 to Bastok Mines Mog HP.
Edit:
HP out of Bastok Mines to Eastern Adoulin / others - works
HP into Bastok Mines from not Eastern Adoulin - works
HP out of Eastern Adoulin into not Bastok Mines - works (for handful of zones tested)
HP from Eastern Adoulin into Bastok Mines (AH or mog) - crash
Ok I applied the auto run flag 0 to these instead
so now they are all the same filesize so less likely to crash and more likely to just cause a graphical bug I would think.
Objects are not mine and I'm not sure whose they are to ask if I am allowed to distribute or not.
But since they make an improvement I included them
They are completely original from when they author(s) posted them and untouched from the perspective of this project.
The current version does not crash in any of the ways I've encountered from previous versions. However, while smoke is gone from Bastok Mines and Upper Jeuno chimneys, it has returned for Mhaura and West Adoulin.
While I have observed changes in Mhaura, Western Adoulin, Upper Jeuno, Bastok Mines, and Bastok Markets, I have not been able to locate / identify the intended changes in the zones listed below. I expect that I simply do not know where to look to find them. If someone knows what smoke effects are being referenced, it would be appreciated and I'll happily test them.
The current version does not crash in any of the ways I've encountered from previous versions. However, while smoke is gone from Bastok Mines and Upper Jeuno chimneys, it has returned for Mhaura and West Adoulin.
They got overwritten, they are back now.
Looking into these others now
Took several attempts to locate all the smoke in Eastern Adoulin.
It would catch one then be like done
I had to go count and tell it at least 7+
While I have observed changes in Mhaura, Western Adoulin, Upper Jeuno, Bastok Mines, and Bastok Markets, I have not been able to locate / identify the intended changes in the zones listed below. I expect that I simply do not know where to look to find them. If someone knows what smoke effects are being referenced, it would be appreciated and I'll happily test them.
Double checking this one now
I am having to walk an alt and do side by sides lol
1. Port Bastok (ROM/1/32.DAT):
- Harbor & Cargo Docks (J-7 / K-7): The metal exhaust pipes and boiler stacks rising behind the warehouses and near the ship loading berths.
- Galka District & Steamer Tavern (E-6 / F-6): The stove chimneys poking out of the residential building roofs.
2. Windurst Woods (ROM/1/34.DAT):
- Mithra Residential Treehouses (G-9 / G-10 & H-10 / I-10): hollowed tree trunks and rounded Tarutaru canopy huts—little wood stove chimneys poke out of the tree bark walls with continuous smoke curls.
- Weavers' Guild & Cooking Stalls (H-11 / I-11): The guild dye/cooking vat chimneys.
3. Windurst Waters (ROM/1/35.DAT):
- Timbre Timbers Tavern & Food Stalls (E-8 / F-8): The kitchen exhaust chimneys above the tavern roof.
- Rhinostery & Aurastery Labs (G-6 / H-6 & J-8 / J-9): The alchemy / research stove chimneys along the canal bank.
4. Port Windurst (ROM/1/37.DAT):
- Fishermen's Guild & Smoke Sheds (C-7 / C-8): The fish curing / smoking sheds along the water.
- Airship Terminal & Pier Cargo Cranes (F-6 / G-6): The boiler exhaust smokestacks near the dock machinery.
There should be minimal to no loading problems, because there is no more outright deletions anymore.
Files are intact byte for byte. If anything does please let me know.
Tried a little experiment with the corrected vanilla version of upper jeuno
Vanilla is 8.29mb (139 emitters)
github dat 8.29mb (0 emitters)
nexthd is 29.47mb (139 emitters)
thiscursedthing 29.47mb (0 emitters)
I first had the AI make changes and feed them back to itself a bunch
At first it could only change lines
Then it started to change color pallets
Then it started to color pallets differently
I then gave it a directive to run each texture through google flow one by one.
Double checking this one now
I am having to walk an alt and do side by sides lol
1. Port Bastok (ROM/1/32.DAT):
- Harbor & Cargo Docks (J-7 / K-7): The metal exhaust pipes and boiler stacks rising behind the warehouses and near the ship loading berths.
- Galka District & Steamer Tavern (E-6 / F-6): The stove chimneys poking out of the residential building roofs.
2. Windurst Woods (ROM/1/34.DAT):
- Mithra Residential Treehouses (G-9 / G-10 & H-10 / I-10): hollowed tree trunks and rounded Tarutaru canopy huts—little wood stove chimneys poke out of the tree bark walls with continuous smoke curls.
- Weavers' Guild & Cooking Stalls (H-11 / I-11): The guild dye/cooking vat chimneys.
3. Windurst Waters (ROM/1/35.DAT):
- Timbre Timbers Tavern & Food Stalls (E-8 / F-8): The kitchen exhaust chimneys above the tavern roof.
- Rhinostery & Aurastery Labs (G-6 / H-6 & J-8 / J-9): The alchemy / research stove chimneys along the canal bank.
4. Port Windurst (ROM/1/37.DAT):
- Fishermen's Guild & Smoke Sheds (C-7 / C-8): The fish curing / smoking sheds along the water.
- Airship Terminal & Pier Cargo Cranes (F-6 / G-6): The boiler exhaust smokestacks near the dock machinery.
Feels like I'm being punked. Out of all these locations / descriptions, the only one that I saw visibly emitting smoke/anything was 2 tall smokestacks on top of the Rhinostery in Windurst Waters (J-8). For which, this mod did nothing (still smokin').
Double checking this one now
I am having to walk an alt and do side by sides lol
1. Port Bastok (ROM/1/32.DAT):
- Harbor & Cargo Docks (J-7 / K-7): The metal exhaust pipes and boiler stacks rising behind the warehouses and near the ship loading berths.
- Galka District & Steamer Tavern (E-6 / F-6): The stove chimneys poking out of the residential building roofs.
2. Windurst Woods (ROM/1/34.DAT):
- Mithra Residential Treehouses (G-9 / G-10 & H-10 / I-10): hollowed tree trunks and rounded Tarutaru canopy huts—little wood stove chimneys poke out of the tree bark walls with continuous smoke curls.
- Weavers' Guild & Cooking Stalls (H-11 / I-11): The guild dye/cooking vat chimneys.
3. Windurst Waters (ROM/1/35.DAT):
- Timbre Timbers Tavern & Food Stalls (E-8 / F-8): The kitchen exhaust chimneys above the tavern roof.
- Rhinostery & Aurastery Labs (G-6 / H-6 & J-8 / J-9): The alchemy / research stove chimneys along the canal bank.
4. Port Windurst (ROM/1/37.DAT):
- Fishermen's Guild & Smoke Sheds (C-7 / C-8): The fish curing / smoking sheds along the water.
- Airship Terminal & Pier Cargo Cranes (F-6 / G-6): The boiler exhaust smokestacks near the dock machinery.
Feels like I'm being punked. Out of all these locations / descriptions, the only one that I saw visibly emitting smoke/anything was 2 tall smokestacks on top of the Rhinostery in Windurst Waters (J-8). For which, this mod did nothing (still smokin').
Port Windy
The Tall White Conical Smokestack Tower:
Rising directly behind the green roof of the Fishermen's Guild is that tall, white conical stone smokestack tower.
Where Emitters #19–#24 Are Attached:
Emitters #19 through #24 (sk01, sk02, sk03, sk04) are attached directly to the top rim of that conical smokestack.
The Retail Bug:
In retail vanilla FFXI, Square Enix programmed those smokestack emitters with Alpha = 0.0% (invisible), but left their Auto-Run flag active (0x10).
Looks like we were supposed to get smoke here and did not.
For the Rhinostry they are a different pattern then normal
"21 chimney sub-emitters (sk00–sk20) inside a parent container at offset 0x831300 using odd byte alignments"
and should now be fixed.
Would that pattern also be responsible for the smoke in Bastok Markets (way up high) on top of what would be the Metalworks as well as the some logical location inside the Metalworks zone itself? Not that we necessarily need every instance of smoke removed but I noticed that those smokestacks were still active in Bastok Markets even though the rest had been quieted.
fish are a new type of texture problem for me as well as is ocean
things got slower for me as I messed with the ocean and fish will need to spend more time on this later
I was thinking, for the lazies like me, if u dont use xipivot and want to change the .dat in your actual folder, you could add the original .dats to the whole thing in case u want to replace them back or something crashes.
I'm gonna go do that myself right now tho
I was thinking, for the lazies like me, if u dont use xipivot and want to change the .dat in your actual folder, you could add the original .dats to the whole thing in case u want to replace them back or something crashes.
I'm gonna go do that myself right now tho
That is a very good point.
However, no one should be directly replacing their direct dat files, especially with experimental ones.
I've crashed to desktop on login for quite a few of these until the methods got cleaner and being able to unload pivot or reload pivot for testing has been ideal.
The only files that should ever be directly swapped are text files because those load as soon as the game starts.
Random thoughts...
Would be funny to make a dat pack randomizer with several flavors of a zone.
For anyone future stuck you can always delete the data I believe and use check files and it should replace it
Hey Jinxs sorry to jump in but I built a zone editor a few months ago which could help you here, if you throw your AI at the docs as well I bet it could speed up development. I went through an huge amount of trail and error removing/editing zones to avoid crashes. It's setup in a way that if you're using something like Claude, GPT or Grok it should be able to build a decent hindsight/memory bank of knowledge.
Hey Jinxs sorry to jump in but I built a zone editor a few months ago which could help you here, if you throw your AI at the docs as well I bet it could speed up development. I went through an huge amount of trail and error removing/editing zones to avoid crashes. It's setup in a way that if you're using something like Claude, GPT or Grok it should be able to build a decent hindsight/memory bank of knowledge.
Hey Jinxs sorry to jump in but I built a zone editor a few months ago which could help you here, if you throw your AI at the docs as well I bet it could speed up development. I went through an huge amount of trail and error removing/editing zones to avoid crashes. It's setup in a way that if you're using something like Claude, GPT or Grok it should be able to build a decent hindsight/memory bank of knowledge.
Nice, that’s a pretty good knowledge doc already, if you clone my docs folder and add it as a memory resource it can help (if you setup hindsight it’ll help a lot!)
The zone editor can do full edits and deletes, I need to make a post on it, I have some edits already live in private servers so very well battle tested.
I got to wondering about a datmod I had in my xipivot arsenal was there and then I remembered how much lag the smoke in Mhaura added and continued to wonder what makes Nashmau terrible.
I had AI (~oooh the boogy man~) look for similar patterns using 2 examples I had and it gave me back this list.
I don't have skills for graphics, nor do I trust AI (a blind man) to do potential in-depth graphics work.
But if we have any graphical talent left out there..
A pack could be made to remove some of the biggest offenders.
Zone
Zone ID
Target DAT Path
Problematic Particle Emitters
Status
Mhaura
Zone 249
ROM/1/44.DAT
Harbor boiler steam, dock haze, chimney smoke
✔ Improved(<Smoke only>)
Western Adoulin
Zone 256
ROM9/0/3.DAT
Hearth smog, residential chimneys, bridge mist
✔ Improved(<Smoke only>)
Nashmau
Zone 53
ROM3/0/3.DAT
Dock boiler steam, Qiqirn cooking kilns, canal mist
✖ Needed
Eastern Adoulin
Zone 257
ROM9/0/4.DAT
Big Bridge water spray, Castle gate braziers, shop steam
✖ Needed
Aht Urhgan Whitegate
Zone 50
ROM3/0/0.DAT
Central fountain water mist, tea house steam vents
✖ Needed
Al Zahbi
Zone 48
ROM3/0/1.DAT
Desert dust storm haze, bazaar firepit smoke
✖ Needed
Lower Jeuno
Zone 245
ROM/1/41.DAT
Alleyway fog layers, door lantern glow emitters
✖ Needed
Port Jeuno
Zone 246
ROM/1/42.DAT
Airship boiler exhaust, cloud backdrop particles
✖ Needed
Bastok Mines / Markets
Zones 234/235
ROM/1/30.DAT, 31.DAT
Industrial smelter smoke, blast furnace plumes
✖ Needed
Kazham / Norg
Zones 250/252
ROM/1/47.DAT, 48.DAT
Jungle humidity mist, massive waterfall water spray
✖ Needed
I don't trust this one as much
Zone
Zone ID
Target DAT Path
Problematic Particle Emitters
Status
Dynamis - San d'Oria [D]
Zone 294
ROM4/4/113.DAT
Burning wreckage, chimney smoke, rain sprites
✖ Needed
Dynamis - Bastok [D]
Zone 295
ROM4/4/114.DAT
Smelter smoke, volcanic ash, divergence haze
✖ Needed
Dynamis - Windurst [D]
Zone 296
ROM4/4/115.DAT
Mana distortion fields, floating magic orbs
✖ Needed
Dynamis - Jeuno [D]
Zone 297
ROM4/4/116.DAT
Dense divergence fog, barrier glow auras, fire braziers
The exact weight of each of these items I do not fully understand, my only experience was with the massive improvement the Mhaura smoke mode made. So a big thank you to whoever made that.
idk how true this explanation is but I do feel inclined to trust the draw-call piece
Why Endgame Zones Suffer Worse Lag:
Endgame lag is a draw-call multiplier. In standard towns, particle smoke only competes with ambient NPCs. In Endgame zones, the engine is forced to depth-sort and alpha-blend:
1. Continuous ambient zone particles (miasma, ash, burning wreckage).
2. 18+ player alliance character models and weapon glow auras.
Because DirectX 8/9 processes transparent alpha passes on a single CPU thread, this combined draw-call overload creates the infamous 10–15 FPS slideshow during large pulls.
Top 3 Biggest Offending Effects in the Game:
[list=1]
[*]Fullscreen Ashfall & Heat Distortion (Mount Zhayolm / Halvung): Unlike localized chimney smoke, ash particles fall across the entire screen. Every single ash flake forces a separate depth-sorting pass against the ground geometry.
[*]Burning Debris & Divergence Barrier Mist (Dynamis-Divergence): Combines base city smoke with continuous fire braziers and glowing barrier distortion shaders right where large alliances pull massive mob groups.
[*]Ambient Floor Miasma & Gate Beacons (Sortie / Outer Ra'Kaznar): Multi-layered transparent mist rolls continuously along the floor meshes, compounding frame drops whenever AOE magic is cast over it.
[/list]
