.
Main JK 202X Download Folder: https://drive.google.com/drive/folders/1f8vfU7kQ642_AeUCsylikFfMRecWG14_

Modified JkGfxMod Download Folder: https://drive.google.com/drive/folders/1GMvq8vvBjukaOFEo-V8ZZrwgcbXKAqA9

Top-level Jedi Knight Folder: https://drive.google.com/drive/folders/1Gz6tDjS3v3LgA__kqelBsYV-d306OFGC
(contains the above two folders, in case I reorganize subfolders, etc.)

If any ZIP files are password protected, the password will be their filename:
e.g. JK_2022.zip password = JK_2022

Passwords are to keep gdrive from mangling or vaporizing DLL and EXE files now or in the future.

---

JK 202X Setup Instructions:

Download the newest version from the "Main JK 202X Download Folder" linked above.
Sort them by modified date (or whatever) to figure out which one is the newest.

Extract the _setup folder into your Jedi Knight folder...

...then go into the _setup folder and run _setup.cmd.

The setup script tries to take a backup of important files and running the setup script again will offer to uninstall back to how it was.

---

The included DirectDraw proxy has special accommodations for the Steam version so that things will work (and even automatically launch JkGfxMod's inject.exe if present) without renaming files, etc. In other words, you can do a fresh Steam install, copy in and run JK 202X's setup script, then launch via Steam and it should just work.

The setup script also undoes some damage Steam's first-run setup does (namely it uninstalls the app compat database for JK.exe that Steam installs).

While you can delete the _setup folder after patching, if you leave it, you can re-run the setup script to verify the patch is still applied, and uninstall the patch.

---

If you need to get ahold of me for anything JK related, you might be able to catch me as "qm" on the Massassi Discord, or via one of the following two websites:
https://www.romhacking.net/community/5289/
https://www.nexusmods.com/users/8096959

I can't promise I frequent any of them, but at least you'll have the possibility of reaching me.

---

I modified this post to contain links to Google Drive folders to hopefully future proof access to files for at least a little while after Massassi is frozen like a contraband smuggler in carbonite.

The original first post is two posts down.

QM

Feel free to bundle the executable with JkGfxMod if you want. It uses Microsoft Detours to hook the executable. Microsoft provides a tool to patch an executable offline, which in your case would let you skip the extra inject.exe stuff.

ORIGINAL FIRST POST:

Jedi Knight Quadranscentennial Edition

i.e. Jedi Knight 25th Anniversary Edition

(starting to get this post created. I'll flesh this out as I'm getting files organized; there's a lot that needs to be documented)

The main JK executable, based on the official 1.01 patch, includes the now-standard assortment of JKUP patches, plus the following features:

• -
• (list to be further populated)
• -
• rotated particles by 45 degrees
• enhanced z-buffer accuracy
• CPU relief (JK should no longer peg a core to 100%)
• toggle enhanced 3D audio without JK restart
• fixed a (big) memory leak in A3D audio code
• fixed a jkSetForceSpeed crash
• added ZeqMacaw's increased synced thing limit
• enabled unused COG extension features:
  
  • hotkey message
  • discrete hotkey filtering
  
  
• added some OpenJKDF2 QOL_IMPROVEMENTS:
  
  • smooth singleplayer aerial physics
  • improved behavior above 100 fps
  • waggle animation scaled to framerate
  • keyboard turning scaled to framerate
  • mouse sensitivity numeric display
  
  
• fixed mouse sensitivity setting drift
• increased mouse sensitivity maximum
• added an adjustable internal framerate (frametime arg and cmd)
• added JkGfxMod detection and compatibility enhancements
• reset debug mode flags at main menu
• tweaked help command to not list itself
• devmode arg skips intro cutscene (can view in cutscene menu)
• added some (all?) OpenJKDF2 memory leak fixes
• added OpenJKDF2 AI awareness fix
• added dual wield weapon mesh rendering
• removed JKUP's increased COG symbol limit:
  
  • it was not actually usable since variables 256 and above disabled COG verbs and rendered a COG functionally useless
  
  
• added mouse wheel support
• made the free look hotkey invert state (since "enable free look" is typical)
• fixed missing F1-F8 hotkeys when player data load fails
• adjustments to default controls (e.g. space = jump, auto run enabled, etc.)
• fixed crash when alt-tabbing during joystick "identify" menu
• added limited controller trigger support (both triggers can't simultaneously cause an action due to limitations in winmm/dinput controller support, but actions that don't really need to occur simultaneously such as "Use Force Power" and "Fire 1" work adequately)
• added left and right stick click-in support (aka L3/R3 or LSB/RSB); these have no defaults bound as I hate them, but they're now available for use
• added rudimentary controller support to the game menus (left stick = move cursor, A = click, B = escape)
• enabled using mouse buttons and joystick (controller) buttons while in-game chat is active

Additionally, external files provide the following features:

• enhanced 3D audio (a3d.dll)
• traditional rendering pipeline (ddraw.dll, wndmode.dll)
• install-free DirectPlay support (dplaysvr.dll, dplayx.dll, dpnhpast.dll, dpnhupnp.dll, dpwsockx.dll)
• CD audio (winmm.dll, libvorbisfile-3.dll, 00_soundtrack.cog, 00_playambient2.cog)
• automatic FOV, mipmap and LOD scaling (item_wrenchyellow.cog)

Where modified COGs are used to provide features, they're designed to match the unmodified vanilla COG in behavior and checksum so as to not cause multiplayer incompatibility.

QM

Edit: I've written down what I've done in the past month, but all prior features are yet to be documented.

---

ORIGINAL SECOND POST:

I actually want to keep JkGfxMod dynamically patched, both so that if it gets further updates a user can simply drop in the new file(s), and because other stuff I've either done personally or adapted from others' (ZeqMacaw et al.) work has the vanilla rendering pipeline functional (including with 32-bit color modes and glow-in-the-dark colors like the 8-bit software renderer) on modern Windows (and still backwards compatible through at least Win98SE; I never tested on 95/98, nor on 11 yet, but I'm optimistic), so having both options seems better than forcing one. The injector, in spite of the anti-virus false positives, also seems to prove handy for launching via Steam.

The glowing Lightsabers and projectiles possible with JkGfxMod are so damn amazing though, forcing that as a standard sure is tempting. I'm experimenting with the displacement map capability as I'm writing this, but not sure if anything will come of it.

For reference, for JkGfxMod compatibility, I did the following:
- detect if JkGfxMod is loaded (via GetModuleHandleA on renderer.dll, just after JK calls CreateWindowExA); I don't like how hacky this is, but it's simple and reliable
- if detected:
-- skip QueryInterface for Viewport3 (JkGfxMod kills itself otherwise, I wish it just returned E_NOINTERFACE)
-- restore the ZBufferMode for rendering layers 3 (transparent level geometry) and 4 (pov weapon model) which are otherwise modified from vanilla
-- modify some window centering code I added to instead snap the client area to 0, 0
-- modify setting load code for
--- 3DDeviceGUID to return JkGfxMod's GUID
--- displayMode to always return 1
--- b3DAccel to always return true (1)
-- modify setting save code for the above three, to skip saving them

The first two are critical for compatibility.

QM

Edit: Playing around with displacement maps. This is the vertical water shaft in Canyon Oasis:


And here's what the mouse config screen looks like now:

I've been kinda going nuts with JkGfxMod. There wasn't a sane way to resolve a compatibility issue between my new code to let the controller drive the cursor in menus and JkGfxMod, so I went ahead and got a build environment functional for JkGfxMod (not difficult, thank you for making a tidy, self-contained project and sharing it with the world) and started making additions.

So far, I've done the following:
- hook SetCursorPos to translate stretched window coordinates
- tweaked the GetCursorPos hook so that it also affects in-game (for 640x480 software mode mouse use)
- hook CreateFileA to load texture replacements during level load to reduce in-game hitching without random pre-caching (I hated writing this hook in C++, I could've made it hyper efficient in assembly)
- level-load-loading the textures necessitated populating a filename-based dictionary during program load to be used during level load (from ["replaces"][0]["name"] entries, accomodating multi-cel textures)
- added a boolean to jkgm.json to control level-load-loading ("per_level_texture_loader")
- avoid crashes for missing albedo_map, emissive_map and displacement_map files
- automatically use current screen resolution if "resolution" isn't defined in jkgm.json (instead of 640x480)
- added "hud_texture_num_slabs" to jkgm.json to specify number of frames per HUD layer refresh (on my computers, at my preferred screen resolutions, updating every frame doesn't clobber framerate, ymmv)
- added additional non-displaced texture passthrough from displacement mapping to screen space shaders
- rewrote SSAO (including aspects in the DLL)
- modified displacement mapping
- made "enable_vsync": true be the default (there were non-zero complaints about videocard heat)

QM

Wow I don't know how I missed that screenshot but it looks amazing!

More JkGfxMod tweaks:
- altered framebuffer clears to restore vanilla JK HOMing; this is more necessary than is obvious to avoid rare-but-consistent partial screen blackouts as the camera passes sector adjoins
- added relaxed cone mapping as an alternative to standard displacement mapping

Relaxed Cone Mapping and fine tuning the new "Fast" Ambient Occlusion have been what I've been working on the past month. Not sure yet how to (or if it's worth) getting screenshots that show off the differences.

super happy to see this is still going on, plus with jkgm integration
any hints as to when we can see a demo?


this is actually super noticeable in levels with 3do skyboxes. also there are some levels and mods that rely on HOM to create certain visual effects, so it's good that consistency with base JK is maintained in this aspect.

I need this!

also, can anything be done to fix this silly wee seaming issue caused by JkGfxMod mipmapping?

I've been working with someone on an all-in-one tool for precedurally generating depth maps (and their matching cone maps). It's technically going to be generically useful and not JkGfxMod-specific. As a baseline, it can be fed a basic (albedo) texture, and will then:
- produce an approximate normal map
- use the normal map to produce a depth map
- apply filters to the depth map (normalizing, changing distribution, blurring, etc.)
- use the (filtered) depth map to produce a cone map
- reconstruct a normal map based on the filtered depth map (not used by JKGM)
It can be fed one of those intermediate data textures, for example, a properly created normal map, or a hand-made depth map, and just start from there to produce the output further down the chain.

While the normal map creation is low cost (computationally) and basically instant (and also really simplistic), calculating a depth map from a normal map is very CPU intensive and can take multiple minutes per texture (once 512x512 or larger), and then generating a cone map from a depth map is very GPU intensive and can also take multiple minutes.

Once this tool is done, and I've processed existing JKGM assets into cone maps, I think I'll be ready to release. All of this already works, but wasn't in a state that could be shared, so it's just been a matter of getting it to that point. No point releasing modified JKGM + shaders if no one can produce the necessary data to feed the shaders.

---

Regarding the mipmapped texture seams, I've definitely witnessed that as well on some of the JK Enhanced models. I'll at least be taking a look to see if there's a practical solution. I assume it's due to the neutral color painted in the "unused" gaps of the UV layout getting blended in during the mipmap generation and affecting visible pixels, so that's where I'll be starting my investigation.

QM

Since you're going to so much effort, it might be worthwhile adding your generated normal maps to the material pack anyway. JKGM could ignore them for now, but normal maps would be useful for adding screen space reflections in the future (or for modernizing the lighting in OpenJKDF2).

Also, if by chance your approach could be applied to it, the same goes for a roughness-metallic texture. It is not useful today, but in any situation where you would be able to use the normal maps you would want to have roughness maps. Mipmapping is destructive to fine detail in normal and displacement maps, and roughness maps lets you treat that lost detail as microfacet scattering (which it technically is). There's a siggraph paper from either valve or square-enix that discusses their approach to computing [an]isotropic roughness from normal mip levels. It's pretty straightforward iirc.

also envmaps when

Merry Christmas! Barely made it before the new year when it would be too late to call it a 25th Anniversary version...

https://drive.google.com/drive/folders/1Gz6tDjS3v3LgA__kqelBsYV-d306OFGC?usp=sharing
JK_2002.ZIP password is: JK_2022

The password is to keep gdrive from mangling or vaporizing DLL and EXE files now or in the future.

---

This is just JK 2022, not the modified JkGfxMod stuff yet.

Extract the _setup folder into your Jedi Knight folder...

...then go into the _setup folder and run _setup.cmd.

For now, do this in a copy of your JK folder. The setup script tries to take a backup of important files, and running the setup script again will offer to uninstall back to how it was, but I only did enough testing to make sure it worked enough to release.

---

The DirectDraw proxy has special accommodations for the Steam version so that things will work (and even automatically launch JkGfxMod's inject.exe if present) without renaming files, etc. In other words, you can do a fresh Steam install, copy in and run JK 2022's setup script, then launch via Steam and it should just work.

The setup script also undoes some damage Steam's first-run setup does (namely it uninstalls the app compat database for JK.exe that Steam installs).

I've only tested the setup script with the original 1997 JK and the Steam version, but it gets the main executable out of the 1.01 patch installer, so it will probably work under most circumstances.

While you can delete the _setup folder after patching, if you leave it, you can re-run the setup script to verify the patch is still applied, and uninstall the patch.

QM

nice
I couldn't figure a way to apply the patch via Linux, so I resorted to an old windows install to do that, but it was super easy then.

I'm mostly familiar with this patch as I'm (supposedly) the one person to have tested its early 2018 build back then on JKHub (and I also applied some of the OpenJKDF2 stuff via manual hex editing hehe), but neat to see this brings some new stuff.
The first, most obvious and important one is JkGfxMod compatibility - not only this works nicely (unlike the 2018 build which simply crashed when trying to run with JKGM) but it also seems to have solved the super annoying keyboard input lag that happened when framerates dropped when using JKGM coupled with JK13 . Speaking of which, JKGM framerates do seem a tad bit better (haven't really compared, but it's better than I remember) on my 7yo PC with integrated Intel graphics.

Hopefully the memory leak fixes will solve the seemingly random crashes that happen in JKGR after some time spent on most maps. I do wonder what "OpenJKDF2 AI awareness fix" is about though, haven't tested thoroughly enough to see a difference. The only negative side effect I've noticed so far is that controlling JKGR cars via arrow keys seems to have been messed up somehow, I believe due to keyboard turning being scaled to framerate. Hopefully I can fix that via some simple cog tweaking.*** [Edit: jesus christ the possibilities with the new hotkey message are scary]

Glad this is out, and hope to see the JKGM tweaks coming out sooner rather than never :downs:

[Edit 2: sadly I don't see any way that can be fixed I don't see a way to tell apart keyboard turning input from mouse turning input, so I'll have to make it be either one or the other. Oh well. It's still worth it for everything else.]

[Edit 3: the keyboard (and only keyboard, doesn't affect mouse) input lag wasn't caused by JKGM, but rather by your JK2018 patch. sadly, it is present on this version.]

Okay, probably have time to pick this up again, so hopefully you'll see this SMLiberator.

As a first test to try and resolve the keyboard input delay, would you try hex editing the patched JK 2022 executable as follows:
- at offset 0x10E016 you should see
6A 01 00 00
- change to
E6 4D EF FF
This should remove a Sleep(1) from the main render loop that might be the problem.

Obviously keep a backup of the executable since this may not help and I'll want to try other things. My current hypothesis is it's a Linux-specific issue either with the Sleep added to the rendering loop to relax CPU load or with how Linux is handling keyboard event messages. Windows 10 (or maybe 8, not sure when window key event messages got so much more computationally expensive) also had some issues in that regard compared to 7, and especially compared to XP and older.

QM

My bad, I just noticed I made a mistake on my previous message - the input lag wasn't being caused by the 2018 patch, it was fixed with the 2018 patch, and was present in JK13 and in the 2022 patch. Will give the hex edit a try ASAP.

[edit] nah, I don't think that fixed it

Well, if you're positive JK13 and 2022 have the input delay and 2018 absolutely doesn't, then I'm not surprised; that hypothesis was based on 2018 and 2022 both having input delay. I'll look through the differences between those three with that in mind, but having you double check that 2018 does not would be appreciated (so I'm not chasing ghosts).

QM

JK22 definitely has it, JK18 definitely does not. For some reason JK13 isn't launching here but I'm 100% positive it also does. Note that it only happens (or is only noticeable?) on low framerates and it can't be replicated by setting frametime to a high value.

Alright, while I'm refamiliarizing myself with JK's guts and carefully analyzing changes I'd made between the 2018 and 2022 versions, here's a bulk archive of all ~50 snapshots between the two in case you're up for some brute force work:
JK_2023_troubleshooting.7z
https://www.sendspace.com/file/mv4bll
Password: SMLiberator
Be careful that you get the correct download link, it'll be ~397 KB and contain over 50 "bak" files. Each is just a prepatched JK executable where you rename it to use it. No patching, etc.

My apologies for the junk file host, but I don't really want this file to persist or be easy to access as it's a confusing mess and I normally never distribute patched executables.

Inside the archive, rev 129 is the 2018 release (with advapi32.dll redirected to portable.dll as in JK2022) so start by making sure it in place of JK2022 works as expected. Rev 180 is the version just before the JK2022 release. All testing should be done in a JK2022 setup.

I'd suggest doing a binary search and see if you can pinpoint which version introduces the input delay. So something like:
Start in the middle and...

- try 155
-- if delay try 145
--- if delay try 140
--- if no delay try 150

- try 155
-- if no delay try 165
--- if delay try 160
--- if no delay try 170

...and so on.

Just if you're up for it. I'm picking away at things on my end, trying to guess which changes might have made sense to affect keyboard input. It could be COG hotkey stuff, or high framerate stuff, various things involving controller support... I'm really not sure yet.

If rev 129 in an otherwise JK2022 setup still has input delay, that would imply that it's not changes to the JK executable itself giving you trouble, but one of the supporting files. So it makes sense to check that first.

And a reminder, keep a backup of your patched JK2022 executable.

QM

Apparently 150 (win10fix) is the last version with no keyboard input lag. 151 (rendersleep) and I'm guessing every version after that has it, but I haven't tested all of them.

Yeah, no need to literally test them all. Having pinpointed 150 -> 151 is likely conclusive, and should be related to my previous theory of it being associated with the sleep calls. Now to figure out why what I had you try before was inadequate (or if there was another change between those versions that I'm forgetting).

For a quick and dirty test while I dig, in the JK2022 executable:
- at offset 0x10E1AA you should see
FF 25 A4 04 8F 00
- change to
C2 04 00 90 90 90

That should disable all sleeps in any main logic loops that I added.

QM

I don't think that did anything I tried changing both 0x10E016 and 0x10E1AA offsets to the values you said but neither worked alone nor together.

Just out of curiosity, what are those sleeps for?

They keep the game from pointlessly pegging a CPU core to 100%; on a CPU from this millennium, JK can use less than 1% at the pause menu if allowed to idle. A 1ms sleep is inserted in 3 places: in the pause menu message pump (both when in a level, or when not) and in the full screen rendering loop. If that is what's giving you trouble, my current hypothesis is that Linux is either pickier about minimum sleep time (JK asks for 1ms accuracy but maybe Linux ignores that?) or is handling Windows' window message emulation in a way that stuffs up the sleep logic. Buuut... the second change I suggested, the one at 0x10E1AA, just disables all 3 sleeps, so behavior should be back like vanilla, so I need to identify what else changed between rev 150 and 151 that might be the cause; it's entirely possible it has nothing to do with the sleeps.

QM

Edit: There isn't a ton changed between 150 and 151; I just reorganized how the sleeps are patched in. At first glance I'm confused how it makes any difference. I'll set up a patch for you to apply tomorrow or something. Are you fine with (i.e. able to use in Linux) the bspatch utility I use in the patch distribution, or would some other patch format be your preference (e.g. IPS)? Or heck, if you want to keep patching these by hand, they are small, the next thing to try will be 20 bytes, so, up to you.

TL;DR version:

On Windows, the "timer" to which timeBeginPeriod/NtSetTimerResolution referred isn't just some "system clock", but rather the programmable interval timer (the clock on the motherboard that generates hardware interrupts, periodically switching control from applications back to the operating system kernel). Calling these functions has a direct effect on latency:

- USB devices are polled by kernel code, which now runs more often.
- Processes end up with slightly less total CPU time, but they wake up more often.

Sleep() of any length blocks the calling process and returns control to the kernel, which will schedule other processes to run. That is the real minimum waiting period, and that's what timeBeginPeriod limits by way of the second point.

That's how things work on Windows. Architecturally, however, Linux is a totally different operating system. There's no way to actually implement this version of timeBeginPeriod/NtSetTimerResolution on Linux, so Wine... doesn't:

https://github.com/wine-mirror/wine/blob/42100f2de6b958f4b6ea9c99dbb9d7c766b6349d/dlls/winmm/time.c#L372

But this is fine, because literally everything I said above is Windows implementation details. You aren't supposed to rely upon any of this stuff, and for the most part, programs don't. The implementation details only start leaking in situations like this, where someone is trying to do something cheesy because doing it the right way is too hard (windows events, swapchain+vsync).

You aren't even supposed to know this stuff. It's forbidden knowledge. I command you to forget it the instant you finish reading this post.

It's odd to me that I also suffer from keyboard input lag with JK13, though. Also odd that it only seems to affect keyboard input but not mouse input.
IIRC last time I had to use an old Windows install for patching the executable, so I think hex editing will do fine.

While sleeps aren't useful for precise timing, yielding for a hopeful 1ms when you know you're not doing anything for 10+ms makes a world of difference for CPU usage. As far as I recall, Sleep(0) behavior changed between the Win9x lineage and WinNT, where on the former it actually yields, while on the latter it basically never yields (there are theoretical circumstances where it will, but they aren't typical real world circumstances), so to give any sort of benefit on modern Windows, when patching sleeps into old software to try and get it to behave better, all you can (realistically) do is Sleep(1) and make sure something called timeBeginPeriod (or equivalent).

To make matters stupider, after all these decades, Windows wedged in a change to timeBeginPeriod maybe to try and have something like program specific accuracy, or maybe some misguided battery saving measure?
https://randomascii.wordpress.com/2020/10/04/windows-timer-resolution-the-great-rule-change/
It only outright breaks things when you relied on, say, a helper program that set the timer accuracy for the benefit of another program, but timeBeginPeriod was documented as having a global effect, so this wasn't some change to implementation details, it was just a breaking change.

offset 0x10E015 before / after
E8 6A 01 00 00 33 C0 39 2D C4 5E 85 00
E8 E6 4D EF FF 33 C0 E8 63 01 00 00 90

offset 0x10E184 before / after
E8 77 4C EF FF E9 78 FF FF FF 90 90
E8 7D FF FF FF 39 2D C4 5E 85 00 C3

Try that both on JK2022 and on rev 151 and test the in-game behavior, if you would please.

QM

Oh yeah, that absolutely fixed it on JK22.

I guess I kind of see it like this: if it's this specific sort of situation, where you're trying to hack it into an old codebase, it's probably fine. I can't imagine spending the effort to do it any other way. If you're starting from scratch, though, Windows already offers a ton of answers for this problem, tailored for specific use cases, and you should definitely use one of those instead of trying to do it yourself. Like presentation intervals in your graphics API, WM_TIMER (these messages coalesce), etc. If nothing else, it makes it less likely that the skeleton crew of interns maintaining the NT kernel will break your app.

Certainly, sleeping and hoping you don't miss a good timing window is a terrible approach, it's just often what's viable when the original code wasn't designed with any yielding. It literally couldn't take certain approaches because they didn't exist yet (like presentation intervals; as far as I know that started with DX9), and if I had to guess, they weren't going to waste any logic on yielding when the target framerate wasn't met on typical hardware of the time anyway. If I had infinite time and interest, I'm not even sure what ideal approach I'd take to retrofit JK.

SMLiberator, It's mildly baffling why that improves things; it's actually more sleep heavy that way. I need to do some testing to make sure it's not one of the variations that hurts framerate on some versions of Windows when holding keypresses. Also, are there still keyboard / mouse sensitivity issues for your driving stuff? I'd like to at least pin down the cause of that (framerate turn speed normalizing from OpenJKDF2?) so it can be toggled with a setting, or something.

QM

I did not manage to fix it (or kind of did by treating any rotational input as keyboard input). Previously, I differentiated keyboard left/right arrow keys from mouse movement by their value, as keyboard input is discrete (either 0 or 90 or -90) and mouse input is somewhat analog. However with keyboard turning speed being linked to FPS (therefore variable) I can no longer differentiate keyboard from mouse input.

A toggle would work well enough I suppose

Okay, let's see if this is all it takes to restore vanilla keyboard turning:
offset 0xD7D15 before / after
E8 26 B6 F6 FF 90
D8 0D 04 2F 52 00
See Edit 4, 4 posts down.

If so, I'll get some sort of toggle hooked up to that. I'm leaning towards a launch argument + in-game console command. I could do it via some UI checkbox or some unused COG accessible setting bit. The UI route is a boatload of work and not something I want to do. The COG route is tempting, but would need to figure out which global setting bit is available. Interested in what you'd actually want to use.

QM

Will test ASAP.

On a completely different note, are you willing to try and fix some issues and limits present in base JK, like actors only rotating using turnright [/SIZE] (submode 31) as a rotate anim regardless of which way they're turning? Also the FOV issue (and 32 iterations limit) with FirstThingInView.

There's also a problem with 0x10 physics flags for actors (Object will align itself with the floor surface it's attached to) but I assume that'd be much much harder to fix

Willing to try? Yes. No promises though.

Actors do only use submode 31 when their AI mode is "turning" (though I think 31 is turnleft, but your point stands regardless). There are a lot of unknowns I'd have to look into here. Does an actor actually give any indication of the direction they're turning (rotational velocity? etc.)? Do actors with a turnleft always have a turnright (32) and if not does the game already have a failsafe or will I have to check? Stuff like that.

What FOV issue with FirstThingInView? I can take a peek at the iteration limit.

What problem with actor physics flag 0x10? Something like it doesn't align them if they're already attached or something else?

Right now for the vanilla keyboard turning, I'm leaning towards using SetSubModeFlags to toggle behavior via COG. Probably 0x80000000. Only 0x1 and 0x8 are used by the game (verified via debugger), and no other values would have side effects or be clobbered by the game as far as I see, with the exception that I don't think they save / load (looks like they always clear to 0), so a singleplayer COG will need to cope with that to be robust.

QM

Edit: To answer some of my own questions:
- Actors don't give an indication of turn direction; the AI code checks if their current orientation matches their desired orientation, and if not, uses their maxrotthrust value to face them (using direct look vector transformations) towards their desired orientation frame by frame until it does. No actual rotation velocity (or thrust) is applied.
- Actors don't always have a turnright puppet entry and there is no fallback to turnleft if turnright isn't specified.
- FirstThingInView populates a static buffer with space for 32 entries. There is no space before or after to expand it so would need to be relocated or re-engineered to allow for more than 32.

You investigated the matters better than I could. I assume the first point explains the problem with 0x10 physics flags, as it probably resets the U vector/thing pitch and roll.

Re.: the FOV issue, FTIV does not work properly for FOV values above 180. I'm not sure what happens then, but values >180 behave as if they were lower. The issue also exists in MotS and is even documented in certain cogs.

Also another question: could you link some other toggles in the UI to SubModeFlags? Like always run, map rotation and the such

If I'm inferring correctly, you're saying the physics flag 0x10 issue is that an actor attached to a non-perfectly-upright surface doesn't stay aligned, that they stay upright? What I saw in the AI turning code definitely explains that, because yes, like when using SetThingLook in COG, PYR are functionally thrown out. To "fix" it would require checking that no other AI/actor code sets look vector, and assuming it doesn't, then making new AI turning code from scratch because what's there probably isn't meaningfully adaptable.

For FTIV's FOV, the values are fed through a trigonometric helper function to get sin/cos. More specifically, it disallows a FOV value less than zero, halves the FOV value, and subtracts it from 90, then passes the result in to a sin/cos function. It's pretty fundamental to how it works and not technically a bug, just doesn't do what pretty much anyone wishes it did (allow searching up to a full 360 degrees). To extend functionality would surely be tricky to not break compatibility with existing COGs, and then COGs using the new behavior wouldn't work correctly vanilla (same input can produce wildly different results). Arguably the same compatibility issue with increasing the find limit, but not quite as bad (same basic results with the same input values, just potentially more results).

Interestingly COG's FirstThingInView doesn't do much itself, and mainly reuses AI target finding code; mentioning because when I was looking at AI turning code, I was often scrolling past AI's FTIV code. Just a happy coincidence that the things you asked about were so intertwined in the executable.

QM

---

Edit: So regarding the UI toggles, what are you looking for? Being able to read them, or read and write, and to some extent, why?

Just brainstorming a little, if I were to hook up more player settings UI stuff to an existing COG verb, it'd probably be the Automatic X Weapon flags because the SetAutoPickup verb is faulty anyway (so would have near zero chance of someone having relied on it historically, more info in a sec), and since the GetAutoX/SetAutoX verbs are related to user settings already.

So I recall, a long time ago, there being some discussions about using the unused Automatic X Weapon setting bits to propagate in-level settings between levels. I don't recall if anyone ever released anything that did so, but I think I recall the JKArena stuff considering the technique. So backwards compatibility concerns come into play.

Since SetAutoPickup and SetAutoReload set the wrong settings (while SetAutoSwitch works), and SetAutoReload has to be used to set the Auto Pickup flags (so it might have been used), SetAutoPickup modifies the Auto Switch flags which is already accounted for by the functional SetAutoSwitch verb... sooo... the least likely verb to have been used is SetAutoPickup.

So what I'd probably do is modify SetAutoPickup so that if the high bit is set (0x80000000), then instead use the low two bits (0x1,0x2) to set the Auto Reload flags (it's not super likely that someone ever used SetAutoPickup(0x80000003) or similar) and then use other bits there for other UI settings, and retrieved via GetAutoReload, which would return the high bit set to indicate patched behavior is active.

So, an example of compatibility-minded COG would be:

Also whoa my COG is rusty. I think that'd be correct.

Since vanilla COG can't touch Auto Reload bits, GetAutoReload vanilla should never return values other than 0-3. Any bits used to mess with other settings wouldn't actually be set / cleared in the Auto Reload bits. The high bit would be used to signal that patched behavior is occurring, and SetAutoPickup, even if just being used to set the two real Auto Reload bits, would still need a GetAutoReload() & 0x80000000 test first to make sure you're not clobbering the Auto Switch bits if run by unpatched JK.

Also note that the SetAutoX verbs suck and overwrite their respective settings, and need the COG author to manually or in existing flags. e.g. To change 0x2 to 0x3 requires: SetAutoSwitch(GetAutoSwitch() | 0x1);

---

Edit 2: Looking at the COG extension verbs, I see:
jkgetwindowgui
jkgetverbosity
jkgetsubtitles
jkgetsfxvolume
jkgetscreensize
jkgetrotatemap
jkgetdisablepageflip
jkgetnohud
jkgetmodpath
jkgetmintexsize
jkgetgammalevel
jkgetframerate
jkgetdrawstatus
jkgetdispstats
jkgetdisplayconfig
jkgetdisablecutscene
jkgetdevmode
jkgetdebugoutput
jkgetcrosshair
jkgetbackbufferinsysmem
jkget3daccel
So at least for reading the settings, everything except auto run might be available?

---

Edit 3: Okay, analyzed the singleplayer save/load code enough to be positive SubModeFlags IS saved and loaded, so that'll make usage for toggling keyboard turning behavior easy. It also means it doesn't make sense to use them for user settings. To clarify, I don't consider framerate compensated keyboard turning a user choice; it's on because vanilla behavior is faulty, and this toggle will be for obscure compatibility reasons (e.g. detecting keyboard vs mouse turning).

---

Edit 4: I've been looking at the keyboard turn speed code in depth; ignore the previous post where I suggest a hex edit.
The logic is probably as follows:
- read raw mouse X axis
- multiply by time since last frame
- value will be 0.0 if no mouse input

- read non-raw mouse X axis and keyboard X axis
- multiply by the player's maxrotthrust (usually 180.0)
- multiply by the current move_multiplier, capped at 1.0 (1.0, 0.5 or 0.25)
- if framerate compensated, multiply by (time since last frame / 50.0)

- add the above two together

Things that affect the move_multiplier are crouching, the "slow" (walk) hotkey with auto run off, and standing on water/deepwater surfaces. Also physics flag 0x200000 but according to the DataMaster that flag gets cleared when attached to a surface (haven't verified).

Of note there's a GetThingRotationalThrustMax COG extension verb, but if you're making a level-specific COG, you'd have control over the player's maxrotthrust anyway (via the walkplayer template).

I've not checked the player's rotational velocity via COG yet, but via the debugger, it looks like I'm generally getting 180, 0 or -180; to get 90/-90 did you meet one of the conditions listed above for a 0.5 move_multiplier?

Also note that disabling framerate compensated keyboard turning will also affect mouse turning (but currently not pitch with either nor the center view hotkey).

---

Edit 5: My apologies for how unwieldy this post has become. There are quite a few questions in there for you SMLiberator; I imagine it'll be a pain to tackle them all.

Also, bad news regarding that input delay fix; what seems to fix it for you, murders framerate on held keypresses on Windows, so the change won't be part of any future releases. It's actually faulty code that puts a sleep between each keypress message (the type that goes to the window's message pump, and that starts spamming after the keypress repeat timeout lapses). Here's an IPS file to streamline applying the patch in the future:
https://drive.google.com/file/d/1ArvnSUHDl7lrx9KFukDVwp50gH_7IFck/view

IPS is a mediocre format, but for such a small patch and considering the availability of tools to apply IPS patches, seems adequate.

Does vanilla JK have input delay for you? Because given what you've mentioned about JK13, 2018 and 2022, I would expect it to.

I'll have a test version where SetSubModeFlags(0x80000000); will disable all framerate compensation for input soon (see next edit). It'll also include a raised transparent surface limit (from vanilla 32 raised to 512).

---

Edit 6: And here's a test version:
https://drive.google.com/file/d/16iKDZb16rfqoGOY597kSsWxJS6baXahK/view

This needs to be applied to a JK 2022 executable, not vanilla. It doesn't include the linuxinputsleep patch from edit 5, so apply that as well (or don't, whatever).

The increased transparent surface limit will be obvious as long as you have a test case to check. I'm fairly certain the test level I have you made for me.

For the framerate compensated input flag, you'll need to rig up a COG to call SetSubModeFlags(0x80000000);. Let's pretend it's to control a drivable vehicle; I'd suggest putting SetSubModeFlags in the part where the player enters the vehicle, and ClearSubModeFlags when they exit, and then PrintVector(GetThingRotVel(player)); while driving? While the flag is set, try a save and load to make sure the flag persists through a load.

Let me know if IPS is no good, but there should be multiple IPS patchers available per platform, plus even browser based options.

You're right about 0x10 physics flags for actors, that's exactly what happens. I assume fixing that would be far more difficult than it's worth.

Re.: FirstThingInView, if the "fixes" are feasible, I don't think it not being backwards compatible with unpatched JK is really an issue as there's so much other stuff that requires a patched .exe to work and this would be just another one. Again, if it's feasible, from what you've described those values seem super super hard coded.

Re.: UI toggle detection, I really only need to "get" values but not "set", and the JK Extension verbs that do exist do that well enough (although I assume reading SubModeFlags should be quicker than running Extension verbs if it's something I'm doing every frame?). I think only the "Controls" toggles are missing from the new verbs.
More specifically, I want to get autorun status as it's an useless option in JKGR which should always be set to On, as the mod sets player's maxthrust to something that only makes sense if autorun is On. I want to be able to compensate player's reduced thrust values if autorun is off and use the UI toggle as a switch for something else.

I made a mistake re: -90/90 values, should've been -180/180 indeed.

Testing if I get input lag with unpatched JK will be hard as I don't have a copy of the original .exe and getting anything to lower my fps to <10 in a level that runs with unpatched JK will be hard

IPS patches are ok, SubModeFlags worked perfectly as you said (enabling and disabling as you get in and out of a vehicle) as did the transparency limits.



Also, I'm sure I've asked this often enough before but is there a reason why Get/Set verbs for thing Puppet, Sprite and Soundclass don't exist nor where added by JK Extension?

My concern with FirstThingInView compatibility (or any compatibility) is a patched JK getting different results with existing COGs than vanilla; I don't care about new COGs written to work with the patched executable not working vanilla, that's on the COG author to include a fallback for if they feel like it. It seems likely safe to increase the find limit (I need to find space to use, but I expect to, like with the transparent surface limit), but I think the FOV will just continue to require the COG author to do two FTIVs if they want more than 180 degrees.

Which UI settings are missing from COG verbs? I may be forgetting something, but my current list is:
Enable auto-aiming
Enable Free look
Enable automatic view centering
Always Run

The simple answer regarding performance is that COG extension verbs should be no different in performance to vanilla ones that do the same thing, and the following hypothetical code:
if(GetAutoReload() & 0x40000000) Print("Always Run is ON");
would be slower than
if(jkGetAlwaysRun()) Print("Always Run is ON");
because "& 0x40000000" in COG is as slow as whatever those verbs do, and because compositing values into the GetAutoReload() result would be the tiniest bit more native code than a single purpose jkGetAlwaysRun().

Saving the results of GetAutoReload() to a variable and doing various bit tests on it would still always result in slower code than single purpose functions because each action in the COG VM is way more native code executed than some simple x86 machine code to read a 32-bit int, maybe do some bitwise instruction on it, and stuff a result on the COG VM stack.

The nuanced answer is that the version of the COG extension DLL included with JK 2023, while having the most features of any I've come across, also is anti-optimized, so in reality any COG extension verb from it is a tiny bit slower than a vanilla one, but not by any amount to consider when writing COG. I wish I had the COG extension DLL source code so I could recompile it.

On that topic, I did finally figure out how it was likely meant to be patched in for the rest of the COG hotkey message hooking so the next version I release will also allow reacting to and blocking axis input (movement & turning).

In the JK_2022 download, in _setup\_files\redist, is Jkupd101.exe and a batch file (extract_jk101.bat) to extract the vanilla JK executable. It rips a CAB file out of the EXE (starts at offset 0xFA00), then uses extrac32.exe (standard in Windows) to extract jk.exe from the CAB.

But instead of doing it manually, 7-Zip can open Jkupd101.exe directly to extract jk.exe. Maybe other multipurpose archive programs can as well.

And finally, not sure regarding Puppet, Sprite and Soundclass. Have you tried using ParseArg to change any of those on a thing? I know I did decades ago but forget the result... According to code, it can try (it knows the keywords "puppet", "sprite" and "soundclass" and has specific code for each), so if it fails in ParseArg, there's likely some complex reason a dedicated verb pair don't exist.

QM

I think also missing are:
Music Volume
Max Sound Channels
Enable A3D Sound

I got the unpatched .exe but can't run here, game either crashes at startup or upon loading a level

ParseArg works fine, but there's still nothing to replace Get verbs. Also ParseArg takes a string as a param, every other thing property verb takes and returns ints

https://drive.google.com/file/d/1QVH55Ps-OW4YR95_b5tQRVEo965gnC6I/view

Added continuous hotkey support for the COG hotkey message. Which means support for "axis" controls such as movement and turning.

Added a hacky magic COG syntax experiment to access the contents of the jkString text buffer for use by other verbs that want a string. A minimal example is:

(Note that the Ü needs to be 1-byte and not unicode. Make sure whatever text editor you use doesn't lose its mind if you add extended ascii to a COG file.)

This example would show the message Test at the top of the screen. Capital U-umlaut (Ü) is actually equivalent to a quotation mark (") to the COG parser due to a flaw in the parser. No sane person was likely even aware, and no one who was also used ÜÜ for an empty string, so I'm using it as a magical value to create a string symbol that points to the jkStringX-verbs string. In vanilla, ÜÜ is treated the same as "" (an empty string).

Was mainly imagining using it to build a string for ParseArg(dummy, ÜÜ);.

Here's an example for the hotkey message in use:
https://drive.google.com/file/d/1vl1RGqT5k83qbR8KLZ1fknib6-b4n1mI/view

Uncomment the following in the hotkey message section to get info about expected values for GetSenderId() and GetParam(0).

...and activate the IR Goggles item in game.

Uhm, try not to go insane if you look at that COG; it has some pretty crazy stuff going on. It's just the only one where I'd used SetHotkeyCog(GetSelfCog()); so it's a functional example. As is it passes checksum.

My point regarding ParseArg was that if all three of those attributes work when set via ParseArg, making a Set verb for them is likely reasonably possible. But if any of them don't work when set via ParseArg, making a Set verb for that attribute may be tricky.

QM

Something in this latest patch seems to have broken the turning function terribly. It's like it's randomly ignoring mouse X axis input if moving mouse leftwards, and even without SubModeFlags set it seems like variations in FPS have a crazy effect in keyboard turning, like the player will almost not turn if framerates are too low. I do not know if that is caused by an incompatibility with the patches I applied previously (and was dumb enough to not make a backup before I applied them)

That behavior is also observed without mods so I know it's not caused by JKGR's implementation of the Hotkey message nor its use of the custom SubMode flags. The unusual keyboard turning behavior does not happen however if the custom SubMode flags are set (maybe that's just the result of scaling turning speed with FPS, and that's how it's supposed to be?), however the mouse X axis issue seems to happen even more frequently (sometimes even when turning right?). Will investigate further.


Edit.: the same sort of -ckery happens when moving mouse cursor in the Y axis as well, and it apparently happens in both directions of the X axis (just maybe more frequently when turning left?)

I'm assuming the keyboard turning behaviour is how it's supposed to be so it's not an actual issue, just different behaviour to what I'm used to.