fix(gta): restore wiped GTA foundation and gameplay path (PPSA04264) (#650)

* fix(kernel): implement APR ResolveFilepathsWithPrefixToIdsAndFileSizes

Resource streamers resolve relative paths against a shared prefix; without
this HLE every call returned NOT_FOUND and assets never got real ids/sizes.

* fix(remoteplay): stub Initialize and GetConnectionStatus as disconnected

Titles probe Remote Play during pad/network bring-up; unresolved imports
returned NOT_FOUND. Report initialized + disconnected so callers take the
normal offline path.

* fix(agc): accept Gen5 hull shaders that omit PGM_LO/HI in CreateShader

Type-5 headers can start with RSRC1/RSRC2; rejecting them left null handles
and Main Thread AVs. Scan the SH table and skip PGM patch when absent.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(kernel): reject getdents on file fds and emit . / .. for empty dirs

Returning rax=0 for non-directory or empty listings looked like EOF and
let GTA treat the fd as a pointer (fiWriteAsyncDataWorker AV at 0xB1).

* fix(hle): enable GuestImageWriteTracker CPU sync on Windows

Windows previously hard-disabled the tracker, so CPU-written guest
planes never marked dirty and host textures stayed empty. Arm pages
with VirtualProtect, handle write AVs in VEH, and warm/test on
VirtualAlloc memory so protect cannot poison the CRT heap.

* fix(agc): skip CB metadata draws for EliminateFastClear/Fmask/DCC

CB_COLOR_CONTROL modes 2/5/6 are colour-buffer metadata ops; applying
the bound shader as a normal colour draw corrupts subsequent composites.
Decode MODE from bits [6:4] and return before translate.

* fix(agc): merge Prospero attrib-table formats onto IR vertex inputs

IR-discovered BufferLoadFormat often keeps a stale float sharp format;
patch DataFormat/offset from the AGC attrib table (semantic index),
allow offen fetches, and map quirks 113/121 through NarrowVk for host
vertex input.

* fix(audio): harden AudioOut2 stack out-buffer writes against canary smash

Titles that stack-allocate AudioOut2 outs next to the frame canary were
corrupted by oversized or mistyped HLE writes; keep ContextPush pacing.

* Revert "fix(memory): reserve only large regions (#608)"

This reverts commit 8f9456229a.

* fix(gpu): decode Gen5 R16 and RG32 render-target formats

* fix(audio): AudioOut2 host beds, deeper waveOut queue, AJM MP3

GTA V Enhanced routes intro/menu audio through AudioOut2 and FMOD's AJM
MP3 path. Wire PortCreate/PortSetAttributes/ContextPush to dual host
stereo streams, deepen WinMM queue to 128KiB, and decode AJM codec 0
with a stateful NLayer helper so menu music is not silent.

* fix(agc): map PS interpolants via SPI_PS_INPUT_CNTL semantics

Identity ATTR→param wiring ignored hardware remapping, so UI draws
got wrong (or empty) interpolants. Pack CNTL from matched PS/GS
semantics, thread it into Vulkan/Metal as Location/Flat, and fingerprint
it in the graphics shader cache key.

* fix(agc): rect-list/NGG strips, Index8 expand, and GE_INDX_OFFSET

NGG single-rect UI needs triangle-strip expansion; Prospero Index8 must
expand to host u16; glyphs need base vertex from GE_INDX_OFFSET. Skip
param-less rect-lists instead of inventing colour draws.

* fix(np): report GTA Story Mode addcont entitlements as owned

NpEntitlementAccess was returning an empty add-on list, so GTA V Enhanced offered Buy Story Mode. Publish the installed license labels and stub premium-event registration so offline sessions take the owned path.

* fix(cpu): prefer native workers for all guest entry stubs

Route thread entry, continuation, and main entry through RunGuestEntryStub so guest stubs are not invoked above CLR-managed frames (UnmanagedCallersOnly FailFast). Keep requireNativeWorker for tbb_thead; other paths prefer workers with calli fallback.

* fix(agc): implement Rewind/Jump writers and IT_REWIND waits

GTA Subrender AVs came from AcbJumpGetSize / DcbRewind returning NOT_FOUND as packet sizes. Add IT_REWIND and INDIRECT_BUFFER writers, patch SetRewindState into the GPU wait registry, and nest-parse 4-dword jumps.

* fix(gpu): use AddrLib ExactXor for Gen5 Standard256B (mode 1)

Mode 5 already had Standard4K ExactXor; mode 1 still used the generic StandardSwizzle block table, which mis-detiles Gen5 UI atlases.

* Revert "fix(cpu): prefer native workers for all guest entry stubs"

This reverts commit 31c4db0d38.

* fix(memory): commit-first large maps; reserve only on failure

Replace the #608 always-reserve-only exact-map path with allocate-first and lazy reserve fallback when a huge non-exec commit cannot be satisfied. Prime and widen GetPointer commit so the fallback path is safer for native walkers. Drops the need for a hard #608 revert.

* [Agc] Implement fused shader half exports

* fix(agc): accept optional hull state in CreatePrimState

Port the CreatePrimState hull-optional path from #583 so fused HS pipelines (GTA) are not rejected with INVALID_ARGUMENT. Geometry-derived CX/UC writes are unchanged; hull is traced only.

* fix(videoout): restore thread-safe VulkanHostBufferPool (#564)

The 6db095e wipe dropped CasualcoderDev's lock-ordering-safe pool. Concurrent Return/TryTake without the gate races after the first present and can hang the submit path.

* Revert "fix(agc): implement Rewind/Jump writers and IT_REWIND waits"

This reverts commit bec77bf083.

* test(memory): align lazy-commit expectations with commit-first policy

Fake hosts must reject Allocate so reserve-only paths still run, and GetPointer asserts the 32 MiB prime range including AlignUp spill.

* diag(gpu): log guest-queue backlog breakdown under backpressure

Rate-limit top work types and ordered debugName prefixes when the Vulkan guest work queue stalls, so North Yankton logs show acquire/label vs draw traffic instead of only VulkanOrderedGuestAction.

* perf(agc): coalesce acquire flushes and batch non-DMA label wakes

Flush pending ACQUIRE_MEM invalidation at draw/dispatch/dma/flip boundaries instead of before every packet, and complete release/write-data producers in the same ordered action so load paths enqueue far fewer VulkanOrderedGuestAction items.

* perf(gpu): wait for ordered-action fences and keep draining sync

On Windows/Linux, block briefly for queue-visibility fences instead of deferring the whole logical queue for the tick. Prefer ordered sync/flip heads under backlog pressure, and keep macOS non-blocking defer behavior.

* perf(gpu): raise sync-item ceiling above payload guest-work cap

Apply SHARPEMU_PENDING_GUEST_WORK_ITEMS mainly to compute/draw/image payload work, and allow a higher SHARPEMU_PENDING_GUEST_SYNC_ITEMS ceiling for zero-payload ordered actions and flip markers. Keep the byte budget as the RAM safety valve.

* fix(gta): stub Voice ports and implement sceKernelCheckReachability

Resolve North Yankton-path Voice Create/Delete/Connect/Disconnect/End NIDs and EnumerationThread reachability checks so leftover unresolved imports are not on the critical path.

* diag(gta): arm flip/present/wait probes after North Audio

Rate-limited load_progress TRACE for flip submit, ordered flip enqueue, present taken/not-taken, and GPU wait backlog so North Yankton freezes can be classified without full AGC tracing.

* fix(ampr): restore sequential offset=-1 reads for streamer packs

Re-wire PakDirectoryTracker into sceAmprAprCommandBufferReadFile (dropped in #216) so RAGE sequential pack reads no longer fail while the North Yankton UI keeps flipping. Also rate-limit CheckReachability miss paths for EnumerationThread diagnosis.

* fix(hle/videoout): Windows GuestImage opt-in and keep GTA intro without sync

Default the tracker off on Windows to avoid VirtualProtect thrash, gate AGC
texel-copy skips on Enabled so guest Bink planes keep shipping pixels, and
drain CPU-written images on the present thread when sync is opted in.

* fix(videoout): probe guest content when tracker off so UI can skip copies

Restores upload-known/texture-cache skips for Dead Cells menus, and uses a
sparse guest-memory fingerprint when GuestImageWriteTracker is disabled so
CPU-updated Bink planes still force texel copies for GTA intro.

* fix(audio): keep 128KiB host queue AudioOut2-only

Restore the default 32 KiB (~171 ms) PCM bed for classic AudioOut so
titles like Dreaming Sarah stay in sync; only AudioOut2 opens the deeper
queue needed for bursty FMOD Push on GTA.

---------

Co-authored-by: samto6 <123419830+samto6@users.noreply.github.com>

tests/SharpEmu.Libs.Tests/Memory/GuestImageWriteTrackerTests.cs

@@ -24,13 +24,53 @@ public sealed unsafe class GuestImageWriteTrackerTests
     // spilling onto neighbouring heap pages.
     private const nuint TrackedByteCount = 4096;
     private const nuint HostPageAlignment = 16384;
+    private const uint MemCommit = 0x1000;
+    private const uint MemReserve = 0x2000;
+    private const uint MemRelease = 0x8000;
+    private const uint PageReadWrite = 0x04;
+
+    [DllImport("kernel32.dll", SetLastError = true)]
+    private static extern nint VirtualAlloc(
+        nint lpAddress,
+        nuint dwSize,
+        uint flAllocationType,
+        uint flProtect);
+
+    [DllImport("kernel32.dll", SetLastError = true)]
+    private static extern int VirtualFree(nint lpAddress, nuint dwSize, uint dwFreeType);
 
     private static ulong AllocateTrackedPages(out void* allocation)
     {
+        // VirtualProtect (Windows) / mprotect (POSIX) must target
+        // VirtualAlloc/mmap pages. Protecting CRT heap pages poisons
+        // neighbouring allocator metadata and crashes the test host.
+        if (OperatingSystem.IsWindows())
+        {
+            var windowsAllocation = VirtualAlloc(
+                0,
+                HostPageAlignment,
+                MemCommit | MemReserve,
+                PageReadWrite);
+            Assert.NotEqual(nint.Zero, windowsAllocation);
+            allocation = (void*)windowsAllocation;
+            return (ulong)windowsAllocation;
+        }
+
         allocation = NativeMemory.AlignedAlloc(2 * HostPageAlignment, HostPageAlignment);
         return (ulong)allocation;
     }
 
+    private static void FreeTrackedPages(void* allocation)
+    {
+        if (OperatingSystem.IsWindows())
+        {
+            _ = VirtualFree((nint)allocation, 0, MemRelease);
+            return;
+        }
+
+        NativeMemory.Free(allocation);
+    }
+
     [Fact]
     public void GenerationSurvivesDirtyConsume()
     {
@@ -58,7 +98,7 @@ public sealed unsafe class GuestImageWriteTrackerTests
         finally
         {
             GuestImageWriteTracker.Untrack(address);
-            NativeMemory.Free(allocation);
+            FreeTrackedPages(allocation);
         }
     }
 
@@ -89,7 +129,7 @@ public sealed unsafe class GuestImageWriteTrackerTests
         finally
         {
             GuestImageWriteTracker.Untrack(address);
-            NativeMemory.Free(allocation);
+            FreeTrackedPages(allocation);
         }
     }
 
@@ -118,7 +158,7 @@ public sealed unsafe class GuestImageWriteTrackerTests
         finally
         {
             GuestImageWriteTracker.Untrack(address);
-            NativeMemory.Free(allocation);
+            FreeTrackedPages(allocation);
         }
     }
 
@@ -132,4 +172,132 @@ public sealed unsafe class GuestImageWriteTrackerTests
 
         Assert.False(GuestImageWriteTracker.TryGetWriteGeneration(0xDEAD_0000_0000UL, out _));
     }
+
+    [Fact]
+    public void WatchOnlyTrackDoesNotArmWriteProtection()
+    {
+        if (!GuestImageWriteTracker.Enabled)
+        {
+            return;
+        }
+
+        var address = AllocateTrackedPages(out var allocation);
+        try
+        {
+            GuestImageWriteTracker.Track(
+                address,
+                TrackedByteCount,
+                source: "test.watch-only",
+                protect: false);
+            Assert.True(
+                GuestImageWriteTracker.TryGetProtectionState(
+                    address,
+                    out var protect,
+                    out var armed));
+            Assert.False(protect);
+            Assert.False(armed);
+
+            // Pages stay writable: a native store must not require a fault handler.
+            *(byte*)address = 0xAB;
+            Assert.Equal(0xAB, *(byte*)address);
+        }
+        finally
+        {
+            GuestImageWriteTracker.Untrack(address);
+            FreeTrackedPages(allocation);
+        }
+    }
+
+    [Fact]
+    public void WatchOnlyRangesAreExcludedFromManagedWriteSnapshot()
+    {
+        if (!GuestImageWriteTracker.Enabled)
+        {
+            return;
+        }
+
+        var address = AllocateTrackedPages(out var allocation);
+        try
+        {
+            GuestImageWriteTracker.Track(
+                address,
+                TrackedByteCount,
+                source: "test.watch-only",
+                protect: false);
+            // Watch-only must not widen the NotifyManagedWrite hot path.
+            GuestImageWriteTracker.NotifyManagedWrite(address, sizeof(uint));
+            Assert.False(GuestImageWriteTracker.ConsumeDirty(address));
+            Assert.True(
+                GuestImageWriteTracker.TryGetProtectionState(
+                    address,
+                    out var protect,
+                    out var armed));
+            Assert.False(protect);
+            Assert.False(armed);
+        }
+        finally
+        {
+            GuestImageWriteTracker.Untrack(address);
+            FreeTrackedPages(allocation);
+        }
+    }
+
+    [Fact]
+    public void ProtectedTrackArmsWriteProtection()
+    {
+        if (!GuestImageWriteTracker.Enabled)
+        {
+            return;
+        }
+
+        var address = AllocateTrackedPages(out var allocation);
+        try
+        {
+            GuestImageWriteTracker.Track(address, TrackedByteCount);
+            Assert.True(
+                GuestImageWriteTracker.TryGetProtectionState(
+                    address,
+                    out var protect,
+                    out var armed));
+            Assert.True(protect);
+            Assert.True(armed);
+        }
+        finally
+        {
+            GuestImageWriteTracker.Untrack(address);
+            FreeTrackedPages(allocation);
+        }
+    }
+
+    [Fact]
+    public void WatchOnlyTrackDoesNotDowngradeProtectedRange()
+    {
+        if (!GuestImageWriteTracker.Enabled)
+        {
+            return;
+        }
+
+        var address = AllocateTrackedPages(out var allocation);
+        try
+        {
+            GuestImageWriteTracker.Track(address, TrackedByteCount, source: "test.rt");
+            GuestImageWriteTracker.Track(
+                address,
+                TrackedByteCount,
+                source: "test.texture-cache",
+                protect: false);
+            Assert.True(
+                GuestImageWriteTracker.TryGetProtectionState(
+                    address,
+                    out var protect,
+                    out var armed));
+            Assert.True(protect);
+            Assert.True(armed);
+        }
+        finally
+        {
+            GuestImageWriteTracker.Untrack(address);
+            FreeTrackedPages(allocation);
+        }
+    }
 }

tests/SharpEmu.Libs.Tests/Memory/GuestMemoryAllocatorTests.cs

@@ -79,7 +79,16 @@ public sealed class GuestMemoryAllocatorTests
         var pointer = memory.GetPointer(address + 0x123);
 
         Assert.Equal(address + 0x123, (ulong)pointer);
-        Assert.Equal([(address, pageSize, HostPageProtection.ReadWrite)], host.CommitCalls);
+        // GetPointer commits a 32 MiB working-set chunk as 4 KiB pages against
+        // this fake host (Query reports 4 KiB reserved regions). Non-page-
+        // aligned start makes AlignUp(addr + chunk) cover one extra page.
+        const ulong lazyPrimeChunkBytes = 0x0200_0000UL;
+        var endPage = (address + 0x123 + lazyPrimeChunkBytes + pageSize - 1) & ~(pageSize - 1);
+        Assert.Equal((int)((endPage - address) / pageSize), host.CommitCalls.Count);
+        Assert.Equal((address, pageSize, HostPageProtection.ReadWrite), host.CommitCalls[0]);
+        Assert.All(
+            host.CommitCalls,
+            call => Assert.Equal(pageSize, call.Size));
     }
 
     [Fact]

tests/SharpEmu.Libs.Tests/Memory/PhysicalVirtualMemoryTests.cs

@@ -9,9 +9,10 @@ using Xunit;
 namespace SharpEmu.Libs.Tests.Memory;
 
 // PhysicalVirtualMemory is the host-backed (identity-mapped) implementation.
-// Reserve-only regions (> 4 GiB, non-executable) defer commit until first
-// access; TryAllocateGuestMemory serves a first-fit free-list with coalescing.
-// These tests pin that behaviour through fake IHostMemory implementations.
+// Huge non-executable maps (> 4 GiB) prefer commit-first, then fall back to
+// reserve-only + lazy commit when Allocate fails. TryAllocateGuestMemory serves
+// a first-fit free-list with coalescing. These tests pin that behaviour through
+// fake IHostMemory implementations that refuse full Allocate for huge sizes.
 public sealed class PhysicalVirtualMemoryTests
 {
     // 1. Lazy commit: a reserve-only region has its pages committed on demand
@@ -22,7 +23,8 @@ public sealed class PhysicalVirtualMemoryTests
         using var host = new LazyZeroedHostMemory();
         using var memory = new PhysicalVirtualMemory(host);
 
-        // > 4 GiB, non-executable -> reserve-only with lazy commit.
+        // > 4 GiB, non-executable; fake host rejects Allocate so reserve-only
+        // + lazy commit is used.
         var address = memory.AllocateAt(0, (4UL << 30) + 0x1000, executable: false);
         Assert.NotEqual(0UL, address);
 
@@ -108,8 +110,17 @@ public sealed class PhysicalVirtualMemoryTests
         Assert.NotEqual(0UL, (ulong)pointer);
         Assert.Equal(address + 0x123, (ulong)pointer);
 
+        // GetPointer primes a 32 MiB working-set chunk (page-sized commits
+        // against this fake host's 4 KiB Query regions). Non-page-aligned
+        // start makes AlignUp(addr + chunk) cover one extra page.
+        const ulong lazyPrimeChunkBytes = 0x0200_0000UL;
         var page = (address + 0x123) & ~0xFFFUL;
-        Assert.Equal([(page, 0x1000UL, HostPageProtection.ReadWrite)], host.CommitCalls);
+        var endPage = (address + 0x123 + lazyPrimeChunkBytes + 0xFFFUL) & ~0xFFFUL;
+        Assert.Equal((int)((endPage - page) / 0x1000UL), host.CommitCalls.Count);
+        Assert.Equal((page, 0x1000UL, HostPageProtection.ReadWrite), host.CommitCalls[0]);
+        Assert.All(
+            host.CommitCalls,
+            call => Assert.Equal(0x1000UL, call.Size));
     }
 
     [Fact]
@@ -187,7 +198,8 @@ public sealed class PhysicalVirtualMemoryTests
 
         public int QueryCalls { get; private set; }
 
-        public ulong Allocate(ulong desiredAddress, ulong size, HostPageProtection protection) => _address;
+        // Force the commit-first → reserve-only fallback for huge maps.
+        public ulong Allocate(ulong desiredAddress, ulong size, HostPageProtection protection) => 0;
 
         public ulong Reserve(ulong desiredAddress, ulong size, HostPageProtection protection) => _address;