fix(gta): restore wiped GTA foundation and gameplay path (PPSA04264) (#650)

* fix(kernel): implement APR ResolveFilepathsWithPrefixToIdsAndFileSizes

Resource streamers resolve relative paths against a shared prefix; without
this HLE every call returned NOT_FOUND and assets never got real ids/sizes.

* fix(remoteplay): stub Initialize and GetConnectionStatus as disconnected

Titles probe Remote Play during pad/network bring-up; unresolved imports
returned NOT_FOUND. Report initialized + disconnected so callers take the
normal offline path.

* fix(agc): accept Gen5 hull shaders that omit PGM_LO/HI in CreateShader

Type-5 headers can start with RSRC1/RSRC2; rejecting them left null handles
and Main Thread AVs. Scan the SH table and skip PGM patch when absent.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(kernel): reject getdents on file fds and emit . / .. for empty dirs

Returning rax=0 for non-directory or empty listings looked like EOF and
let GTA treat the fd as a pointer (fiWriteAsyncDataWorker AV at 0xB1).

* fix(hle): enable GuestImageWriteTracker CPU sync on Windows

Windows previously hard-disabled the tracker, so CPU-written guest
planes never marked dirty and host textures stayed empty. Arm pages
with VirtualProtect, handle write AVs in VEH, and warm/test on
VirtualAlloc memory so protect cannot poison the CRT heap.

* fix(agc): skip CB metadata draws for EliminateFastClear/Fmask/DCC

CB_COLOR_CONTROL modes 2/5/6 are colour-buffer metadata ops; applying
the bound shader as a normal colour draw corrupts subsequent composites.
Decode MODE from bits [6:4] and return before translate.

* fix(agc): merge Prospero attrib-table formats onto IR vertex inputs

IR-discovered BufferLoadFormat often keeps a stale float sharp format;
patch DataFormat/offset from the AGC attrib table (semantic index),
allow offen fetches, and map quirks 113/121 through NarrowVk for host
vertex input.

* fix(audio): harden AudioOut2 stack out-buffer writes against canary smash

Titles that stack-allocate AudioOut2 outs next to the frame canary were
corrupted by oversized or mistyped HLE writes; keep ContextPush pacing.

* Revert "fix(memory): reserve only large regions (#608)"

This reverts commit 8f9456229a.

* fix(gpu): decode Gen5 R16 and RG32 render-target formats

* fix(audio): AudioOut2 host beds, deeper waveOut queue, AJM MP3

GTA V Enhanced routes intro/menu audio through AudioOut2 and FMOD's AJM
MP3 path. Wire PortCreate/PortSetAttributes/ContextPush to dual host
stereo streams, deepen WinMM queue to 128KiB, and decode AJM codec 0
with a stateful NLayer helper so menu music is not silent.

* fix(agc): map PS interpolants via SPI_PS_INPUT_CNTL semantics

Identity ATTR→param wiring ignored hardware remapping, so UI draws
got wrong (or empty) interpolants. Pack CNTL from matched PS/GS
semantics, thread it into Vulkan/Metal as Location/Flat, and fingerprint
it in the graphics shader cache key.

* fix(agc): rect-list/NGG strips, Index8 expand, and GE_INDX_OFFSET

NGG single-rect UI needs triangle-strip expansion; Prospero Index8 must
expand to host u16; glyphs need base vertex from GE_INDX_OFFSET. Skip
param-less rect-lists instead of inventing colour draws.

* fix(np): report GTA Story Mode addcont entitlements as owned

NpEntitlementAccess was returning an empty add-on list, so GTA V Enhanced offered Buy Story Mode. Publish the installed license labels and stub premium-event registration so offline sessions take the owned path.

* fix(cpu): prefer native workers for all guest entry stubs

Route thread entry, continuation, and main entry through RunGuestEntryStub so guest stubs are not invoked above CLR-managed frames (UnmanagedCallersOnly FailFast). Keep requireNativeWorker for tbb_thead; other paths prefer workers with calli fallback.

* fix(agc): implement Rewind/Jump writers and IT_REWIND waits

GTA Subrender AVs came from AcbJumpGetSize / DcbRewind returning NOT_FOUND as packet sizes. Add IT_REWIND and INDIRECT_BUFFER writers, patch SetRewindState into the GPU wait registry, and nest-parse 4-dword jumps.

* fix(gpu): use AddrLib ExactXor for Gen5 Standard256B (mode 1)

Mode 5 already had Standard4K ExactXor; mode 1 still used the generic StandardSwizzle block table, which mis-detiles Gen5 UI atlases.

* Revert "fix(cpu): prefer native workers for all guest entry stubs"

This reverts commit 31c4db0d38.

* fix(memory): commit-first large maps; reserve only on failure

Replace the #608 always-reserve-only exact-map path with allocate-first and lazy reserve fallback when a huge non-exec commit cannot be satisfied. Prime and widen GetPointer commit so the fallback path is safer for native walkers. Drops the need for a hard #608 revert.

* [Agc] Implement fused shader half exports

* fix(agc): accept optional hull state in CreatePrimState

Port the CreatePrimState hull-optional path from #583 so fused HS pipelines (GTA) are not rejected with INVALID_ARGUMENT. Geometry-derived CX/UC writes are unchanged; hull is traced only.

* fix(videoout): restore thread-safe VulkanHostBufferPool (#564)

The 6db095e wipe dropped CasualcoderDev's lock-ordering-safe pool. Concurrent Return/TryTake without the gate races after the first present and can hang the submit path.

* Revert "fix(agc): implement Rewind/Jump writers and IT_REWIND waits"

This reverts commit bec77bf083.

* test(memory): align lazy-commit expectations with commit-first policy

Fake hosts must reject Allocate so reserve-only paths still run, and GetPointer asserts the 32 MiB prime range including AlignUp spill.

* diag(gpu): log guest-queue backlog breakdown under backpressure

Rate-limit top work types and ordered debugName prefixes when the Vulkan guest work queue stalls, so North Yankton logs show acquire/label vs draw traffic instead of only VulkanOrderedGuestAction.

* perf(agc): coalesce acquire flushes and batch non-DMA label wakes

Flush pending ACQUIRE_MEM invalidation at draw/dispatch/dma/flip boundaries instead of before every packet, and complete release/write-data producers in the same ordered action so load paths enqueue far fewer VulkanOrderedGuestAction items.

* perf(gpu): wait for ordered-action fences and keep draining sync

On Windows/Linux, block briefly for queue-visibility fences instead of deferring the whole logical queue for the tick. Prefer ordered sync/flip heads under backlog pressure, and keep macOS non-blocking defer behavior.

* perf(gpu): raise sync-item ceiling above payload guest-work cap

Apply SHARPEMU_PENDING_GUEST_WORK_ITEMS mainly to compute/draw/image payload work, and allow a higher SHARPEMU_PENDING_GUEST_SYNC_ITEMS ceiling for zero-payload ordered actions and flip markers. Keep the byte budget as the RAM safety valve.

* fix(gta): stub Voice ports and implement sceKernelCheckReachability

Resolve North Yankton-path Voice Create/Delete/Connect/Disconnect/End NIDs and EnumerationThread reachability checks so leftover unresolved imports are not on the critical path.

* diag(gta): arm flip/present/wait probes after North Audio

Rate-limited load_progress TRACE for flip submit, ordered flip enqueue, present taken/not-taken, and GPU wait backlog so North Yankton freezes can be classified without full AGC tracing.

* fix(ampr): restore sequential offset=-1 reads for streamer packs

Re-wire PakDirectoryTracker into sceAmprAprCommandBufferReadFile (dropped in #216) so RAGE sequential pack reads no longer fail while the North Yankton UI keeps flipping. Also rate-limit CheckReachability miss paths for EnumerationThread diagnosis.

* fix(hle/videoout): Windows GuestImage opt-in and keep GTA intro without sync

Default the tracker off on Windows to avoid VirtualProtect thrash, gate AGC
texel-copy skips on Enabled so guest Bink planes keep shipping pixels, and
drain CPU-written images on the present thread when sync is opted in.

* fix(videoout): probe guest content when tracker off so UI can skip copies

Restores upload-known/texture-cache skips for Dead Cells menus, and uses a
sparse guest-memory fingerprint when GuestImageWriteTracker is disabled so
CPU-updated Bink planes still force texel copies for GTA intro.

* fix(audio): keep 128KiB host queue AudioOut2-only

Restore the default 32 KiB (~171 ms) PCM bed for classic AudioOut so
titles like Dreaming Sarah stay in sync; only AudioOut2 opens the deeper
queue needed for bursty FMOD Push on GTA.

---------

Co-authored-by: samto6 <123419830+samto6@users.noreply.github.com>

src/SharpEmu.Core/Cpu/Native/DirectExecutionBackend.Exceptions.cs

@@ -55,6 +55,7 @@ public sealed partial class DirectExecutionBackend
 		}
 		_exceptionHandler = (nint)AddVectoredExceptionHandler(1u, _exceptionHandlerStub);
 		Console.Error.WriteLine($"[LOADER][INFO] Exception handler installed: 0x{_exceptionHandler:X16}");
+		SharpEmu.HLE.GuestImageWriteTracker.WarmUp();
 
 		_unhandledFilterDelegate = UnhandledExceptionFilter;
 		_unhandledFilterHandle = GCHandle.Alloc(_unhandledFilterDelegate);
@@ -117,6 +118,13 @@ public sealed partial class DirectExecutionBackend
 			{
 				return -1;
 			}
+			if (exceptionCode == 3221225477u &&
+				exceptionRecord->NumberParameters >= 2 &&
+				SharpEmu.HLE.GuestImageWriteTracker.TryHandleWriteFault(
+					exceptionRecord->ExceptionInformation[1]))
+			{
+				return -1;
+			}
 			if (TryRecoverAuxiliaryThreadExecuteFault(exceptionRecord, contextRecord, rip))
 			{
 				return -1;

src/SharpEmu.Core/Cpu/Native/DirectExecutionBackend.Imports.cs

@@ -69,6 +69,15 @@ public sealed partial class DirectExecutionBackend
 
 	private unsafe static int RawVectoredHandlerManaged(void* exceptionInfo)
 	{
+		EXCEPTION_RECORD* exceptionRecord = ((EXCEPTION_POINTERS*)exceptionInfo)->ExceptionRecord;
+		if (exceptionRecord->ExceptionCode == 3221225477u &&
+			exceptionRecord->NumberParameters >= 2 &&
+			SharpEmu.HLE.GuestImageWriteTracker.TryHandleWriteFault(
+				exceptionRecord->ExceptionInformation[1]))
+		{
+			return -1;
+		}
+
 		return TryRecoverUnresolvedSentinel(exceptionInfo);
 	}
 

src/SharpEmu.Core/Cpu/Native/DirectExecutionBackend.cs

@@ -13,6 +13,7 @@ using SharpEmu.Core.Cpu.Debugging;
 using SharpEmu.Core.Loader;
 using SharpEmu.Core.Memory;
 using SharpEmu.HLE;
+using SharpEmu.Libs.Diagnostics;
 
 namespace SharpEmu.Core.Cpu.Native;
 
@@ -3651,6 +3652,7 @@ public sealed unsafe partial class DirectExecutionBackend : INativeCpuBackend, I
 			$"[LOADER][INFO] Scheduled guest thread '{thread.Name}' handle=0x{thread.ThreadHandle:X16} " +
 			$"entry=0x{thread.EntryPoint:X16} arg=0x{thread.Argument:X16} priority={thread.Priority} " +
 			$"host_priority={MapGuestThreadPriority(thread.Priority)} affinity=0x{thread.AffinityMask:X}");
+		LoadProgressDiagnostics.ArmIfNorthAudioThread(thread.Name);
 		Pump(creatorContext, "pthread_create");
 		// Pump is suppressed while another cooperative dispatch is active. The
 		// background dispatcher would eventually observe this thread, but an

src/SharpEmu.Core/Memory/PhysicalVirtualMemory.cs

@@ -238,15 +238,22 @@ public sealed unsafe class PhysicalVirtualMemory : IVirtualMemory, IGuestMemoryA
         var alignedSize = (size + 0xFFF) & ~0xFFFUL;
         var protection = executable ? PAGE_EXECUTE_READWRITE : PAGE_READWRITE;
         var hostProtection = executable ? HostPageProtection.ReadWriteExecute : HostPageProtection.ReadWrite;
-
-        // Reserve address space only for very large non-executable regions; commit is done lazily later.
-        var reservedOnly = !executable &&
+        var allowLazyReserve = !executable &&
             alignedSize >= LargeDataReserveThreshold &&
             alignedSize > FullCommitRegionLimit;
 
-        var result = reservedOnly
-            ? _hostMemory.Reserve(desiredAddress, alignedSize, HostPageProtection.ReadWrite)
-            : _hostMemory.Allocate(desiredAddress, alignedSize, hostProtection);
+        // Commit first so titles that walk guest memory via raw host pointers
+        // (GTA post-RenderThread workers) keep fully backed pages. Fall back to
+        // reserve-only + lazy commit only when a huge non-exec commit fails —
+        // that is the Poppy / large-reservation path #608 was aiming for.
+        var reservedOnly = false;
+        var result = _hostMemory.Allocate(desiredAddress, alignedSize, hostProtection);
+        if (result == 0 && allowLazyReserve)
+        {
+            result = _hostMemory.Reserve(desiredAddress, alignedSize, HostPageProtection.ReadWrite);
+            reservedOnly = result != 0;
+        }
+
         if (result == 0)
         {
             return false;
@@ -260,7 +267,7 @@ public sealed unsafe class PhysicalVirtualMemory : IVirtualMemory, IGuestMemoryA
             return false;
         }
 
-        var state = reservedOnly ? ReserveRegion(actualAddress, alignedSize) : "n/a";
+        var lazyPrimeState = reservedOnly ? PrimeLazyReserveRegion(actualAddress, alignedSize) : "n/a";
 
         _gate.EnterWriteLock();
         try
@@ -279,9 +286,12 @@ public sealed unsafe class PhysicalVirtualMemory : IVirtualMemory, IGuestMemoryA
             _gate.ExitWriteLock();
         }
 
-
-        var allocationKind = executable ? "executable memory" : "data memory";
-        TraceVmem($"Allocated exact {allocationKind}: 0x{actualAddress:X16} - 0x{actualAddress + alignedSize:X16} ({alignedSize} bytes)");
+        var allocationKind = reservedOnly
+            ? "reserved data memory (lazy commit)"
+            : (executable ? "executable memory" : "data memory");
+        TraceVmem(
+            $"Allocated exact {allocationKind}: 0x{actualAddress:X16} - 0x{actualAddress + alignedSize:X16} " +
+            $"({alignedSize} bytes) lazy_prime={lazyPrimeState}");
         return true;
     }
 
@@ -312,55 +322,44 @@ public sealed unsafe class PhysicalVirtualMemory : IVirtualMemory, IGuestMemoryA
 
         var protection = executable ? PAGE_EXECUTE_READWRITE : PAGE_READWRITE;
         var hostProtection = executable ? HostPageProtection.ReadWriteExecute : HostPageProtection.ReadWrite;
-        var reservedOnly = false;
-        var preferReserveOnly = !executable &&
+        var allowLazyReserve = !executable &&
             alignedSize >= LargeDataReserveThreshold &&
             alignedSize > FullCommitRegionLimit;
+        var reservedOnly = false;
 
-        ulong result = 0;
-        if (preferReserveOnly)
-        {
-            result = _hostMemory.Reserve(desiredAddress, alignedSize, HostPageProtection.ReadWrite);
-            if (result == 0 && allowAlternative)
-            {
-                result = _hostMemory.Reserve(0, alignedSize, HostPageProtection.ReadWrite);
-            }
-
-            if (result != 0)
-            {
-                reservedOnly = true;
-            }
-        }
-
-        if (result == 0)
-        {
-            result = _hostMemory.Allocate(desiredAddress, alignedSize, hostProtection);
-        }
+        // Prefer a full commit. Only fall back to reserve-only when a large
+        // non-executable commit cannot be satisfied (see TryAllocateAtExact).
+        ulong result = _hostMemory.Allocate(desiredAddress, alignedSize, hostProtection);
 
         if (result == 0)
         {
             if (!allowAlternative)
             {
-                throw new InvalidOperationException($"Failed to allocate exact mapping at 0x{desiredAddress:X16} ({alignedSize} bytes)");
-            }
-
-            TraceVmem($"Could not allocate at 0x{desiredAddress:X16}, trying any address...");
-            result = _hostMemory.Allocate(0, alignedSize, hostProtection);
-
-            if (result == 0)
-            {
-                if (!executable)
+                if (allowLazyReserve)
                 {
                     result = _hostMemory.Reserve(desiredAddress, alignedSize, HostPageProtection.ReadWrite);
-                    if (result == 0 && allowAlternative)
+                    reservedOnly = result != 0;
+                }
+
+                if (result == 0)
+                {
+                    throw new InvalidOperationException($"Failed to allocate exact mapping at 0x{desiredAddress:X16} ({alignedSize} bytes)");
+                }
+            }
+            else
+            {
+                TraceVmem($"Could not allocate at 0x{desiredAddress:X16}, trying any address...");
+                result = _hostMemory.Allocate(0, alignedSize, hostProtection);
+
+                if (result == 0 && allowLazyReserve)
+                {
+                    result = _hostMemory.Reserve(desiredAddress, alignedSize, HostPageProtection.ReadWrite);
+                    if (result == 0)
                     {
                         result = _hostMemory.Reserve(0, alignedSize, HostPageProtection.ReadWrite);
                     }
 
-                    if (result != 0)
-                    {
-                        reservedOnly = true;
-                    }
+                    reservedOnly = result != 0;
                 }
 
                 if (result == 0)
@@ -371,8 +370,7 @@ public sealed unsafe class PhysicalVirtualMemory : IVirtualMemory, IGuestMemoryA
         }
 
         var actualAddress = result;
-
-        var lazyPrimeState = reservedOnly ? ReserveRegion(actualAddress, alignedSize) : "n/a";
+        var lazyPrimeState = reservedOnly ? PrimeLazyReserveRegion(actualAddress, alignedSize) : "n/a";
 
         _gate.EnterWriteLock();
         try
@@ -399,7 +397,11 @@ public sealed unsafe class PhysicalVirtualMemory : IVirtualMemory, IGuestMemoryA
         return actualAddress;
     }
 
-    private string ReserveRegion(ulong actualAddress, ulong alignedSize)
+    /// <summary>
+    /// Commits the leading slice of a reserve-only region so early guest touches
+    /// succeed before on-demand <see cref="EnsureRangeCommitted"/> runs.
+    /// </summary>
+    private string PrimeLazyReserveRegion(ulong actualAddress, ulong alignedSize)
     {
         var primeBytes = Math.Min(alignedSize, LazyReservePrimeBytes);
         if (primeBytes == 0)
@@ -426,11 +428,11 @@ public sealed unsafe class PhysicalVirtualMemory : IVirtualMemory, IGuestMemoryA
             var state = committedBytes == primeBytes
                 ? $"ok:{committedBytes:X}"
                 : $"partial:{committedBytes:X}/{primeBytes:X}";
-            TraceVmem($"region: 0x{actualAddress:X16} - 0x{actualAddress + committedBytes:X16} ({committedBytes} bytes)");
+            TraceVmem($"Primed lazy region: 0x{actualAddress:X16} - 0x{actualAddress + committedBytes:X16} ({committedBytes} bytes)");
             return state;
         }
 
-        TraceVmem($"Failed to reserve region at 0x{actualAddress:X16} ({primeBytes} bytes)!");
+        TraceVmem($"Failed to prime lazy region at 0x{actualAddress:X16} ({primeBytes} bytes), continuing with on-demand commit");
         return $"fail:{primeBytes:X}";
     }
 
@@ -1316,12 +1318,26 @@ public sealed unsafe class PhysicalVirtualMemory : IVirtualMemory, IGuestMemoryA
         try
         {
             var region = FindRegion(virtualAddress, 1);
-            if (region is null ||
-                (region.IsReservedOnly && !EnsureRangeCommitted(virtualAddress, 1, region)))
+            if (region is null)
             {
                 return null;
             }
 
+            // Raw host pointers are walked by native/JIT code without further
+            // EnsureRangeCommitted calls. For reserve-only regions, commit a
+            // leading working-set chunk from this address so the common case
+            // does not immediately AV on the next page.
+            if (region.IsReservedOnly)
+            {
+                var regionEnd = region.VirtualAddress + region.Size;
+                var remaining = regionEnd > virtualAddress ? regionEnd - virtualAddress : 0;
+                var commitBytes = Math.Min(remaining, LazyReservePrimeChunkBytes);
+                if (commitBytes == 0 || !EnsureRangeCommitted(virtualAddress, commitBytes, region))
+                {
+                    return null;
+                }
+            }
+
             return (void*)virtualAddress;
         }
         finally