fix(gta): restore wiped GTA foundation and gameplay path (PPSA04264) (#650)

* fix(kernel): implement APR ResolveFilepathsWithPrefixToIdsAndFileSizes Resource streamers resolve relative paths against a shared prefix; without this HLE every call returned NOT_FOUND and assets never got real ids/sizes. * fix(remoteplay): stub Initialize and GetConnectionStatus as disconnected Titles probe Remote Play during pad/network bring-up; unresolved imports returned NOT_FOUND. Report initialized + disconnected so callers take the normal offline path. * fix(agc): accept Gen5 hull shaders that omit PGM_LO/HI in CreateShader Type-5 headers can start with RSRC1/RSRC2; rejecting them left null handles and Main Thread AVs. Scan the SH table and skip PGM patch when absent. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(kernel): reject getdents on file fds and emit . / .. for empty dirs Returning rax=0 for non-directory or empty listings looked like EOF and let GTA treat the fd as a pointer (fiWriteAsyncDataWorker AV at 0xB1). * fix(hle): enable GuestImageWriteTracker CPU sync on Windows Windows previously hard-disabled the tracker, so CPU-written guest planes never marked dirty and host textures stayed empty. Arm pages with VirtualProtect, handle write AVs in VEH, and warm/test on VirtualAlloc memory so protect cannot poison the CRT heap. * fix(agc): skip CB metadata draws for EliminateFastClear/Fmask/DCC CB_COLOR_CONTROL modes 2/5/6 are colour-buffer metadata ops; applying the bound shader as a normal colour draw corrupts subsequent composites. Decode MODE from bits [6:4] and return before translate. * fix(agc): merge Prospero attrib-table formats onto IR vertex inputs IR-discovered BufferLoadFormat often keeps a stale float sharp format; patch DataFormat/offset from the AGC attrib table (semantic index), allow offen fetches, and map quirks 113/121 through NarrowVk for host vertex input. * fix(audio): harden AudioOut2 stack out-buffer writes against canary smash Titles that stack-allocate AudioOut2 outs next to the frame canary were corrupted by oversized or mistyped HLE writes; keep ContextPush pacing. * Revert "fix(memory): reserve only large regions (#608)" This reverts commit 8f9456229a. * fix(gpu): decode Gen5 R16 and RG32 render-target formats * fix(audio): AudioOut2 host beds, deeper waveOut queue, AJM MP3 GTA V Enhanced routes intro/menu audio through AudioOut2 and FMOD's AJM MP3 path. Wire PortCreate/PortSetAttributes/ContextPush to dual host stereo streams, deepen WinMM queue to 128KiB, and decode AJM codec 0 with a stateful NLayer helper so menu music is not silent. * fix(agc): map PS interpolants via SPI_PS_INPUT_CNTL semantics Identity ATTR→param wiring ignored hardware remapping, so UI draws got wrong (or empty) interpolants. Pack CNTL from matched PS/GS semantics, thread it into Vulkan/Metal as Location/Flat, and fingerprint it in the graphics shader cache key. * fix(agc): rect-list/NGG strips, Index8 expand, and GE_INDX_OFFSET NGG single-rect UI needs triangle-strip expansion; Prospero Index8 must expand to host u16; glyphs need base vertex from GE_INDX_OFFSET. Skip param-less rect-lists instead of inventing colour draws. * fix(np): report GTA Story Mode addcont entitlements as owned NpEntitlementAccess was returning an empty add-on list, so GTA V Enhanced offered Buy Story Mode. Publish the installed license labels and stub premium-event registration so offline sessions take the owned path. * fix(cpu): prefer native workers for all guest entry stubs Route thread entry, continuation, and main entry through RunGuestEntryStub so guest stubs are not invoked above CLR-managed frames (UnmanagedCallersOnly FailFast). Keep requireNativeWorker for tbb_thead; other paths prefer workers with calli fallback. * fix(agc): implement Rewind/Jump writers and IT_REWIND waits GTA Subrender AVs came from AcbJumpGetSize / DcbRewind returning NOT_FOUND as packet sizes. Add IT_REWIND and INDIRECT_BUFFER writers, patch SetRewindState into the GPU wait registry, and nest-parse 4-dword jumps. * fix(gpu): use AddrLib ExactXor for Gen5 Standard256B (mode 1) Mode 5 already had Standard4K ExactXor; mode 1 still used the generic StandardSwizzle block table, which mis-detiles Gen5 UI atlases. * Revert "fix(cpu): prefer native workers for all guest entry stubs" This reverts commit 31c4db0d38. * fix(memory): commit-first large maps; reserve only on failure Replace the #608 always-reserve-only exact-map path with allocate-first and lazy reserve fallback when a huge non-exec commit cannot be satisfied. Prime and widen GetPointer commit so the fallback path is safer for native walkers. Drops the need for a hard #608 revert. * [Agc] Implement fused shader half exports * fix(agc): accept optional hull state in CreatePrimState Port the CreatePrimState hull-optional path from #583 so fused HS pipelines (GTA) are not rejected with INVALID_ARGUMENT. Geometry-derived CX/UC writes are unchanged; hull is traced only. * fix(videoout): restore thread-safe VulkanHostBufferPool (#564) The 6db095e wipe dropped CasualcoderDev's lock-ordering-safe pool. Concurrent Return/TryTake without the gate races after the first present and can hang the submit path. * Revert "fix(agc): implement Rewind/Jump writers and IT_REWIND waits" This reverts commit bec77bf083. * test(memory): align lazy-commit expectations with commit-first policy Fake hosts must reject Allocate so reserve-only paths still run, and GetPointer asserts the 32 MiB prime range including AlignUp spill. * diag(gpu): log guest-queue backlog breakdown under backpressure Rate-limit top work types and ordered debugName prefixes when the Vulkan guest work queue stalls, so North Yankton logs show acquire/label vs draw traffic instead of only VulkanOrderedGuestAction. * perf(agc): coalesce acquire flushes and batch non-DMA label wakes Flush pending ACQUIRE_MEM invalidation at draw/dispatch/dma/flip boundaries instead of before every packet, and complete release/write-data producers in the same ordered action so load paths enqueue far fewer VulkanOrderedGuestAction items. * perf(gpu): wait for ordered-action fences and keep draining sync On Windows/Linux, block briefly for queue-visibility fences instead of deferring the whole logical queue for the tick. Prefer ordered sync/flip heads under backlog pressure, and keep macOS non-blocking defer behavior. * perf(gpu): raise sync-item ceiling above payload guest-work cap Apply SHARPEMU_PENDING_GUEST_WORK_ITEMS mainly to compute/draw/image payload work, and allow a higher SHARPEMU_PENDING_GUEST_SYNC_ITEMS ceiling for zero-payload ordered actions and flip markers. Keep the byte budget as the RAM safety valve. * fix(gta): stub Voice ports and implement sceKernelCheckReachability Resolve North Yankton-path Voice Create/Delete/Connect/Disconnect/End NIDs and EnumerationThread reachability checks so leftover unresolved imports are not on the critical path. * diag(gta): arm flip/present/wait probes after North Audio Rate-limited load_progress TRACE for flip submit, ordered flip enqueue, present taken/not-taken, and GPU wait backlog so North Yankton freezes can be classified without full AGC tracing. * fix(ampr): restore sequential offset=-1 reads for streamer packs Re-wire PakDirectoryTracker into sceAmprAprCommandBufferReadFile (dropped in #216) so RAGE sequential pack reads no longer fail while the North Yankton UI keeps flipping. Also rate-limit CheckReachability miss paths for EnumerationThread diagnosis. * fix(hle/videoout): Windows GuestImage opt-in and keep GTA intro without sync Default the tracker off on Windows to avoid VirtualProtect thrash, gate AGC texel-copy skips on Enabled so guest Bink planes keep shipping pixels, and drain CPU-written images on the present thread when sync is opted in. * fix(videoout): probe guest content when tracker off so UI can skip copies Restores upload-known/texture-cache skips for Dead Cells menus, and uses a sparse guest-memory fingerprint when GuestImageWriteTracker is disabled so CPU-updated Bink planes still force texel copies for GTA intro. * fix(audio): keep 128KiB host queue AudioOut2-only Restore the default 32 KiB (~171 ms) PCM bed for classic AudioOut so titles like Dreaming Sarah stay in sync; only AudioOut2 opens the deeper queue needed for bursty FMOD Push on GTA. --------- Co-authored-by: samto6 <123419830+samto6@users.noreply.github.com>
2026-08-01 15:39:47 +08:00 · 2026-07-26 23:58:55 +01:00
parent 0535783f46
commit db4339f698
58 changed files with 7301 additions and 642 deletions
@@ -1,4 +1,4 @@
-// Copyright (C) 2026 SharpEmu Emulator Project
+// Copyright (C) 2026 SharpEmu Emulator Project
 // SPDX-License-Identifier: GPL-2.0-or-later

 using SharpEmu.HLE;
@@ -1773,6 +1773,113 @@ public static partial class KernelMemoryCompatExports
        return (int)OrbisGen2Result.ORBIS_GEN2_OK;
    }

+    // WithPrefix sibling of sceKernelAprResolveFilepathsToIdsAndFileSizes.
+    // Resource streamers resolve relative asset paths against a shared directory
+    // prefix. Without HLE, every call returned the generic NOT_FOUND sentinel
+    // and no asset received a real file id/size. Signature inferred from
+    // observed guest registers (rdi=prefix, rsi=path list, rdx=count, rcx=ids,
+    // r8=sizes, r9=error index): the no-prefix sibling's args shifted right by
+    // one with a leading `const char* prefix`.
+    [SysAbiExport(
+        Nid = "w5fcCG+t31g",
+        ExportName = "sceKernelAprResolveFilepathsWithPrefixToIdsAndFileSizes",
+        Target = Generation.Gen4 | Generation.Gen5,
+        LibraryName = "libKernel")]
+    public static int KernelAprResolveFilepathsWithPrefixToIdsAndFileSizes(CpuContext ctx)
+    {
+        var prefixAddress = ctx[CpuRegister.Rdi];
+        var pathListAddress = ctx[CpuRegister.Rsi];
+        var count = ctx[CpuRegister.Rdx];
+        var idsAddress = ctx[CpuRegister.Rcx];
+        var sizesAddress = ctx[CpuRegister.R8];
+        var errorIndexAddress = ctx[CpuRegister.R9];
+        if (pathListAddress == 0 || count == 0 || sizesAddress == 0 || count > 1024)
+        {
+            KernelRuntimeCompatExports.TrySetErrno(ctx, Einval);
+            return (int)OrbisGen2Result.ORBIS_GEN2_ERROR_INVALID_ARGUMENT;
+        }
+
+        var prefix = string.Empty;
+        if (prefixAddress != 0)
+        {
+            _ = TryReadNullTerminatedUtf8(ctx, prefixAddress, MaxGuestStringLength, out prefix);
+        }
+
+        for (ulong i = 0; i < count; i++)
+        {
+            if (idsAddress != 0 &&
+                !TryWriteUInt32Compat(ctx, idsAddress + (i * sizeof(uint)), uint.MaxValue))
+            {
+                KernelRuntimeCompatExports.TrySetErrno(ctx, Efault);
+                return (int)OrbisGen2Result.ORBIS_GEN2_ERROR_MEMORY_FAULT;
+            }
+
+            if (!TryResolveAprFilepath(ctx, pathListAddress, i, out var relativePath))
+            {
+                KernelRuntimeCompatExports.TrySetErrno(ctx, Efault);
+                return (int)OrbisGen2Result.ORBIS_GEN2_ERROR_MEMORY_FAULT;
+            }
+
+            var guestPath = CombineAprPrefixedPath(prefix, relativePath);
+            var hostPath = ResolveGuestPath(guestPath);
+            if (!TryGetAprFileSize(hostPath, out var fileSize))
+            {
+                LogIoTrace("apr_resolve_with_prefix", guestPath, $"host='{hostPath}' index={i} count={count} result=not_found");
+                if (sizesAddress != 0 &&
+                    !TryWriteUInt64Compat(ctx, sizesAddress + (i * sizeof(ulong)), 0))
+                {
+                    KernelRuntimeCompatExports.TrySetErrno(ctx, Efault);
+                    return (int)OrbisGen2Result.ORBIS_GEN2_ERROR_MEMORY_FAULT;
+                }
+
+                if (errorIndexAddress != 0 &&
+                    !TryWriteUInt32Compat(ctx, errorIndexAddress, (uint)i))
+                {
+                    KernelRuntimeCompatExports.TrySetErrno(ctx, Efault);
+                    return (int)OrbisGen2Result.ORBIS_GEN2_ERROR_MEMORY_FAULT;
+                }
+
+                KernelRuntimeCompatExports.TrySetErrno(ctx, 2); // ENOENT
+                ctx[CpuRegister.Rax] = ulong.MaxValue;
+                return -1;
+            }
+
+            var fileId = AmprFileRegistry.Register(guestPath, hostPath);
+            LogIoTrace("apr_resolve_with_prefix", guestPath, $"host='{hostPath}' index={i} count={count} id=0x{fileId:X8} size={fileSize}");
+
+            if (idsAddress != 0 &&
+                !TryWriteUInt32Compat(ctx, idsAddress + (i * sizeof(uint)), fileId))
+            {
+                KernelRuntimeCompatExports.TrySetErrno(ctx, Efault);
+                return (int)OrbisGen2Result.ORBIS_GEN2_ERROR_MEMORY_FAULT;
+            }
+
+            if (!TryWriteUInt64Compat(ctx, sizesAddress + (i * sizeof(ulong)), fileSize))
+            {
+                KernelRuntimeCompatExports.TrySetErrno(ctx, Efault);
+                return (int)OrbisGen2Result.ORBIS_GEN2_ERROR_MEMORY_FAULT;
+            }
+        }
+
+        ctx[CpuRegister.Rax] = 0;
+        return (int)OrbisGen2Result.ORBIS_GEN2_OK;
+    }
+
+    private static string CombineAprPrefixedPath(string prefix, string relative)
+    {
+        if (string.IsNullOrEmpty(prefix))
+        {
+            return relative;
+        }
+
+        if (string.IsNullOrEmpty(relative))
+        {
+            return prefix;
+        }
+
+        return $"{prefix.TrimEnd('/')}/{relative.TrimStart('/')}";
+    }
+
    // The IDs-only sibling of sceKernelAprResolveFilepathsToIdsAndFileSizes.
    // Games that stream via AMPR APR call this to turn asset paths into file
    // IDs, then hand those IDs to sceAmprAprCommandBufferReadFile. Without it
@@ -7182,28 +7289,41 @@ public static partial class KernelMemoryCompatExports
    {
        if (fd < 0 || bufferAddress == 0 || requested < 512)
        {
+            ctx[CpuRegister.Rax] = unchecked((ulong)(int)OrbisGen2Result.ORBIS_GEN2_ERROR_INVALID_ARGUMENT);
            return (int)OrbisGen2Result.ORBIS_GEN2_ERROR_INVALID_ARGUMENT;
        }

        OpenDirectory? directory;
+        bool isOpenFile;
        lock (_fdGate)
        {
            _openDirectories.TryGetValue(fd, out directory);
+            isOpenFile = directory is null && _openFiles.ContainsKey(fd);
        }

        if (directory is null)
        {
-            return (int)OrbisGen2Result.ORBIS_GEN2_ERROR_NOT_FOUND;
+            // A regular file fd used with getdents must not look like EOF (rax=0);
+            // that path has caused GTA's fiWriteAsyncDataWorker to treat the fd
+            // integer as a pointer and AV at address 0xB1.
+            var error = isOpenFile
+                ? OrbisGen2Result.ORBIS_GEN2_ERROR_INVALID_ARGUMENT
+                : OrbisGen2Result.ORBIS_GEN2_ERROR_NOT_FOUND;
+            LogIoTrace("getdents", $"fd:{fd}", $"result={(isOpenFile ? "not_directory" : "badfd")}");
+            ctx[CpuRegister.Rax] = unchecked((ulong)(int)error);
+            return (int)error;
        }

        var currentIndex = directory.NextIndex;
        if (basePointerAddress != 0 && !TryWriteUInt64Compat(ctx, basePointerAddress, (ulong)currentIndex))
        {
+            ctx[CpuRegister.Rax] = unchecked((ulong)(int)OrbisGen2Result.ORBIS_GEN2_ERROR_MEMORY_FAULT);
            return (int)OrbisGen2Result.ORBIS_GEN2_ERROR_MEMORY_FAULT;
        }

        if (currentIndex >= directory.Entries.Length)
        {
+            LogIoTrace("getdents", directory.Path, $"fd={fd} result=eof entries={directory.Entries.Length}");
            ctx[CpuRegister.Rax] = 0;
            return (int)OrbisGen2Result.ORBIS_GEN2_OK;
        }
@@ -7234,11 +7354,16 @@ public static partial class KernelMemoryCompatExports

    private static string[] EnumerateDirectoryEntries(string hostPath)
    {
-        return Directory.EnumerateFileSystemEntries(hostPath)
+        // Real getdents always yields "." / ".." before other names. An empty
+        // host dir previously returned EOF on the first call (rax=0), which
+        // sent GTA's fiWriteAsyncDataWorker down a path that treated the fd as
+        // a pointer (AV at 0xB1 on /download0/cloudcache/).
+        var children = Directory.EnumerateFileSystemEntries(hostPath)
            .Select(Path.GetFileName)
            .Where(static name => !string.IsNullOrEmpty(name))
-            .OrderBy(static name => name, StringComparer.OrdinalIgnoreCase)
-            .ToArray()!;
+            .OrderBy(static name => name, StringComparer.OrdinalIgnoreCase);
+
+        return new[] { ".", ".." }.Concat(children).ToArray()!;
    }

    private static uint ComputeDirectoryEntryHash(ReadOnlySpan<byte> utf8Name)
@@ -7792,6 +7917,47 @@ public static partial class KernelMemoryCompatExports
        return (int)OrbisGen2Result.ORBIS_GEN2_OK;
    }

+    private static long _checkReachabilityMissTraceCount;
+
+    // POSIX access(2)-style path reachability probe used by RAGE EnumerationThread.
+    [SysAbiExport(
+        Nid = "uWyW3v98sU4",
+        ExportName = "sceKernelCheckReachability",
+        Target = Generation.Gen4 | Generation.Gen5,
+        LibraryName = "libKernel")]
+    public static int KernelCheckReachability(CpuContext ctx)
+    {
+        var pathAddress = ctx[CpuRegister.Rdi];
+        if (pathAddress == 0)
+        {
+            return (int)OrbisGen2Result.ORBIS_GEN2_ERROR_INVALID_ARGUMENT;
+        }
+
+        if (!TryReadNullTerminatedUtf8(ctx, pathAddress, MaxGuestStringLength, out var guestPath))
+        {
+            return (int)OrbisGen2Result.ORBIS_GEN2_ERROR_MEMORY_FAULT;
+        }
+
+        var hostPath = ResolveGuestPath(guestPath);
+        if (File.Exists(hostPath) || Directory.Exists(hostPath))
+        {
+            ctx[CpuRegister.Rax] = 0;
+            return (int)OrbisGen2Result.ORBIS_GEN2_OK;
+        }
+
+        // EnumerationThread probes missing mounts during load; surface guest/host
+        // paths so North Yankton stalls are diagnosable without SHARPEMU_LOG_IO.
+        var miss = Interlocked.Increment(ref _checkReachabilityMissTraceCount);
+        if (miss <= 16 || (miss & (miss - 1)) == 0)
+        {
+            Console.Error.WriteLine(
+                $"[LOADER][TRACE] kernel.check_reachability_miss count={miss} " +
+                $"guest='{guestPath}' host='{hostPath}'");
+        }
+
+        return (int)OrbisGen2Result.ORBIS_GEN2_ERROR_NOT_FOUND;
+    }
+
    [SysAbiExport(
        Nid = "fgIsQ10xYVA",
        ExportName = "sceKernelChmod",