fix(gta): restore wiped GTA foundation and gameplay path (PPSA04264) (#650)

* fix(kernel): implement APR ResolveFilepathsWithPrefixToIdsAndFileSizes

Resource streamers resolve relative paths against a shared prefix; without
this HLE every call returned NOT_FOUND and assets never got real ids/sizes.

* fix(remoteplay): stub Initialize and GetConnectionStatus as disconnected

Titles probe Remote Play during pad/network bring-up; unresolved imports
returned NOT_FOUND. Report initialized + disconnected so callers take the
normal offline path.

* fix(agc): accept Gen5 hull shaders that omit PGM_LO/HI in CreateShader

Type-5 headers can start with RSRC1/RSRC2; rejecting them left null handles
and Main Thread AVs. Scan the SH table and skip PGM patch when absent.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(kernel): reject getdents on file fds and emit . / .. for empty dirs

Returning rax=0 for non-directory or empty listings looked like EOF and
let GTA treat the fd as a pointer (fiWriteAsyncDataWorker AV at 0xB1).

* fix(hle): enable GuestImageWriteTracker CPU sync on Windows

Windows previously hard-disabled the tracker, so CPU-written guest
planes never marked dirty and host textures stayed empty. Arm pages
with VirtualProtect, handle write AVs in VEH, and warm/test on
VirtualAlloc memory so protect cannot poison the CRT heap.

* fix(agc): skip CB metadata draws for EliminateFastClear/Fmask/DCC

CB_COLOR_CONTROL modes 2/5/6 are colour-buffer metadata ops; applying
the bound shader as a normal colour draw corrupts subsequent composites.
Decode MODE from bits [6:4] and return before translate.

* fix(agc): merge Prospero attrib-table formats onto IR vertex inputs

IR-discovered BufferLoadFormat often keeps a stale float sharp format;
patch DataFormat/offset from the AGC attrib table (semantic index),
allow offen fetches, and map quirks 113/121 through NarrowVk for host
vertex input.

* fix(audio): harden AudioOut2 stack out-buffer writes against canary smash

Titles that stack-allocate AudioOut2 outs next to the frame canary were
corrupted by oversized or mistyped HLE writes; keep ContextPush pacing.

* Revert "fix(memory): reserve only large regions (#608)"

This reverts commit 8f9456229a.

* fix(gpu): decode Gen5 R16 and RG32 render-target formats

* fix(audio): AudioOut2 host beds, deeper waveOut queue, AJM MP3

GTA V Enhanced routes intro/menu audio through AudioOut2 and FMOD's AJM
MP3 path. Wire PortCreate/PortSetAttributes/ContextPush to dual host
stereo streams, deepen WinMM queue to 128KiB, and decode AJM codec 0
with a stateful NLayer helper so menu music is not silent.

* fix(agc): map PS interpolants via SPI_PS_INPUT_CNTL semantics

Identity ATTR→param wiring ignored hardware remapping, so UI draws
got wrong (or empty) interpolants. Pack CNTL from matched PS/GS
semantics, thread it into Vulkan/Metal as Location/Flat, and fingerprint
it in the graphics shader cache key.

* fix(agc): rect-list/NGG strips, Index8 expand, and GE_INDX_OFFSET

NGG single-rect UI needs triangle-strip expansion; Prospero Index8 must
expand to host u16; glyphs need base vertex from GE_INDX_OFFSET. Skip
param-less rect-lists instead of inventing colour draws.

* fix(np): report GTA Story Mode addcont entitlements as owned

NpEntitlementAccess was returning an empty add-on list, so GTA V Enhanced offered Buy Story Mode. Publish the installed license labels and stub premium-event registration so offline sessions take the owned path.

* fix(cpu): prefer native workers for all guest entry stubs

Route thread entry, continuation, and main entry through RunGuestEntryStub so guest stubs are not invoked above CLR-managed frames (UnmanagedCallersOnly FailFast). Keep requireNativeWorker for tbb_thead; other paths prefer workers with calli fallback.

* fix(agc): implement Rewind/Jump writers and IT_REWIND waits

GTA Subrender AVs came from AcbJumpGetSize / DcbRewind returning NOT_FOUND as packet sizes. Add IT_REWIND and INDIRECT_BUFFER writers, patch SetRewindState into the GPU wait registry, and nest-parse 4-dword jumps.

* fix(gpu): use AddrLib ExactXor for Gen5 Standard256B (mode 1)

Mode 5 already had Standard4K ExactXor; mode 1 still used the generic StandardSwizzle block table, which mis-detiles Gen5 UI atlases.

* Revert "fix(cpu): prefer native workers for all guest entry stubs"

This reverts commit 31c4db0d38.

* fix(memory): commit-first large maps; reserve only on failure

Replace the #608 always-reserve-only exact-map path with allocate-first and lazy reserve fallback when a huge non-exec commit cannot be satisfied. Prime and widen GetPointer commit so the fallback path is safer for native walkers. Drops the need for a hard #608 revert.

* [Agc] Implement fused shader half exports

* fix(agc): accept optional hull state in CreatePrimState

Port the CreatePrimState hull-optional path from #583 so fused HS pipelines (GTA) are not rejected with INVALID_ARGUMENT. Geometry-derived CX/UC writes are unchanged; hull is traced only.

* fix(videoout): restore thread-safe VulkanHostBufferPool (#564)

The 6db095e wipe dropped CasualcoderDev's lock-ordering-safe pool. Concurrent Return/TryTake without the gate races after the first present and can hang the submit path.

* Revert "fix(agc): implement Rewind/Jump writers and IT_REWIND waits"

This reverts commit bec77bf083.

* test(memory): align lazy-commit expectations with commit-first policy

Fake hosts must reject Allocate so reserve-only paths still run, and GetPointer asserts the 32 MiB prime range including AlignUp spill.

* diag(gpu): log guest-queue backlog breakdown under backpressure

Rate-limit top work types and ordered debugName prefixes when the Vulkan guest work queue stalls, so North Yankton logs show acquire/label vs draw traffic instead of only VulkanOrderedGuestAction.

* perf(agc): coalesce acquire flushes and batch non-DMA label wakes

Flush pending ACQUIRE_MEM invalidation at draw/dispatch/dma/flip boundaries instead of before every packet, and complete release/write-data producers in the same ordered action so load paths enqueue far fewer VulkanOrderedGuestAction items.

* perf(gpu): wait for ordered-action fences and keep draining sync

On Windows/Linux, block briefly for queue-visibility fences instead of deferring the whole logical queue for the tick. Prefer ordered sync/flip heads under backlog pressure, and keep macOS non-blocking defer behavior.

* perf(gpu): raise sync-item ceiling above payload guest-work cap

Apply SHARPEMU_PENDING_GUEST_WORK_ITEMS mainly to compute/draw/image payload work, and allow a higher SHARPEMU_PENDING_GUEST_SYNC_ITEMS ceiling for zero-payload ordered actions and flip markers. Keep the byte budget as the RAM safety valve.

* fix(gta): stub Voice ports and implement sceKernelCheckReachability

Resolve North Yankton-path Voice Create/Delete/Connect/Disconnect/End NIDs and EnumerationThread reachability checks so leftover unresolved imports are not on the critical path.

* diag(gta): arm flip/present/wait probes after North Audio

Rate-limited load_progress TRACE for flip submit, ordered flip enqueue, present taken/not-taken, and GPU wait backlog so North Yankton freezes can be classified without full AGC tracing.

* fix(ampr): restore sequential offset=-1 reads for streamer packs

Re-wire PakDirectoryTracker into sceAmprAprCommandBufferReadFile (dropped in #216) so RAGE sequential pack reads no longer fail while the North Yankton UI keeps flipping. Also rate-limit CheckReachability miss paths for EnumerationThread diagnosis.

* fix(hle/videoout): Windows GuestImage opt-in and keep GTA intro without sync

Default the tracker off on Windows to avoid VirtualProtect thrash, gate AGC
texel-copy skips on Enabled so guest Bink planes keep shipping pixels, and
drain CPU-written images on the present thread when sync is opted in.

* fix(videoout): probe guest content when tracker off so UI can skip copies

Restores upload-known/texture-cache skips for Dead Cells menus, and uses a
sparse guest-memory fingerprint when GuestImageWriteTracker is disabled so
CPU-updated Bink planes still force texel copies for GTA intro.

* fix(audio): keep 128KiB host queue AudioOut2-only

Restore the default 32 KiB (~171 ms) PCM bed for classic AudioOut so
titles like Dreaming Sarah stay in sync; only AudioOut2 opens the deeper
queue needed for bursty FMOD Push on GTA.

---------

Co-authored-by: samto6 <123419830+samto6@users.noreply.github.com>
This commit is contained in:
MarcelMediaDev
2026-07-26 23:58:55 +01:00
committed by GitHub
parent 0535783f46
commit db4339f698
58 changed files with 7301 additions and 642 deletions
+155 -24
View File
@@ -1,6 +1,7 @@
// Copyright (C) 2026 SharpEmu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
using System.Diagnostics;
using System.Globalization;
using System.Runtime.InteropServices;
@@ -30,6 +31,12 @@ public static unsafe class GuestImageWriteTracker
public ulong End;
public int Dirty;
public int Armed;
/// <summary>
/// When false the range is watch-only: managed writes still dirty it via
/// <see cref="NotifyManagedWrite"/>, but pages are never write-protected
/// so native CPU stores do not fault.
/// </summary>
public bool Protect;
public int FirstCpuWriteSeen;
public int PendingFirstCpuWrite;
public long WriteGeneration;
@@ -80,8 +87,17 @@ public static unsafe class GuestImageWriteTracker
private static RangeSnapshot _rangeSnapshot = RangeSnapshot.Empty;
private static readonly bool _enabled = !OperatingSystem.IsWindows() &&
Environment.GetEnvironmentVariable("SHARPEMU_GUEST_IMAGE_CPU_SYNC") != "0";
// Windows defaults off: VirtualProtect fault sync still regresses titles
// like Dead Cells / Demon's Souls. Opt in with SHARPEMU_GUEST_IMAGE_CPU_SYNC=1
// when a title needs CPU-written guest planes (e.g. GTA intro). Linux/macOS
// keep the historical opt-out (=0 disables).
private static readonly bool _enabled =
OperatingSystem.IsWindows()
? string.Equals(
Environment.GetEnvironmentVariable("SHARPEMU_GUEST_IMAGE_CPU_SYNC"),
"1",
StringComparison.Ordinal)
: Environment.GetEnvironmentVariable("SHARPEMU_GUEST_IMAGE_CPU_SYNC") != "0";
private static readonly (bool Wildcard, ulong[] Addresses) _lifetimeTraceFilter =
ParseAddressList(Environment.GetEnvironmentVariable("SHARPEMU_TRACE_GUEST_IMAGE_ADDRS"));
private static readonly (bool Wildcard, string[] Sources) _lifetimeSourceTraceFilter =
@@ -95,14 +111,67 @@ public static unsafe class GuestImageWriteTracker
_enabled && _lifetimeTraceEnabled ? GetMonotonicNanoseconds() : 0;
private static long _lifetimeTraceSequence;
private const uint PageReadonly = 0x02;
private const uint PageReadWrite = 0x04;
[DllImport("libc", EntryPoint = "mprotect", SetLastError = true)]
private static extern int Mprotect(nint address, nuint length, int protection);
[DllImport("libc", EntryPoint = "clock_gettime", SetLastError = false)]
private static extern int ClockGetTime(int clockId, Timespec* time);
[DllImport("kernel32.dll", SetLastError = true)]
private static extern int VirtualProtect(
nint lpAddress,
nuint dwSize,
uint flNewProtect,
out uint lpflOldProtect);
[DllImport("kernel32.dll", SetLastError = true)]
private static extern nint VirtualAlloc(
nint lpAddress,
nuint dwSize,
uint flAllocationType,
uint flProtect);
[DllImport("kernel32.dll", SetLastError = true)]
private static extern int VirtualFree(nint lpAddress, nuint dwSize, uint dwFreeType);
private const uint MemCommit = 0x1000;
private const uint MemReserve = 0x2000;
private const uint MemRelease = 0x8000;
public static bool Enabled => _enabled;
/// <summary>
/// Test/diagnostics helper: whether <paramref name="address"/> is tracked
/// with write protection armed (watch-only ranges report protect=false).
/// </summary>
public static bool TryGetProtectionState(
ulong address,
out bool protect,
out bool armed)
{
protect = false;
armed = false;
if (!_enabled)
{
return false;
}
lock (_gate)
{
if (!_rangesByAddress.TryGetValue(address, out var range))
{
return false;
}
protect = range.Protect;
armed = Volatile.Read(ref range.Armed) != 0;
return true;
}
}
/// <summary>
/// Exercises the fault-handling path once outside signal context so every
/// branch is JIT-compiled (and, under Rosetta 2, translated) before a real
@@ -115,7 +184,17 @@ public static unsafe class GuestImageWriteTracker
return;
}
var scratch = NativeMemory.AllocZeroed(4096);
// VirtualProtect only belongs on VirtualAlloc/mmap pages. Warming on
// CRT heap memory makes neighbouring heap metadata read-only and
// crashes the process on Windows.
var scratch = OperatingSystem.IsWindows()
? VirtualAlloc(0, 4096, MemCommit | MemReserve, PageReadWrite)
: (nint)NativeMemory.AllocZeroed(4096);
if (scratch == 0)
{
return;
}
try
{
// Warm the timestamp P/Invoke used by the signal-safe scalar
@@ -129,16 +208,29 @@ public static unsafe class GuestImageWriteTracker
}
finally
{
NativeMemory.Free(scratch);
if (OperatingSystem.IsWindows())
{
_ = VirtualFree(scratch, 0, MemRelease);
}
else
{
NativeMemory.Free((void*)scratch);
}
}
}
/// <summary>Registers a range and arms write protection on it.</summary>
/// <summary>
/// Registers a range. When <paramref name="protect"/> is true, arms write
/// protection so native stores fault and mark the range dirty. When false,
/// the range is watch-only (managed HLE writes still dirty via
/// <see cref="NotifyManagedWrite"/>) and never <c>VirtualProtect</c>'d.
/// </summary>
public static void Track(
ulong address,
ulong byteCount,
long sourceSequence = 0,
string source = "unspecified")
string source = "unspecified",
bool protect = true)
{
if (!_enabled || address == 0 || byteCount == 0)
{
@@ -159,6 +251,7 @@ public static unsafe class GuestImageWriteTracker
// a fresh immutable range, carrying the write generation so
// resizes do not hide guest CPU rewrites from cache owners.
var writeGeneration = Volatile.Read(ref range.WriteGeneration);
var keepProtect = range.Protect || protect;
DisarmLocked(range, "replace-range");
_rangesByAddress.Remove(address);
range = new TrackedRange
@@ -167,6 +260,7 @@ public static unsafe class GuestImageWriteTracker
ByteCount = byteCount,
Start = start,
End = start + length,
Protect = keepProtect,
WriteGeneration = writeGeneration,
};
_rangesByAddress[address] = range;
@@ -181,6 +275,7 @@ public static unsafe class GuestImageWriteTracker
ByteCount = byteCount,
Start = start,
End = start + length,
Protect = protect,
TraceLifetime =
ShouldTraceRange(start, start + length) || ShouldTraceSource(source),
SourceSequence = sourceSequence,
@@ -192,13 +287,22 @@ public static unsafe class GuestImageWriteTracker
else
{
FlushPendingFirstCpuWrite(range);
// Protect is sticky: a later watch-only Track (texture cache)
// must not disarm an RT that already needs page faults.
if (protect && !range.Protect)
{
range.Protect = true;
}
}
range.SourceSequence = sourceSequence;
range.Source = source;
range.TraceLifetime =
ShouldTraceRange(range.Start, range.End) || ShouldTraceSource(source);
ArmLocked(range, "arm");
if (range.Protect)
{
ArmLocked(range, "arm");
}
}
}
@@ -277,7 +381,8 @@ public static unsafe class GuestImageWriteTracker
lock (_gate)
{
if (_rangesByAddress.TryGetValue(address, out var range))
if (_rangesByAddress.TryGetValue(address, out var range) &&
range.Protect)
{
ArmLocked(range, "rearm");
}
@@ -445,10 +550,7 @@ public static unsafe class GuestImageWriteTracker
}
if (needsUnprotect &&
Mprotect(
(nint)writableStart,
(nuint)(writableEnd - writableStart),
ProtRead | ProtWrite) != 0)
!TrySetProtection(writableStart, writableEnd - writableStart, writable: true))
{
return false;
}
@@ -462,7 +564,11 @@ public static unsafe class GuestImageWriteTracker
}
var wasArmed = Interlocked.Exchange(ref range.Armed, 0) != 0;
if (wasArmed)
var wasDirty = Interlocked.Exchange(ref range.Dirty, 1) != 0;
// Protected ranges bump generation once per arm/fault cycle.
// Watch-only ranges never arm, so bump on the first dirty mark
// (NotifyManagedWrite) so cache owners still see a rewrite.
if (wasArmed || (!range.Protect && !wasDirty))
{
Interlocked.Increment(ref range.WriteGeneration);
}
@@ -480,8 +586,6 @@ public static unsafe class GuestImageWriteTracker
Volatile.Write(ref range.PendingFirstCpuWrite, 1);
Volatile.Write(ref range.FirstCpuWriteSeen, 2);
}
Volatile.Write(ref range.Dirty, 1);
}
return true;
@@ -497,10 +601,7 @@ public static unsafe class GuestImageWriteTracker
// A new publication/rearm starts a new first-write lifetime.
Volatile.Write(ref range.FirstCpuWriteSeen, 0);
var failed = Mprotect(
(nint)range.Start,
(nuint)(range.End - range.Start),
ProtRead) != 0;
var failed = !TrySetProtection(range.Start, range.End - range.Start, writable: false);
if (failed)
{
Volatile.Write(ref range.Armed, 0);
@@ -520,10 +621,7 @@ public static unsafe class GuestImageWriteTracker
var wasArmed = Interlocked.Exchange(ref range.Armed, 0) == 1;
if (wasArmed)
{
_ = Mprotect(
(nint)range.Start,
(nuint)(range.End - range.Start),
ProtRead | ProtWrite);
_ = TrySetProtection(range.Start, range.End - range.Start, writable: true);
}
if (range.TraceLifetime)
@@ -534,7 +632,13 @@ public static unsafe class GuestImageWriteTracker
private static void RebuildSnapshotLocked()
{
Volatile.Write(ref _rangeSnapshot, new RangeSnapshot(_rangesByAddress.Values.ToArray()));
// Fault / NotifyManagedWrite hot paths must only see protected ranges.
// Watch-only texture-cache registrations used to widen Start..End across
// nearly all GPU memory so every managed guest write walked this path.
var protectedRanges = _rangesByAddress.Values
.Where(static range => range.Protect)
.ToArray();
Volatile.Write(ref _rangeSnapshot, new RangeSnapshot(protectedRanges));
}
private static (ulong Start, ulong Length) PageAlign(ulong address, ulong byteCount)
@@ -679,8 +783,35 @@ public static unsafe class GuestImageWriteTracker
$"fault=0x{faultAddress:X16} page=0x{faultPage:X16}");
}
private static bool TrySetProtection(ulong start, ulong length, bool writable)
{
if (length == 0)
{
return true;
}
if (OperatingSystem.IsWindows())
{
return VirtualProtect(
(nint)start,
(nuint)length,
writable ? PageReadWrite : PageReadonly,
out _) != 0;
}
return Mprotect(
(nint)start,
(nuint)length,
writable ? ProtRead | ProtWrite : ProtRead) == 0;
}
private static long GetMonotonicNanoseconds()
{
if (OperatingSystem.IsWindows())
{
return Stopwatch.GetTimestamp() * 1_000_000_000L / Stopwatch.Frequency;
}
Timespec time;
return ClockGetTime(ClockMonotonicRaw, &time) == 0
? unchecked((time.Seconds * 1_000_000_000L) + time.Nanoseconds)