fix(gta): restore wiped GTA foundation and gameplay path (PPSA04264) (#650)

* fix(kernel): implement APR ResolveFilepathsWithPrefixToIdsAndFileSizes

Resource streamers resolve relative paths against a shared prefix; without
this HLE every call returned NOT_FOUND and assets never got real ids/sizes.

* fix(remoteplay): stub Initialize and GetConnectionStatus as disconnected

Titles probe Remote Play during pad/network bring-up; unresolved imports
returned NOT_FOUND. Report initialized + disconnected so callers take the
normal offline path.

* fix(agc): accept Gen5 hull shaders that omit PGM_LO/HI in CreateShader

Type-5 headers can start with RSRC1/RSRC2; rejecting them left null handles
and Main Thread AVs. Scan the SH table and skip PGM patch when absent.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(kernel): reject getdents on file fds and emit . / .. for empty dirs

Returning rax=0 for non-directory or empty listings looked like EOF and
let GTA treat the fd as a pointer (fiWriteAsyncDataWorker AV at 0xB1).

* fix(hle): enable GuestImageWriteTracker CPU sync on Windows

Windows previously hard-disabled the tracker, so CPU-written guest
planes never marked dirty and host textures stayed empty. Arm pages
with VirtualProtect, handle write AVs in VEH, and warm/test on
VirtualAlloc memory so protect cannot poison the CRT heap.

* fix(agc): skip CB metadata draws for EliminateFastClear/Fmask/DCC

CB_COLOR_CONTROL modes 2/5/6 are colour-buffer metadata ops; applying
the bound shader as a normal colour draw corrupts subsequent composites.
Decode MODE from bits [6:4] and return before translate.

* fix(agc): merge Prospero attrib-table formats onto IR vertex inputs

IR-discovered BufferLoadFormat often keeps a stale float sharp format;
patch DataFormat/offset from the AGC attrib table (semantic index),
allow offen fetches, and map quirks 113/121 through NarrowVk for host
vertex input.

* fix(audio): harden AudioOut2 stack out-buffer writes against canary smash

Titles that stack-allocate AudioOut2 outs next to the frame canary were
corrupted by oversized or mistyped HLE writes; keep ContextPush pacing.

* Revert "fix(memory): reserve only large regions (#608)"

This reverts commit 8f9456229a.

* fix(gpu): decode Gen5 R16 and RG32 render-target formats

* fix(audio): AudioOut2 host beds, deeper waveOut queue, AJM MP3

GTA V Enhanced routes intro/menu audio through AudioOut2 and FMOD's AJM
MP3 path. Wire PortCreate/PortSetAttributes/ContextPush to dual host
stereo streams, deepen WinMM queue to 128KiB, and decode AJM codec 0
with a stateful NLayer helper so menu music is not silent.

* fix(agc): map PS interpolants via SPI_PS_INPUT_CNTL semantics

Identity ATTR→param wiring ignored hardware remapping, so UI draws
got wrong (or empty) interpolants. Pack CNTL from matched PS/GS
semantics, thread it into Vulkan/Metal as Location/Flat, and fingerprint
it in the graphics shader cache key.

* fix(agc): rect-list/NGG strips, Index8 expand, and GE_INDX_OFFSET

NGG single-rect UI needs triangle-strip expansion; Prospero Index8 must
expand to host u16; glyphs need base vertex from GE_INDX_OFFSET. Skip
param-less rect-lists instead of inventing colour draws.

* fix(np): report GTA Story Mode addcont entitlements as owned

NpEntitlementAccess was returning an empty add-on list, so GTA V Enhanced offered Buy Story Mode. Publish the installed license labels and stub premium-event registration so offline sessions take the owned path.

* fix(cpu): prefer native workers for all guest entry stubs

Route thread entry, continuation, and main entry through RunGuestEntryStub so guest stubs are not invoked above CLR-managed frames (UnmanagedCallersOnly FailFast). Keep requireNativeWorker for tbb_thead; other paths prefer workers with calli fallback.

* fix(agc): implement Rewind/Jump writers and IT_REWIND waits

GTA Subrender AVs came from AcbJumpGetSize / DcbRewind returning NOT_FOUND as packet sizes. Add IT_REWIND and INDIRECT_BUFFER writers, patch SetRewindState into the GPU wait registry, and nest-parse 4-dword jumps.

* fix(gpu): use AddrLib ExactXor for Gen5 Standard256B (mode 1)

Mode 5 already had Standard4K ExactXor; mode 1 still used the generic StandardSwizzle block table, which mis-detiles Gen5 UI atlases.

* Revert "fix(cpu): prefer native workers for all guest entry stubs"

This reverts commit 31c4db0d38.

* fix(memory): commit-first large maps; reserve only on failure

Replace the #608 always-reserve-only exact-map path with allocate-first and lazy reserve fallback when a huge non-exec commit cannot be satisfied. Prime and widen GetPointer commit so the fallback path is safer for native walkers. Drops the need for a hard #608 revert.

* [Agc] Implement fused shader half exports

* fix(agc): accept optional hull state in CreatePrimState

Port the CreatePrimState hull-optional path from #583 so fused HS pipelines (GTA) are not rejected with INVALID_ARGUMENT. Geometry-derived CX/UC writes are unchanged; hull is traced only.

* fix(videoout): restore thread-safe VulkanHostBufferPool (#564)

The 6db095e wipe dropped CasualcoderDev's lock-ordering-safe pool. Concurrent Return/TryTake without the gate races after the first present and can hang the submit path.

* Revert "fix(agc): implement Rewind/Jump writers and IT_REWIND waits"

This reverts commit bec77bf083.

* test(memory): align lazy-commit expectations with commit-first policy

Fake hosts must reject Allocate so reserve-only paths still run, and GetPointer asserts the 32 MiB prime range including AlignUp spill.

* diag(gpu): log guest-queue backlog breakdown under backpressure

Rate-limit top work types and ordered debugName prefixes when the Vulkan guest work queue stalls, so North Yankton logs show acquire/label vs draw traffic instead of only VulkanOrderedGuestAction.

* perf(agc): coalesce acquire flushes and batch non-DMA label wakes

Flush pending ACQUIRE_MEM invalidation at draw/dispatch/dma/flip boundaries instead of before every packet, and complete release/write-data producers in the same ordered action so load paths enqueue far fewer VulkanOrderedGuestAction items.

* perf(gpu): wait for ordered-action fences and keep draining sync

On Windows/Linux, block briefly for queue-visibility fences instead of deferring the whole logical queue for the tick. Prefer ordered sync/flip heads under backlog pressure, and keep macOS non-blocking defer behavior.

* perf(gpu): raise sync-item ceiling above payload guest-work cap

Apply SHARPEMU_PENDING_GUEST_WORK_ITEMS mainly to compute/draw/image payload work, and allow a higher SHARPEMU_PENDING_GUEST_SYNC_ITEMS ceiling for zero-payload ordered actions and flip markers. Keep the byte budget as the RAM safety valve.

* fix(gta): stub Voice ports and implement sceKernelCheckReachability

Resolve North Yankton-path Voice Create/Delete/Connect/Disconnect/End NIDs and EnumerationThread reachability checks so leftover unresolved imports are not on the critical path.

* diag(gta): arm flip/present/wait probes after North Audio

Rate-limited load_progress TRACE for flip submit, ordered flip enqueue, present taken/not-taken, and GPU wait backlog so North Yankton freezes can be classified without full AGC tracing.

* fix(ampr): restore sequential offset=-1 reads for streamer packs

Re-wire PakDirectoryTracker into sceAmprAprCommandBufferReadFile (dropped in #216) so RAGE sequential pack reads no longer fail while the North Yankton UI keeps flipping. Also rate-limit CheckReachability miss paths for EnumerationThread diagnosis.

* fix(hle/videoout): Windows GuestImage opt-in and keep GTA intro without sync

Default the tracker off on Windows to avoid VirtualProtect thrash, gate AGC
texel-copy skips on Enabled so guest Bink planes keep shipping pixels, and
drain CPU-written images on the present thread when sync is opted in.

* fix(videoout): probe guest content when tracker off so UI can skip copies

Restores upload-known/texture-cache skips for Dead Cells menus, and uses a
sparse guest-memory fingerprint when GuestImageWriteTracker is disabled so
CPU-updated Bink planes still force texel copies for GTA intro.

* fix(audio): keep 128KiB host queue AudioOut2-only

Restore the default 32 KiB (~171 ms) PCM bed for classic AudioOut so
titles like Dreaming Sarah stay in sync; only AudioOut2 opens the deeper
queue needed for bursty FMOD Push on GTA.

---------

Co-authored-by: samto6 <123419830+samto6@users.noreply.github.com>
This commit is contained in:
MarcelMediaDev
2026-07-26 23:58:55 +01:00
committed by GitHub
parent 0535783f46
commit db4339f698
58 changed files with 7301 additions and 642 deletions
+130 -22
View File
@@ -23,13 +23,32 @@ public static class AjmExports
private static int _nextContextId;
private static int _nextBatchId;
private const uint AjmCodecMp3 = 0;
private sealed class AjmInstanceState
{
public required uint Codec { get; init; }
public required ulong Flags { get; init; }
public AjmMp3Decoder? Mp3 { get; init; }
public bool PreferPcm16
{
get
{
// AjmInstanceFlags: version:3, channels:4, format:3
var encoding = (Flags >> 7) & 0x7;
return encoding is 0 or 1; // S16 / S32 — we emit S16 for both
}
}
}
private sealed class AjmContextState
{
public object Gate { get; } = new();
public HashSet<uint> RegisteredCodecs { get; } = new();
public Dictionary<uint, uint> InstancesBySlot { get; } = new();
public Dictionary<uint, AjmInstanceState> InstancesBySlot { get; } = new();
public int NextInstanceIndex { get; set; }
}
@@ -169,7 +188,14 @@ public static class AjmExports
}
state.NextInstanceIndex = nextInstanceIndex;
state.InstancesBySlot.Add(instanceSlot, instanceId);
state.InstancesBySlot.Add(
instanceSlot,
new AjmInstanceState
{
Codec = codecType,
Flags = flags,
Mp3 = codecType == AjmCodecMp3 ? new AjmMp3Decoder() : null,
});
}
Trace($"instance_create context={contextId} codec={codecType} flags=0x{flags:X} instance=0x{instanceId:X8}");
@@ -229,11 +255,9 @@ public static class AjmExports
}
/// <summary>
/// Enqueues a decode job on a batch. Titles call this on the Bink/AJM hot
/// path; leaving it unresolved floods Import WARN spam. This is a silence
/// stub, not a codec: advance the batch cursor and report the input as
/// consumed with silence produced so the title does not spin on the same
/// packet.
/// Enqueues a decode job on a batch. GTA V Enhanced streams menu music
/// through AJM MP3 (codec 0); we decode eagerly here so BatchStart/Wait
/// stay synchronous no-ops.
/// </summary>
[SysAbiExport(
Nid = "39WxhR-ePew",
@@ -255,37 +279,120 @@ public static class AjmExports
return ctx.SetReturn(OrbisAjmErrorInvalidParameter);
}
// Best-effort: bump the batch cursor when the guest filled AjmBatchInfo.
// Still succeed without it — the unresolved stub returned 0 and titles
// keep calling; failing here would reintroduce hot-path spam via retries.
_ = TryAppendBatchJob(ctx, infoAddress, AjmJobRunSize);
// Silence: clear PCM out and claim full input consumed so the guest
// advances its bitstream cursor instead of re-submitting forever.
if (outputAddress != 0 && outputSize != 0 && outputSize <= MaxSilentPcmBytes)
var inputConsumed = 0;
var outputWritten = 0;
ulong totalSamples = 0;
var frames = 0u;
var decoded = false;
if (TryGetInstance(instanceId, out var instance) &&
instance.Mp3 is not null &&
inputAddress != 0 &&
inputSize is > 0 and <= MaxSilentPcmBytes &&
outputAddress != 0 &&
outputSize is > 0 and <= MaxSilentPcmBytes)
{
ClearGuestMemory(ctx, outputAddress, outputSize);
var input = new byte[inputSize];
var output = new byte[outputSize];
if (ctx.Memory.TryRead(inputAddress, input))
{
var result = instance.Mp3.Decode(input, output, pcm16: instance.PreferPcm16);
if (result.OutputWritten > 0)
{
if (!ctx.Memory.TryWrite(outputAddress, output.AsSpan(0, result.OutputWritten)))
{
return ctx.SetReturn(OrbisAjmErrorInvalidParameter);
}
// Zero any remainder so stale PCM does not leak into FMOD.
if ((ulong)result.OutputWritten < outputSize)
{
ClearGuestMemory(
ctx,
outputAddress + (ulong)result.OutputWritten,
outputSize - (ulong)result.OutputWritten);
}
decoded = true;
inputConsumed = result.InputConsumed;
outputWritten = result.OutputWritten;
frames = result.Frames;
totalSamples = instance.Mp3.TotalDecodedSamples;
}
else
{
inputConsumed = result.InputConsumed;
totalSamples = instance.Mp3.TotalDecodedSamples;
}
}
}
if (!decoded)
{
// Fallback: silence + consume input so the guest does not spin.
if (outputAddress != 0 && outputSize != 0 && outputSize <= MaxSilentPcmBytes)
{
ClearGuestMemory(ctx, outputAddress, outputSize);
}
if (inputConsumed == 0)
{
inputConsumed = inputSize > int.MaxValue ? int.MaxValue : (int)inputSize;
}
if (frames == 0 && (inputSize != 0 || outputSize != 0))
{
frames = 1;
}
}
WriteDecodeStreamResult(
ctx,
resultAddress,
inputConsumed: inputSize > int.MaxValue ? int.MaxValue : (int)inputSize,
outputWritten: 0,
totalDecodedSamples: 0,
frames: inputSize != 0 || outputSize != 0 ? 1u : 0u);
inputConsumed,
outputWritten,
totalSamples,
frames);
Trace(
$"batch_job_decode info=0x{infoAddress:X16} instance=0x{instanceId:X8} " +
$"in=0x{inputAddress:X16}+0x{inputSize:X} out=0x{outputAddress:X16}+0x{outputSize:X} " +
$"result=0x{resultAddress:X16}");
$"written={outputWritten} frames={frames} result=0x{resultAddress:X16}");
return ctx.SetReturn(0);
}
private static bool TryGetInstance(uint instanceId, out AjmInstanceState instance)
{
instance = null!;
var codec = instanceId >> 14;
var slot = instanceId & 0x3FFF;
if (slot == 0)
{
return false;
}
foreach (var context in Contexts.Values)
{
lock (context.Gate)
{
if (context.InstancesBySlot.TryGetValue(slot, out var found) &&
found.Codec == codec)
{
instance = found;
return true;
}
}
}
return false;
}
/// <summary>
/// Submits a built batch. Instant-complete silence stub: publish a batch id
/// and clear any error out. Decode sidebands were already filled at
/// job-enqueue time.
/// Submits a built batch. Hot path after BatchJobDecode; unresolved WARNs
/// dominate the log. Instant-complete: publish a batch id and clear any
/// error out. Decode sidebands were already filled at job-enqueue time.
/// </summary>
[SysAbiExport(
Nid = "5tOfnaClcqM",
@@ -363,6 +470,7 @@ public static class AjmExports
private const ulong AjmBatchInfoSizeField = 16;
private const ulong AjmBatchInfoLastGoodJobField = 24;
private const ulong AjmJobRunSize = 64;
private const int OrbisAjmErrorJobCreation = unchecked((int)0x80930012);
private const ulong MaxSilentPcmBytes = 1 << 20;
// AjmSidebandResult (8) + AjmSidebandStream (16) + AjmSidebandMFrame (8).
private const int DecodeSidebandBytes = 32;
+231
View File
@@ -0,0 +1,231 @@
// Copyright (C) 2026 SharpEmu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
using NLayer;
using System.Buffers.Binary;
using System.Reflection;
namespace SharpEmu.Libs.Audio;
/// <summary>
/// Stateful AJM MP3 (codec 0) decoder. GTA menu music arrives as ~960-byte
/// packets that NLayer must decode with a persistent bit-reservoir.
/// </summary>
internal sealed class AjmMp3Decoder
{
private static readonly Type? MpegStreamReaderType =
typeof(MpegFrameDecoder).Assembly.GetType("NLayer.Decoder.MpegStreamReader");
private static readonly MethodInfo? NextFrameMethod =
MpegStreamReaderType?.GetMethod(
"NextFrame",
BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic);
private static readonly MethodInfo? ClearBufferMethod =
typeof(MpegFrameDecoder).Assembly.GetType("NLayer.Decoder.FrameBase")
?.GetMethod("ClearBuffer", BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic);
private readonly MpegFrameDecoder _decoder = new();
private readonly object _gate = new();
private byte[] _pending = Array.Empty<byte>();
private readonly float[] _floatScratch = new float[1152 * 2];
public ulong TotalDecodedSamples { get; private set; }
public void Reset()
{
lock (_gate)
{
_decoder.Reset();
_pending = Array.Empty<byte>();
TotalDecodedSamples = 0;
}
}
public DecodeResult Decode(ReadOnlySpan<byte> input, Span<byte> output, bool pcm16)
{
lock (_gate)
{
if (MpegStreamReaderType is null || NextFrameMethod is null)
{
return DecodeResult.Failed;
}
var merged = new byte[_pending.Length + input.Length];
if (_pending.Length != 0)
{
_pending.CopyTo(merged, 0);
}
input.CopyTo(merged.AsSpan(_pending.Length));
using var stream = new MemoryStream(merged, writable: false);
object? reader;
try
{
reader = Activator.CreateInstance(
MpegStreamReaderType,
BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic,
binder: null,
args: [stream],
culture: null);
}
catch
{
return DecodeResult.Failed;
}
if (reader is null)
{
return DecodeResult.Failed;
}
var outputOffset = 0;
var inputConsumed = 0;
var frames = 0u;
var samplesThisCall = 0u;
while (outputOffset < output.Length)
{
object? frameObj;
try
{
frameObj = NextFrameMethod.Invoke(reader, null);
}
catch
{
break;
}
if (frameObj is not IMpegFrame frame)
{
break;
}
try
{
var frameOffset = GetFrameOffset(frameObj);
var frameLength = frame.FrameLength;
if (frameLength <= 0 || frameOffset + frameLength > merged.Length)
{
break;
}
int sampleCount;
try
{
sampleCount = _decoder.DecodeFrame(frame, _floatScratch, 0);
}
catch
{
_decoder.Reset();
inputConsumed = frameOffset + frameLength;
continue;
}
if (sampleCount <= 0)
{
inputConsumed = frameOffset + frameLength;
continue;
}
var channels = frame.ChannelMode == MpegChannelMode.Mono ? 1 : 2;
var bytesPerSample = pcm16 ? 2 : 4;
var byteCount = sampleCount * bytesPerSample;
if (outputOffset + byteCount > output.Length)
{
// Not enough room for this frame — leave it for next job.
break;
}
if (pcm16)
{
WritePcm16(_floatScratch.AsSpan(0, sampleCount), output[outputOffset..]);
}
else
{
WriteFloat(_floatScratch.AsSpan(0, sampleCount), output[outputOffset..]);
}
outputOffset += byteCount;
inputConsumed = frameOffset + frameLength;
frames++;
samplesThisCall += (uint)(sampleCount / Math.Max(channels, 1));
TotalDecodedSamples += (ulong)(sampleCount / Math.Max(channels, 1));
}
finally
{
try
{
ClearBufferMethod?.Invoke(frameObj, null);
}
catch
{
// best-effort
}
}
}
_pending = inputConsumed < merged.Length
? merged[inputConsumed..]
: Array.Empty<byte>();
// Consume the portion of *this* input that left the pending window.
var pendingBefore = merged.Length - input.Length;
var consumedFromInput = Math.Clamp(inputConsumed - pendingBefore, 0, input.Length);
return new DecodeResult(
Success: frames > 0 || consumedFromInput > 0,
InputConsumed: consumedFromInput,
OutputWritten: outputOffset,
Frames: frames,
SamplesThisCall: samplesThisCall);
}
}
private static int GetFrameOffset(object frameObj)
{
for (var type = frameObj.GetType(); type is not null; type = type.BaseType)
{
var prop = type.GetProperty(
"Offset",
BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.DeclaredOnly);
if (prop?.GetValue(frameObj) is long offset)
{
return checked((int)offset);
}
}
return 0;
}
private static void WritePcm16(ReadOnlySpan<float> samples, Span<byte> destination)
{
for (var i = 0; i < samples.Length; i++)
{
var sample = samples[i];
var scaled = sample < 0f ? sample * 32768f : sample * 32767f;
var value = (short)Math.Clamp(MathF.Round(scaled), short.MinValue, short.MaxValue);
BinaryPrimitives.WriteInt16LittleEndian(destination[(i * 2)..], value);
}
}
private static void WriteFloat(ReadOnlySpan<float> samples, Span<byte> destination)
{
for (var i = 0; i < samples.Length; i++)
{
var bits = BitConverter.SingleToInt32Bits(samples[i]);
BinaryPrimitives.WriteInt32LittleEndian(destination[(i * 4)..], bits);
}
}
internal readonly record struct DecodeResult(
bool Success,
int InputConsumed,
int OutputWritten,
uint Frames,
uint SamplesThisCall)
{
public static DecodeResult Failed { get; } = new(false, 0, 0, 0, 0);
}
}
File diff suppressed because it is too large Load Diff
@@ -212,6 +212,14 @@ public static class AudioOutExports
return ctx.SetReturn((int)OrbisGen2Result.ORBIS_GEN2_ERROR_INVALID_ARGUMENT);
}
// Same rule as AudioOut2 PortGetState: never bulk-write onto the caller
// stack. Some titles place small locals next to the canary; a full
// SceAudioOutPortState write smashes it.
if (IsGuestStackAddress(stateAddress))
{
return ctx.SetReturn(0);
}
// SceAudioOutPortState: report a connected primary output at full volume
// so pacing/mixing code sees a live port. We do no host rerouting, so
// rerouteCounter and flag stay zero.
@@ -229,6 +237,9 @@ public static class AudioOutExports
return ctx.SetReturn(0);
}
private static bool IsGuestStackAddress(ulong value) =>
value >= 0x0000_7FF0_0000_0000UL && value <= 0x0000_7FFF_FFFF_FFFFUL;
[SysAbiExport(
Nid = "w3PdaSTSwGE",
ExportName = "sceAudioOutOutputs",