[HLE] Fix guest-thread sync and boot for Unreal Engine titles (#102)

* [HLE] Fix guest-thread sync and boot for Unreal Engine titles

Silent Hill: The Short Message (and other UE titles) now boot the full
engine thread graph instead of hanging early. Four related fixes:

- pthread cond/mutex semantics: retain a signal raised with no waiter as
  pending, and key block/wake on the state's identity rather than a
  resolved address that could differ between lock and unlock. This ends
  the ~1.5M-call cond_wait busy-spin.

- Warm HLE type initializers and force-JIT their methods on a host thread
  at Freeze(). A .cctor or first-time JIT running on a guest thread's
  hijacked stack fail-fasts the CLR as "Invalid Program: attempted to
  call a UnmanagedCallersOnly method from managed code".

- Guest thread scheduling: pump after a wake so a readied thread actually
  runs, add a dispatcher thread for when every guest thread is parked,
  and make the pump-depth guard an atomic CAS.

- Route mutex/rwlock lock/unlock off the non-blocking leaf-import fast
  path so a contended lock can deschedule its guest thread.

Ported from the unreal-boot-fixes branch.

* [HLE] Keep mutex/rwlock unlock on the leaf-import fast path

The previous change routed all mutex/rwlock lock and unlock NIDs off the
leaf fast path so a contended lock could deschedule its guest thread. But
unlock never blocks, and taking it off the fast path made it slow enough
that Demon's Souls' job workers livelocked in a guest spinlock (millions
of mutex_unlock calls, no import progress, main thread stuck in
sceKernelWaitEventFlag).

Only *lock* needs to leave the leaf path. Restore the four unlock NIDs
(mutex + rwlock) so guest spinlocks stay cheap, while lock/rd/wrlock
remain off it for the blocking case Silent Hill needs.

* [HLE] Gate pthread_mutex_lock guest-thread blocking (fixes Demon's Souls)

Re-enabling cooperative deschedule on a contended pthread_mutex_lock
regressed Demon's Souls: its job workers run on libSceFiber, and blocking
a guest thread mid-fiber left sceFiberSwitch returning ESRCH followed by
a null fiber-context deref (0xC0000005). Bisect confirmed the pthread
change as the cause; the game reaches the same point as before it once
the block is skipped.

Gate the block behind SHARPEMU_MUTEX_LOCK_BLOCKING (off by default) so
contended locks fall through to the synchronous host-thread wait. The
rest of the pthread fixes (cond_wait pending signals, identity wake keys)
are unaffected.
This commit is contained in:
Spooks
2026-07-13 09:59:38 -06:00
committed by GitHub
parent 0565d01744
commit 63b440efcd
5 changed files with 753 additions and 66 deletions
@@ -2782,11 +2782,10 @@ public sealed unsafe partial class DirectExecutionBackend : INativeCpuBackend, I
{
return;
}
if (_guestThreadPumpDepth != 0)
if (Interlocked.CompareExchange(ref _guestThreadPumpDepth, 1, 0) != 0)
{
return;
}
_guestThreadPumpDepth++;
try
{
for (int i = 0; i < 8; i++)
@@ -2832,7 +2831,7 @@ public sealed unsafe partial class DirectExecutionBackend : INativeCpuBackend, I
}
finally
{
_guestThreadPumpDepth--;
Volatile.Write(ref _guestThreadPumpDepth, 0);
}
}
@@ -2875,9 +2874,18 @@ public sealed unsafe partial class DirectExecutionBackend : INativeCpuBackend, I
}
}
if (wakeCount != 0 && _logGuestThreads)
if (wakeCount != 0)
{
Console.Error.WriteLine($"[LOADER][INFO] guest_threads.wake key={wakeKey} count={wakeCount}");
if (_logGuestThreads)
{
Console.Error.WriteLine($"[LOADER][INFO] guest_threads.wake key={wakeKey} count={wakeCount}");
}
// Pump or the readied thread waits for an import dispatch that never comes.
if (_cpuContext is { } wakeContext)
{
Pump(wakeContext, "wake");
}
}
return wakeCount;
@@ -4422,6 +4430,26 @@ public sealed unsafe partial class DirectExecutionBackend : INativeCpuBackend, I
return;
}
_stallWatchdogStop = false;
// Drives woken threads when every guest thread is parked (nothing dispatches then).
var dispatcherThread = new Thread(new ThreadStart(delegate
{
while (!_stallWatchdogStop)
{
Thread.Sleep(1);
WakeExpiredBlockedGuestThreads();
if (Volatile.Read(ref _readyGuestThreadCount) > 0 && _cpuContext is { } dispatchContext)
{
Pump(dispatchContext, "dispatcher");
}
}
}))
{
IsBackground = true,
Name = "SharpEmu-GuestThreadDispatcher"
};
dispatcherThread.Start();
long num = (long)((double)stallWatchdogSeconds * Stopwatch.Frequency);
_stallWatchdogThread = new Thread(new ThreadStart(delegate
{
@@ -4450,6 +4478,15 @@ public sealed unsafe partial class DirectExecutionBackend : INativeCpuBackend, I
MarkExecutionProgress();
continue;
}
if (IsExpectedBlockingImportStall(out var blockingNid, out var blockingName))
{
Console.Error.WriteLine(
$"[LOADER][WARN] No import progress for {stallWatchdogSeconds}s while waiting in {blockingName} ({blockingNid}); continuing.");
LogStallWatchdogSnapshot();
Console.Error.Flush();
MarkExecutionProgress();
continue;
}
if (Interlocked.Exchange(ref _stallWatchdogTriggered, 1) != 0)
{
continue;
@@ -4485,6 +4522,38 @@ public sealed unsafe partial class DirectExecutionBackend : INativeCpuBackend, I
return false;
}
// A thread parked in a blocking wait is idle by design, not stalled.
private bool IsExpectedBlockingImportStall(out string nid, out string name)
{
nid = string.Empty;
name = string.Empty;
var cpuContext = _cpuContext;
if (cpuContext is null)
{
return false;
}
var importAddress = cpuContext.Rip & 0xFFFFFFFFFFFFFFF0uL;
foreach (var entry in _importEntries)
{
if (entry.Address != importAddress)
{
continue;
}
nid = entry.Nid;
name = _moduleManager.TryGetExport(nid, out var export)
? $"{export.LibraryName}:{export.Name}"
: nid;
return nid is
"Op8TBGY5KHg" or // pthread_cond_wait
"27bAgiJmOh0" or // pthread_cond_timedwait
"fzyMKs9kim0"; // sceKernelWaitEqueue
}
return false;
}
private void StopStallWatchdog()
{
_stallWatchdogStop = true;