* [HLE] Fix guest-thread sync and boot for Unreal Engine titles
Silent Hill: The Short Message (and other UE titles) now boot the full
engine thread graph instead of hanging early. Four related fixes:
- pthread cond/mutex semantics: retain a signal raised with no waiter as
pending, and key block/wake on the state's identity rather than a
resolved address that could differ between lock and unlock. This ends
the ~1.5M-call cond_wait busy-spin.
- Warm HLE type initializers and force-JIT their methods on a host thread
at Freeze(). A .cctor or first-time JIT running on a guest thread's
hijacked stack fail-fasts the CLR as "Invalid Program: attempted to
call a UnmanagedCallersOnly method from managed code".
- Guest thread scheduling: pump after a wake so a readied thread actually
runs, add a dispatcher thread for when every guest thread is parked,
and make the pump-depth guard an atomic CAS.
- Route mutex/rwlock lock/unlock off the non-blocking leaf-import fast
path so a contended lock can deschedule its guest thread.
Ported from the unreal-boot-fixes branch.
* [HLE] Keep mutex/rwlock unlock on the leaf-import fast path
The previous change routed all mutex/rwlock lock and unlock NIDs off the
leaf fast path so a contended lock could deschedule its guest thread. But
unlock never blocks, and taking it off the fast path made it slow enough
that Demon's Souls' job workers livelocked in a guest spinlock (millions
of mutex_unlock calls, no import progress, main thread stuck in
sceKernelWaitEventFlag).
Only *lock* needs to leave the leaf path. Restore the four unlock NIDs
(mutex + rwlock) so guest spinlocks stay cheap, while lock/rd/wrlock
remain off it for the blocking case Silent Hill needs.
* [HLE] Gate pthread_mutex_lock guest-thread blocking (fixes Demon's Souls)
Re-enabling cooperative deschedule on a contended pthread_mutex_lock
regressed Demon's Souls: its job workers run on libSceFiber, and blocking
a guest thread mid-fiber left sceFiberSwitch returning ESRCH followed by
a null fiber-context deref (0xC0000005). Bisect confirmed the pthread
change as the cause; the game reaches the same point as before it once
the block is skipped.
Gate the block behind SHARPEMU_MUTEX_LOCK_BLOCKING (off by default) so
contended locks fall through to the synchronous host-thread wait. The
rest of the pthread fixes (cond_wait pending signals, identity wake keys)
are unaffected.
* fix(core): implement 4-tier lazy dlsym stub materialization and argument normalization
Enforce transactional and thread-safe resolution for standalone ELF bootstrapper pipelines. - Implement a 4-tier additive fallback cascade (T0: runtime symbols, T1: import entries scan, T2: Aerolib mapping, T3: runtime slack-pool lazy stub allocation at 0x7000_0000_0000). - Fix UnmanagedCallersOnly CLR runtime crashes on second bootstrap by adding NormalizeKernelDynlibDlsymArguments to detect and swap mirrored (symbol, handle) register inputs via rigorous pointer bounds verification. - Protect failure paths via CompleteKernelDynlibDlsymFailure, cleanly zero-filling target outputAddress buffers and returns Rax = -1 with zero managed logging execution in hot native paths.
* fix(core): harden lazy-stub diagnostics with COW snapshots and deferred bootstrap logging
Follow-up to lazy stub pool copy-on-write publishing in TryGetOrCreateLazyImportStub. - Snapshot _importEntries in ProbeReturnRip before near-call and PLT import lookup loops to prevent torn iteration during concurrent array replacement. - Defer SHARPEMU_LOG_BOOTSTRAP output: hot path records raw register slots in a ring buffer under lock; TryReadAsciiZ and Console.Error run only after import handler completion via DrainDeferredBootstrapTraces. - Normalize bootstrap dynlib register order at DispatchImport gateway entry before any logging or trace reads, so swapped RDI/RSI on Import#2 cannot fault the native gateway when bootstrap tracing is enabled. - Resolve lazy stub pool bounds from the full SelfLoader-mapped import region via VirtualQuery instead of a hardcoded 4 KiB cap. - Use ConcurrentDictionary for runtime symbol registration during concurrent dlsym. - Emit distinct [LOADER][WARN] reasons when import stub region resolution fails versus lazy stub pool exhaustion.
* [agc] WAIT_REG_MEM suspend/resume, draw packet fixes, new HLE exports, debug cleanup
Rebased onto upstream 79a7437 (par274/sharpemu, rewritten history).
- GpuWaitRegistry: DCBs suspended on unsatisfied WAIT_REG_MEM are re-polled
against guest memory on every submit; fixed 64-bit and standard packet parse
offsets, apply the mask, treat PM4 compare function 0 as "always".
- TryReadSubmittedDrawCount: accept the 5-dword ItDrawIndex2 form emitted by
DcbDrawIndex (count at +4); menu draws were silently discarded before.
- sceAgcDriverSubmitMultiDcbs: reversed ABI (rdi=address array, rsi=dword
sizes, rdx=count).
- VideoOut: vblank events, sceVideoOutGetFlipStatus, buffers registered via
sceVideoOutRegisterBuffers are valid flip targets.
- New HLE: libc stdio (fopen/fread/fseek/ftell/fclose/fgets), Dinkumware
_Getpctype ctype table, NpTrophy2 stubs, AMPR PAK sequential-read tracker,
MsgDialog lifecycle, NGS2 alt NIDs + dummy vtable for handle objects,
guarded memset intrinsic, abort()/strcasecmp null-arg recovery.
- Removed investigation-only code (INT3 breakpoints, qfont/mcpp dumps,
error-candidate printf traces, unconditional debug logs).
First rendered frame: Quake (PPSA01880) presents a 1920x1080 guest frame.
* Implemented a guarded native intrinsic (rep movsb) in DirectExecutionBackend to bypass HLE dispatch overhead, while preserving memory safety checks.
* [hle] clock_gettime clock ids, AudioOut2 canary fix, NID rebinds, new offline stubs
- clock_gettime (lLMT9vJAck0): support CLOCK_SECOND and the *_PRECISE/*_FAST
variants instead of returning EINVAL, which games treated as fatal and
retried in a tight loop.
- AudioOut2: context param writes shrunk to the guest-observed layout (the
old 0x80-byte reset smashed the stack canary at +0x60 and killed audio
init); ContextQueryMemory writes the single u64 the caller expects.
- NGS2: dropped wrong alt-NID aliases (they hash to sceImeUpdate,
sceMouseRead, sceSystemGestureUpdateAllTouchRecognizer - now bound in
their real libraries); added sceNgs2PanInit; fixed VoiceGetState NIDs.
- New verified stubs: sceUltInitialize, sceNpUniversalDataSystemDestroyHandle,
sceNpGetOnlineId, sceNpGetNpReachabilityState, sceImeKeyboardOpen,
sceImeKeyboardGetResourceId, sceMouseOpen, sceKernelAprGetFileSize.
- Import gateway: unwind guest workers at dispatch during backend teardown;
env-gated SHARPEMU_LOG_THREAD_MODE tracing.
* [cpu] Isolate guest execution on native worker threads
Guest entry stubs no longer run above CLR-managed frames: each run is handed
to a pooled raw OS thread whose loop is emitted native code. While guest code
executes there is not a single managed frame on the thread and it stays in
preemptive GC mode, so the GC never walks a frame chain interleaved with
guest stubs that carry no CLR unwind info (the ReversePInvokeBadTransition /
UnmanagedCallersOnly FailFast class of crashes on pumped guest threads).
- NativeGuestExecutor: CreateThread + emitted run loop (WaitForSingleObject,
UnmanagedCallersOnly prologue/epilogue, entry stub call, SetEvent). The
prologue rebinds guest TLS, the host-RSP slot, thread affinity and the
Active* ambient per run, so workers carry no guest identity and pool
freely; the orchestrating managed thread parks in a preemptive wait.
- All three entry sites route through RunGuestEntryStub: guest thread
entries, blocked-continuation resumes, and the main ExecuteEntry.
- Teardown stops workers before any executable stub or TLS index they
reference is freed; a worker that will not stop leaks its loop instead of
freeing running code.
- Kill switch: SHARPEMU_DISABLE_NATIVE_GUEST_WORKERS=1 restores the inline
calli path.
* [cpu] Implement SysV variadic float ABI (xmm0-7 capture, float returns, printf %f)
The import trampoline spilled only xmm0 and never reloaded a return xmm0. The
guest uses the System V AMD64 ABI: variadic float args pass in xmm0..xmm7 and
float/double returns come back in xmm0. As a result variadic float args past
the first were unavailable to HLE handlers, float returns never reached the
guest, and direct printf read %f/%e/%g from GP registers instead of XMM,
printing garbage and desynchronizing every following argument.
- Trampoline: spill xmm0..xmm7 into a 0x80-byte save area below the GP argpack
(r12 stays at the argpack base) and reload the return xmm0 in the epilogue.
- Gateway: read xmm0..7 from the save area into CpuContext and write the
handler's xmm0 back. XMM is caller-saved in SysV, so restoring xmm0 on return
is safe for non-float imports too.
- RegisterPrintfArgumentSource: read float args from xmm0..7 with independent
GP/FP counters and a shared stack-overflow cursor.
Every emitted byte was decoded; a unit test confirms float args read xmm0..7
(not GP) and interleaved "%d %f %d %f" stays synchronized. Build 0/0.
* [cpu] Document the scalar-only leaf-import constraint at its registration site
- IsLeafImport: spell out the no-XMM-args / no-XMM-return invariant the fast
path relies on and what breaks if it is violated; record the 2026-07-11 audit.
- Name every previously uncommented NID in the leaf list (mutex lock/unlock,
usleep, the Ampr/Apr command-buffer block, the unknown AGC packet NID).
- IsNoBlockLeafImport: document that it is a sub-filter of IsLeafImport and
that its five extra entries currently take the full gateway path; fix the
mislabeled K-jXhbt2gn4 comment (pthread_mutex_trylock, not
scePthreadMutexTrylock, which is upoVrzMHFeE).
- Point the DispatchImport call-site note at the audited list.
Comment-only change: the comment-stripped diff is empty and the solution
builds with 0 warnings / 0 errors.
* [agc] Add shader type 4 (GS) and register defaults v13 support
Astro Bot (#11) crashes on boot due to two missing GPU features:
1. Shader type 4 (Geometry Shader) — SPI_SHADER_PGM_LO/HI register
offsets 0x8A/0x8B were missing. Added constants and switch cases
for shader type 4 in GetExpectedSpiShaderPgmLo/Hi. Also added
type 4 to IsEsGeometryShaderType (2 or 4 or 6).
2. Register defaults version 13 — was not recognized as supported.
Added RegisterDefaultsVersion13 constant and included it in
IsSupportedRegisterDefaultsVersion.
* [kernel] Add POSIX pthread_cond_timedwait export
SILENT HILL (#4) and Poppy Playtime (#3) crash on boot due to
missing POSIX pthread_cond_timedwait (NID 27bAgiJmOh0).
The Sony wrapper scePthreadCondTimedwait (NID BmMjYxmew1w) was
already implemented, but the raw POSIX symbol was not exported.
Added [SysAbiExport] for pthread_cond_timedwait delegating to
existing PthreadCondWaitCore with timed: true.
* [memory] Fix FlushInstructionCache null process handle
PhysicalVirtualMemory.cs called FlushInstructionCache with null as the
process handle in two places (SetProtection and TryWriteExclusive).
On Windows, a null handle does not reliably resolve to the current
process — the correct call is GetCurrentProcess() (pseudo-handle -1).
Also corrected the P/Invoke signature:
- Changed return type from void to bool with [return: MarshalAs(Bool)]
- Added SetLastError = true
- Added GetCurrentProcess() P/Invoke import
This matches the pattern already used in DirectExecutionBackend.cs
which correctly passes GetCurrentProcess() to all FlushInstructionCache
calls.
* [hle] Distinguish NOT_FOUND from NOT_IMPLEMENTED and log duplicate NIDs
Three diagnostic improvements to the HLE dispatch path:
1. ModuleManager.RegisterFromAssembly — duplicate NID registration was
silently skipped (dispatchTable first-wins, exportTable last-wins,
causing metadata divergence). Now logs a warning with the NID and
export name so conflicts are visible.
2. ModuleManager.TryDispatch — generation mismatch returned
ORBIS_GEN2_ERROR_NOT_FOUND, conflating 'function does not exist'
with 'function exists but not for this generation'. Now returns
ORBIS_GEN2_ERROR_NOT_IMPLEMENTED for generation mismatch, matching
the existing convention in CpuDispatcher. Also adds debug logging
for both NOT_FOUND and NOT_IMPLEMENTED paths.
3. DirectExecutionBackend.Imports.cs — the import dispatch else-branch
(the actual hot path that bypasses ModuleManager.TyDispatch via
cached export) had the same conflation. Split into:
- else if (export exists but generation mismatch) → NOT_IMPLEMENTED
- else (no export at all) → NOT_FOUND
This makes runtime diagnostics correctly distinguish missing exports
from generation-unsupported exports.
* [cpu] Check VirtualProtect return values in all stub creation paths
9 VirtualProtect calls in DirectExecutionBackend.cs had unchecked
return values. If VirtualProtect silently fails, memory protection
remains incorrect — stubs allocated with PAGE_EXECUTE_READWRITE (0x40)
never get downgraded to PAGE_EXECUTE_READ (0x20), or guest thread
entry stubs never get upgraded to writable. This causes access
violations on next execution or silent data corruption.
Fixed all 9 sites with proper error handling:
- 6 stub creation methods (return 0 on failure + log error)
- 2 guest thread entry methods (set reason + return Exception)
- 1 guest entry method (set LastError + return MEMORY_FAULT)
Stub creation sites fixed:
- CreateImportDispatchStub (line ~1683)
- EnsureTlsHandler (void, log + return)
- CreateUnresolvedReturnStub (return 0)
- CreateGuestReturnStub (return 0)
- CreateExceptionHandlerTrampoline (return 0)
- CreateTlsStoreHelperStub (return 0)
Guest thread entry sites fixed:
- StartGuestThreadNativeCall (return Exception)
- StartGuestContinuationNativeCall (return Exception)
- RunGuestEntryPoint (return MEMORY_FAULT)
* [kernel] Remove unused duplicate _nextFileDescriptor field
KernelExports.cs declared _nextFileDescriptor but never used it.
The actual field used for file descriptor allocation lives in
KernelMemoryCompatExports.cs (lines 1314, 1337). This was a dead
duplicate causing CS0414 warning.
Build is now 0 errors, 0 warnings.
---------
Co-authored-by: Hermes Atlas <hermesatlas@example.com>
Spooks