* [Tests] Add SharpEmu.Libs.Tests project
Introduce an xunit project for the HLE libs with a minimal ICpuMemory fake,
so library-level exports and helpers can be exercised without a live guest.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [Ampr] Disambiguate pak size-collisions by read locality
PakDirectoryTracker resolves a sequential AMPR read (offset -1) back to an
absolute pak offset by matching the requested byte count against the PACK
directory. When several files share that byte count it took the first
unconsumed match in directory order, which mis-resolves out-of-order reads:
progs/h_ogre.mdl and bots/navigation/death32c.nav are both 0x3A34 bytes, and
death32c.nav sits earlier in the directory and is never read during Quake's
intro demo, so requesting h_ogre.mdl returned the nav file's bytes. The engine
then parsed "NAV2" as a brush model, failed the version check and aborted.
Pick the unconsumed same-size entry nearest the running read cursor instead.
id archives cluster related assets and the guest streams them with locality,
so this lands on the intended file; contiguous same-size runs (the
gfx/weapons/ww_*.lmp icons) still resolve in packed order.
Verified against a Quake dump: the abort is gone, h_ogre.mdl reads correctly,
and the intro demo reaches its main loop and renders instead of dying at the
error dialog.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* [Json] Implement sce::Json::Value and Json::String construct/set/destroy
libSceJson previously only had the Initializer/MemAllocator setup path.
The Value and String classes themselves were entirely absent, so a
Prospero title that builds a JSON tree (Quake PPSA01880 does, to shape
a web-API request) hit unresolved imports and faulted on the call. The
imports it left unresolved right before its access violation are exactly
these Value ctors/setters and String ctor/dtor.
Model the Value/String payload host-side (JsonObjectHeap), keyed by the
guest `this` pointer, following the handle-shadow pattern already used
by Ngs2Exports. The guest object bytes are deliberately not written:
these objects are usually stack-allocated with an unknown real layout,
and writing a guessed layout risks smashing an adjacent stack canary
(the same hazard the AudioOut2 context-param note in this tree records).
Constructors and setters follow the Itanium ABI and return `this` in rax,
which is correct whether the real setter returns void or Value&.
Covered NIDs (complete-object C1/D1 variants, matching the observed
imports): Value(default/bool/long/ulong/double/ValueType/char*/String),
Value::~Value, Value::set(bool/long/ulong/double/ValueType/char*/String),
Value::clear, String(char*/default/copy), String::~String.
Only the payload the guest can reach through library methods is modelled;
direct guest reads of the object bytes are out of scope and would need
observed layout evidence.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [Tests] Add SharpEmu.Libs.Tests covering the Json Value/String exports
First test project for SharpEmu.Libs (xunit), the SharpEmu.Libs.Tests
layout the maintainer already agreed to in issue #36.
- A FakeCpuMemory (single contiguous region) drives the exports at the
CpuContext level with no live guest.
- Direct-call tests: ctor/setter round-trips for bool/int/uint/double
(read from xmm0)/char*/String/ValueType, destructor cleanup, and the
graceful-degradation paths (missing String shadow and a faulting char*
pointer both fall back to the empty string instead of throwing).
- Registration test: a real ModuleManager scans SharpEmu.Libs and the
nine NIDs Quake left unresolved now resolve to the libSceJson exports
and dispatch cleanly (returns `this` in rax).
InternalsVisibleTo exposes JsonObjectHeap to the test assembly. The test
project's packages.lock.json is committed for CI locked-mode restore;
CI does not run tests yet, left as a maintainer decision.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [Json] Add Initializer::setGlobalNullAccessCallback
Quake calls it during kexPSNWebAPI::Initialize and treats the
not-found error as fatal for the whole Np Web API bring-up. Store the
guest hook (never invoked by this HLE: shadows degrade to defaults
instead of dereferencing missing members) and return success.
Verified against the dump: the "setGlobalNullAccessCallback failed
(0x80020002)" line is gone and kexPSNWebAPI::Initialize now logs
"Np Web API Initialized"; the next blockers are sceNpAuthCreateRequest
and sceUserServiceInitialize ordering, outside libSceJson.
Also pins both Json test classes to one xunit collection: they share
JsonObjectHeap statics and parallel class execution raced ResetForTests
against a running test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Add SaveData transaction and NP UDS layout HLE stubs
Wire Prepare, Commit, and Umount2 for implicit save transactions,
unregister guest mounts on Umount2, and add NP UDS CreateEvent,
DestroyEvent, and EventPropertyObjectSetString for layout-load imports.
* Add NP UDS SetArray and PostEvent layout HLE stubs
Add sceNpUniversalDataSystemEventPropertyObjectSetArray and
sceNpUniversalDataSystemPostEvent for layout-load imports on PPSA02929.
Move KMcEa+rHsIo from libKernel MapMemory mislabel to sceAvPlayerAddSource.
Align WV1GwM32NgY ExportName with sceNpWebApi2PushEventCreateHandle. Behavior unchanged.
TryCompileVertexShader gained an optional scalarRegisterBufferIndex
parameter (#156), and reflection Invoke does not apply C# default
parameter values, so ShaderDump crashed with
TargetParameterCountException. Pad trailing optional parameters with
Type.Missing under BindingFlags.OptionalParamBinding so the declared
defaults are used; only a new required parameter now needs a tool
update, and that fails with a named error instead of a crash.
Verified: all five programs behave as expected (exit 0), all eight
emitted blobs pass spirv-val --target-env vulkan1.3.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Rework the sceMsgDialog and sceSaveDataDialog HLE state machines so the full
Initialize -> Open -> poll -> GetResult -> Close/Terminate lifecycle honors the
common-dialog contract, and add the three missing sceMsgDialogProgressBar* exports.
- Fix an unreachable close path: sceSaveDataDialogClose already did a
RUNNING -> FINISHED compare-exchange, but Open jumped straight to FINISHED, so
RUNNING never existed and Close could only return NOT_RUNNING. Open now enters
RUNNING and the first status poll advances it to FINISHED. Same model applied to
sceMsgDialog.
- Return the real SCE_COMMON_DIALOG_ERROR_* codes (0x80B8xxxx) from sceMsgDialog*
instead of emulator-internal result codes, with the missing argument/state guards
(ARG_NULL, NOT_INITIALIZED, BUSY, NOT_FINISHED, NOT_RUNNING).
- GetResult reports buttonId = 1 (affirmative) instead of 0, the invalid sentinel a
yes/no prompt could mis-branch on.
- Add sceMsgDialogProgressBarSetValue, sceMsgDialogProgressBarInc and
sceMsgDialogProgressBarSetMsg (NIDs wTpfglkmv34, Gc5k1qcK4fs, 6H-71OdrpXM), gated
on the service being initialized.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Resolves _sceUlobjmgrRegisterObject (BG26hBGiNlw) and
_sceUlobjmgrUnregisterObject (Smf+fUNblPc), reported as unresolved by
testers, plus four sceNpEAAccess exports. Names taken from shadPS4's
NID tables and each verified by recomputing the NID with the repo's
name2nid derivation before inclusion. aerolib.bin regenerated with
scripts/generate_aerolib_binary.py.
Co-authored-by: tensorcrush <tensorcrush@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
SharpEmu.Tools.GpuConformance executes the exec-cs.spv blob produced by
SharpEmu.Tools.ShaderDump on a real Vulkan device (preferring a discrete
GPU) and compares every word of the 64-byte storage buffer against
CPU-computed expectations, bit for bit. Creating the compute pipeline
doubles as a driver-acceptance check for SharpEmu's emitted SPIR-V.
The checks cover the three ALU results, the store attempted with EXEC=0
(its destination must keep the sentinel), the store after EXEC is
restored, and all trailing sentinel words. Any mismatch counts toward the
failure total and makes the tool exit non-zero.
Verified on an RTX 3060 Laptop GPU (NVIDIA) with all values matching, and
the failure path verified to exit 1 by running a non-storing blob.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Added about tab with github and discord
* Added discord & github svgs and svg support
* Changed svg to pngs and localization text in english & spanish
* [AGC] Complete gfx10 v_cmpx_f32 decode and emit ordered/unordered float compares
Add the missing v_cmpx_*_f32 VOPC decode entries (0x17-0x1C, 0x1F) and
emission for the ordered/unordered predicates: nlg maps to OpFUnordEqual,
while o/u are lowered from OpIsNan (unordered = isnan(a) || isnan(b),
ordered = !unordered) because SPIR-V's OpOrdered/OpUnordered require the
Kernel capability and are invalid in Vulkan shader modules.
Opcode numbers cross-checked against LLVM's llvm-mc regression tests
(llvm/test/MC/AMDGPU/gfx10_asm_vopc.s, gfx10_asm_vopcx.s); emitted
lowering validated with spirv-val --target-env vulkan1.1.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [AGC] Write VCC only for non-X vector compares
On gfx10 the VCmpx encodings have no sdst and define EXEC only, so the
unconditional VCC store clobbered VCC on every VCmpx. Move the VCC store
to the non-X path; EXEC keeps the existing old-EXEC & condition update.
Addresses review feedback on #122.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: tensorcrush <tensorcrush@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Implement six missing libc string/memory search and concatenation
routines in the kernel compat layer. Titles frequently call these
during startup string handling (path parsing, config lookups, format
string assembly), and without them the loader currently falls through
to unresolved-import handling.
The implementations follow the existing byte-at-a-time compat helpers
(TryReadCompat/TryWriteCompat) already used by strcpy/strncpy/memcmp,
matching native semantics: strchr/strrchr scan through and including
the terminator, memchr is bounded strictly by count, strcat/strncat
overwrite the destination terminator and re-terminate, and strstr
returns the haystack pointer for an empty needle. NIDs are the
libSceLibcInternal/libc symbol hashes for each name.
SharpEmu.Tools.ShaderDump feeds hand-assembled Gen5 (gfx10) instruction
words — cross-checked against LLVM's AMDGPU target definitions — through
the real Gen5ShaderTranslator -> Gen5SpirvTranslator pipeline via
reflection (no emulator source changes; the project is not in the main
solution) and dumps the resulting vertex/compute SPIR-V blobs for
inspection with spirv-val / spirv-dis.
Each bundled program carries an expectation: fmac/muls/sopp-hints/exec
must decode and emit both stages, while sopp-mode (s_round_mode,
s_denorm_mode) pins the loud unknown-sopp decode failure those FP MODE
writes must keep producing until their semantics are modeled (#108). Any
unexpected outcome makes the tool exit non-zero, so it can gate scripts
or CI.
The exec program computes real ALU results and stores them with
buffer_store_dword, toggling EXEC off and on around a pair of stores; its
exec-cs.spv blob is designed for numeric verification on a real Vulkan
device (follow-up tool).
All dumped blobs pass spirv-val --target-env vulkan1.3.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Expand Aerolib catalog from nids.csv and wire socket/net NID handlers
Load authoritative NID pairs from scripts/nids.csv with ps5_names fallback.
Replace mislabeled kernel zero stubs with socket/connect/bind/getsockname HLE
and sceNet byte-order exports backed by the CSV symbol names.
* Add inet_pton, htons, and bzero kernel compat with CSV NIDs
Wire libc network helpers using authoritative NID names from nids.csv
instead of synthetic Gst* exports used on the crt-loader branch.
* Fix REUSE annotation for scripts/nids.csv
* Drop bundled nids.csv; extend ps5_names and regenerate Aerolib
Remove scripts/nids.csv from the repository and fold csv-only symbol names
into scripts/ps5_names.txt so Aerolib keeps the full catalog via name2nid.
Two follow-ups to upstream #102's condition-variable changes, as analyzed
in upstream issue #113:
- The pending-signal consume path reacquired the guest mutex while still
holding the condition state lock, inverting lock order against
cond-signal (mutex -> SyncRoot) and deadlocking both threads. Leave the
condition lock before relocking, matching the normal wake path.
This unfroze Dreaming Sarah (PPSA02929) at its title screen.
- pthread_cond_timedwait's third argument is a pointer to an absolute
CLOCK_REALTIME timespec, not a relative microsecond count; the guest
address was being truncated into a duration, yielding arbitrary
timeouts. Read the timespec and convert to a relative wait.
scePthreadCondTimedwait keeps its separate relative-time ABI.
* [AGC] Decode gfx10 SOPP hint instructions
s_clause (0x21), s_waitcnt_depctr (0x23), s_round_mode (0x24) and s_denorm_mode (0x25) were missing from the SOPP decode table, so any shader containing one of these scheduling/mode hints failed to decode entirely with unknown-sopp. No emitter changes are needed: non-branch SOPP instructions are already emitted as no-ops. Opcodes verified against LLVM SOPInstructions.td (SOPP_Real_32_gfx10); decode and end-to-end SPIR-V compilation verified with a synthetic program containing all four hints.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [AGC] Narrow SOPP additions to scheduling hints only
Per review: s_round_mode (0x24) and s_denorm_mode (0x25) write the
shader floating-point MODE state, and the emitter's blanket SOPP no-op
would have silently ignored their simm16 payloads, trading a loud
decode failure for a potential floating-point semantics mismatch. They
are removed and keep failing decode explicitly until their semantics
are modeled or conservatively validated.
s_clause (0x21) and s_waitcnt_depctr (0x23) remain: they are pure
scheduler/dependency hints with no value semantics.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* [HLE] Fix guest-thread sync and boot for Unreal Engine titles
Silent Hill: The Short Message (and other UE titles) now boot the full
engine thread graph instead of hanging early. Four related fixes:
- pthread cond/mutex semantics: retain a signal raised with no waiter as
pending, and key block/wake on the state's identity rather than a
resolved address that could differ between lock and unlock. This ends
the ~1.5M-call cond_wait busy-spin.
- Warm HLE type initializers and force-JIT their methods on a host thread
at Freeze(). A .cctor or first-time JIT running on a guest thread's
hijacked stack fail-fasts the CLR as "Invalid Program: attempted to
call a UnmanagedCallersOnly method from managed code".
- Guest thread scheduling: pump after a wake so a readied thread actually
runs, add a dispatcher thread for when every guest thread is parked,
and make the pump-depth guard an atomic CAS.
- Route mutex/rwlock lock/unlock off the non-blocking leaf-import fast
path so a contended lock can deschedule its guest thread.
Ported from the unreal-boot-fixes branch.
* [HLE] Keep mutex/rwlock unlock on the leaf-import fast path
The previous change routed all mutex/rwlock lock and unlock NIDs off the
leaf fast path so a contended lock could deschedule its guest thread. But
unlock never blocks, and taking it off the fast path made it slow enough
that Demon's Souls' job workers livelocked in a guest spinlock (millions
of mutex_unlock calls, no import progress, main thread stuck in
sceKernelWaitEventFlag).
Only *lock* needs to leave the leaf path. Restore the four unlock NIDs
(mutex + rwlock) so guest spinlocks stay cheap, while lock/rd/wrlock
remain off it for the blocking case Silent Hill needs.
* [HLE] Gate pthread_mutex_lock guest-thread blocking (fixes Demon's Souls)
Re-enabling cooperative deschedule on a contended pthread_mutex_lock
regressed Demon's Souls: its job workers run on libSceFiber, and blocking
a guest thread mid-fiber left sceFiberSwitch returning ESRCH followed by
a null fiber-context deref (0xC0000005). Bisect confirmed the pthread
change as the cause; the game reaches the same point as before it once
the block is skipped.
Gate the block behind SHARPEMU_MUTEX_LOCK_BLOCKING (off by default) so
contended locks fall through to the synchronous host-thread wait. The
rest of the pthread fixes (cond_wait pending signals, identity wake keys)
are unaffected.
José Luis Caravaca Carretero