mirror of
https://github.com/par274/sharpemu.git
synced 2026-08-01 23:49:44 +08:00
f2d90513582bb2b235004a7f67c20e249518221e
29 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Peter Bonanni
|
f2d9051358 | Fix macOS fixed-address allocation collisions (#246) | ||
|
Miguel Cruz
|
864cbb0fa0 |
[AGC/Vulkan] Extend PS5 runtime and rendering compatibility (#216)
* [Core] Add POSIX native execution and PS5 SELF support Extend the native backend, guest TLS, fixed-address memory, and loader paths needed by PS5 titles on Windows, Linux, and macOS. Keep workstation GC so high-core-count hosts do not reserve over fixed guest image bases. * [HLE] Expand PS5 service and media compatibility Add the kernel, threading, save-data, networking, audio, video-codec, font, dialog, and service exports required by newer PS5 software. Preserve every SysAbi NID currently registered by main while adding the compatibility surface used by ASTRO BOT. * [AGC/Vulkan] Extend Gen5 shader and presentation support Expand PM4 handling, Gen5 shader translation, MRT and packed export support, guest image tracking, depth initialization, texture aliasing, and Vulkan presentation. Add the performance overlay and address-filtered diagnostics used to validate ASTRO BOT with original shaders. * [Core] Align static TLS reservation across hosts * [Pad] Align primary user ID with UserService * [Gpu] Preserve runtime scalar buffers across renderer seam * [AGC] Restore omitted command helper exports * [Vulkan] Reuse primary views for promoted MRT targets * [Vulkan] Preserve scratch storage bindings in compute dispatches |
||
|
Nicola Pomarico
|
90651f26a9 |
Fix macOS exact-address mmap falling back to MAP_FIXED (#214)
The PS5 image loader requires guest images to land at fixed virtual addresses (e.g. the 32 GiB main image base). On macOS the exact-address mmap path only ever passed that address as a hint, never MAP_FIXED, to avoid clobbering untracked host mappings (dyld, JIT heap, Rosetta). On Apple Silicon under Rosetta 2 the kernel does not reliably honor that hint, so the allocation failed deterministically before a single guest instruction ran (reported in #194). Retry with true MAP_FIXED as a fallback only when the hint-only attempt doesn't land at the requested address, so the safer hint path is still tried first. |
||
|
Gutemberg Ribeiro
|
320dbcacba |
[SourceGenerators] Compile-time SysAbi export registry, analyzers, and build-generated aerolib.bin (#204)
* [SourceGenerators] Add the SysAbi export generator and analyzers (phase 0) New SharpEmu.SourceGenerators Roslyn component, complete and tested but consumed by nothing yet — the emulator projects adopt it in the following commits. Ps5Nid ports the PS NID derivation (base64 of the byte-reversed first eight SHA1 bytes of name + fixed suffix) from scripts/generate_aerolib_binary.py to C#, so what has always been a manual, out-of-band computation becomes a compile-time capability. SysAbiExportGenerator emits a per-assembly SysAbiExportRegistry whose CreateExports(Generation) reproduces ModuleManager's reflection scan exactly — same generation inheritance and filtering, same method-name fallback, same libKernel default — with attribute-omitted NIDs derived algorithmically (equivalent to the runtime catalog lookup, which is built from the same computation). Parameterless handlers are adapted to the SysAbiFunction shape; invalid declarations are skipped here because the analyzer rejects them as build errors, so nothing drops silently. SysAbiExportAnalyzer turns the runtime failure modes into diagnostics: SHEM001 duplicate NID (across declared and derived forms), SHEM002 malformed NID, SHEM003 uncallable handler signature, SHEM004 NID contradicting its export name (the class of drift previously fixed by hand), SHEM005 unresolvable export, SHEM006 export name unknown to ps5_names.txt when the catalog is wired as an AdditionalFile, SHEM007 handler not reachable by generated code. The self-contained test suite drives both in-process against the real SharpEmu.HLE metadata: known catalog NID pairs pin the algorithm, the generated registry must itself compile, and each diagnostic has a triggering fixture. Fittingly, the NID pinning test caught a wrong pair in its own first draft — the exact mistake SHEM004 exists to stop. * [SourceGenerators] Adopt the generated export registry in the emulator (phase 1) SharpEmu.Libs consumes the generator and analyzers, with scripts/ps5_names.txt wired as the AdditionalFile catalog. The runtime now registers exports from the compile-time SysAbiExportRegistry instead of the boot-time reflection scan; RegisterFromAssembly is retained solely as the arbiter for a parity test that pins the two tables identical — same NIDs, names, libraries, targets, and handler methods — across Gen4, Gen5, and combined registration. First contact between the analyzer and all 715 existing exports surfaced real drift the old offline checker structurally missed (scripts/check_sysabi_aerolib.py skipped any NID absent from aerolib.bin): three exports whose friendly names collide with real catalog symbols of different NIDs, now suppressed at-site with reasons pending AGC API confirmation, alongside the established synthetic Unknown* labels for uncatalogued NIDs, which prompted a rule refinement — SHEM004 only hard-errors when the export name is a real catalog symbol, since synthetic labels cannot be validated by hashing and the NID is authoritative for them. The two allowlisted mismatches in the python checker no longer trigger anything, and the checker is deleted: the analyzer subsumes it with the semantic model instead of regex, and validates every declared pair rather than only catalog-known NIDs. * [SourceGenerators] Generate aerolib.bin at build time from ps5_names.txt The runtime NID -> name catalog is derived data and no longer lives in the repository: a Framework-only MSBuild task (GenerateAerolibBinaryTask, sharing the same Ps5Nid implementation the analyzers use) builds it into the intermediate directory from scripts/ps5_names.txt — now the single source of truth — and SharpEmu.HLE embeds it from there. The output is byte-identical to the previously committed binary, verified with cmp against git history; a new test pins that the embedded catalog loads and resolves a known symbol both directions. scripts/generate_aerolib_binary.py is deleted (its algorithm lives in Ps5Nid, its invocation in the build); the REUSE annotation for the binary goes with it. MSBuild's Inputs/Outputs check means the ~154k NID hashes only recompute when the names file actually changes. The task implements ITask against Microsoft.Build.Framework directly, keeping the vulnerable-flagged Utilities.Core package out and the analyzer project's file-IO ban suppressed only inside the task itself. * [SourceGenerators] Emit typed-signature register thunks (phase 2) [SysAbiExport] handlers can now be written with real signatures — a CpuContext followed by up to six int/uint/long/ulong parameters — and the generator emits the SysV unmarshalling thunk, mapping parameters positionally to RDI/RSI/RDX/RCX/R8/R9 with the same unchecked-cast idiom hand-written handlers use. SHEM003 accepts the new shape and rejects register overflow and non-register-representable types. Both shapes coexist, so migration is per-handler; sceKernelPollSema, sceKernelSignalSema, and sceKernelCancelSema migrate as the demonstration (the last showing raw ulong guest-address passthrough). The reflection scan cannot represent typed handlers, so it retires here: RegisterFromAssembly, its signature validation, and ResolveExportInfo are deleted, and the parity test that pinned the generated registry to the scan is replaced by content-invariant tests (duplicate-free, full 715-export surface, catalog identity). Deleting the scan surfaced a phase-1 latent regression — the pre-JIT warm sweep enumerated only reflection-scanned assemblies, so the generated registration path warmed nothing and re-exposed the guest-thread fail-fast risk; the warm set is now derived from the registered handler delegates themselves. * [SourceGenerators] Marshal guest strings declaratively with [GuestCString] (phase 3) A string parameter on a typed [SysAbiExport] handler, annotated [GuestCString(maxLength)], now makes the generated thunk read the null-terminated UTF-8 string from the argument register's guest address before the handler runs, returning ORBIS_GEN2_ERROR_MEMORY_FAULT to the guest when the read fails — the exact prologue nearly every string-taking handler writes by hand. The attribute lives in SharpEmu.HLE next to SysAbiExportAttribute; SHEM008 rejects misuse (non-string parameter, non-positive MaxLength) while a bare string parameter stays a SHEM003 signature error. _open, open, and sceKernelOpen migrate as the demonstration; they were chosen because their hand-written prologue faulted on a null pointer the same way the thunk does (handlers that return INVALID_ARGUMENT for null pointers, like sceKernelCreateSema, keep the raw shape so guest- visible semantics stay untouched). * [SourceGenerators] Apply review findings across the branch Behavior: the open/_open/sceKernelOpen [GuestCString] demo migration is reverted — the local compat reader falls back to host memory for paths in loader-mapped regions that ctx.Memory cannot see, so the generated thunk would have turned recoverable reads into MEMORY_FAULT. The marshalling infrastructure stays, proven by generator/analyzer tests; production migration waits for a handler whose semantics the thunk reproduces exactly. A comment on the handler records why. Build robustness: the aerolib target is skipped for design-time builds (the IDE resolves project references without compiling them, so on a fresh clone the task assembly does not exist yet), and the task/names paths are centralized in properties. The generator now emits no registry for export-free assemblies, so referencing the analyzer can never mint a colliding SharpEmu.Generated type. Cleanup and perf: the pragma-suppression sites left mis-indented by the phase-1 relocation are reformatted and the restores moved after the method body; the dead ExportsForTesting hook and its InternalsVisibleTo are deleted; the aerolib task reuses one SHA1 instance across ~150k names; the analyzer caches the parsed catalog per file snapshot instead of re-parsing 150k lines every compilation start, shares the attribute name constant with the generator, and computes the catalog-membership check once. * [CI] Run the test suites in the build workflow The workflow compiled the test projects (they are in SharpEmu.slnx) but never executed them. A solution-level dotnet test now runs between build and publish, so any test failure fails the build — including the AerolibCatalogTests/SysAbiRegistryTests that guard the build-generated aerolib.bin and the generated export registry. Generation failures of aerolib.bin itself already fail the build step: the MSBuild task logs an error event and returns false, and a missing task assembly or missing embedded output are hard MSBuild errors. The NuGet cache key now also tracks the test projects' lock files. * [SourceGenerators] Address review feedback Multi-diagnostic analyzer tests no longer assume a stable diagnostic order (analyzer execution is concurrent), and the aerolib task logs the full exception instead of only its message so build failures keep the type and stack trace. * [SourceGenerators] Address second review round Symbol-name comparisons in the shape rules and analyzer now pin an explicit SymbolDisplayFormat.FullyQualifiedFormat instead of relying on the display-format default, and the aerolib task fails loudly on a symbol name that would overflow the format's ushort length prefix instead of silently truncating it, with null-safe output-directory handling made explicit. * [SourceGenerators] Embed aerolib.bin via a target so design-time builds never reference it The static EmbeddedResource item referenced the generated file even in design-time builds, where the generation target is skipped — on a fresh clone the IDE would try to embed a file that never existed. The item is now created inside an EmbedAerolibBinary target gated on DesignTimeBuild, separate from the generation target so an up-to-date skip of GenerateAerolibBinary cannot drop the item with the rest of its body, and hooked before AssignTargetPaths since dynamic resource items added later miss the resource pipeline. Verified fresh build, incremental rebuild (embedded catalog test both times), and a simulated design-time compile with no artifacts present. * [SourceGenerators] Regenerate test lock file after rebase onto main Rebase fallout: main's package graph shifted under #200, so the SourceGenerators.Tests lock file is re-evaluated to keep --locked-mode restore green at the branch tip. * [Build] Drop NuGet lock files; rely on central package management Central package management was already in effect (ManagePackageVersionsCentrally with all versions in Directory.Packages.props and no inline PackageReference versions), so the per-project packages.lock.json files and the lock-mode workflow only added maintenance overhead. This removes all eleven lock files, drops RestorePackagesWithLockFile so restore no longer regenerates them, and takes --locked-mode off the CI restore steps (re-keying the NuGet cache on the central props files). Package versions remain centrally pinned in Directory.Packages.props. |
||
|
kuba
|
fa2616d224 |
Linux and macOS support (#47)
* [macos/linux] Cross-platform host memory, TLS, and ABI layer for POSIX Introduces the foundation for running SharpEmu on macOS (osx-x64 under Rosetta 2) and Linux (linux-x64). The CPU backend executes guest x86-64 code natively, so these targets run the whole process as x86-64; this commit replaces the Windows-only host primitives with platform-dispatched equivalents so the guest boots and services HLE calls off Windows. Memory (HostMemory.cs, new): a Win32-semantics facade over mmap/mprotect/munmap with a shadow region table answering VirtualQuery. PhysicalVirtualMemory, DirectExecutionBackend, StubManager, and the two Kernel*CompatExports now go through it instead of kernel32 P/Invokes. Exact-address requests use MAP_FIXED_NOREPLACE (Linux) / guarded MAP_FIXED (macOS) so they match Win32 "map there or fail" semantics. TLS + host helpers (PosixHostStubs.cs, new): pthread-backed TLS and Win64-ABI-compatible stubs for the kernel32 helpers the backend embeds into emitted x86-64 code (TlsGetValue, QueryPerformanceCounter, SwitchToThread, Sleep). A Win64->SysV thunk wraps managed callbacks, since .NET on POSIX compiles them for the SysV ABI while the emitted call sites use Win64. Guest address layout: the 0x7FFx window is Windows-only (dyld shared cache / Rosetta runtime live there on POSIX), so stack/TLS/stub regions relocate to 0x6FFx off Windows. Vectored exception handling is gated off on POSIX for now (guest faults are not yet recovered) — the signal-based bridge is the next step. Also adds osx-x64 to the RID list and a Docker-based Linux smoke-test script. Status: on both macOS (Rosetta) and Linux (amd64), the guest now boots, runs native x86-64 code, and dispatches HLE imports. macOS stops at a Rosetta translation-cache issue; Linux runs ~252 imports through C++ static-init before hitting the missing fault handler (SIGSEGV). * [posix] Bridge the vectored exception handler to sigaction(SIGSEGV/SIGBUS/SIGILL) Guest faults on macOS/Linux previously terminated the process because the recovery logic in DirectExecutionBackend.Exceptions.cs was Windows-only. This adds a POSIX front-end that reuses the existing handler bodies: - DirectExecutionBackend.PosixSignals.cs installs SA_SIGINFO handlers via an [UnmanagedCallersOnly] entry, rebuilds the Win64 EXCEPTION_POINTERS / CONTEXT view from the platform mcontext (Darwin __ss thread state via the mcontext pointer at ucontext+48, Linux glibc gregs at ucontext+40 -- offsets verified against the headers on both platforms), runs the same chain as the VEH path (TryRecoverUnresolvedSentinel trap-sentinel recovery, TryHandleLazyCommittedPage demand paging, VectoredHandler diagnostics incl. FS/GS TLS-fault detection), and writes register changes back into the mcontext so sigreturn resumes the repaired guest. Unrecovered faults chain to the previously installed handler so the .NET runtime keeps mapping its own faults to managed exceptions. - The whole recovery path is warmed up with fabricated inputs before the handlers are installed. This is required under Rosetta 2: the signal trampoline cannot enter x86 code that has never been executed (and so never translated) -- a cold handler is silently never invoked and the faulting instruction retries forever (reproduced and verified in an isolated .NET test under Rosetta for Linux). It also keeps first-fault JIT work out of the signal frame. - Handlers run without SA_ONSTACK: the runtime's alternate stacks are too small for the diagnostic path, while guest (2MB) and host thread stacks match where Windows dispatches exceptions anyway. - The raw reads in the shared fault diagnostics (stack qwords, RBP walk, code bytes at RIP) now probe the region table on POSIX before touching memory, since a nested SIGSEGV inside the handler would kill the process before diagnostics finish. Windows keeps its try/catch reads. - Escape hatches: SHARPEMU_DISABLE_POSIX_SIGNALS=1 skips installation, SHARPEMU_DISABLE_RAW_HANDLER=1 disables sentinel recovery (parity with Windows), SHARPEMU_LOG_POSIX_SIGNALS=1 traces every delivery (first 16 and every 1024th are always traced). Verified with the test game: Linux (amd64 container) previously died with SIGSEGV right after import #252; it now recovers/diagnoses signals and the run proceeds to the real next blocker, an unpatched negative-offset guest TLS read (fault at TLS base - 0x1708), which gets the full NATIVE EXCEPTION dump before terminating. macOS is unchanged: the bridge installs and the game still stops at the known Rosetta translation-cache error at import 12, which is the next work item. * [posix] Fix guest memory layout faults: TLS prefix, exact mmap, map search base Three fixes that take the test game from dying during libc init to running its full main loop on macOS and Linux: - Static TLS blocks live below the TCB (FreeBSD amd64 variant II) and libc.prx reaches past -0x1700, but only a 4KB prefix was mapped below the TLS base. The prefix is now 64KB on POSIX (Windows keeps 4KB); the fault was a read at TLS base - 0x1708 during libc init. - HostMemory exact allocation on macOS used MAP_FIXED, which silently maps over untracked host memory. The direct-memory allocator's address scan walked into the .NET runtime's JIT heap and replaced live code, which under Rosetta 2 surfaced as "no code fragment associated with the given arm pc". Exact placement now passes the address as a hint and fails on relocation, like MAP_FIXED_NOREPLACE does on Linux. - sceKernelMapDirectMemory/MapFlexibleMemory searched for free space starting at 4GB, which is the Mach-O image base on macOS. The default search base is 0x20_0000_0000 on POSIX, and TryAllocateAtOrAbove now asks the kernel for a placement instead of page-stepping through host- owned address space (Rosetta ignores mmap hints for whole VA windows), over-allocating when the caller needs more than page alignment. Windows behavior is unchanged; all divergences are platform-guarded. * [macos] Video presenter on the main thread, MoltenVK support, window keyboard input Gets the test game from a headless loop to a playable window on macOS: - AppKit traps with SIGILL ("NSUpdateCycleInitialize() is called off the main thread") when GLFW runs on a worker thread. The CLI now moves emulation onto a worker thread on macOS and parks the real main thread in HostMainThread.Pump(); the presenter posts its whole window loop there instead of spawning a thread, and a shutdown handler asks the render loop to close the window so the pump unwinds on guest exit. - MoltenVK: enable VK_KHR_portability_enumeration (+ the portability instance flag) and VK_KHR_portability_subset when advertised, and gate robustBufferAccess2 on the device actually supporting it (Metal does not; the old code keyed it off robustImageAccess2 and vkCreateDevice failed with ErrorFeatureNotPresent). - Input: pad exports polled user32 GetAsyncKeyState, so POSIX hosts threw DllNotFoundException per scePadReadState call. The presenter now attaches the window's keyboard via Silk.NET.Input into HostWindowInput, and the pad exports map the existing VK-code layout onto it off Windows. Headless hosts (Linux containers) report a disconnected keyboard and fall back to neutral pad data silently. GLFW needs an x86-64 Vulkan loader under Rosetta: place a universal libMoltenVK.dylib next to SharpEmu named libvulkan.1.dylib (Homebrew's arm64-only copy cannot load into the x86-64 process) and export DYLD_LIBRARY_PATH to that directory. Verified: Dreaming Sarah boots to a MoltenVK-backed 2560x1440 window on macOS (Apple M4, Rosetta 2), renders the intro, title, and menus, and keyboard input drives it into gameplay. Linux (amd64 container) runs the same build headless through millions of imports with no faults. Windows paths unchanged; arm64 and x64 builds clean. * [posix] CoreAudio playback, self-contained MoltenVK loading, input/log polish - Audio: sceAudioOut ports now play through an AudioQueue backend on macOS (stereo PCM16 with the same 32KB backpressure pacing as the WinMM path). The WinMM port and the new CoreAudio port share an IHostAudioPort interface and sample converter; hosts without a backend (Linux containers) keep the silent fallback. - MoltenVK: GLFW resolves Vulkan with dlopen("libvulkan.1.dylib"), which cannot see the app-local universal MoltenVK build, so the presenter now feeds vkGetInstanceProcAddr straight into glfwInitVulkanLoader (GLFW 3.4) before creating the window. No DYLD_LIBRARY_PATH needed; the CLI also preloads the dylib for Silk.NET and prints setup hints when it is missing. scripts/fetch-macos-moltenvk.sh stages the official universal dylib next to a build. - The virtual-range allocator's failure trace now names the address and length instead of "AllocateAt invocation threw". Investigated and documented (not port defects): the savedata transaction failure is identical on Linux and macOS (HLE argument-register mapping for sceSaveDataCreateTransactionResource), and the in-game tile speckling has no platform-specific code in its path - the one macOS-only delta is that MoltenVK lacks robustBufferAccess2, so out-of-bounds shader reads return garbage instead of zeros. Verified on macOS: window, audio backend, and keyboard input all come up with zero environment configuration; the game runs to gameplay. Linux headless run unchanged (silent audio, no faults). Windows paths untouched; arm64 and x64 builds clean. * [cpu] Preserve guest registers and flags across patched TLS accesses The TLS patch handler replaces guest `mov reg, fs:[...]` instructions, which preserve every other register and the flags - but the handler loaded the TLS index into ecx and called TlsGetValue (Win64: clobbers rcx/rdx/r8-r11) with `sub/add rsp` trashing the arithmetic flags. Guest code that keeps live values or comparison results across a TLS access computed garbage deterministically. The handler now saves rcx, rdx, r8-r11, and the flags around the call, keeping the same inner stack alignment. This applies to the load patches and both store-helper stubs, on every platform. Also in this change, from the rendering-artifact investigation: - The present blit picks linear filtering for any fractional scale (nearest only for integer upscales): a 3840x2160 guest frame blitted into a 2560x1440 swapchain with nearest silently dropped every third row/column. - ClampViewport no longer trims the guest viewport rectangle to the render target; trimming changed the guest's scale/offset and skewed texel addressing. Vulkan permits viewports beyond the framebuffer (the scissor confines rendering), so only spec bounds are enforced. - Env-gated diagnostics grown during the investigation: guest texture dumps (SHARPEMU_TEXTURE_DUMP_DIR), aliased guest-image readback dumps (SHARPEMU_TRACE_GUEST_IMAGES=alias), small-render-target write movies (SHARPEMU_TRACE_GUEST_WRITES=small), unattended input injection (SHARPEMU_AUTO_CROSS=secs,...), viewport nudging (SHARPEMU_VIEWPORT_EPSILON), chunked-draw toggle (SHARPEMU_DISABLE_CHUNKED_DRAWS), and rect-list/draw vertex traces. Known remaining issue (root cause narrowed, not yet fixed): the game's terrain texture pages are corrupted in guest memory before any GPU work - the mound's solid-fill 32x32 tiles decode to fully transparent texels and the grass page has deterministic gaps, byte-identical across runs. Ruled out: memcpy/memmove/memset/realloc HLE semantics, sampler wrap modes, texel-boundary rounding, chunked draws, viewport handling. Next step is auditing the Chowdren asset decode path (custom compressed images) against the emulator's import surface. * [linux] ALSA playback backend for sceAudioOut sceAudioOut ports on Linux now play through libasound instead of the silent fallback. The PCM device opens in blocking mode with ~170ms of device buffer (the time-equivalent of the 32KB queue the WinMM and CoreAudio ports keep), so snd_pcm_writei provides the same backpressure pacing without a managed queue. Underruns and suspend/resume go through snd_pcm_recover with one retry per submit; anything else drops the buffer rather than stalling the guest. The "default" device routes through PulseAudio/PipeWire on desktops and straight to hardware on bare ALSA; SHARPEMU_ALSA_DEVICE overrides it (the null device makes the path testable in containers). A missing libasound or device fails port creation and lands in the existing silent fallback. Verified in an amd64 container: the test game opens the port (backend=alsa, 48kHz stereo float32) and streams sceAudioOutOutput through the null device for a full run; without a usable device the port logs a warning and falls back to silent. Playback on real Linux audio hardware has not been tested. * [fixes] Address review feedback: commit bounds, CoreAudio shutdown, dump errors - HostMemory: a MEM_COMMIT that runs past its reservation now fails like Win32 instead of committing a prefix and reporting success. All current callers already clamp their ranges to the region, so this only guards future callers. - CoreAudioPort: Dispose wakes a submitter waiting on backpressure and the wait treats ObjectDisposedException as a timed-out wait, so closing a port during playback can no longer throw. A failed AudioQueueStart tears the queue down and fails fast instead of leaving an undrainable queue that stalls every later submit on its timeout. - AgcExports: texture dumping catches all write failures (bad path, permissions), logging a warning instead of crashing when SHARPEMU_TEXTURE_DUMP_DIR points somewhere unusable. Verified with the Linux container run: game boots and streams audio with the stricter commit check, and a dump dir under /proc produces warnings instead of taking the process down. * [ci] Build linux-x64 and osx-x64 archives Adds a build-posix matrix job (ubuntu-latest / macos-latest) mirroring the Windows build: locked restore, Release build, self-contained CLI publish, and a tar.gz artifact per RID (tar keeps the executable bit). The macOS archive also stages the universal MoltenVK dylib via scripts/fetch-macos-moltenvk.sh so the artifact runs without any manual Vulkan setup. The release job still only ships the Windows archive. * [cli] Keep POSIX glfw natives outside the single-file bundle The KeepGlfwOutsideSingleFile target only matched filenames starting with 'glfw', which covers Windows (glfw3.dll) but not libglfw.3.dylib / libglfw.so.3. Those got embedded into the single-file bundle, and Silk.NET's library loader does not probe the bundle extraction directory, so a published build died with "Couldn't find a suitable window platform" (and the glfwInitVulkanLoader wiring, which loads the library from AppContext.BaseDirectory, could not run either). Keeping the POSIX names loose next to the executable fixes both, the same way the Windows build already handled it. Found by running the CI-built osx-x64 archive: video failed while local loose-file builds worked. With the fix the published single-file build opens the MoltenVK window, wires the loader, and reaches gameplay. * [ci] Publish linux-x64 and osx-x64 release archives The build-posix artifacts now ship as per-RID GitHub releases on main pushes and manual dispatches, tagged the same way as the win64 ones (<rid>-<ref>-<sha>). Archives stay tar.gz so the executable bit survives extraction. * [cli] Fail early on non-x86-64 host processes The CPU backend executes guest x86-64 code natively, so the process must be x86-64 (win-x64/linux-x64 on x64 hardware, osx-x64 under Rosetta 2 on Apple Silicon). An arm64 process previously failed deep inside emulation startup, indistinguishable from MoltenVK, signal handler, or guest memory problems. CLI mode now checks the process architecture up front and exits with a message naming the supported execution model (and the Rosetta install command on macOS). The GUI-only path stays usable on arm64. * [video] Log the selected Vulkan device name and API version The presenter never named the GPU it picked, so a 'no video' report could not be told apart from a real windowing failure without guessing. It now logs the device name, type, and API version right after selection. A software rasterizer (llvmpipe/lavapipe/SwiftShader) shows up here and typically lacks the device features the translated shaders need, which is the likely cause when a window opens and presents frames but nothing draws. * [video] Steer GLFW to XWayland on Wayland sessions GLFW's native Wayland backend does not reliably map the Vulkan window with some drivers (NVIDIA in particular): frames present but the window never becomes visible, so the game runs with audio and no picture. A report on an RTX 5080 showed exactly this — all device features present, frames presenting, but the log had 'libdecor-gtk.so failed to init' and a 1.4x-scaled window, both Wayland tells. On a Wayland session that also exposes an X server (DISPLAY set), the presenter now clears WAYLAND_DISPLAY for its own process before GLFW initializes, so GLFW selects its dependable X11/XWayland backend. SHARPEMU_ENABLE_WAYLAND=1 opts back into native Wayland. Headless (no DISPLAY) and non-Linux hosts are unaffected. * [video] Force GLFW X11 backend via the platform init hint, log the platform The previous attempt cleared WAYLAND_DISPLAY to steer GLFW off Wayland, but a reporter still hit the native-Wayland path (the Wayland-only libdecor error persisted), so that env trick doesn't switch GLFW. Use GLFW's supported mechanism instead: glfwInitHint(GLFW_PLATFORM, GLFW_PLATFORM_X11) before GLFW initializes, called into the same libglfw GLFW itself loads (the pattern InitializeMacVulkanLoader already uses). Still gated on a Wayland session with an X server present (DISPLAY set) so we never force X11 where XWayland can't catch it, and still overridable with SHARPEMU_ENABLE_WAYLAND=1. Also logs 'GLFW windowing platform in use: <backend>' after init via glfwGetPlatform, so a 'no window' report shows X11 vs Wayland outright. Verified on macOS: the readback correctly reports Cocoa and the presenter is unaffected (the fix is a no-op off Linux). * [video] Run the GLFW window on the main thread on Linux too GLFW requires window creation and event processing on the process main thread on every platform: initialization, window creation, and glfwPollEvents are main-thread-only, and X11 in particular has a single event queue that must be serviced there. A window created and polled on another thread may never map — which is why the game ran (audio, imports, even Vulkan present) with no visible window on Linux. macOS already routed the window loop to the main-thread pump (AppKit needs it); Windows is fine because it has a per-thread event queue. Linux was the gap: it spawned a background thread for the presenter. Extend the existing HostMainThread pattern to Linux — emulation runs on a worker, the main thread pumps the window work the presenter posts. Refs GLFW intro guide (thread-safety): init, window creation, and event processing are restricted to the main thread. Verified: macOS still boots to its window unchanged; the Linux headless container runs to millions of imports with no deadlock or regression. On-screen confirmation on a real Linux desktop is still pending, but this is the documented root cause for a windowless-but-running Linux session. * [posix] Skip Win32 native guest workers * [vulkan] Synchronize offscreen targets before present * [vulkan] Transition fresh textures from undefined layout * [vulkan] Report swapchain pixels before source readback * [vulkan] Emit requested guest image diagnostics * [agc] Diagnose guest texture fallbacks * [linux] Keep guest GPU mappings in low address space * [video] Reduce diagnostic stalls and drain complete frames * [memory] Harden packed GPU address handling * [readme] Document Linux and macOS release support * [posix] Integrate the host platform abstraction * [posix] Restore guest thread address window * [video] Run the performance HUD on POSIX hosts The FPS/CPU/work HUD bailed out unless the host was Windows; only the per-thread hottest-thread scan actually needs Windows APIs. Keep that scan Windows-only (POSIX reports 'idle') and let the rest of the HUD run everywhere — the title is already set from the render thread, which owns the window on macOS and Linux. * [posix] Implement native guest worker threads Guest entry stubs must not run above CLR-managed frames on CLR-created threads (see the NativeWorker preamble); the PR previously fell back to the inline calli path on POSIX, which reproduced the documented 'attempted to call a UnmanagedCallersOnly method from managed code' fail-fast (observed after Dreaming Sarah's menu select) and left the runtime's suspension machinery walking guest frames. Provide the missing POSIX half of the worker loop: - PosixHostStubs grows Win64-convention WaitForSingleObject/SetEvent/ ExitThread stubs backed by dispatch semaphores (macOS) / unnamed POSIX semaphores (Linux) plus pthread_exit, with EINTR retry in the wait. - Worker events are creatable/signalable/waitable from managed code too, so NativeGuestExecutor.Run keeps its handshake (AutoResetEvent stays on Windows byte-for-byte). - PosixHostThreading implements CreateNativeThread/WaitForThreadExit/ CloseThreadHandle over pthreads (liveness probed with pthread_kill(0), then joined). - RunPrologue/RunEpilogue are routed through the existing Win64->SysV thunks, so the emitted loop stays identical across platforms. * [macos] Disable concurrent GC under Rosetta's write-watch hazard Background GC's write-watch revisit (SoftwareWriteWatch::GetDirty -> FlushProcessWriteBuffers) calls thread_get_register_pointer_values on every thread; under Rosetta 2 that Mach call stalls indefinitely on threads executing translated guest code. The background mark phase then never finishes and every allocating or Monitor-taking thread wedges behind it — observed as Dreaming Sarah freezing at the menu/loading screen with FPS 0 in 5 of 7 runs, dispatcher/watchdog parked in Monitor.Enter and all BGC threads waiting in t_join. Non-concurrent GC never takes that path; a 5-minute soak now holds 22-31 fps in-game with zero stalls. Windows and Linux keep concurrent GC. * [diag] Periodic guest-thread snapshots with gate-owner tracking SHARPEMU_PERIODIC_SNAPSHOT_SECONDS=N dumps the stall snapshot every N seconds even while imports are progressing, for soft stalls where the game stops advancing but threads keep spinning. The periodic dump never touches the guest-thread gate (it must keep reporting when the gate is what's wedged): it reads a lock-free owner record — every gate acquisition now goes through LockGate(site), which notes site/thread — and walks the thread table without the lock, tolerating torn reads. SHARPEMU_PERIODIC_SNAPSHOT_FILE redirects the dump to a side file for the case where the console itself is wedged (frozen log mirror was one of the observed failure modes). * [nuget] Add osx-x64 RID targets to lock files * [cpu] Back off the guest join poll TryJoinThread polled the host thread at a fixed 1ms; a game main thread joining a long-lived worker (Dreaming Sarah parks there for the whole session) burned ~5% of managed CPU in Join/Sleep syscalls. Ramp the poll interval to 10ms once the join is clearly long-lived — exit detection latency for long joins moves from ~1ms to at most 10ms, and short-lived joins still resolve on the first 1ms polls. * [nuget] Add linux-x64/win-x64 RID targets to lock files * [posix] Keep guest stacks clear of the import-stub descent The import-stub region descends from 0x7000_0000_0000 on the same 16MB grid as the guest thread windows; moving stacks to 0x6FFF_E000_0000 put them inside the stub region's 64-module descent range (floor 0x6FFF_C000_0000), silently consuming the top ~32 stack slots on hosts with many loaded modules. Drop the POSIX stack base to 0x6FFF_A000_0000: 512MB below the stub floor, still 2.5GB above the TLS window. Windows keeps 0x7FFF_E000_0000 (its bands are ~15TB apart). * [pad] Read window gamepads on POSIX hosts XInput and the DualSense hid reader are Windows-only, which left macOS/Linux with keyboard input only. The presenter's Silk/GLFW input context already enumerates gamepads on both platforms, so track their state in HostWindowInput (event-driven on the window thread, snapshot guarded like the key set) translated to ORBIS conventions: GLFW's Xbox layout maps A/B/X/Y to Cross/Circle/Square/Triangle, sticks bias from -1..1 to 0..255 with Y growing down, and triggers rescale from GLFW's -1..1 resting-at--1 range with digital L2/R2 bits past 25%. The merge into ReadHostInputState is gated to non-Windows so a physical pad is never sampled twice through both a native reader and GLFW. Hotplug is handled via ConnectionChanged; with no pad connected the path is inert. Untested against a physical controller (none attached to the dev host); axis conventions follow the GLFW gamepad-mapping contract. * [nuget] Refresh lock files after cross-RID restores * [posix] Adopt the host audio/input seams from main Main's #192 abstracted audio output and pad/keyboard input behind IHostAudioOutput/IHostInput; re-express the POSIX backends behind them: - CoreAudioPort/AlsaAudioPort move to Host/Posix as PosixCoreAudioStream/PosixAlsaAudioStream implementing IHostAudioStream. The seam converts to stereo PCM16 before Submit, so the ports' own conversion (and IHostAudioPort/AudioSampleConverter) is gone; queueing and backpressure are unchanged. - PosixHostAudio selects CoreAudio (macOS) / ALSA (Linux) as the platform's IHostAudioOutput. - PosixHostInput implements IHostInput over an IPosixWindowInputSource that HostWindowInput registers when the presenter attaches the window's GLFW input context: keyboard with virtual-key translation, the window gamepad snapshot (now in seam HostGamepadState/HostGamepadButtons terms), and keyboard-connected as the focus signal. Rumble/lightbar no-op (GLFW has no such API). - PadExports drops its direct HostWindowInput gamepad merge — pads now flow through IHostInput.GetGamepadStates like every platform. - PosixHostThreading.RequestTimerResolution is a documented no-op. All three RIDs build; SharpEmu.Libs.Tests pass (26/26). * [nuget] Regenerate GUI lock file for RID-less locked restore Local cross-RID builds stamped a win-x64 runtimes section into SharpEmu.GUI's lock file; the project declares no RuntimeIdentifiers, so CI's 'dotnet restore --locked-mode' failed with NU1004 on every platform. Regenerated via a plain solution restore (--force-evaluate), matching what the workflow validates. |
||
|
Gutemberg Ribeiro
|
72645cb373 |
[Host] Abstract audio output and pad/keyboard input behind the host platform seam (#192)
* [Host] Abstract audio output behind IHostAudioOutput Add IHostAudioOutput (opens streams, names the backend for diagnostics) and IHostAudioStream (submit interleaved stereo 16-bit PCM, Dispose) to the host seam, with the winmm waveOut implementation moving whole into Host/Windows/WindowsWaveOutAudio — same device open, queueing, 32 KB backpressure wait, and buffer lifetime as WinMmAudioPort had. The DllImports become source-generated LibraryImports in the move, matching the other Windows backends. The guest-format conversion (mono/stereo/7.1, s16/float32 -> stereo PCM16) is platform policy, not device code, so it stays in Libs as AudioPcmConversion; AudioOutOutput converts into a pooled buffer and submits the result through the stream. Open failures still degrade to the silent paced port with the same warning, and the port log line now takes its backend name from the platform instead of a hardcoded string. * [Host] Abstract pad and keyboard input behind IHostInput Add IHostInput to the host seam: gamepad state snapshots, rumble / trigger-rumble / lightbar sinks, and the keyboard-fallback queries (window focus, key state). Gamepad state crosses the seam as the new unmanaged HostGamepadState with HostGamepadButtons flags — named after the PlayStation layout the guest API exposes but with the seam's own values, so SCE_PAD_BUTTON bits never leak into host backends and the per-frame poll can stackalloc its snapshot buffer. The DualSense raw-HID reader, the XInput reader, and the Win32 HID interop move whole into Host/Windows (report parsing, hot-plug loops, rumble/lightbar output reports, and log strings unchanged), translating to the neutral flags instead of ORBIS bits and converting their DllImports to source-generated LibraryImports. WindowsHostInput composes them plus the user32 keyboard queries; rumble still fans out to both readers, trigger rumble stays XInput-only, lightbar stays DualSense-only. PadExports keeps all policy: the keyboard mapping (now via named OrbisPadButton constants instead of raw hex), the controller-beats- keyboard-past-deadzone merge, and the new host->ORBIS button translation. The GUI's source-linked reader copies re-point to the moved files (it still cannot reference SharpEmu.HLE wholesale), which requires AllowUnsafeBlocks for the generated marshalling stubs; its navigation code switches to the neutral flags. * [Host] Move the timer-resolution request behind IHostThreading IHostThreading gains RequestTimerResolution (idempotent, best-effort ~1 ms timed-wait granularity; a no-op wherever the platform default is already fine). The winmm timeBeginPeriod call, its once-only latch, and both warning strings move from the Libs-level HostTimerResolution helper into WindowsHostThreading as a source-generated LibraryImport; the vblank pump requests it through the platform instead. HostSystemInfo in SharpEmu.Logging keeps its direct user32/kernel32 imports deliberately: Logging sits below HLE in the dependency chain so it cannot see the host seam, every path is already OS-gated with fallbacks, and it only runs once for the diagnostics banner. |
||
|
Gutemberg Ribeiro
|
62e1775c5c |
[HLE] Remove steady-state allocations from the hot HLE paths (#190)
* [HLE] Stop allocating on the memcpy/memset and trace hot paths memcpy/memmove no longer allocate a bounce buffer sized to the whole copy (large copies previously landed on the LOH); they loop through a single pooled 256 KB rental, copying high-to-low when the destination overlaps above the source so memmove semantics survive the chunking. memset reuses a shared zero chunk for the dominant zero-fill case and rents/fills only min(length, 16K) bytes for non-zero values instead of allocating and filling a fresh 16 KB array per call; the map-time zero-fill loop shares the same zero chunk. SHARPEMU_LOG_SEMA / SHARPEMU_LOG_VIDEOOUT are now read once into cached bools and every TraceSemaphore/TraceVideoOut call site is guarded, so trace messages are no longer interpolated (and the env var no longer queried) on every semaphore op and every flip with tracing off. Trace output when the flags are set is unchanged. * [HLE] Remove per-frame allocations from the vblank/flip/equeue plumbing The 60 Hz vblank pump no longer allocates per edge: PumpVblanks reuses a pump-thread-only port list instead of a LINQ Where/ToArray, and SignalVblank/SubmitFlip snapshot their event registrations into pooled rentals instead of copying the List on every edge and every flip (the snapshot must still be taken, since triggers run outside _stateGate and a per-port reusable buffer would race the pump thread against a guest thread's first-edge signal). sceKernelWaitEqueue delivery rents the dequeue buffer from the pool instead of allocating an array per wait, and event-queue wake keys are formatted once per handle (cached in a ConcurrentDictionary, dropped on queue delete) instead of building the string on every enqueue. The semaphore wake key moves onto KernelSemaphoreState at creation, the same pattern the pthread mutex state already uses, removing the per-signal/per-wait formatting. SHARPEMU_LOG_EQUEUE is read once into a cached bool like the sema/videoout flags. * [HLE] Read guest C-strings without per-call buffer allocations CpuContext.TryReadNullTerminatedUtf8 allocated a byte[capacity] and issued one TryRead per byte for every string-argument import. It now reads through a stack buffer (pooled above 512 bytes) in 128-byte bulk chunks, falling back to per-byte reads only when a chunk touches an unreadable range so a terminator sitting just before unmapped memory still resolves exactly as before. The chunk bound also keeps the overread past the terminator smaller than the old loop's worst case is wide, so no fault can appear where the byte loop succeeded. TryReadAsciiZ (dlsym/symbol resolution) drops its List<byte> + ToArray round-trip for the same stack/pooled buffer, keeping the byte-by-byte TryReadByteCompat reads because their Marshal.ReadByte fallback must probe exactly up to the terminator. Only the final string is allocated on either path now. * [HLE] Replace blocking-wait closures with waiter continuation objects Every wait that actually parked a guest thread allocated two capturing lambdas (plus their display classes) for the scheduler's resume/wake callbacks. RequestCurrentThreadBlock and the backend's blocked-thread state now carry a single IGuestThreadBlockWaiter instead of the Func<int>/Func<bool> pair: TryWake keeps the run-under-the-scheduler- gate contract and Resume still produces the guest's RAX on the woken thread. The waiter stays attached through the wake transition (the old code nulled only the wake handler there) and is consumed at resume. The existing waiter objects absorb the captured state as fields, so a blocking wait now allocates exactly one object: SemaphoreWaiter, PthreadMutexWaiter, and EventFlagWaiter implement the interface directly, and the equeue, cond, and rwlock waits get small waiter classes replacing their closures. Handler bodies delegate to the same static methods with the same arguments as before; the untimed event flag wait's mutable captured result becomes a field on its waiter. * [HLE] Back pending event queues with a ring deque instead of LinkedList LinkedList<KernelQueuedEvent> allocated a node object on every non-coalesced enqueue — one per vblank/flip edge per registered queue, 60+ times a second in steady state. KernelEventDeque is a grow-only ring buffer over a KernelQueuedEvent[] with the three operations the queue actually uses (AddLast, RemoveFirst, find-and-update-in-place by ident/filter), so steady-state enqueue/dequeue allocates nothing and the coalescing update writes the struct back through an indexer instead of a node reference. All accesses stay under _eventQueueGate, matching the LinkedList usage it replaces. * [HLE] Cap memcpy chunk iterations at the requested size, not the rented length Address Copilot review: ArrayPool.Rent may return a larger array than requested, so sizing each iteration by chunk.Length let the copy granularity depend on pool bucketing internals instead of the intended 256 KB chunking. Behavior was already correct for any chunk size (each iteration re-reads the source, and the overlap ordering is size- independent), but the loop now mins against the requested chunkLength, matching what memset already does. * [HLE] Skip the flip/vblank snapshot rental when no events are registered Address Copilot review: SignalVblank and SubmitFlip rented (and returned) a pooled snapshot even with zero registrations — steady per-frame pool traffic for games that never register flip events and only poll flip status. Zero-count signals now skip the rental, the copy, and the trigger loop entirely, which also retires the Math.Max(count, 1) minimum-rent guard. |
||
|
Spooks
|
9d88542efd |
Fix virtual memory allocation and access (#193)
* Fix virtual memory allocation and access * Update test dependency lock file |
||
|
Gutemberg Ribeiro
|
f23161be9a |
Host platform abstraction layer for the execution engine (#181)
* [Host] Introduce host platform abstraction with IHostMemory Add SharpEmu.HLE/Host with IHostPlatform/IHostMemory interfaces, neutral page-protection/region enums, and a HostPlatform.Current factory that resolves the Windows backend (or throws PlatformNotSupportedException on other OSes, matching today's de-facto behavior). WindowsHostMemory wraps the exact VirtualAlloc/VirtualFree/VirtualProtect/VirtualQuery calls used across the engine today, with identical MEM_*/PAGE_* constants. Migrate StubManager as the first consumer: its private kernel32 P/Invokes and enums are replaced by IHostMemory calls that issue the same two native operations (RWX commit+reserve of the PLT arena, release on Dispose). No behavior change. This is the first step toward supporting non-Windows hosts; subsequent commits move the remaining direct P/Invokes in Core and Libs behind the same seam. * [Host] Route PhysicalVirtualMemory through IHostMemory Replace the class's private VirtualAlloc/VirtualFree/VirtualProtect/ VirtualQuery P/Invokes with IHostMemory calls. Every site maps 1:1 onto the exact native call it issued before: MEM_COMMIT|MEM_RESERVE -> Allocate, MEM_RESERVE -> Reserve, fault-path commits -> Commit, and MEM_RELEASE -> Free, with identical protection values produced by the Windows backend. IHostMemory gains ProtectRaw so the save/restore protection sequences in TryWriteExclusive and TryTemporarilyProtectForRead round-trip the raw OS protection word (including modifier bits the neutral enum cannot represent) exactly as before. Raw PAGE_* constants remain only for the internal region-classification helpers, which only ever see values this class itself assigned. The exact-address free-on-mismatch, lazy reserve-only threshold, prime loop, and all trace strings are unchanged. * [Host] Add IGuestAddressSpace and retire the reflection-based allocator lookup Introduce IGuestAddressSpace in SharpEmu.HLE (fixed-address AllocateAt / TryAllocateAtOrAbove and guest mprotect via TryProtect) with signatures copied from PhysicalVirtualMemory, which now implements it. TryProtect reproduces the read/write/execute decomposition that KernelMemoryCompatExports.ResolveHostProtection performs, yielding the same PAGE_* values through the Windows backend. KernelVirtualRangeAllocator previously located AllocateAt via cached MethodInfo reflection (because SharpEmu.Libs cannot see Core types) and walked wrapper memories through an untyped 'Inner' property. Both are now typed: ICpuMemoryWrapper exposes the decorated memory (implemented by TrackedCpuMemory, whose Inner property already existed) and the allocator type-tests for IGuestAddressSpace with the same bounded unwrap depth. Failure paths keep the exact [LOADER][TRACE] strings. * [Host] Move Kernel HLE memory exports off direct kernel32 P/Invokes KernelMemoryCompatExports loses its private VirtualQuery/VirtualProtect/ VirtualAlloc/VirtualFree declarations and MemoryBasicInformation struct: - Guest mprotect (sceKernelMprotect/sceKernelMtypeprotect) now routes through IGuestAddressSpace.TryProtect resolved from ctx.Memory. The orbis read/write/execute decomposition moves into a GuestPageProtection conversion whose mapping is value-identical to the removed ResolveHostProtection. - The guarded libc heap and host-page accessibility checks go through IHostMemory (same commit+reserve/protect/free sequence; guard-page and protection-mask checks compare HostRegionInfo.RawProtection against the same PAGE_* literals as before). - HostMemory is exposed as a property so merely loading the type never resolves the platform backend on non-Windows hosts. KernelRuntimeCompatExports' RDTSC stub allocates its 16-byte RWX page via IHostMemory.Allocate; the OperatingSystem.IsWindows() gate returning null is unchanged. * [Host] Abstract thread, TLS, and symbol primitives in the execution backend Add IHostThreading (native TLS slots, current-thread id, affinity, raw thread create/join, diagnostic register capture) and IHostSymbolResolver (enum-keyed host function addresses baked into emitted stubs), with Windows implementations wrapping the exact kernel32 calls the backend made directly before. DirectExecutionBackend takes an optional IHostPlatform (defaulting to HostPlatform.Current) and routes every TlsAlloc/TlsFree/TlsSet/GetValue, GetCurrentThreadId, SetThreadAffinityMask, GetModuleHandle/GetProcAddress and the suspend+GetThreadContext diagnostic snapshot through it. The snapshot moves wholesale into WindowsHostThreading (including the Win64 CONTEXT size/flags/offsets, which are Windows-specific by nature) and returns a neutral HostCapturedRegisters. NativeGuestExecutor resolves WaitForSingleObject/SetEvent/ExitThread via the symbol resolver — the same addresses end up in the emitted run loop, so stub bytes are unchanged — and creates/joins its raw worker thread through IHostThreading with the same stack-reservation semantics. The run-loop emitter itself does not move. Marshal.GetLastWin32Error() in the affinity-failure log still observes SetThreadAffinityMask's error because the wrapper makes no intervening SetLastError call. * [Host] Move fault handling and remaining backend memory ops behind the seam Add IHostFaultHandling (handler-thunk creation, first-chance handler install/remove, unhandled-filter set) with WindowsFaultHandling in a new Cpu/Native/Windows/ folder. The exception-handler trampoline emitter moves there whole — same pre-filtered NTSTATUS codes, same TEB gs:[8]/ gs:[0x10] stack-limit reads, same host-RSP TLS switch — parameterized only by (managed callback, TLS slot, TlsGetValue address), which is exactly what SetupExceptionHandler passed it before. Handler installation order, the AddVectoredExceptionHandler(first=1) flag, the SHARPEMU_DISABLE_RAW_HANDLER gate, and all install/teardown log strings are unchanged. Every remaining VirtualAlloc/VirtualProtect/VirtualFree/VirtualQuery/ FlushInstructionCache in the backend partials routes through IHostMemory with 1:1 call mapping (RWX emit -> RX downgrade -> flush for stub emission, reserve/commit for the PRT aperture and lazy-commit fault path, raw-protection round-trips via ProtectRaw). HostRegionInfo gains RawState/RawAllocationProtection so the lazy-commit trace lines and protection-mask checks keep printing and comparing the exact native values. Windows semantics leaked as bare literals become named constants with identical values: NTSTATUS codes (WindowsFaultCodes) and Win64 CONTEXT byte offsets (Win64ContextOffsets, with the existing CTX_* constants aliased to it and handler-local numeric offsets replaced by the names). * [Host] Resolve the host platform explicitly at the composition root SharpEmuRuntime.CreateDefault() now resolves HostPlatform.Current once and passes it explicitly to PhysicalVirtualMemory and (via a new optional CpuDispatcher parameter) to DirectExecutionBackend, replacing the implicit default-argument fallbacks. On unsupported OSes boot now fails at the root with PlatformNotSupportedException and a clear message instead of on the first native call. A future Linux/macOS backend plugs in by returning a different IHostPlatform here. * [Host] Convert the platform backends to source-generated P/Invokes Replace [DllImport] with [LibraryImport] in the four Windows backend files added by this branch (WindowsHostMemory, WindowsHostThreading, WindowsHostSymbolResolver, WindowsFaultHandling). Marshalling stubs are now generated at compile time instead of JIT-emitted at runtime, which fits the pre-JIT-everything boot model and keeps the backends NativeAOT/trimming ready. Interop stays zero-copy: all signatures are blittable, GetModuleHandleW now pins the managed string via Utf16 marshalling instead of copying, and GetProcAddress names marshal through a stack-allocated Utf8 buffer. Implicit contracts become explicit where LibraryImport requires it: TlsFree/TlsSetValue gain [MarshalAs(UnmanagedType.Bool)] (the 4-byte Win32 BOOL DllImport assumed silently), and GetModuleHandle targets the W entry point directly since LibraryImport never probes suffixes. The CONTEXT snapshot buffer stays a NativeMemory allocation rather than stackalloc: CONTEXT requires 16-byte alignment, now documented at the call site. Native call sequences are unchanged. * [Host] Address Copilot review: harden failure paths, honor injected platform - Free the handler thunk page when the RX protection downgrade fails (the leak predates this branch, but the failure path is boot-fatal so releasing the page is unobservable). - TraceThreadMode and the static diagnostics helpers now resolve host primitives through the backend bound to the current thread, falling back to HostPlatform.Current only when no run is active (identical on supported configs, honors injection everywhere a backend exists). - HostPlatform.Create additionally requires an x64 process so native Windows ARM64 fails with the promised PlatformNotSupportedException instead of emitting x86-64 stubs into an ARM64 process. |
||
|
tensorcrush
|
c06c70cad7 |
[Aerolib] Add ulobjmgr and NpEAAccess symbol names (#150)
Resolves _sceUlobjmgrRegisterObject (BG26hBGiNlw) and _sceUlobjmgrUnregisterObject (Smf+fUNblPc), reported as unresolved by testers, plus four sceNpEAAccess exports. Names taken from shadPS4's NID tables and each verified by recomputing the NID with the repo's name2nid derivation before inclusion. aerolib.bin regenerated with scripts/generate_aerolib_binary.py. Co-authored-by: tensorcrush <tensorcrush@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Mike Saito
|
caf859cc52 |
Fix guest shutdown when VideoOut window is closed (#184)
Propagate Silk window close to runtime teardown so audio and CPU workers stop instead of continuing after the presentation window is dismissed. |
||
|
Spooks
|
d8397b022e |
Performance Improvements and Optimization Tweaks (#156)
* Improve Gen5 rendering performance and compatibility * Pin .NET SDK for locked restore --------- Co-authored-by: Spooks4576 <Spooks4576@users.noreply.github.com> |
||
|
Mike Saito
|
4f028d0483 |
Expand Aerolib from nids.csv and wire socket/net kernel NID handlers (#128)
* Expand Aerolib catalog from nids.csv and wire socket/net NID handlers Load authoritative NID pairs from scripts/nids.csv with ps5_names fallback. Replace mislabeled kernel zero stubs with socket/connect/bind/getsockname HLE and sceNet byte-order exports backed by the CSV symbol names. * Add inet_pton, htons, and bzero kernel compat with CSV NIDs Wire libc network helpers using authoritative NID names from nids.csv instead of synthetic Gst* exports used on the crt-loader branch. * Fix REUSE annotation for scripts/nids.csv * Drop bundled nids.csv; extend ps5_names and regenerate Aerolib Remove scripts/nids.csv from the repository and fold csv-only symbol names into scripts/ps5_names.txt so Aerolib keeps the full catalog via name2nid. |
||
|
Spooks
|
63b440efcd |
[HLE] Fix guest-thread sync and boot for Unreal Engine titles (#102)
* [HLE] Fix guest-thread sync and boot for Unreal Engine titles Silent Hill: The Short Message (and other UE titles) now boot the full engine thread graph instead of hanging early. Four related fixes: - pthread cond/mutex semantics: retain a signal raised with no waiter as pending, and key block/wake on the state's identity rather than a resolved address that could differ between lock and unlock. This ends the ~1.5M-call cond_wait busy-spin. - Warm HLE type initializers and force-JIT their methods on a host thread at Freeze(). A .cctor or first-time JIT running on a guest thread's hijacked stack fail-fasts the CLR as "Invalid Program: attempted to call a UnmanagedCallersOnly method from managed code". - Guest thread scheduling: pump after a wake so a readied thread actually runs, add a dispatcher thread for when every guest thread is parked, and make the pump-depth guard an atomic CAS. - Route mutex/rwlock lock/unlock off the non-blocking leaf-import fast path so a contended lock can deschedule its guest thread. Ported from the unreal-boot-fixes branch. * [HLE] Keep mutex/rwlock unlock on the leaf-import fast path The previous change routed all mutex/rwlock lock and unlock NIDs off the leaf fast path so a contended lock could deschedule its guest thread. But unlock never blocks, and taking it off the fast path made it slow enough that Demon's Souls' job workers livelocked in a guest spinlock (millions of mutex_unlock calls, no import progress, main thread stuck in sceKernelWaitEventFlag). Only *lock* needs to leave the leaf path. Restore the four unlock NIDs (mutex + rwlock) so guest spinlocks stay cheap, while lock/rd/wrlock remain off it for the blocking case Silent Hill needs. * [HLE] Gate pthread_mutex_lock guest-thread blocking (fixes Demon's Souls) Re-enabling cooperative deschedule on a contended pthread_mutex_lock regressed Demon's Souls: its job workers run on libSceFiber, and blocking a guest thread mid-fiber left sceFiberSwitch returning ESRCH followed by a null fiber-context deref (0xC0000005). Bisect confirmed the pthread change as the cause; the game reaches the same point as before it once the block is skipped. Gate the block behind SHARPEMU_MUTEX_LOCK_BLOCKING (off by default) so contended locks fall through to the synchronous host-thread wait. The rest of the pthread fixes (cond_wait pending signals, identity wake keys) are unaffected. |
||
|
Digote
|
4bd42795c7 |
[logging] Migrate HLE diagnostics to SharpEmuLog (#80)
Signed-off-by: Digote <45742711+Digote@users.noreply.github.com> |
||
|
PandaCatz
|
edb4eb86a2 |
[kernel] Wake blocked waiters on semaphore signal, cancel, and delete (#67)
sceKernelWaitSema parks a guest thread on the scheduler when the count is not yet available, but sceKernelSignalSema only incremented the count and returned: there was no WakeBlockedThreads call anywhere in the file, so a thread blocked in WaitSema was never woken and the game hung there. sceKernelCancelSema and sceKernelDeleteSema left parked waiters stranded the same way. Give each semaphore a per-handle wake key and each waiter a small record with the count it needs and a result slot. Signal, cancel, and delete wake the waiters through the scheduler after releasing the semaphore lock, matching the lock order the event flag and event queue paths already use. The wake handler runs under the scheduler gate and consumes the count under the semaphore lock, so a waiter needing more than is available stays parked while a smaller waiter can still proceed; the resume handler hands the recorded result back as the guest's return value. Cancel bumps an epoch and delete sets a flag so woken waiters return what the kernel returns in those cases: ECANCELED (0x80020055) for a canceled wait and the EACCES-class 0x8002000D for a deleted semaphore. Delete succeeds even with waiters present. Only the woken waiter's own handler adjusts the waiting-thread count, so a waiter that parks during a cancel is not double-counted, and the create path now wakes a waiter that raced onto the handle if the handle write-back fails instead of stranding it. This does not change the immediate paths: an available count is still consumed inline, and a wait with a timeout pointer still returns immediately (honoring the timeout through the scheduler is a separate change). Verified with a block/wake harness that drives real guest threads through the real import trampolines: signal-after-block, signal racing the park, multi-waiter signal, need-count gating with a smaller waiter slipping past, and cancel and delete with parked waiters including the reported waiter count, plus event flag and event queue regression checks. Builds clean on Windows and Linux. |
||
|
Dawid
|
29021b5a71 | [fixes] move repeating methods into CpuContext (#41) | ||
|
kostyaff
|
c0fd6a80e8 |
Astro Bot shader type 4, pthread_cond_timedwait, and HLE/memory/cpu bug fixes (#40)
* [agc] Add shader type 4 (GS) and register defaults v13 support Astro Bot (#11) crashes on boot due to two missing GPU features: 1. Shader type 4 (Geometry Shader) — SPI_SHADER_PGM_LO/HI register offsets 0x8A/0x8B were missing. Added constants and switch cases for shader type 4 in GetExpectedSpiShaderPgmLo/Hi. Also added type 4 to IsEsGeometryShaderType (2 or 4 or 6). 2. Register defaults version 13 — was not recognized as supported. Added RegisterDefaultsVersion13 constant and included it in IsSupportedRegisterDefaultsVersion. * [kernel] Add POSIX pthread_cond_timedwait export SILENT HILL (#4) and Poppy Playtime (#3) crash on boot due to missing POSIX pthread_cond_timedwait (NID 27bAgiJmOh0). The Sony wrapper scePthreadCondTimedwait (NID BmMjYxmew1w) was already implemented, but the raw POSIX symbol was not exported. Added [SysAbiExport] for pthread_cond_timedwait delegating to existing PthreadCondWaitCore with timed: true. * [memory] Fix FlushInstructionCache null process handle PhysicalVirtualMemory.cs called FlushInstructionCache with null as the process handle in two places (SetProtection and TryWriteExclusive). On Windows, a null handle does not reliably resolve to the current process — the correct call is GetCurrentProcess() (pseudo-handle -1). Also corrected the P/Invoke signature: - Changed return type from void to bool with [return: MarshalAs(Bool)] - Added SetLastError = true - Added GetCurrentProcess() P/Invoke import This matches the pattern already used in DirectExecutionBackend.cs which correctly passes GetCurrentProcess() to all FlushInstructionCache calls. * [hle] Distinguish NOT_FOUND from NOT_IMPLEMENTED and log duplicate NIDs Three diagnostic improvements to the HLE dispatch path: 1. ModuleManager.RegisterFromAssembly — duplicate NID registration was silently skipped (dispatchTable first-wins, exportTable last-wins, causing metadata divergence). Now logs a warning with the NID and export name so conflicts are visible. 2. ModuleManager.TryDispatch — generation mismatch returned ORBIS_GEN2_ERROR_NOT_FOUND, conflating 'function does not exist' with 'function exists but not for this generation'. Now returns ORBIS_GEN2_ERROR_NOT_IMPLEMENTED for generation mismatch, matching the existing convention in CpuDispatcher. Also adds debug logging for both NOT_FOUND and NOT_IMPLEMENTED paths. 3. DirectExecutionBackend.Imports.cs — the import dispatch else-branch (the actual hot path that bypasses ModuleManager.TyDispatch via cached export) had the same conflation. Split into: - else if (export exists but generation mismatch) → NOT_IMPLEMENTED - else (no export at all) → NOT_FOUND This makes runtime diagnostics correctly distinguish missing exports from generation-unsupported exports. * [cpu] Check VirtualProtect return values in all stub creation paths 9 VirtualProtect calls in DirectExecutionBackend.cs had unchecked return values. If VirtualProtect silently fails, memory protection remains incorrect — stubs allocated with PAGE_EXECUTE_READWRITE (0x40) never get downgraded to PAGE_EXECUTE_READ (0x20), or guest thread entry stubs never get upgraded to writable. This causes access violations on next execution or silent data corruption. Fixed all 9 sites with proper error handling: - 6 stub creation methods (return 0 on failure + log error) - 2 guest thread entry methods (set reason + return Exception) - 1 guest entry method (set LastError + return MEMORY_FAULT) Stub creation sites fixed: - CreateImportDispatchStub (line ~1683) - EnsureTlsHandler (void, log + return) - CreateUnresolvedReturnStub (return 0) - CreateGuestReturnStub (return 0) - CreateExceptionHandlerTrampoline (return 0) - CreateTlsStoreHelperStub (return 0) Guest thread entry sites fixed: - StartGuestThreadNativeCall (return Exception) - StartGuestContinuationNativeCall (return Exception) - RunGuestEntryPoint (return MEMORY_FAULT) * [kernel] Remove unused duplicate _nextFileDescriptor field KernelExports.cs declared _nextFileDescriptor but never used it. The actual field used for file descriptor allocation lives in KernelMemoryCompatExports.cs (lines 1314, 1337). This was a dead duplicate causing CS0414 warning. Build is now 0 errors, 0 warnings. --------- Co-authored-by: Hermes Atlas <hermesatlas@example.com> |
||
|
Dawid
|
7337683c16 |
[fixes] stackalloc warnings, consolidate duplicated methods, minor adjustments in project settings (#39)
* [fixes] stackalloc warnings, consolidate duplicated methods * [fix] remove unnecessary edit in .slnx file |
||
|
Berk
|
16c1b74636 | [core] Update native execution and kernel exports, phtread improvement (#13) | ||
|
ParantezTech
|
7f85971e39 | [kernel] carry pthread scheduling | ||
|
ParantezTech
|
0e922a73ee | [core] Improve native execution and guest threading | ||
|
ParantezTech
|
d134f9b9f6 |
[fiber] synchronization problems have been fixed for such a titles: Demon's Souls
[ampr] new exports [memory] trampoline fixes |
||
|
ParantezTech
|
2a52ccfe6a | [cpu] Add guest thread continuation | ||
|
ParantezTech
|
08b315b5fc | [kernel] Add guest memory support | ||
|
ParantezTech
|
bbf5ff7be8 | more HLE's and fix cpu execution some titles | ||
|
ParantezTech
|
1f71c970d9 | A dozen changes; new HLEs, AV fixes, ELF loader fixes, new return codes, etc. | ||
|
ParantezTech
|
028494a83b | another hle implements, especially for mutex calls | ||
|
ParantezTech
|
4d73f469bc | initial commit |