mirror of
https://github.com/par274/sharpemu.git
synced 2026-08-02 16:09:47 +08:00
b804c84ad8df696c712973542b9265dfc249cb6b
12 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 h4sht
|
2a4da8c0a9 |
[Kernel/Semaphore] Close race between sceKernelWaitSema and sceKernelSignalSema (#504)
When sceKernelWaitSema finds the count insufficient it increments WaitingThreads, releases the semaphore gate, and calls RequestCurrentThreadBlock to set the thread-static block flags. A signal arriving before the scheduler registers the block metadata is missed by WakeBlockedThreads — the waiter has not been registered yet and the signal's wake iteration skips it. The scheduler's exit handler already re-checks TryWake() after setting the thread to Blocked, but that requires the thread to fully exit to the scheduler and back. Instead, re-check the semaphore count under the gate immediately after the block request: if the count is now sufficient, consume the tokens, cancel the pending block via TryConsumeCurrentThreadBlock, and return without ever yielding to the scheduler. Co-authored-by: tru3 <tru3@tru3.com> |
||
|
wearr
|
a60bfc9c83 | [Kernel] Implement pthread semaphore exports (#424) | ||
|
Berk
|
fbafd3f429 |
[GUI] Add native Vulkan host surface support and more (#337)
* [GUI] Add native Vulkan host surface support and more * reuse * [GUI] set width session menu |
||
|
Miguel Cruz
|
864cbb0fa0 |
[AGC/Vulkan] Extend PS5 runtime and rendering compatibility (#216)
* [Core] Add POSIX native execution and PS5 SELF support Extend the native backend, guest TLS, fixed-address memory, and loader paths needed by PS5 titles on Windows, Linux, and macOS. Keep workstation GC so high-core-count hosts do not reserve over fixed guest image bases. * [HLE] Expand PS5 service and media compatibility Add the kernel, threading, save-data, networking, audio, video-codec, font, dialog, and service exports required by newer PS5 software. Preserve every SysAbi NID currently registered by main while adding the compatibility surface used by ASTRO BOT. * [AGC/Vulkan] Extend Gen5 shader and presentation support Expand PM4 handling, Gen5 shader translation, MRT and packed export support, guest image tracking, depth initialization, texture aliasing, and Vulkan presentation. Add the performance overlay and address-filtered diagnostics used to validate ASTRO BOT with original shaders. * [Core] Align static TLS reservation across hosts * [Pad] Align primary user ID with UserService * [Gpu] Preserve runtime scalar buffers across renderer seam * [AGC] Restore omitted command helper exports * [Vulkan] Reuse primary views for promoted MRT targets * [Vulkan] Preserve scratch storage bindings in compute dispatches |
||
|
Gutemberg Ribeiro
|
320dbcacba |
[SourceGenerators] Compile-time SysAbi export registry, analyzers, and build-generated aerolib.bin (#204)
* [SourceGenerators] Add the SysAbi export generator and analyzers (phase 0) New SharpEmu.SourceGenerators Roslyn component, complete and tested but consumed by nothing yet — the emulator projects adopt it in the following commits. Ps5Nid ports the PS NID derivation (base64 of the byte-reversed first eight SHA1 bytes of name + fixed suffix) from scripts/generate_aerolib_binary.py to C#, so what has always been a manual, out-of-band computation becomes a compile-time capability. SysAbiExportGenerator emits a per-assembly SysAbiExportRegistry whose CreateExports(Generation) reproduces ModuleManager's reflection scan exactly — same generation inheritance and filtering, same method-name fallback, same libKernel default — with attribute-omitted NIDs derived algorithmically (equivalent to the runtime catalog lookup, which is built from the same computation). Parameterless handlers are adapted to the SysAbiFunction shape; invalid declarations are skipped here because the analyzer rejects them as build errors, so nothing drops silently. SysAbiExportAnalyzer turns the runtime failure modes into diagnostics: SHEM001 duplicate NID (across declared and derived forms), SHEM002 malformed NID, SHEM003 uncallable handler signature, SHEM004 NID contradicting its export name (the class of drift previously fixed by hand), SHEM005 unresolvable export, SHEM006 export name unknown to ps5_names.txt when the catalog is wired as an AdditionalFile, SHEM007 handler not reachable by generated code. The self-contained test suite drives both in-process against the real SharpEmu.HLE metadata: known catalog NID pairs pin the algorithm, the generated registry must itself compile, and each diagnostic has a triggering fixture. Fittingly, the NID pinning test caught a wrong pair in its own first draft — the exact mistake SHEM004 exists to stop. * [SourceGenerators] Adopt the generated export registry in the emulator (phase 1) SharpEmu.Libs consumes the generator and analyzers, with scripts/ps5_names.txt wired as the AdditionalFile catalog. The runtime now registers exports from the compile-time SysAbiExportRegistry instead of the boot-time reflection scan; RegisterFromAssembly is retained solely as the arbiter for a parity test that pins the two tables identical — same NIDs, names, libraries, targets, and handler methods — across Gen4, Gen5, and combined registration. First contact between the analyzer and all 715 existing exports surfaced real drift the old offline checker structurally missed (scripts/check_sysabi_aerolib.py skipped any NID absent from aerolib.bin): three exports whose friendly names collide with real catalog symbols of different NIDs, now suppressed at-site with reasons pending AGC API confirmation, alongside the established synthetic Unknown* labels for uncatalogued NIDs, which prompted a rule refinement — SHEM004 only hard-errors when the export name is a real catalog symbol, since synthetic labels cannot be validated by hashing and the NID is authoritative for them. The two allowlisted mismatches in the python checker no longer trigger anything, and the checker is deleted: the analyzer subsumes it with the semantic model instead of regex, and validates every declared pair rather than only catalog-known NIDs. * [SourceGenerators] Generate aerolib.bin at build time from ps5_names.txt The runtime NID -> name catalog is derived data and no longer lives in the repository: a Framework-only MSBuild task (GenerateAerolibBinaryTask, sharing the same Ps5Nid implementation the analyzers use) builds it into the intermediate directory from scripts/ps5_names.txt — now the single source of truth — and SharpEmu.HLE embeds it from there. The output is byte-identical to the previously committed binary, verified with cmp against git history; a new test pins that the embedded catalog loads and resolves a known symbol both directions. scripts/generate_aerolib_binary.py is deleted (its algorithm lives in Ps5Nid, its invocation in the build); the REUSE annotation for the binary goes with it. MSBuild's Inputs/Outputs check means the ~154k NID hashes only recompute when the names file actually changes. The task implements ITask against Microsoft.Build.Framework directly, keeping the vulnerable-flagged Utilities.Core package out and the analyzer project's file-IO ban suppressed only inside the task itself. * [SourceGenerators] Emit typed-signature register thunks (phase 2) [SysAbiExport] handlers can now be written with real signatures — a CpuContext followed by up to six int/uint/long/ulong parameters — and the generator emits the SysV unmarshalling thunk, mapping parameters positionally to RDI/RSI/RDX/RCX/R8/R9 with the same unchecked-cast idiom hand-written handlers use. SHEM003 accepts the new shape and rejects register overflow and non-register-representable types. Both shapes coexist, so migration is per-handler; sceKernelPollSema, sceKernelSignalSema, and sceKernelCancelSema migrate as the demonstration (the last showing raw ulong guest-address passthrough). The reflection scan cannot represent typed handlers, so it retires here: RegisterFromAssembly, its signature validation, and ResolveExportInfo are deleted, and the parity test that pinned the generated registry to the scan is replaced by content-invariant tests (duplicate-free, full 715-export surface, catalog identity). Deleting the scan surfaced a phase-1 latent regression — the pre-JIT warm sweep enumerated only reflection-scanned assemblies, so the generated registration path warmed nothing and re-exposed the guest-thread fail-fast risk; the warm set is now derived from the registered handler delegates themselves. * [SourceGenerators] Marshal guest strings declaratively with [GuestCString] (phase 3) A string parameter on a typed [SysAbiExport] handler, annotated [GuestCString(maxLength)], now makes the generated thunk read the null-terminated UTF-8 string from the argument register's guest address before the handler runs, returning ORBIS_GEN2_ERROR_MEMORY_FAULT to the guest when the read fails — the exact prologue nearly every string-taking handler writes by hand. The attribute lives in SharpEmu.HLE next to SysAbiExportAttribute; SHEM008 rejects misuse (non-string parameter, non-positive MaxLength) while a bare string parameter stays a SHEM003 signature error. _open, open, and sceKernelOpen migrate as the demonstration; they were chosen because their hand-written prologue faulted on a null pointer the same way the thunk does (handlers that return INVALID_ARGUMENT for null pointers, like sceKernelCreateSema, keep the raw shape so guest- visible semantics stay untouched). * [SourceGenerators] Apply review findings across the branch Behavior: the open/_open/sceKernelOpen [GuestCString] demo migration is reverted — the local compat reader falls back to host memory for paths in loader-mapped regions that ctx.Memory cannot see, so the generated thunk would have turned recoverable reads into MEMORY_FAULT. The marshalling infrastructure stays, proven by generator/analyzer tests; production migration waits for a handler whose semantics the thunk reproduces exactly. A comment on the handler records why. Build robustness: the aerolib target is skipped for design-time builds (the IDE resolves project references without compiling them, so on a fresh clone the task assembly does not exist yet), and the task/names paths are centralized in properties. The generator now emits no registry for export-free assemblies, so referencing the analyzer can never mint a colliding SharpEmu.Generated type. Cleanup and perf: the pragma-suppression sites left mis-indented by the phase-1 relocation are reformatted and the restores moved after the method body; the dead ExportsForTesting hook and its InternalsVisibleTo are deleted; the aerolib task reuses one SHA1 instance across ~150k names; the analyzer caches the parsed catalog per file snapshot instead of re-parsing 150k lines every compilation start, shares the attribute name constant with the generator, and computes the catalog-membership check once. * [CI] Run the test suites in the build workflow The workflow compiled the test projects (they are in SharpEmu.slnx) but never executed them. A solution-level dotnet test now runs between build and publish, so any test failure fails the build — including the AerolibCatalogTests/SysAbiRegistryTests that guard the build-generated aerolib.bin and the generated export registry. Generation failures of aerolib.bin itself already fail the build step: the MSBuild task logs an error event and returns false, and a missing task assembly or missing embedded output are hard MSBuild errors. The NuGet cache key now also tracks the test projects' lock files. * [SourceGenerators] Address review feedback Multi-diagnostic analyzer tests no longer assume a stable diagnostic order (analyzer execution is concurrent), and the aerolib task logs the full exception instead of only its message so build failures keep the type and stack trace. * [SourceGenerators] Address second review round Symbol-name comparisons in the shape rules and analyzer now pin an explicit SymbolDisplayFormat.FullyQualifiedFormat instead of relying on the display-format default, and the aerolib task fails loudly on a symbol name that would overflow the format's ushort length prefix instead of silently truncating it, with null-safe output-directory handling made explicit. * [SourceGenerators] Embed aerolib.bin via a target so design-time builds never reference it The static EmbeddedResource item referenced the generated file even in design-time builds, where the generation target is skipped — on a fresh clone the IDE would try to embed a file that never existed. The item is now created inside an EmbedAerolibBinary target gated on DesignTimeBuild, separate from the generation target so an up-to-date skip of GenerateAerolibBinary cannot drop the item with the rest of its body, and hooked before AssignTargetPaths since dynamic resource items added later miss the resource pipeline. Verified fresh build, incremental rebuild (embedded catalog test both times), and a simulated design-time compile with no artifacts present. * [SourceGenerators] Regenerate test lock file after rebase onto main Rebase fallout: main's package graph shifted under #200, so the SourceGenerators.Tests lock file is re-evaluated to keep --locked-mode restore green at the branch tip. * [Build] Drop NuGet lock files; rely on central package management Central package management was already in effect (ManagePackageVersionsCentrally with all versions in Directory.Packages.props and no inline PackageReference versions), so the per-project packages.lock.json files and the lock-mode workflow only added maintenance overhead. This removes all eleven lock files, drops RestorePackagesWithLockFile so restore no longer regenerates them, and takes --locked-mode off the CI restore steps (re-keying the NuGet cache on the central props files). Package versions remain centrally pinned in Directory.Packages.props. |
||
|
Gutemberg Ribeiro
|
62e1775c5c |
[HLE] Remove steady-state allocations from the hot HLE paths (#190)
* [HLE] Stop allocating on the memcpy/memset and trace hot paths memcpy/memmove no longer allocate a bounce buffer sized to the whole copy (large copies previously landed on the LOH); they loop through a single pooled 256 KB rental, copying high-to-low when the destination overlaps above the source so memmove semantics survive the chunking. memset reuses a shared zero chunk for the dominant zero-fill case and rents/fills only min(length, 16K) bytes for non-zero values instead of allocating and filling a fresh 16 KB array per call; the map-time zero-fill loop shares the same zero chunk. SHARPEMU_LOG_SEMA / SHARPEMU_LOG_VIDEOOUT are now read once into cached bools and every TraceSemaphore/TraceVideoOut call site is guarded, so trace messages are no longer interpolated (and the env var no longer queried) on every semaphore op and every flip with tracing off. Trace output when the flags are set is unchanged. * [HLE] Remove per-frame allocations from the vblank/flip/equeue plumbing The 60 Hz vblank pump no longer allocates per edge: PumpVblanks reuses a pump-thread-only port list instead of a LINQ Where/ToArray, and SignalVblank/SubmitFlip snapshot their event registrations into pooled rentals instead of copying the List on every edge and every flip (the snapshot must still be taken, since triggers run outside _stateGate and a per-port reusable buffer would race the pump thread against a guest thread's first-edge signal). sceKernelWaitEqueue delivery rents the dequeue buffer from the pool instead of allocating an array per wait, and event-queue wake keys are formatted once per handle (cached in a ConcurrentDictionary, dropped on queue delete) instead of building the string on every enqueue. The semaphore wake key moves onto KernelSemaphoreState at creation, the same pattern the pthread mutex state already uses, removing the per-signal/per-wait formatting. SHARPEMU_LOG_EQUEUE is read once into a cached bool like the sema/videoout flags. * [HLE] Read guest C-strings without per-call buffer allocations CpuContext.TryReadNullTerminatedUtf8 allocated a byte[capacity] and issued one TryRead per byte for every string-argument import. It now reads through a stack buffer (pooled above 512 bytes) in 128-byte bulk chunks, falling back to per-byte reads only when a chunk touches an unreadable range so a terminator sitting just before unmapped memory still resolves exactly as before. The chunk bound also keeps the overread past the terminator smaller than the old loop's worst case is wide, so no fault can appear where the byte loop succeeded. TryReadAsciiZ (dlsym/symbol resolution) drops its List<byte> + ToArray round-trip for the same stack/pooled buffer, keeping the byte-by-byte TryReadByteCompat reads because their Marshal.ReadByte fallback must probe exactly up to the terminator. Only the final string is allocated on either path now. * [HLE] Replace blocking-wait closures with waiter continuation objects Every wait that actually parked a guest thread allocated two capturing lambdas (plus their display classes) for the scheduler's resume/wake callbacks. RequestCurrentThreadBlock and the backend's blocked-thread state now carry a single IGuestThreadBlockWaiter instead of the Func<int>/Func<bool> pair: TryWake keeps the run-under-the-scheduler- gate contract and Resume still produces the guest's RAX on the woken thread. The waiter stays attached through the wake transition (the old code nulled only the wake handler there) and is consumed at resume. The existing waiter objects absorb the captured state as fields, so a blocking wait now allocates exactly one object: SemaphoreWaiter, PthreadMutexWaiter, and EventFlagWaiter implement the interface directly, and the equeue, cond, and rwlock waits get small waiter classes replacing their closures. Handler bodies delegate to the same static methods with the same arguments as before; the untimed event flag wait's mutable captured result becomes a field on its waiter. * [HLE] Back pending event queues with a ring deque instead of LinkedList LinkedList<KernelQueuedEvent> allocated a node object on every non-coalesced enqueue — one per vblank/flip edge per registered queue, 60+ times a second in steady state. KernelEventDeque is a grow-only ring buffer over a KernelQueuedEvent[] with the three operations the queue actually uses (AddLast, RemoveFirst, find-and-update-in-place by ident/filter), so steady-state enqueue/dequeue allocates nothing and the coalescing update writes the struct back through an indexer instead of a node reference. All accesses stay under _eventQueueGate, matching the LinkedList usage it replaces. * [HLE] Cap memcpy chunk iterations at the requested size, not the rented length Address Copilot review: ArrayPool.Rent may return a larger array than requested, so sizing each iteration by chunk.Length let the copy granularity depend on pool bucketing internals instead of the intended 256 KB chunking. Behavior was already correct for any chunk size (each iteration re-reads the source, and the overlap ordering is size- independent), but the loop now mins against the requested chunkLength, matching what memset already does. * [HLE] Skip the flip/vblank snapshot rental when no events are registered Address Copilot review: SignalVblank and SubmitFlip rented (and returned) a pooled snapshot even with zero registrations — steady per-frame pool traffic for games that never register flip events and only poll flip status. Zero-count signals now skip the rental, the copy, and the trigger loop entirely, which also retires the Math.Max(count, 1) minimum-rent guard. |
||
|
Berk
|
cf6964710a | [emulator] Improve emulator performance by optimizing memory access and reducing unnecessary overhead in kernel and CPU execution paths (#131) | ||
|
PandaCatz
|
edb4eb86a2 |
[kernel] Wake blocked waiters on semaphore signal, cancel, and delete (#67)
sceKernelWaitSema parks a guest thread on the scheduler when the count is not yet available, but sceKernelSignalSema only incremented the count and returned: there was no WakeBlockedThreads call anywhere in the file, so a thread blocked in WaitSema was never woken and the game hung there. sceKernelCancelSema and sceKernelDeleteSema left parked waiters stranded the same way. Give each semaphore a per-handle wake key and each waiter a small record with the count it needs and a result slot. Signal, cancel, and delete wake the waiters through the scheduler after releasing the semaphore lock, matching the lock order the event flag and event queue paths already use. The wake handler runs under the scheduler gate and consumes the count under the semaphore lock, so a waiter needing more than is available stays parked while a smaller waiter can still proceed; the resume handler hands the recorded result back as the guest's return value. Cancel bumps an epoch and delete sets a flag so woken waiters return what the kernel returns in those cases: ECANCELED (0x80020055) for a canceled wait and the EACCES-class 0x8002000D for a deleted semaphore. Delete succeeds even with waiters present. Only the woken waiter's own handler adjusts the waiting-thread count, so a waiter that parks during a cancel is not double-counted, and the create path now wakes a waiter that raced onto the handle if the handle write-back fails instead of stranding it. This does not change the immediate paths: an available count is still consumed inline, and a wait with a timeout pointer still returns immediately (honoring the timeout through the scheduler is a separate change). Verified with a block/wake harness that drives real guest threads through the real import trampolines: signal-after-block, signal racing the park, multi-waiter signal, need-count gating with a smaller waiter slipping past, and cancel and delete with parked waiters including the reported waiter count, plus event flag and event queue regression checks. Builds clean on Windows and Linux. |
||
|
Dawid
|
29021b5a71 | [fixes] move repeating methods into CpuContext (#41) | ||
|
Dawid
|
7337683c16 |
[fixes] stackalloc warnings, consolidate duplicated methods, minor adjustments in project settings (#39)
* [fixes] stackalloc warnings, consolidate duplicated methods * [fix] remove unnecessary edit in .slnx file |
||
|
ParantezTech
|
a5172fd2c0 | [kernel] Guest-thread blocking for pthread_mutex_lock is currently disabled | ||
|
ParantezTech
|
1ea1396979 | [kernel/videoOut] extend memory managements and videoOut (this is not a swapchain) |