mirror of
https://github.com/par274/sharpemu.git
synced 2026-09-21 08:22:37 +08:00
a8fa9c96dce5fe7e3424f811cab8e15790a11339
36
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
ecd657006a |
AGC: retain label producers and report failed release writes (#719)
* AGC: retain active label producer records
Ported from the archived silent-hill-minimal branch (274ccfdf).
RegisterLabelProducer bounded its history with RemoveRange(0, 1024) once it
reached 4096 entries, which evicts the oldest records regardless of whether
their label write has completed. Those records are not a diagnostic cache:
a suspended DCB resolves its wait_reg_mem by finding the producer that will
write the watched label. Dropping an active record hides an earlier
same-submission label write, so a legitimate in-stream fence suspends
forever and the graphics queue stops.
Compact only completed history, and let the list exceed the soft bound when
every record is still active — correctness over a diagnostic limit.
Two hardening changes over the ported version, because a suspended queue is
exactly when every record is active:
- compaction is a single order-preserving pass instead of repeated RemoveAt,
which would shift the tail per eviction (quadratic) while the label gate is
held;
- when a pass frees nothing the bound is raised to twice the current count, so
registration does not rescan the whole list on every subsequent add, and it
is reset once compaction can make progress again.
Tests: 664 pass, 0 fail (SharpEmu.Libs.Tests 598 -> 600).
* AGC: keep produced label values a suspended wait still needs
GpuWaitRegistry.RecordProduced dropped the entire produced-value table once it
reached 8192 entries:
if (_lastProduced.Count >= 8192) { _lastProduced.Clear(); }
Those entries are release state, not a cache. CollectDeadlockBroken is the only
way out for a DCB suspended on a WAIT_REG_MEM whose label the guest has since
recycled for other data: it replays the value a real producer already wrote to
that label. Clearing the table wholesale erases exactly the records live waiters
depend on, so every such waiter is stranded permanently — the suspended graphics
queue never resumes, and with it the render thread and the whole engine thread
graph park.
That is what Silent Hill (PPSA10112) hits. Its final unsatisfied waits watch
labels holding guest allocator metadata (each reads a pointer into its own
region rather than the expected 1), i.e. recycled memory, which is precisely
the case the deadlock breaker exists to handle — yet agc.deadlock_break fires 0
times across a run with 490 wait suspensions and 1068 successful release_mem
writes, because the producing values had already been cleared.
Prune only values no registered waiter is watching, and let the table exceed the
soft bound while they are all watched. Same reasoning as the label-producer
history in the preceding commit: correctness of synchronization state takes
precedence over a diagnostic bound.
Tests: 666 pass, 0 fail. The regression test fails against the previous
wholesale Clear() and passes with the prune, verified by reverting the fix.
* AGC: report release-label writes that never reach guest memory
Both release_mem handlers recorded the produced label value only on success:
if (wroteData && dataSelection is 1 or 2) GpuWaitRegistry.RecordProduced(...)
with no else. A failed write is not benign here. That packet is the producer a
suspended WAIT_REG_MEM is waiting for, so when it fails the label is neither
written nor recorded, CollectDeadlockBroken has no value to replay, and the
graphics queue stays suspended forever — the render thread and the engine
thread graph park behind it with nothing in the log to say why.
Measured on Silent Hill (PPSA10112): the fatal AccessViolation lands exactly
here, on the presenter thread, in ApplySubmittedReleaseMem's label write into a
protected guest page. The faulting variant kills the process; the
returns-false variant wedges the GPU silently. Only the first was ever visible.
Report the failure (rate-limited: first 16, then powers of two) instead of
dropping it. This does not fix the underlying write failure — it makes a
permanently suspended queue diagnosable rather than silent.
Tests: 666 pass, 0 fail.
|
||
|
|
97bd8c422e |
Combined: Pad, AGC, Metal, Vulkan, Ampr, overlay, and CPU hot-path fixes (#727)
* VideoOut: track guest-flip FPS and label load/stall gaps Headline FPS follows VideoOut submit cadence rather than host presents, and long gaps show LOAD/STALL instead of a stale multi-second MS average. * Metal: skip waitUntilCompleted when the command buffer is already Completed Tiny write-back batches often finish before the wait; checking status avoids redundant ordered-queue round-trips. * Metal: opt-in long-edge drawable cap via SHARPEMU_METAL_CAP_DRAWABLE Default presentation resolution is unchanged; set SHARPEMU_METAL_CAP_DRAWABLE=1 to cap the drawable long edge at 1920. * Pad: implement scePadGetTriggerEffectState under its own NID NID znaWI0gpuo8 was mapped to sceUserServiceGetUserName as a "title-captured alias". It is not that symbol. Recomputing the NID of every catalogued name (base64 of the reversed first eight sha1 bytes of name+salt) resolves znaWI0gpuo8 to scePadGetTriggerEffectState, and sceUserServiceGetUserName hashes to 1xxcMiGu2fo instead. Auditing all 1087 export declarations the same way found this to be the only NID whose declared name is wrong. The consequence was not a missing export but a wrong one: the user-service handler rejected the pad's arguments and returned SCE_USER_SERVICE_ERROR_INVALID_PARAMETER about eighteen thousand times per run in PPSA10112, so every poll fell back to a cached button bitmask. It also hid the calls from every search for pad activity, which is why this title was believed never to touch scePad at all. The state size is taken from the caller's own frame rather than assumed: the guest points the out-param at rbp-0x30 and stores its stack cookie at rbp-0x28, leaving eight bytes for the state. Writing the sixteen the frame superficially suggests would land on the cookie and fail the guest's stack check - the same failure this codebase has already hit three times from oversized HLE writes - so the test pins the size by asserting the cookie survives. No host pad exposes DualSense adaptive-trigger feedback, so the neutral all-zero state is reported as success, which lets the caller take its normal path instead of the fallback. * Apply SDWA ABS/NEG as float sign-bit, not integer, modifiers SDWA's ABS and NEG source modifiers are floating-point sign-bit operations on GCN: ABS clears the sign bit, NEG flips it. We applied them as integer operations instead - SAbs, and a two's-complement negate of the raw bit pattern. That turns 1.0 into -4.0 and -3.0 into 1.5. UE4 compiles the final line of DrawRectangle, OutPosition.xy *= float2(1,-1), into a single V_MOV_B32 with SDWA NEG, so every UE fullscreen pass had its clip-space Y silently skewed. The canonical fullscreen triangle (1,-1) (-3,-1) (1,3) became (1,-4) (-3,-4) (1,1.5), which covers 6/11 of the viewport instead of all of it. That reproduces the measured defect exactly, on four independent quantities: hypotenuse slope 8/11, crossings of y=+1 and y=-1 at x=+7/11 and x=-9/11, and covered area 6/11 = 54.55% of the 2304x1296 viewport. It also explains why the edge was resolution-independent and identical across six unrelated shaders - it is the same instruction in every one of them. The distinguishing evidence is the transform's fixed point. A wrong scale would hold NDC -1 in x and +1 in y; the observed transform holds the opposite corner in both. Independently, and using only the measured line rather than any assumed vertex position: a wrong multiplier alone leaves a residual of -18 whatever the multiplier, and a wrong addend alone forces slope 1, not 8/11. Both terms had to be wrong at once, which only a Y-only sign-bit corruption produces. Float instructions are unaffected: GetFloatSource passes applySdwaIntegerModifiers: false and applies its own modifiers, so this path only ever fed raw-source reads - where the hardware behaviour is the sign-bit one regardless of the opcode being a bit-move. The sign bit is selected by the SDWA source-select width so a 16-bit select flips bit 15 rather than bit 31. Verified: emitted SPIR-V for the same shader changes from OpISub %uint %uint_0 %2147 to OpBitwiseXor %uint %2147 %uint_2147483648; 25-program synthetic conformance gate passes; 805 tests green; three 100-110s live runs with no crashes and no new shader failures. The end-to-end pixel re-measurement is NOT yet closed - see task #26. * gpu: size single-channel 16-bit and two-channel 8-bit formats GetFormatCompatibilityClass listed only R16Sfloat in the 16-bit class, so GetVulkanImageByteCount computed zero bytes for R16Unorm, R16SNorm, R16Uint, R16Sint and the R8G8 family. UploadGuestImageInitialData treats a zero expected size as an incompatible upload and drops it, which leaves the texture blank for the rest of the run rather than failing loudly. Silent Hill uploads R16Unorm at 144x81, 240x135, 256x256, 512x512 and 1024x1024, and every one was rejected: the guest supplied exactly width*height*2 bytes each time (23328 for 144x81) against an expected zero. Also adds R8SNorm to the 8-bit class, which was missing for the same reason. * Vulkan: enable textureCompressionBC when the device supports it Guest BC1–BC7 textures can be sampled directly when the feature is available; warn when it is not. * Share one guest image across sRGB/UNORM aliases Ported from origin/fix/view-compatible-guest-images 7fb8fdf. Rendering as sRGB and ImageLoad/Store-ing as UNORM at the same guest address are the same surface accessed through different number formats. Recreating the guest image per number format ping-pongs content between two VkImages and loses the rendered pixels on every transition; the mutable-format image now accepts the counterpart identity and serves it through alias views. The commit names AvPlayer movie copies as the pattern that needs this. Adapted for this base: GetOrCreateGuestImage has since grown resolution scaling and 3D/array support, so the alias accept is folded into the current predicate (LogicalWidth/LogicalHeight/LogicalDepth/Type) rather than the old Width/Height pair, and the storage-counterpart widening is placed before the physical-dimension computation. The helper functions it relies on (GetStorageImageFormat, IsCompatibleViewFormat) already existed. * fix(gpu): one vertex attribute per guest stream view The scalar evaluator gave every buffer_load_format instruction its own attribute location. Two things multiply those: the CFG walk visits one instruction on several paths, and an uber vertex shader fetches the same stream from every material branch. UE's larger shaders reached 56 bindings from 8 distinct views, and one reached 701 from 5. Metal caps a vertex function at 31 attributes, so MoltenVK failed the MSL compile with "'attribute' attribute parameter is out of bounds" and the surrounding vkCreateGraphicsPipelines returned ErrorInitializationFailed. Every draw using those pipelines was dropped, which is why Silent Hill: The Short Message rendered a black scene. The vertex buffer count drove Metal's buffer indices out of range too, giving the companion "cannot reserve 'buffer' resource location at index 0" failures. Key attributes by the guest stream view they read - absolute element address, record stride and format - and alias every other fetch that resolves to the same view onto that binding, so both translators map those instruction PCs to one input variable. On PPSA10112 this takes the worst shader from 56 attributes to 8 and pipeline failures from 840 to 0. * AGC: deliver compute-queue completion events at the queue fence sceAgcDriverSubmitAcb submissions never produced a completion interrupt. NotifySubmittedDcbCompleted returned early for anything that was not the graphics queue, so an ACB reaching its ordered-queue fence published nothing. UE 4.27's dynamic-resolution GPU-timing heuristic parks the game thread on exactly that interrupt, so the render side never advanced and Silent Hill: The Short Message deadlocked after its first frame. Give every queue a CompletionEventId: 0 for graphics (what it already published) and the owner handle from sceAgcDriverSubmitAcb rdi for a compute queue, which is the same value the guest passes to sceAgcDriverAddEqEvent. Publish under that ident from the existing fence point, which both PumpSubmittedQueue and ResumeSuspendedDcb already reach only after the submission is fully parsed. Delivery is synchronous on the ordered guest-action queue rather than on a ThreadPool hop with a sleep. That action runs after the logical queue has flushed and waited for its latest fence, which is the moment hardware would raise end-of-pipe. Deferring past it can only make the interrupt late and reorder it against registration changes. Gating: the per-queue completion event is unconditional, because completion interrupts do fire on real hardware and because delivery is registration-gated -- TriggerRegisteredEvents only queues onto equeues that registered this exact (ident, graphics filter) pair, and sceAgcDriverAddEqEvent is the only producer of graphics registrations. A title that never registers its ACB owner handle observes no change. SHARPEMU_AGC_SUBMIT_COMPLETION_EVENT is left to gate only the broad ident-ignoring fan-out (TriggerRegisteredEventsDistinct), which is a compatibility guess rather than hardware behavior; it also stays scoped to the graphics queue where it was measured, so enabling the flag does not newly fan out across compute queues. * fix(gpu): reflect guest CPU writes into large and render-target-aliased images Ports the still-applicable half of the archived fork's stale-texture fix (a42ccae) onto current main. Silent Hill: The Short Message (PPSA10112) shows both faces: black title-screen UI (glyph atlas frozen at its first upload) and stale/garbage rows on the brightness screen (a 3840x2160 UI sheet whose backing bytes were never re-read). 1. Write tracking is armed under a byte budget, not a resolution cap. GetOrCreateGuestImage armed GuestImageWriteTracker only when target.Width <= 1920 && target.Height <= 1080. Silent Hill renders at 3840x2160, so every one of its render targets was excluded from CPU-write tracking and no guest rewrite of one could ever invalidate it: SyncCpuWrittenGuestImages (the flip / ACQUIRE_MEM re-upload path) only ever visits ranges the tracker armed. The cap presumably existed as a perf guard, but resolution is the wrong proxy for the cost. Arming is one mprotect over the range, and the fault handler unprotects the whole range on the first store, so a write burst costs one fault regardless of size. What actually scales with the surface is the dirty re-upload: one byte[byteCount] allocation plus a full guest-memory read per dirty flip. So the guard is now a byte budget, set equal to the 128 MiB limit that SyncCpuWrittenGuestImages itself enforces before re-uploading. Above that, arming can only cost faults; it can never produce a re-upload. That is generous for 4K (RGBA8 32 MiB, RGBA16F 63 MiB, RGBA32F 127 MiB all fit) while still excluding volume textures that a resolution cap could not see at all (512^3 RGBA8 is 512 MiB behind a "512x512" surface). Both sites now also arm the exact extent recorded in _guestImageExtents (GetTextureByteCount) instead of Width*Height*depth*GetTextureBytesPerPixel, so the armed range and the range the sync path reads back are the same bytes; the old expression over-counted block-compressed and unknown formats. 2. The CPU-texture refresh path no longer gates on IsCpuBacked alone. TryCreateCpuTextureRefreshResource bailed on !guestImage.IsCpuBacked. That flag is a latch: it flips false the first time an address is used as a render target and never flips back. A surface that was rendered into once and is afterwards rewritten by the guest CPU was therefore frozen at its last GPU content forever, even when the parse thread had already shipped fresh texels for it. The gate is replaced by ShouldRefreshGuestImageFromCpu: CPU-backed, or the parse-time write generation is above zero and differs from the generation recorded by the last upload. Keeping the positive- generation requirement preserves the pure GPU-feedback case (render into an image, then sample it) that IsCpuBacked used to protect: such a surface is now tracked (change 1) but never CPU-written, so its generation stays zero and its live image is left alone. This matters more than it did in the fork, precisely because change 1 arms tracking on far more render targets. Dropping the gate outright, as the fork did, would let a target sampled under a format tag that the availability map does not match be overwritten once with whatever sits in guest memory. The existing content fingerprint still suppresses redundant uploads, and MarkSampledImagesInitialized records the uploaded generation, so a rewritten surface re-uploads exactly once per guest write burst. Not ported: the fork's third part added a PeekDirty guard to the parse-time snapshot fast path in AgcExports. It is superseded. That path now calls IsGuestImageUploadKnown, which already compares _cpuBackedUploadGenerations against the tracker's write generation - a monotonic value that survives another owner consuming the dirty flag, unlike PeekDirty, which both EvictDirtyCachedTextures and SyncCpuWrittenGuestImages clear. Guest images with no generation entry are covered instead by SyncCpuWrittenGuestImages. Adding a non-consuming PeekDirty there would also make every draw between a CPU write and the next flip fall through to a full texel re-read of the surface (33 MiB for a 4K sheet), since Track re-arms without clearing the dirty flag. The fork's promotion-path Track ("vulkan.cpu-backed-image") is likewise superseded: AgcExports already arms every sampled texture's backing extent as "agc.decoded-texture" before reading its texels. Both decisions are extracted as pure internal predicates so they can be unit-tested; the Vulkan device code around them needs a real device. * Kernel: correct equeue event delivery and waiter lifetime Ports the event-queue rework from the archived silent-hill-minimal branch (968e9606, 9e3abb12, 2be9cfbf, f157a115) onto current upstream. The equeue implementation had not been touched upstream since that branch forked, so none of it had landed. Behaviour fixed: - Per-waiter event reservation. A blocked waiter used to wake on "the queue has any pending event" (TryWake => HasPendingEvents) and then re-read the queue on resume, so a woken waiter could find the event already drained by another waiter and park again with the wake consumed. Events are now reserved to the waiter they are delivered to. - Level-triggered events are preserved instead of being cleared by an unrelated read; only events that declare clear-on-read reset their trigger state. - Queued interrupts are bound to the registration generation that produced them, so an event registered after a queue was reused cannot consume an interrupt raised for the previous registration. - Deleting an equeue now terminates its waiters instead of leaving them blocked on a handle that no longer resolves. - sceKernelTriggerUserEvent stores its third argument in the event's udata (0x18) rather than its data word (0x10). The guest reads it back with sceKernelGetEventUserData, which loads 0x18, so every triggered user event previously read back as 0. This matches the reference behaviour in shadPS4, where TriggerEvent takes udata and sceKernelGetEventUserData returns ev->udata. The upstream test asserting the data word is updated, since it encoded the inconsistency rather than the ABI. KernelPthreadState gains TryGetCurrentThreadIdentity and a new KernelSyncTraceFormatter carries the shared, opt-in diagnostic formatting the ported code calls; both are gated behind the existing trace flag and do no work when it is off. Tests: 662 pass, 0 fail (SharpEmu.Libs.Tests 588 -> 598). * Implement VOP3 0x149 V_BFE_I32 and 0x36A V_CVT_PKRTZ_F16_F32 PPSA10112 dropped three shaders per run on two unimplemented VOP3 opcodes. The translator fails a whole shader on an opcode it does not know, so each one costs a dropped draw rather than wrong pixels. 0x149 sits between 0x148 V_BFE_U32 and 0x14A V_BFI_B32, and 0x36A between 0x369 V_CVT_PKNORM_U16_F32 and 0x36D V_ADD3_U32; the surrounding table entries already match the canonical map densely on both sides of each gap. V_CVT_PKRTZ_F16_F32 needed no emitter - it was already implemented for the VOP2 form at 0x2F and its VOP3 alias at 0x12F, and only the VOP3-only opcode number was missing. V_BFE_I32 mirrors its unsigned sibling, masking offset and width to 5 bits, and differs only in extracting through a signed type so the field sign-extends. GCN defines width 0 as returning 0 where SPIR-V leaves a zero Count unspecified; VBfeU32 has the same gap, so this matches it deliberately rather than diverging - fix both together if it matters. Also widen ReportGuestPointerSplit, which filtered to an incoming value of exactly 1. The crash being hunted leaves 0x0000007000000000, whose low dword is 0, so the one detector built to catch this bug class could never have reported it. Verified: unsupported-opcode errors 3-4 per run -> 0, over a 130s run that survives to the same stage. Metal translator still lacks VBfeI32; Vulkan/MoltenVK is the macOS path so it is a divergence, not a blocker. * AGC: retain active label producer records Ported from the archived silent-hill-minimal branch (274ccfdf). RegisterLabelProducer bounded its history with RemoveRange(0, 1024) once it reached 4096 entries, which evicts the oldest records regardless of whether their label write has completed. Those records are not a diagnostic cache: a suspended DCB resolves its wait_reg_mem by finding the producer that will write the watched label. Dropping an active record hides an earlier same-submission label write, so a legitimate in-stream fence suspends forever and the graphics queue stops. Compact only completed history, and let the list exceed the soft bound when every record is still active — correctness over a diagnostic limit. Two hardening changes over the ported version, because a suspended queue is exactly when every record is active: - compaction is a single order-preserving pass instead of repeated RemoveAt, which would shift the tail per eviction (quadratic) while the label gate is held; - when a pass frees nothing the bound is raised to twice the current count, so registration does not rescan the whole list on every subsequent add, and it is reset once compaction can make progress again. Tests: 664 pass, 0 fail (SharpEmu.Libs.Tests 598 -> 600). * AGC: keep produced label values a suspended wait still needs GpuWaitRegistry.RecordProduced dropped the entire produced-value table once it reached 8192 entries: if (_lastProduced.Count >= 8192) { _lastProduced.Clear(); } Those entries are release state, not a cache. CollectDeadlockBroken is the only way out for a DCB suspended on a WAIT_REG_MEM whose label the guest has since recycled for other data: it replays the value a real producer already wrote to that label. Clearing the table wholesale erases exactly the records live waiters depend on, so every such waiter is stranded permanently — the suspended graphics queue never resumes, and with it the render thread and the whole engine thread graph park. That is what Silent Hill (PPSA10112) hits. Its final unsatisfied waits watch labels holding guest allocator metadata (each reads a pointer into its own region rather than the expected 1), i.e. recycled memory, which is precisely the case the deadlock breaker exists to handle — yet agc.deadlock_break fires 0 times across a run with 490 wait suspensions and 1068 successful release_mem writes, because the producing values had already been cleared. Prune only values no registered waiter is watching, and let the table exceed the soft bound while they are all watched. Same reasoning as the label-producer history in the preceding commit: correctness of synchronization state takes precedence over a diagnostic bound. Tests: 666 pass, 0 fail. The regression test fails against the previous wholesale Clear() and passes with the prune, verified by reverting the fix. * Correct VOP3 0x36A: V_CVT_PK_U16_U32, not V_CVT_PKRTZ_F16_F32 a2d186e mapped 0x36A to V_CVT_PKRTZ_F16_F32 on the strength of a gap in our own opcode table. That was wrong, and wrong in the worst available way: an emitter for V_CVT_PKRTZ_F16_F32 already existed, so instead of failing loudly like an unknown opcode, the mapping would have emitted float-pack semantics for an integer-pack instruction and produced silently incorrect results. LLVM is unambiguous: defm V_CVT_PK_U16_U32 : VOP3Only_Real_gfx10<0x36a>; defm V_CVT_PKRTZ_F16_F32 : VOP2_Real_gfx6_gfx7_gfx10<0x02f>; so on Gen5 the float pack is VOP2 0x2F with VOP3 alias 0x12F - both of which our table already had - and there is no VOP3-only encoding of it to add. 0x36A sits with the other integer/normalised pack conversions at 0x368/0x369/0x36B. silent-hill-minimal-rebased had this right all along. The V_BFE_I32 half of a2d186e stands: LLVM confirms 0x149, and that branch maps it identically. Lesson, since I had just warned someone else about exactly this: a gap in a table is evidence about numbering, not about identity. Inference from neighbouring entries is fine for narrowing candidates and worth nothing as a conclusion - especially when a plausible emitter already exists to swallow the mistake quietly. * AGC: report release-label writes that never reach guest memory Both release_mem handlers recorded the produced label value only on success: if (wroteData && dataSelection is 1 or 2) GpuWaitRegistry.RecordProduced(...) with no else. A failed write is not benign here. That packet is the producer a suspended WAIT_REG_MEM is waiting for, so when it fails the label is neither written nor recorded, CollectDeadlockBroken has no value to replay, and the graphics queue stays suspended forever — the render thread and the engine thread graph park behind it with nothing in the log to say why. Measured on Silent Hill (PPSA10112): the fatal AccessViolation lands exactly here, on the presenter thread, in ApplySubmittedReleaseMem's label write into a protected guest page. The faulting variant kills the process; the returns-false variant wedges the GPU silently. Only the first was ever visible. Report the failure (rate-limited: first 16, then powers of two) instead of dropping it. This does not fix the underlying write failure — it makes a permanently suspended queue diagnosable rather than silent. Tests: 666 pass, 0 fail. * CPU: gate periodic import logs and fast-path memcpy/memmove leaves SHARPEMU_LOG_IMPORT_PERIODIC=1 re-enables periodic Import# tracing. memcpy/memmove use a thin TryCopy leaf that skips register marshalling. * Ampr: cooked-id index preload and host FD LRU cache Preload the app0 APR index during bind and bound the open host-file cache so streaming reads stay responsive under large title archives. * Vulkan: abandon timed-out guest fences without blocking later frames Move fence-timeout submissions out of the blocking queue, keep GPU objects alive until the fence signals, and retire them from an abandoned list so one hung dispatch cannot re-block capacity waits. * AGC: follow command-buffer branches across arena switches Implement sceAgcCbBranch and walk INDIRECT_BUFFER so submissions that continue in a linked buffer keep their flip and end-of-frame labels. |
||
|
|
c4ae4a2059 |
Vulkan: abandon timed-out guest fences without blocking later frames (#726)
Move fence-timeout submissions out of the blocking queue, keep GPU objects alive until the fence signals, and retire them from an abandoned list so one hung dispatch cannot re-block capacity waits. |
||
|
|
93c9f14081 |
Ampr: cooked-id index preload and host FD LRU cache (#725)
Preload the app0 APR index during bind and bound the open host-file cache so streaming reads stay responsive under large title archives. |
||
|
|
532251c0c3 |
CPU: gate periodic import logs and fast-path memcpy/memmove leaves (#724)
SHARPEMU_LOG_IMPORT_PERIODIC=1 re-enables periodic Import# tracing. memcpy/memmove use a thin TryCopy leaf that skips register marshalling. |
||
|
|
816ec4ad27 |
Kernel: correct equeue event delivery and waiter lifetime (#723)
Ports the event-queue rework from the archived silent-hill-minimal branch (968e9606, 9e3abb12, 2be9cfbf, f157a115) onto current upstream. The equeue implementation had not been touched upstream since that branch forked, so none of it had landed. Behaviour fixed: - Per-waiter event reservation. A blocked waiter used to wake on "the queue has any pending event" (TryWake => HasPendingEvents) and then re-read the queue on resume, so a woken waiter could find the event already drained by another waiter and park again with the wake consumed. Events are now reserved to the waiter they are delivered to. - Level-triggered events are preserved instead of being cleared by an unrelated read; only events that declare clear-on-read reset their trigger state. - Queued interrupts are bound to the registration generation that produced them, so an event registered after a queue was reused cannot consume an interrupt raised for the previous registration. - Deleting an equeue now terminates its waiters instead of leaving them blocked on a handle that no longer resolves. - sceKernelTriggerUserEvent stores its third argument in the event's udata (0x18) rather than its data word (0x10). The guest reads it back with sceKernelGetEventUserData, which loads 0x18, so every triggered user event previously read back as 0. This matches the reference behaviour in shadPS4, where TriggerEvent takes udata and sceKernelGetEventUserData returns ev->udata. The upstream test asserting the data word is updated, since it encoded the inconsistency rather than the ABI. KernelPthreadState gains TryGetCurrentThreadIdentity and a new KernelSyncTraceFormatter carries the shared, opt-in diagnostic formatting the ported code calls; both are gated behind the existing trace flag and do no work when it is off. Tests: 662 pass, 0 fail (SharpEmu.Libs.Tests 588 -> 598). |
||
|
|
82c2c7f48c |
fix(gpu): reflect guest CPU writes into large and render-target-aliased images (#722)
Ports the still-applicable half of the archived fork's stale-texture fix
(a42ccae) onto current main. Silent Hill: The Short Message (PPSA10112)
shows both faces: black title-screen UI (glyph atlas frozen at its first
upload) and stale/garbage rows on the brightness screen (a 3840x2160 UI
sheet whose backing bytes were never re-read).
1. Write tracking is armed under a byte budget, not a resolution cap.
GetOrCreateGuestImage armed GuestImageWriteTracker only when
target.Width <= 1920 && target.Height <= 1080. Silent Hill renders at
3840x2160, so every one of its render targets was excluded from
CPU-write tracking and no guest rewrite of one could ever invalidate
it: SyncCpuWrittenGuestImages (the flip / ACQUIRE_MEM re-upload path)
only ever visits ranges the tracker armed.
The cap presumably existed as a perf guard, but resolution is the
wrong proxy for the cost. Arming is one mprotect over the range, and
the fault handler unprotects the whole range on the first store, so a
write burst costs one fault regardless of size. What actually scales
with the surface is the dirty re-upload: one byte[byteCount]
allocation plus a full guest-memory read per dirty flip. So the guard
is now a byte budget, set equal to the 128 MiB limit that
SyncCpuWrittenGuestImages itself enforces before re-uploading. Above
that, arming can only cost faults; it can never produce a re-upload.
That is generous for 4K (RGBA8 32 MiB, RGBA16F 63 MiB, RGBA32F
127 MiB all fit) while still excluding volume textures that a
resolution cap could not see at all (512^3 RGBA8 is 512 MiB behind a
"512x512" surface).
Both sites now also arm the exact extent recorded in
_guestImageExtents (GetTextureByteCount) instead of
Width*Height*depth*GetTextureBytesPerPixel, so the armed range and
the range the sync path reads back are the same bytes; the old
expression over-counted block-compressed and unknown formats.
2. The CPU-texture refresh path no longer gates on IsCpuBacked alone.
TryCreateCpuTextureRefreshResource bailed on !guestImage.IsCpuBacked.
That flag is a latch: it flips false the first time an address is
used as a render target and never flips back. A surface that was
rendered into once and is afterwards rewritten by the guest CPU was
therefore frozen at its last GPU content forever, even when the parse
thread had already shipped fresh texels for it.
The gate is replaced by ShouldRefreshGuestImageFromCpu: CPU-backed,
or the parse-time write generation is above zero and differs from the
generation recorded by the last upload. Keeping the positive-
generation requirement preserves the pure GPU-feedback case (render
into an image, then sample it) that IsCpuBacked used to protect: such
a surface is now tracked (change 1) but never CPU-written, so its
generation stays zero and its live image is left alone. This matters
more than it did in the fork, precisely because change 1 arms
tracking on far more render targets. Dropping the gate outright, as
the fork did, would let a target sampled under a format tag that the
availability map does not match be overwritten once with whatever
sits in guest memory.
The existing content fingerprint still suppresses redundant uploads,
and MarkSampledImagesInitialized records the uploaded generation, so
a rewritten surface re-uploads exactly once per guest write burst.
Not ported: the fork's third part added a PeekDirty guard to the
parse-time snapshot fast path in AgcExports. It is superseded. That path
now calls IsGuestImageUploadKnown, which already compares
_cpuBackedUploadGenerations against the tracker's write generation - a
monotonic value that survives another owner consuming the dirty flag,
unlike PeekDirty, which both EvictDirtyCachedTextures and
SyncCpuWrittenGuestImages clear. Guest images with no generation entry
are covered instead by SyncCpuWrittenGuestImages. Adding a non-consuming
PeekDirty there would also make every draw between a CPU write and the
next flip fall through to a full texel re-read of the surface (33 MiB
for a 4K sheet), since Track re-arms without clearing the dirty flag.
The fork's promotion-path Track ("vulkan.cpu-backed-image") is likewise
superseded: AgcExports already arms every sampled texture's backing
extent as "agc.decoded-texture" before reading its texels.
Both decisions are extracted as pure internal predicates so they can be
unit-tested; the Vulkan device code around them needs a real device.
|
||
|
|
531e35b6d5 |
AGC: deliver compute-queue completion events at the queue fence (#721)
sceAgcDriverSubmitAcb submissions never produced a completion interrupt. NotifySubmittedDcbCompleted returned early for anything that was not the graphics queue, so an ACB reaching its ordered-queue fence published nothing. UE 4.27's dynamic-resolution GPU-timing heuristic parks the game thread on exactly that interrupt, so the render side never advanced and Silent Hill: The Short Message deadlocked after its first frame. Give every queue a CompletionEventId: 0 for graphics (what it already published) and the owner handle from sceAgcDriverSubmitAcb rdi for a compute queue, which is the same value the guest passes to sceAgcDriverAddEqEvent. Publish under that ident from the existing fence point, which both PumpSubmittedQueue and ResumeSuspendedDcb already reach only after the submission is fully parsed. Delivery is synchronous on the ordered guest-action queue rather than on a ThreadPool hop with a sleep. That action runs after the logical queue has flushed and waited for its latest fence, which is the moment hardware would raise end-of-pipe. Deferring past it can only make the interrupt late and reorder it against registration changes. Gating: the per-queue completion event is unconditional, because completion interrupts do fire on real hardware and because delivery is registration-gated -- TriggerRegisteredEvents only queues onto equeues that registered this exact (ident, graphics filter) pair, and sceAgcDriverAddEqEvent is the only producer of graphics registrations. A title that never registers its ACB owner handle observes no change. SHARPEMU_AGC_SUBMIT_COMPLETION_EVENT is left to gate only the broad ident-ignoring fan-out (TriggerRegisteredEventsDistinct), which is a compatibility guess rather than hardware behavior; it also stays scoped to the graphics queue where it was measured, so enabling the flag does not newly fan out across compute queues. |
||
|
|
3f9bd2b92b |
AGC: follow command-buffer branches across arena switches (#720)
Implement sceAgcCbBranch and walk INDIRECT_BUFFER so submissions that continue in a linked buffer keep their flip and end-of-frame labels. |
||
|
|
eb0653eded |
fix(gpu): one vertex attribute per guest stream view (#718)
The scalar evaluator gave every buffer_load_format instruction its own attribute location. Two things multiply those: the CFG walk visits one instruction on several paths, and an uber vertex shader fetches the same stream from every material branch. UE's larger shaders reached 56 bindings from 8 distinct views, and one reached 701 from 5. Metal caps a vertex function at 31 attributes, so MoltenVK failed the MSL compile with "'attribute' attribute parameter is out of bounds" and the surrounding vkCreateGraphicsPipelines returned ErrorInitializationFailed. Every draw using those pipelines was dropped, which is why Silent Hill: The Short Message rendered a black scene. The vertex buffer count drove Metal's buffer indices out of range too, giving the companion "cannot reserve 'buffer' resource location at index 0" failures. Key attributes by the guest stream view they read - absolute element address, record stride and format - and alias every other fetch that resolves to the same view onto that binding, so both translators map those instruction PCs to one input variable. On PPSA10112 this takes the worst shader from 56 attributes to 8 and pipeline failures from 840 to 0. |
||
|
|
e1695cf87f |
Share one guest image across sRGB/UNORM aliases (#717)
Ported from origin/fix/view-compatible-guest-images 7fb8fdf. Rendering as sRGB and ImageLoad/Store-ing as UNORM at the same guest address are the same surface accessed through different number formats. Recreating the guest image per number format ping-pongs content between two VkImages and loses the rendered pixels on every transition; the mutable-format image now accepts the counterpart identity and serves it through alias views. The commit names AvPlayer movie copies as the pattern that needs this. Adapted for this base: GetOrCreateGuestImage has since grown resolution scaling and 3D/array support, so the alias accept is folded into the current predicate (LogicalWidth/LogicalHeight/LogicalDepth/Type) rather than the old Width/Height pair, and the storage-counterpart widening is placed before the physical-dimension computation. The helper functions it relies on (GetStorageImageFormat, IsCompatibleViewFormat) already existed. |
||
|
|
0dd543354d |
Vulkan: enable textureCompressionBC when the device supports it (#716)
Guest BC1–BC7 textures can be sampled directly when the feature is available; warn when it is not. |
||
|
|
fc5b6baaa7 |
gpu: size single-channel 16-bit and two-channel 8-bit formats (#715)
GetFormatCompatibilityClass listed only R16Sfloat in the 16-bit class, so GetVulkanImageByteCount computed zero bytes for R16Unorm, R16SNorm, R16Uint, R16Sint and the R8G8 family. UploadGuestImageInitialData treats a zero expected size as an incompatible upload and drops it, which leaves the texture blank for the rest of the run rather than failing loudly. Silent Hill uploads R16Unorm at 144x81, 240x135, 256x256, 512x512 and 1024x1024, and every one was rejected: the guest supplied exactly width*height*2 bytes each time (23328 for 144x81) against an expected zero. Also adds R8SNorm to the 8-bit class, which was missing for the same reason. |
||
|
|
e5e02c0908 |
Shader: implement V_BFE_I32 and correct VOP3 0x36A mapping (#714)
* Implement VOP3 0x149 V_BFE_I32 and 0x36A V_CVT_PKRTZ_F16_F32
PPSA10112 dropped three shaders per run on two unimplemented VOP3
opcodes. The translator fails a whole shader on an opcode it does not
know, so each one costs a dropped draw rather than wrong pixels.
0x149 sits between 0x148 V_BFE_U32 and 0x14A V_BFI_B32, and 0x36A
between 0x369 V_CVT_PKNORM_U16_F32 and 0x36D V_ADD3_U32; the
surrounding table entries already match the canonical map densely on
both sides of each gap. V_CVT_PKRTZ_F16_F32 needed no emitter - it was
already implemented for the VOP2 form at 0x2F and its VOP3 alias at
0x12F, and only the VOP3-only opcode number was missing.
V_BFE_I32 mirrors its unsigned sibling, masking offset and width to 5
bits, and differs only in extracting through a signed type so the field
sign-extends. GCN defines width 0 as returning 0 where SPIR-V leaves a
zero Count unspecified; VBfeU32 has the same gap, so this matches it
deliberately rather than diverging - fix both together if it matters.
Also widen ReportGuestPointerSplit, which filtered to an incoming value
of exactly 1. The crash being hunted leaves 0x0000007000000000, whose
low dword is 0, so the one detector built to catch this bug class could
never have reported it.
Verified: unsupported-opcode errors 3-4 per run -> 0, over a 130s run
that survives to the same stage. Metal translator still lacks VBfeI32;
Vulkan/MoltenVK is the macOS path so it is a divergence, not a blocker.
* Correct VOP3 0x36A: V_CVT_PK_U16_U32, not V_CVT_PKRTZ_F16_F32
a2d186e mapped 0x36A to V_CVT_PKRTZ_F16_F32 on the strength of a gap in
our own opcode table. That was wrong, and wrong in the worst available
way: an emitter for V_CVT_PKRTZ_F16_F32 already existed, so instead of
failing loudly like an unknown opcode, the mapping would have emitted
float-pack semantics for an integer-pack instruction and produced
silently incorrect results.
LLVM is unambiguous:
defm V_CVT_PK_U16_U32 : VOP3Only_Real_gfx10<0x36a>;
defm V_CVT_PKRTZ_F16_F32 : VOP2_Real_gfx6_gfx7_gfx10<0x02f>;
so on Gen5 the float pack is VOP2 0x2F with VOP3 alias 0x12F - both of
which our table already had - and there is no VOP3-only encoding of it
to add. 0x36A sits with the other integer/normalised pack conversions
at 0x368/0x369/0x36B.
silent-hill-minimal-rebased had this right all along.
The V_BFE_I32 half of a2d186e stands: LLVM confirms 0x149, and that
branch maps it identically.
Lesson, since I had just warned someone else about exactly this: a gap
in a table is evidence about numbering, not about identity. Inference
from neighbouring entries is fine for narrowing candidates and worth
nothing as a conclusion - especially when a plausible emitter already
exists to swallow the mistake quietly.
|
||
|
|
539baa66e7 |
Apply SDWA ABS/NEG as float sign-bit, not integer, modifiers (#713)
SDWA's ABS and NEG source modifiers are floating-point sign-bit operations on GCN: ABS clears the sign bit, NEG flips it. We applied them as integer operations instead - SAbs, and a two's-complement negate of the raw bit pattern. That turns 1.0 into -4.0 and -3.0 into 1.5. UE4 compiles the final line of DrawRectangle, OutPosition.xy *= float2(1,-1), into a single V_MOV_B32 with SDWA NEG, so every UE fullscreen pass had its clip-space Y silently skewed. The canonical fullscreen triangle (1,-1) (-3,-1) (1,3) became (1,-4) (-3,-4) (1,1.5), which covers 6/11 of the viewport instead of all of it. That reproduces the measured defect exactly, on four independent quantities: hypotenuse slope 8/11, crossings of y=+1 and y=-1 at x=+7/11 and x=-9/11, and covered area 6/11 = 54.55% of the 2304x1296 viewport. It also explains why the edge was resolution-independent and identical across six unrelated shaders - it is the same instruction in every one of them. The distinguishing evidence is the transform's fixed point. A wrong scale would hold NDC -1 in x and +1 in y; the observed transform holds the opposite corner in both. Independently, and using only the measured line rather than any assumed vertex position: a wrong multiplier alone leaves a residual of -18 whatever the multiplier, and a wrong addend alone forces slope 1, not 8/11. Both terms had to be wrong at once, which only a Y-only sign-bit corruption produces. Float instructions are unaffected: GetFloatSource passes applySdwaIntegerModifiers: false and applies its own modifiers, so this path only ever fed raw-source reads - where the hardware behaviour is the sign-bit one regardless of the opcode being a bit-move. The sign bit is selected by the SDWA source-select width so a 16-bit select flips bit 15 rather than bit 31. Verified: emitted SPIR-V for the same shader changes from OpISub %uint %uint_0 %2147 to OpBitwiseXor %uint %2147 %uint_2147483648; 25-program synthetic conformance gate passes; 805 tests green; three 100-110s live runs with no crashes and no new shader failures. The end-to-end pixel re-measurement is NOT yet closed - see task #26. |
||
|
|
b572738547 |
Pad: implement scePadGetTriggerEffectState under its own NID (#712)
NID znaWI0gpuo8 was mapped to sceUserServiceGetUserName as a "title-captured alias". It is not that symbol. Recomputing the NID of every catalogued name (base64 of the reversed first eight sha1 bytes of name+salt) resolves znaWI0gpuo8 to scePadGetTriggerEffectState, and sceUserServiceGetUserName hashes to 1xxcMiGu2fo instead. Auditing all 1087 export declarations the same way found this to be the only NID whose declared name is wrong. The consequence was not a missing export but a wrong one: the user-service handler rejected the pad's arguments and returned SCE_USER_SERVICE_ERROR_INVALID_PARAMETER about eighteen thousand times per run in PPSA10112, so every poll fell back to a cached button bitmask. It also hid the calls from every search for pad activity, which is why this title was believed never to touch scePad at all. The state size is taken from the caller's own frame rather than assumed: the guest points the out-param at rbp-0x30 and stores its stack cookie at rbp-0x28, leaving eight bytes for the state. Writing the sixteen the frame superficially suggests would land on the cookie and fail the guest's stack check - the same failure this codebase has already hit three times from oversized HLE writes - so the test pins the size by asserting the cookie survives. No host pad exposes DualSense adaptive-trigger feedback, so the neutral all-zero state is reported as success, which lets the caller take its normal path instead of the fallback. |
||
|
|
b75e4e01a0 |
Metal: opt-in long-edge drawable cap via SHARPEMU_METAL_CAP_DRAWABLE (#711)
Default presentation resolution is unchanged; set SHARPEMU_METAL_CAP_DRAWABLE=1 to cap the drawable long edge at 1920. |
||
|
|
79aa764d03 |
Metal: skip waitUntilCompleted when the command buffer is already Completed (#709)
Tiny write-back batches often finish before the wait; checking status avoids redundant ordered-queue round-trips. |
||
|
|
ec65419c0a |
VideoOut: track guest-flip FPS and label load/stall gaps (#708)
Headline FPS follows VideoOut submit cadence rather than host presents, and long gaps show LOAD/STALL instead of a stale multi-second MS average. |
||
|
|
3574a3b145 |
Shader: lower VOP3P V_FMA_MIX_F32/LO/HI (was dropping Unity HDR shaders) (#466)
The decoder recognises the VOP3P mix ops (0x20 V_FMA_MIX_F32, 0x21 V_FMA_MIXLO_F16, 0x22 V_FMA_MIXHI_F16) but left them opaque (Vop3pRaw20/21/22), so at SPIR-V emission they fell through the vector-ALU switch to the default and failed with "unsupported vector opcode". A single unhandled instruction fails the whole compile, so any shader using fma_mix was dropped entirely. Unity's built-in-RP / PostProcessing v2 HDR, tone-mapping and auto-exposure shaders emit V_FMA_MIX_F32, so those passes never translated (this is what kept Superliminal's auto-exposure luminance chain from running). Name the three opcodes in DecodeVop3p (like the packed v_pk_* ops) and lower them in the SPIR-V translator. Each mix op computes a single f32 fma(a, b, c) where every source is read *independently* as either a full f32 register/constant or one f16 half widened to f32. Per operand, op_sel_hi selects f16-vs-f32 and op_sel picks which f16 half; the neg_hi field is repurposed as an absolute-value modifier and neg negates, applied abs-then-neg. This reuses the VOP3P op_sel/op_sel_hi/neg/neg_hi bit layout with the mix-specific meaning, not the packed-math meaning. The result is a scalar f32 for V_FMA_MIX_F32; _MIXLO/_MIXHI narrow it back to f16 (exact round-to-nearest-even, via the existing EmitFloatToHalf) and write it into the low/high 16 bits of vdst, preserving the other half. The clamp modifier saturates to [0, 1] consistently with the other VOP3P ops. Per-operand F16/F32 select and the abs/neg modifiers follow shadPS4's GetSrcMix, the authoritative reference for the mix semantics. Adds Gen5FmaMixSpirvTests: assembles V_FMA_MIX_F32 (with a representative op_sel/op_sel_hi/neg/abs) and V_FMA_MIXLO_F16 compute shaders and asserts they translate to GPU SPIR-V without hitting the drop path and emit a GLSL.std.450 Fma (and an FAbs for the neg_hi modifier). Both fail against the pre-fix tree with "unsupported vector opcode Vop3pRaw20/21". |
||
|
|
20eda4443c |
Shader: test a wave mask consumed as a per-lane predicate at the lane bit (#465)
* Shader: read a wave mask consumed as a per-lane predicate at the lane bit A VCC/EXEC wave mask consumed as a per-lane predicate (the VCndmask condition, a VCC/EXEC branch, or the derived _vcc/_exec bool) was tested in single-lane emulation with a whole-word non-zero test (IsNotZero64) instead of the current lane's bit. That is correct for comparison results (only the lane's own bit is ever set) but wrong for bitwise-complement wave-mask idioms (S_NOT / S_ORN2 / S_ANDN2 / S_NAND / S_NOR), which set the unused upper 63 bits: a whole-word test then reports the lane active even when its bit is clear. Unity's PostProcessing NaN killer does exactly this: per channel it computes isNaN = NLT AND NGT AND NEQ (against 0), then combines the channels as anyNaN OR NOT(v3-is-finite) via S_ORN2_B64. The complement set the upper mask bits, so every valid pixel read as NaN and was replaced with 0, zeroing the whole HDR scene before Bloom/Uber/tonemap. The 3D scene therefore rendered black behind the menu while the UI survived. Extract the current lane's bit in both single-lane and subgroup modes so IsWaveMaskActive matches the hardware. Fixes Superliminal (PPSA06084) black 3D scene: the storage room now renders behind the menu with natural exposure and no forced values. (cherry picked from commit 7af6f4b6f314fe302619c0d44f4db00971c5bf24) * test: wave-mask predicate is tested at the current lane bit Regression test for the wave-mask lane-bit fix. Compiles a shader that writes VCC at run time (V_CMP_EQ_F32) and asserts the emitted SPIR-V tests the wave mask at the current lane's bit (mask & lane_bit) rather than with a whole-word non-zero test. Fails against the previous IsNotZero64(mask) path, which zeroed complement wave-mask idioms (S_ORN2/S_NOT, e.g. Unity's NaN killer) across every lane. |
||
|
|
327018e80a |
Encode linear-float flips to sRGB at present (#448)
PS5 float VideoOut buffers (A16B16G16R16F flips) hold linear scRGB light where 1.0 is SDR white; hardware scan-out applies the display transfer function. vkCmdBlitImage converts numerically only, so raw-blitting a linear-float guest frame into a UNORM swapchain crushes dim scenes to near-black. Blit float flip sources through a cached swapchain-sized sRGB intermediate (the sRGB store performs the linear->sRGB encode), then raw vkCmdCopyImage the encoded bytes into the same-compatibility-class UNORM swapchain image. Swapchains that are already sRGB keep the direct blit (their store encodes), and swapchain formats without an sRGB counterpart keep today's raw blit unchanged. |
||
|
|
04557fd250 |
Refresh CPU-rewritten guest textures by write generation (#447)
* Track guest CPU write generations * Refresh CPU-rewritten guest textures by write generation |
||
|
|
24b82a7f1c |
ci: rebuild the website when a release is published (#312)
* ci: rebuild the website when a release is published * ci: add REUSE license header to notify-site workflow |
||
|
|
f544146d6d | AGC: reduce inactive trace and draw allocations (#308) | ||
|
|
dbd74654c4 | Vulkan: trim disabled draw diagnostics overhead (#307) | ||
|
|
28485b60e2 | CPU: avoid continuation emitter closure allocations (#306) | ||
|
|
8f405caebe | Vulkan: reduce guest buffer upload allocations (#301) | ||
|
|
dabf723b3e | [Vulkan] Honor guest depth clear state (#290) | ||
|
|
2db1fae282 | [Tests/HLE] Cover APR resolve, stat, and streaming flow (#272) | ||
|
|
33f96252da | [CPU] Reject context transfers to unmapped guest addresses (#273) | ||
|
|
f7981a7ed7 | [Tests/CPU] Verify import trampoline volatile-state ABI (#274) | ||
|
|
e10efa3ae1 | [HLE] Make guest printf formatting locale-invariant (#271) | ||
|
|
fa2616d224 |
Linux and macOS support (#47)
* [macos/linux] Cross-platform host memory, TLS, and ABI layer for POSIX Introduces the foundation for running SharpEmu on macOS (osx-x64 under Rosetta 2) and Linux (linux-x64). The CPU backend executes guest x86-64 code natively, so these targets run the whole process as x86-64; this commit replaces the Windows-only host primitives with platform-dispatched equivalents so the guest boots and services HLE calls off Windows. Memory (HostMemory.cs, new): a Win32-semantics facade over mmap/mprotect/munmap with a shadow region table answering VirtualQuery. PhysicalVirtualMemory, DirectExecutionBackend, StubManager, and the two Kernel*CompatExports now go through it instead of kernel32 P/Invokes. Exact-address requests use MAP_FIXED_NOREPLACE (Linux) / guarded MAP_FIXED (macOS) so they match Win32 "map there or fail" semantics. TLS + host helpers (PosixHostStubs.cs, new): pthread-backed TLS and Win64-ABI-compatible stubs for the kernel32 helpers the backend embeds into emitted x86-64 code (TlsGetValue, QueryPerformanceCounter, SwitchToThread, Sleep). A Win64->SysV thunk wraps managed callbacks, since .NET on POSIX compiles them for the SysV ABI while the emitted call sites use Win64. Guest address layout: the 0x7FFx window is Windows-only (dyld shared cache / Rosetta runtime live there on POSIX), so stack/TLS/stub regions relocate to 0x6FFx off Windows. Vectored exception handling is gated off on POSIX for now (guest faults are not yet recovered) — the signal-based bridge is the next step. Also adds osx-x64 to the RID list and a Docker-based Linux smoke-test script. Status: on both macOS (Rosetta) and Linux (amd64), the guest now boots, runs native x86-64 code, and dispatches HLE imports. macOS stops at a Rosetta translation-cache issue; Linux runs ~252 imports through C++ static-init before hitting the missing fault handler (SIGSEGV). * [posix] Bridge the vectored exception handler to sigaction(SIGSEGV/SIGBUS/SIGILL) Guest faults on macOS/Linux previously terminated the process because the recovery logic in DirectExecutionBackend.Exceptions.cs was Windows-only. This adds a POSIX front-end that reuses the existing handler bodies: - DirectExecutionBackend.PosixSignals.cs installs SA_SIGINFO handlers via an [UnmanagedCallersOnly] entry, rebuilds the Win64 EXCEPTION_POINTERS / CONTEXT view from the platform mcontext (Darwin __ss thread state via the mcontext pointer at ucontext+48, Linux glibc gregs at ucontext+40 -- offsets verified against the headers on both platforms), runs the same chain as the VEH path (TryRecoverUnresolvedSentinel trap-sentinel recovery, TryHandleLazyCommittedPage demand paging, VectoredHandler diagnostics incl. FS/GS TLS-fault detection), and writes register changes back into the mcontext so sigreturn resumes the repaired guest. Unrecovered faults chain to the previously installed handler so the .NET runtime keeps mapping its own faults to managed exceptions. - The whole recovery path is warmed up with fabricated inputs before the handlers are installed. This is required under Rosetta 2: the signal trampoline cannot enter x86 code that has never been executed (and so never translated) -- a cold handler is silently never invoked and the faulting instruction retries forever (reproduced and verified in an isolated .NET test under Rosetta for Linux). It also keeps first-fault JIT work out of the signal frame. - Handlers run without SA_ONSTACK: the runtime's alternate stacks are too small for the diagnostic path, while guest (2MB) and host thread stacks match where Windows dispatches exceptions anyway. - The raw reads in the shared fault diagnostics (stack qwords, RBP walk, code bytes at RIP) now probe the region table on POSIX before touching memory, since a nested SIGSEGV inside the handler would kill the process before diagnostics finish. Windows keeps its try/catch reads. - Escape hatches: SHARPEMU_DISABLE_POSIX_SIGNALS=1 skips installation, SHARPEMU_DISABLE_RAW_HANDLER=1 disables sentinel recovery (parity with Windows), SHARPEMU_LOG_POSIX_SIGNALS=1 traces every delivery (first 16 and every 1024th are always traced). Verified with the test game: Linux (amd64 container) previously died with SIGSEGV right after import #252; it now recovers/diagnoses signals and the run proceeds to the real next blocker, an unpatched negative-offset guest TLS read (fault at TLS base - 0x1708), which gets the full NATIVE EXCEPTION dump before terminating. macOS is unchanged: the bridge installs and the game still stops at the known Rosetta translation-cache error at import 12, which is the next work item. * [posix] Fix guest memory layout faults: TLS prefix, exact mmap, map search base Three fixes that take the test game from dying during libc init to running its full main loop on macOS and Linux: - Static TLS blocks live below the TCB (FreeBSD amd64 variant II) and libc.prx reaches past -0x1700, but only a 4KB prefix was mapped below the TLS base. The prefix is now 64KB on POSIX (Windows keeps 4KB); the fault was a read at TLS base - 0x1708 during libc init. - HostMemory exact allocation on macOS used MAP_FIXED, which silently maps over untracked host memory. The direct-memory allocator's address scan walked into the .NET runtime's JIT heap and replaced live code, which under Rosetta 2 surfaced as "no code fragment associated with the given arm pc". Exact placement now passes the address as a hint and fails on relocation, like MAP_FIXED_NOREPLACE does on Linux. - sceKernelMapDirectMemory/MapFlexibleMemory searched for free space starting at 4GB, which is the Mach-O image base on macOS. The default search base is 0x20_0000_0000 on POSIX, and TryAllocateAtOrAbove now asks the kernel for a placement instead of page-stepping through host- owned address space (Rosetta ignores mmap hints for whole VA windows), over-allocating when the caller needs more than page alignment. Windows behavior is unchanged; all divergences are platform-guarded. * [macos] Video presenter on the main thread, MoltenVK support, window keyboard input Gets the test game from a headless loop to a playable window on macOS: - AppKit traps with SIGILL ("NSUpdateCycleInitialize() is called off the main thread") when GLFW runs on a worker thread. The CLI now moves emulation onto a worker thread on macOS and parks the real main thread in HostMainThread.Pump(); the presenter posts its whole window loop there instead of spawning a thread, and a shutdown handler asks the render loop to close the window so the pump unwinds on guest exit. - MoltenVK: enable VK_KHR_portability_enumeration (+ the portability instance flag) and VK_KHR_portability_subset when advertised, and gate robustBufferAccess2 on the device actually supporting it (Metal does not; the old code keyed it off robustImageAccess2 and vkCreateDevice failed with ErrorFeatureNotPresent). - Input: pad exports polled user32 GetAsyncKeyState, so POSIX hosts threw DllNotFoundException per scePadReadState call. The presenter now attaches the window's keyboard via Silk.NET.Input into HostWindowInput, and the pad exports map the existing VK-code layout onto it off Windows. Headless hosts (Linux containers) report a disconnected keyboard and fall back to neutral pad data silently. GLFW needs an x86-64 Vulkan loader under Rosetta: place a universal libMoltenVK.dylib next to SharpEmu named libvulkan.1.dylib (Homebrew's arm64-only copy cannot load into the x86-64 process) and export DYLD_LIBRARY_PATH to that directory. Verified: Dreaming Sarah boots to a MoltenVK-backed 2560x1440 window on macOS (Apple M4, Rosetta 2), renders the intro, title, and menus, and keyboard input drives it into gameplay. Linux (amd64 container) runs the same build headless through millions of imports with no faults. Windows paths unchanged; arm64 and x64 builds clean. * [posix] CoreAudio playback, self-contained MoltenVK loading, input/log polish - Audio: sceAudioOut ports now play through an AudioQueue backend on macOS (stereo PCM16 with the same 32KB backpressure pacing as the WinMM path). The WinMM port and the new CoreAudio port share an IHostAudioPort interface and sample converter; hosts without a backend (Linux containers) keep the silent fallback. - MoltenVK: GLFW resolves Vulkan with dlopen("libvulkan.1.dylib"), which cannot see the app-local universal MoltenVK build, so the presenter now feeds vkGetInstanceProcAddr straight into glfwInitVulkanLoader (GLFW 3.4) before creating the window. No DYLD_LIBRARY_PATH needed; the CLI also preloads the dylib for Silk.NET and prints setup hints when it is missing. scripts/fetch-macos-moltenvk.sh stages the official universal dylib next to a build. - The virtual-range allocator's failure trace now names the address and length instead of "AllocateAt invocation threw". Investigated and documented (not port defects): the savedata transaction failure is identical on Linux and macOS (HLE argument-register mapping for sceSaveDataCreateTransactionResource), and the in-game tile speckling has no platform-specific code in its path - the one macOS-only delta is that MoltenVK lacks robustBufferAccess2, so out-of-bounds shader reads return garbage instead of zeros. Verified on macOS: window, audio backend, and keyboard input all come up with zero environment configuration; the game runs to gameplay. Linux headless run unchanged (silent audio, no faults). Windows paths untouched; arm64 and x64 builds clean. * [cpu] Preserve guest registers and flags across patched TLS accesses The TLS patch handler replaces guest `mov reg, fs:[...]` instructions, which preserve every other register and the flags - but the handler loaded the TLS index into ecx and called TlsGetValue (Win64: clobbers rcx/rdx/r8-r11) with `sub/add rsp` trashing the arithmetic flags. Guest code that keeps live values or comparison results across a TLS access computed garbage deterministically. The handler now saves rcx, rdx, r8-r11, and the flags around the call, keeping the same inner stack alignment. This applies to the load patches and both store-helper stubs, on every platform. Also in this change, from the rendering-artifact investigation: - The present blit picks linear filtering for any fractional scale (nearest only for integer upscales): a 3840x2160 guest frame blitted into a 2560x1440 swapchain with nearest silently dropped every third row/column. - ClampViewport no longer trims the guest viewport rectangle to the render target; trimming changed the guest's scale/offset and skewed texel addressing. Vulkan permits viewports beyond the framebuffer (the scissor confines rendering), so only spec bounds are enforced. - Env-gated diagnostics grown during the investigation: guest texture dumps (SHARPEMU_TEXTURE_DUMP_DIR), aliased guest-image readback dumps (SHARPEMU_TRACE_GUEST_IMAGES=alias), small-render-target write movies (SHARPEMU_TRACE_GUEST_WRITES=small), unattended input injection (SHARPEMU_AUTO_CROSS=secs,...), viewport nudging (SHARPEMU_VIEWPORT_EPSILON), chunked-draw toggle (SHARPEMU_DISABLE_CHUNKED_DRAWS), and rect-list/draw vertex traces. Known remaining issue (root cause narrowed, not yet fixed): the game's terrain texture pages are corrupted in guest memory before any GPU work - the mound's solid-fill 32x32 tiles decode to fully transparent texels and the grass page has deterministic gaps, byte-identical across runs. Ruled out: memcpy/memmove/memset/realloc HLE semantics, sampler wrap modes, texel-boundary rounding, chunked draws, viewport handling. Next step is auditing the Chowdren asset decode path (custom compressed images) against the emulator's import surface. * [linux] ALSA playback backend for sceAudioOut sceAudioOut ports on Linux now play through libasound instead of the silent fallback. The PCM device opens in blocking mode with ~170ms of device buffer (the time-equivalent of the 32KB queue the WinMM and CoreAudio ports keep), so snd_pcm_writei provides the same backpressure pacing without a managed queue. Underruns and suspend/resume go through snd_pcm_recover with one retry per submit; anything else drops the buffer rather than stalling the guest. The "default" device routes through PulseAudio/PipeWire on desktops and straight to hardware on bare ALSA; SHARPEMU_ALSA_DEVICE overrides it (the null device makes the path testable in containers). A missing libasound or device fails port creation and lands in the existing silent fallback. Verified in an amd64 container: the test game opens the port (backend=alsa, 48kHz stereo float32) and streams sceAudioOutOutput through the null device for a full run; without a usable device the port logs a warning and falls back to silent. Playback on real Linux audio hardware has not been tested. * [fixes] Address review feedback: commit bounds, CoreAudio shutdown, dump errors - HostMemory: a MEM_COMMIT that runs past its reservation now fails like Win32 instead of committing a prefix and reporting success. All current callers already clamp their ranges to the region, so this only guards future callers. - CoreAudioPort: Dispose wakes a submitter waiting on backpressure and the wait treats ObjectDisposedException as a timed-out wait, so closing a port during playback can no longer throw. A failed AudioQueueStart tears the queue down and fails fast instead of leaving an undrainable queue that stalls every later submit on its timeout. - AgcExports: texture dumping catches all write failures (bad path, permissions), logging a warning instead of crashing when SHARPEMU_TEXTURE_DUMP_DIR points somewhere unusable. Verified with the Linux container run: game boots and streams audio with the stricter commit check, and a dump dir under /proc produces warnings instead of taking the process down. * [ci] Build linux-x64 and osx-x64 archives Adds a build-posix matrix job (ubuntu-latest / macos-latest) mirroring the Windows build: locked restore, Release build, self-contained CLI publish, and a tar.gz artifact per RID (tar keeps the executable bit). The macOS archive also stages the universal MoltenVK dylib via scripts/fetch-macos-moltenvk.sh so the artifact runs without any manual Vulkan setup. The release job still only ships the Windows archive. * [cli] Keep POSIX glfw natives outside the single-file bundle The KeepGlfwOutsideSingleFile target only matched filenames starting with 'glfw', which covers Windows (glfw3.dll) but not libglfw.3.dylib / libglfw.so.3. Those got embedded into the single-file bundle, and Silk.NET's library loader does not probe the bundle extraction directory, so a published build died with "Couldn't find a suitable window platform" (and the glfwInitVulkanLoader wiring, which loads the library from AppContext.BaseDirectory, could not run either). Keeping the POSIX names loose next to the executable fixes both, the same way the Windows build already handled it. Found by running the CI-built osx-x64 archive: video failed while local loose-file builds worked. With the fix the published single-file build opens the MoltenVK window, wires the loader, and reaches gameplay. * [ci] Publish linux-x64 and osx-x64 release archives The build-posix artifacts now ship as per-RID GitHub releases on main pushes and manual dispatches, tagged the same way as the win64 ones (<rid>-<ref>-<sha>). Archives stay tar.gz so the executable bit survives extraction. * [cli] Fail early on non-x86-64 host processes The CPU backend executes guest x86-64 code natively, so the process must be x86-64 (win-x64/linux-x64 on x64 hardware, osx-x64 under Rosetta 2 on Apple Silicon). An arm64 process previously failed deep inside emulation startup, indistinguishable from MoltenVK, signal handler, or guest memory problems. CLI mode now checks the process architecture up front and exits with a message naming the supported execution model (and the Rosetta install command on macOS). The GUI-only path stays usable on arm64. * [video] Log the selected Vulkan device name and API version The presenter never named the GPU it picked, so a 'no video' report could not be told apart from a real windowing failure without guessing. It now logs the device name, type, and API version right after selection. A software rasterizer (llvmpipe/lavapipe/SwiftShader) shows up here and typically lacks the device features the translated shaders need, which is the likely cause when a window opens and presents frames but nothing draws. * [video] Steer GLFW to XWayland on Wayland sessions GLFW's native Wayland backend does not reliably map the Vulkan window with some drivers (NVIDIA in particular): frames present but the window never becomes visible, so the game runs with audio and no picture. A report on an RTX 5080 showed exactly this — all device features present, frames presenting, but the log had 'libdecor-gtk.so failed to init' and a 1.4x-scaled window, both Wayland tells. On a Wayland session that also exposes an X server (DISPLAY set), the presenter now clears WAYLAND_DISPLAY for its own process before GLFW initializes, so GLFW selects its dependable X11/XWayland backend. SHARPEMU_ENABLE_WAYLAND=1 opts back into native Wayland. Headless (no DISPLAY) and non-Linux hosts are unaffected. * [video] Force GLFW X11 backend via the platform init hint, log the platform The previous attempt cleared WAYLAND_DISPLAY to steer GLFW off Wayland, but a reporter still hit the native-Wayland path (the Wayland-only libdecor error persisted), so that env trick doesn't switch GLFW. Use GLFW's supported mechanism instead: glfwInitHint(GLFW_PLATFORM, GLFW_PLATFORM_X11) before GLFW initializes, called into the same libglfw GLFW itself loads (the pattern InitializeMacVulkanLoader already uses). Still gated on a Wayland session with an X server present (DISPLAY set) so we never force X11 where XWayland can't catch it, and still overridable with SHARPEMU_ENABLE_WAYLAND=1. Also logs 'GLFW windowing platform in use: <backend>' after init via glfwGetPlatform, so a 'no window' report shows X11 vs Wayland outright. Verified on macOS: the readback correctly reports Cocoa and the presenter is unaffected (the fix is a no-op off Linux). * [video] Run the GLFW window on the main thread on Linux too GLFW requires window creation and event processing on the process main thread on every platform: initialization, window creation, and glfwPollEvents are main-thread-only, and X11 in particular has a single event queue that must be serviced there. A window created and polled on another thread may never map — which is why the game ran (audio, imports, even Vulkan present) with no visible window on Linux. macOS already routed the window loop to the main-thread pump (AppKit needs it); Windows is fine because it has a per-thread event queue. Linux was the gap: it spawned a background thread for the presenter. Extend the existing HostMainThread pattern to Linux — emulation runs on a worker, the main thread pumps the window work the presenter posts. Refs GLFW intro guide (thread-safety): init, window creation, and event processing are restricted to the main thread. Verified: macOS still boots to its window unchanged; the Linux headless container runs to millions of imports with no deadlock or regression. On-screen confirmation on a real Linux desktop is still pending, but this is the documented root cause for a windowless-but-running Linux session. * [posix] Skip Win32 native guest workers * [vulkan] Synchronize offscreen targets before present * [vulkan] Transition fresh textures from undefined layout * [vulkan] Report swapchain pixels before source readback * [vulkan] Emit requested guest image diagnostics * [agc] Diagnose guest texture fallbacks * [linux] Keep guest GPU mappings in low address space * [video] Reduce diagnostic stalls and drain complete frames * [memory] Harden packed GPU address handling * [readme] Document Linux and macOS release support * [posix] Integrate the host platform abstraction * [posix] Restore guest thread address window * [video] Run the performance HUD on POSIX hosts The FPS/CPU/work HUD bailed out unless the host was Windows; only the per-thread hottest-thread scan actually needs Windows APIs. Keep that scan Windows-only (POSIX reports 'idle') and let the rest of the HUD run everywhere — the title is already set from the render thread, which owns the window on macOS and Linux. * [posix] Implement native guest worker threads Guest entry stubs must not run above CLR-managed frames on CLR-created threads (see the NativeWorker preamble); the PR previously fell back to the inline calli path on POSIX, which reproduced the documented 'attempted to call a UnmanagedCallersOnly method from managed code' fail-fast (observed after Dreaming Sarah's menu select) and left the runtime's suspension machinery walking guest frames. Provide the missing POSIX half of the worker loop: - PosixHostStubs grows Win64-convention WaitForSingleObject/SetEvent/ ExitThread stubs backed by dispatch semaphores (macOS) / unnamed POSIX semaphores (Linux) plus pthread_exit, with EINTR retry in the wait. - Worker events are creatable/signalable/waitable from managed code too, so NativeGuestExecutor.Run keeps its handshake (AutoResetEvent stays on Windows byte-for-byte). - PosixHostThreading implements CreateNativeThread/WaitForThreadExit/ CloseThreadHandle over pthreads (liveness probed with pthread_kill(0), then joined). - RunPrologue/RunEpilogue are routed through the existing Win64->SysV thunks, so the emitted loop stays identical across platforms. * [macos] Disable concurrent GC under Rosetta's write-watch hazard Background GC's write-watch revisit (SoftwareWriteWatch::GetDirty -> FlushProcessWriteBuffers) calls thread_get_register_pointer_values on every thread; under Rosetta 2 that Mach call stalls indefinitely on threads executing translated guest code. The background mark phase then never finishes and every allocating or Monitor-taking thread wedges behind it — observed as Dreaming Sarah freezing at the menu/loading screen with FPS 0 in 5 of 7 runs, dispatcher/watchdog parked in Monitor.Enter and all BGC threads waiting in t_join. Non-concurrent GC never takes that path; a 5-minute soak now holds 22-31 fps in-game with zero stalls. Windows and Linux keep concurrent GC. * [diag] Periodic guest-thread snapshots with gate-owner tracking SHARPEMU_PERIODIC_SNAPSHOT_SECONDS=N dumps the stall snapshot every N seconds even while imports are progressing, for soft stalls where the game stops advancing but threads keep spinning. The periodic dump never touches the guest-thread gate (it must keep reporting when the gate is what's wedged): it reads a lock-free owner record — every gate acquisition now goes through LockGate(site), which notes site/thread — and walks the thread table without the lock, tolerating torn reads. SHARPEMU_PERIODIC_SNAPSHOT_FILE redirects the dump to a side file for the case where the console itself is wedged (frozen log mirror was one of the observed failure modes). * [nuget] Add osx-x64 RID targets to lock files * [cpu] Back off the guest join poll TryJoinThread polled the host thread at a fixed 1ms; a game main thread joining a long-lived worker (Dreaming Sarah parks there for the whole session) burned ~5% of managed CPU in Join/Sleep syscalls. Ramp the poll interval to 10ms once the join is clearly long-lived — exit detection latency for long joins moves from ~1ms to at most 10ms, and short-lived joins still resolve on the first 1ms polls. * [nuget] Add linux-x64/win-x64 RID targets to lock files * [posix] Keep guest stacks clear of the import-stub descent The import-stub region descends from 0x7000_0000_0000 on the same 16MB grid as the guest thread windows; moving stacks to 0x6FFF_E000_0000 put them inside the stub region's 64-module descent range (floor 0x6FFF_C000_0000), silently consuming the top ~32 stack slots on hosts with many loaded modules. Drop the POSIX stack base to 0x6FFF_A000_0000: 512MB below the stub floor, still 2.5GB above the TLS window. Windows keeps 0x7FFF_E000_0000 (its bands are ~15TB apart). * [pad] Read window gamepads on POSIX hosts XInput and the DualSense hid reader are Windows-only, which left macOS/Linux with keyboard input only. The presenter's Silk/GLFW input context already enumerates gamepads on both platforms, so track their state in HostWindowInput (event-driven on the window thread, snapshot guarded like the key set) translated to ORBIS conventions: GLFW's Xbox layout maps A/B/X/Y to Cross/Circle/Square/Triangle, sticks bias from -1..1 to 0..255 with Y growing down, and triggers rescale from GLFW's -1..1 resting-at--1 range with digital L2/R2 bits past 25%. The merge into ReadHostInputState is gated to non-Windows so a physical pad is never sampled twice through both a native reader and GLFW. Hotplug is handled via ConnectionChanged; with no pad connected the path is inert. Untested against a physical controller (none attached to the dev host); axis conventions follow the GLFW gamepad-mapping contract. * [nuget] Refresh lock files after cross-RID restores * [posix] Adopt the host audio/input seams from main Main's #192 abstracted audio output and pad/keyboard input behind IHostAudioOutput/IHostInput; re-express the POSIX backends behind them: - CoreAudioPort/AlsaAudioPort move to Host/Posix as PosixCoreAudioStream/PosixAlsaAudioStream implementing IHostAudioStream. The seam converts to stereo PCM16 before Submit, so the ports' own conversion (and IHostAudioPort/AudioSampleConverter) is gone; queueing and backpressure are unchanged. - PosixHostAudio selects CoreAudio (macOS) / ALSA (Linux) as the platform's IHostAudioOutput. - PosixHostInput implements IHostInput over an IPosixWindowInputSource that HostWindowInput registers when the presenter attaches the window's GLFW input context: keyboard with virtual-key translation, the window gamepad snapshot (now in seam HostGamepadState/HostGamepadButtons terms), and keyboard-connected as the focus signal. Rumble/lightbar no-op (GLFW has no such API). - PadExports drops its direct HostWindowInput gamepad merge — pads now flow through IHostInput.GetGamepadStates like every platform. - PosixHostThreading.RequestTimerResolution is a documented no-op. All three RIDs build; SharpEmu.Libs.Tests pass (26/26). * [nuget] Regenerate GUI lock file for RID-less locked restore Local cross-RID builds stamped a win-x64 runtimes section into SharpEmu.GUI's lock file; the project declares no RuntimeIdentifiers, so CI's 'dotnet restore --locked-mode' failed with NU1004 on every platform. Regenerated via a plain solution restore (--force-evaluate), matching what the workflow validates. |
||
|
|
6e2878f2ff | Add commit hash to video window title (#93) | ||
|
|
8c1507777c |
[agc] Reset transparent Chowdren effect-layer fills (#83)
Treat the exact untextured transparent-black premultiplied fill used by Chowdren as an overwrite. This prevents Dreaming Sarah fog and vignette render targets from accumulating across frames; SHARPEMU_DISABLE_TRANSPARENT_FILL_CLEAR=1 restores prior behavior. |