5 Commits

Author	SHA1	Message	Date
Miguel Cruz	864cbb0fa0	[AGC/Vulkan] Extend PS5 runtime and rendering compatibility (#216) ... * [Core] Add POSIX native execution and PS5 SELF support Extend the native backend, guest TLS, fixed-address memory, and loader paths needed by PS5 titles on Windows, Linux, and macOS. Keep workstation GC so high-core-count hosts do not reserve over fixed guest image bases. * [HLE] Expand PS5 service and media compatibility Add the kernel, threading, save-data, networking, audio, video-codec, font, dialog, and service exports required by newer PS5 software. Preserve every SysAbi NID currently registered by main while adding the compatibility surface used by ASTRO BOT. * [AGC/Vulkan] Extend Gen5 shader and presentation support Expand PM4 handling, Gen5 shader translation, MRT and packed export support, guest image tracking, depth initialization, texture aliasing, and Vulkan presentation. Add the performance overlay and address-filtered diagnostics used to validate ASTRO BOT with original shaders. * [Core] Align static TLS reservation across hosts * [Pad] Align primary user ID with UserService * [Gpu] Preserve runtime scalar buffers across renderer seam * [AGC] Restore omitted command helper exports * [Vulkan] Reuse primary views for promoted MRT targets * [Vulkan] Preserve scratch storage bindings in compute dispatches	2026-07-16 02:02:34 +03:00
kostyaff	b5930465f2	[AGC] Correct VReadlaneB32 decode and emission (#237) ... VReadlaneB32 (VOP3 0x360) had two bugs: 1. Decode: VOP3 decode always set destinations = Vector(word & 0xFF). For VReadlaneB32, bits 0-7 are unused — the scalar destination is in bits 8-14. Now decodes as Scalar((word >> 8) & 0x7F). 2. Emission: VReadlaneB32 was in the VMovB32 fall-through group, just returning GetRawSource(instruction, 0) — reading the current lane's value. By ISA, sdst = vsrc0[lane(src1)], which requires reading a different lane's value. Now uses SPIR-V GroupNonUniformBroadcast(scope=Subgroup, value=src0, lane=src1) when subgroup operations are available, with a fallback to the current-lane simplification when not. 3. Routing: TryEmitVectorAlu called TryGetVectorDestination first, which checks for VectorRegister kind. With the scalar destination fix, VReadlaneB32 now routes to a new TryEmitReadlane handler before the vector destination check. 4. Subgroup capability: UsesSubgroupShuffle now includes VReadlaneB32 so GroupNonUniform capability is enabled when needed. Verified: dotnet build 0 errors/0 warnings, 26/26 tests pass, ShaderDump all programs behaved as expected.	2026-07-16 00:27:26 +03:00
kostyaff	92497689ab	[AGC] Fix VOP3 decode for V_READLANE_B32 and V_WRITELANE_B32 (#232) ... PR #200 moved shader files from SharpEmu.Libs/Agc/ to new projects SharpEmu.ShaderCompiler and SharpEmu.ShaderCompiler.Vulkan. This re-ports the VOP3 decode fix from PR #226 to the new file paths. Decode table (Gen5ShaderTranslator.cs): - 0x360: VMadU32U16 -> VReadlaneB32 (per RDNA2 ISA) - 0x361: VMulLoU32 -> VWritelaneB32 (0x361 was a duplicate of 0x169) - 0x373: added VMadU32U16 at its correct opcode Emission (Gen5SpirvTranslator.Alu.cs): - VWritelaneB32: per-lane conditional write via IEqual+Select, stores with guardWithExec:false (writelane bypasses exec mask) - VReadlaneB32: kept as GetRawSource(instruction, 0) simplification (correct emission with src1 lane select is a follow-up) Verified: dotnet build 0 errors/0 warnings, 26/26 tests pass, ShaderDump all programs behaved as expected.	2026-07-16 00:25:59 +03:00
Gutemberg Ribeiro	320dbcacba	[SourceGenerators] Compile-time SysAbi export registry, analyzers, and build-generated aerolib.bin (#204) ... * [SourceGenerators] Add the SysAbi export generator and analyzers (phase 0) New SharpEmu.SourceGenerators Roslyn component, complete and tested but consumed by nothing yet — the emulator projects adopt it in the following commits. Ps5Nid ports the PS NID derivation (base64 of the byte-reversed first eight SHA1 bytes of name + fixed suffix) from scripts/generate_aerolib_binary.py to C#, so what has always been a manual, out-of-band computation becomes a compile-time capability. SysAbiExportGenerator emits a per-assembly SysAbiExportRegistry whose CreateExports(Generation) reproduces ModuleManager's reflection scan exactly — same generation inheritance and filtering, same method-name fallback, same libKernel default — with attribute-omitted NIDs derived algorithmically (equivalent to the runtime catalog lookup, which is built from the same computation). Parameterless handlers are adapted to the SysAbiFunction shape; invalid declarations are skipped here because the analyzer rejects them as build errors, so nothing drops silently. SysAbiExportAnalyzer turns the runtime failure modes into diagnostics: SHEM001 duplicate NID (across declared and derived forms), SHEM002 malformed NID, SHEM003 uncallable handler signature, SHEM004 NID contradicting its export name (the class of drift previously fixed by hand), SHEM005 unresolvable export, SHEM006 export name unknown to ps5_names.txt when the catalog is wired as an AdditionalFile, SHEM007 handler not reachable by generated code. The self-contained test suite drives both in-process against the real SharpEmu.HLE metadata: known catalog NID pairs pin the algorithm, the generated registry must itself compile, and each diagnostic has a triggering fixture. Fittingly, the NID pinning test caught a wrong pair in its own first draft — the exact mistake SHEM004 exists to stop. * [SourceGenerators] Adopt the generated export registry in the emulator (phase 1) SharpEmu.Libs consumes the generator and analyzers, with scripts/ps5_names.txt wired as the AdditionalFile catalog. The runtime now registers exports from the compile-time SysAbiExportRegistry instead of the boot-time reflection scan; RegisterFromAssembly is retained solely as the arbiter for a parity test that pins the two tables identical — same NIDs, names, libraries, targets, and handler methods — across Gen4, Gen5, and combined registration. First contact between the analyzer and all 715 existing exports surfaced real drift the old offline checker structurally missed (scripts/check_sysabi_aerolib.py skipped any NID absent from aerolib.bin): three exports whose friendly names collide with real catalog symbols of different NIDs, now suppressed at-site with reasons pending AGC API confirmation, alongside the established synthetic Unknown* labels for uncatalogued NIDs, which prompted a rule refinement — SHEM004 only hard-errors when the export name is a real catalog symbol, since synthetic labels cannot be validated by hashing and the NID is authoritative for them. The two allowlisted mismatches in the python checker no longer trigger anything, and the checker is deleted: the analyzer subsumes it with the semantic model instead of regex, and validates every declared pair rather than only catalog-known NIDs. * [SourceGenerators] Generate aerolib.bin at build time from ps5_names.txt The runtime NID -> name catalog is derived data and no longer lives in the repository: a Framework-only MSBuild task (GenerateAerolibBinaryTask, sharing the same Ps5Nid implementation the analyzers use) builds it into the intermediate directory from scripts/ps5_names.txt — now the single source of truth — and SharpEmu.HLE embeds it from there. The output is byte-identical to the previously committed binary, verified with cmp against git history; a new test pins that the embedded catalog loads and resolves a known symbol both directions. scripts/generate_aerolib_binary.py is deleted (its algorithm lives in Ps5Nid, its invocation in the build); the REUSE annotation for the binary goes with it. MSBuild's Inputs/Outputs check means the ~154k NID hashes only recompute when the names file actually changes. The task implements ITask against Microsoft.Build.Framework directly, keeping the vulnerable-flagged Utilities.Core package out and the analyzer project's file-IO ban suppressed only inside the task itself. * [SourceGenerators] Emit typed-signature register thunks (phase 2) [SysAbiExport] handlers can now be written with real signatures — a CpuContext followed by up to six int/uint/long/ulong parameters — and the generator emits the SysV unmarshalling thunk, mapping parameters positionally to RDI/RSI/RDX/RCX/R8/R9 with the same unchecked-cast idiom hand-written handlers use. SHEM003 accepts the new shape and rejects register overflow and non-register-representable types. Both shapes coexist, so migration is per-handler; sceKernelPollSema, sceKernelSignalSema, and sceKernelCancelSema migrate as the demonstration (the last showing raw ulong guest-address passthrough). The reflection scan cannot represent typed handlers, so it retires here: RegisterFromAssembly, its signature validation, and ResolveExportInfo are deleted, and the parity test that pinned the generated registry to the scan is replaced by content-invariant tests (duplicate-free, full 715-export surface, catalog identity). Deleting the scan surfaced a phase-1 latent regression — the pre-JIT warm sweep enumerated only reflection-scanned assemblies, so the generated registration path warmed nothing and re-exposed the guest-thread fail-fast risk; the warm set is now derived from the registered handler delegates themselves. * [SourceGenerators] Marshal guest strings declaratively with [GuestCString] (phase 3) A string parameter on a typed [SysAbiExport] handler, annotated [GuestCString(maxLength)], now makes the generated thunk read the null-terminated UTF-8 string from the argument register's guest address before the handler runs, returning ORBIS_GEN2_ERROR_MEMORY_FAULT to the guest when the read fails — the exact prologue nearly every string-taking handler writes by hand. The attribute lives in SharpEmu.HLE next to SysAbiExportAttribute; SHEM008 rejects misuse (non-string parameter, non-positive MaxLength) while a bare string parameter stays a SHEM003 signature error. _open, open, and sceKernelOpen migrate as the demonstration; they were chosen because their hand-written prologue faulted on a null pointer the same way the thunk does (handlers that return INVALID_ARGUMENT for null pointers, like sceKernelCreateSema, keep the raw shape so guest- visible semantics stay untouched). * [SourceGenerators] Apply review findings across the branch Behavior: the open/_open/sceKernelOpen [GuestCString] demo migration is reverted — the local compat reader falls back to host memory for paths in loader-mapped regions that ctx.Memory cannot see, so the generated thunk would have turned recoverable reads into MEMORY_FAULT. The marshalling infrastructure stays, proven by generator/analyzer tests; production migration waits for a handler whose semantics the thunk reproduces exactly. A comment on the handler records why. Build robustness: the aerolib target is skipped for design-time builds (the IDE resolves project references without compiling them, so on a fresh clone the task assembly does not exist yet), and the task/names paths are centralized in properties. The generator now emits no registry for export-free assemblies, so referencing the analyzer can never mint a colliding SharpEmu.Generated type. Cleanup and perf: the pragma-suppression sites left mis-indented by the phase-1 relocation are reformatted and the restores moved after the method body; the dead ExportsForTesting hook and its InternalsVisibleTo are deleted; the aerolib task reuses one SHA1 instance across ~150k names; the analyzer caches the parsed catalog per file snapshot instead of re-parsing 150k lines every compilation start, shares the attribute name constant with the generator, and computes the catalog-membership check once. * [CI] Run the test suites in the build workflow The workflow compiled the test projects (they are in SharpEmu.slnx) but never executed them. A solution-level dotnet test now runs between build and publish, so any test failure fails the build — including the AerolibCatalogTests/SysAbiRegistryTests that guard the build-generated aerolib.bin and the generated export registry. Generation failures of aerolib.bin itself already fail the build step: the MSBuild task logs an error event and returns false, and a missing task assembly or missing embedded output are hard MSBuild errors. The NuGet cache key now also tracks the test projects' lock files. * [SourceGenerators] Address review feedback Multi-diagnostic analyzer tests no longer assume a stable diagnostic order (analyzer execution is concurrent), and the aerolib task logs the full exception instead of only its message so build failures keep the type and stack trace. * [SourceGenerators] Address second review round Symbol-name comparisons in the shape rules and analyzer now pin an explicit SymbolDisplayFormat.FullyQualifiedFormat instead of relying on the display-format default, and the aerolib task fails loudly on a symbol name that would overflow the format's ushort length prefix instead of silently truncating it, with null-safe output-directory handling made explicit. * [SourceGenerators] Embed aerolib.bin via a target so design-time builds never reference it The static EmbeddedResource item referenced the generated file even in design-time builds, where the generation target is skipped — on a fresh clone the IDE would try to embed a file that never existed. The item is now created inside an EmbedAerolibBinary target gated on DesignTimeBuild, separate from the generation target so an up-to-date skip of GenerateAerolibBinary cannot drop the item with the rest of its body, and hooked before AssignTargetPaths since dynamic resource items added later miss the resource pipeline. Verified fresh build, incremental rebuild (embedded catalog test both times), and a simulated design-time compile with no artifacts present. * [SourceGenerators] Regenerate test lock file after rebase onto main Rebase fallout: main's package graph shifted under #200, so the SourceGenerators.Tests lock file is re-evaluated to keep --locked-mode restore green at the branch tip. * [Build] Drop NuGet lock files; rely on central package management Central package management was already in effect (ManagePackageVersionsCentrally with all versions in Directory.Packages.props and no inline PackageReference versions), so the per-project packages.lock.json files and the lock-mode workflow only added maintenance overhead. This removes all eleven lock files, drops RestorePackagesWithLockFile so restore no longer regenerates them, and takes --locked-mode off the CI restore steps (re-keying the NuGet cache on the central props files). Package versions remain centrally pinned in Directory.Packages.props.	2026-07-16 00:00:32 +03:00
Gutemberg Ribeiro	30fdd8d6ed	[Gpu] Backend-neutral shader compiler and guest-GPU renderer seam (#200) ... * [ShaderCompiler] Extract the backend-neutral shader compiler project Move the Gen5 (gfx10) microcode decoder, the scalar evaluator, the shader IR, and the metadata reader out of SharpEmu.Libs/Agc into a new SharpEmu.ShaderCompiler project — the half of shader compilation every codegen backend (SPIR-V today; MSL and DXIL later) consumes. Types go public: they are the contract now. Nothing in the project may depend on a host graphics API; the SPIR-V-specific artifact types (Gen5SpirvShader, Gen5SpirvStage) stay beside the emitter in Libs. Three couplings surfaced by the move, each resolved at the right depth: GuestDrawKind was defined inside VulkanVideoPresenter despite being a guest-domain, decoder-produced concept — it moves to the shared project; the evaluator's one HLE dependency (the tracked-libc-heap read fallback) becomes an injectable hook that a Libs module initializer installs before any caller can reach the evaluator; and the inline- constant table is promoted to a shared Gen5InlineConstants so backends cannot drift on constant semantics (the SPIR-V translator now delegates to it). The ShaderDump tool drops its reflection over the moved types in favor of direct typed calls; only the SPIR-V emitter, still internal to Libs until it moves to its own backend project, is reached via reflection. Verified by a clean solution build, the existing test suite, and a full ShaderDump conformance run. * [ShaderCompiler] Move the SPIR-V emitter into SharpEmu.ShaderCompiler.Vulkan Gen5SpirvTranslator (with its ALU partial), SpirvModuleBuilder, SpirvFixedShaders, and the Gen5SpirvShader/Gen5SpirvStage artifact types move whole from SharpEmu.Libs/Agc into the first per-backend codegen project. Notably it needs no Vulkan bindings reference: emitters produce bytes from the shared IR; renderers own graphics APIs. Types go public as the backend's contract; AgcExports and the presenter consume them exactly as before. The ShaderDump tool drops its last reflection: with both halves of the pipeline public it drives decode and all three emit entry points with direct typed calls, retiring the PadWithDefaults invoke shim — and it no longer references SharpEmu.Libs at all, making the conformance tool emulator-independent by design. Verified by a clean solution build, the test suite, a full ShaderDump conformance run, and a locked-mode restore under the pinned SDK. * [Gpu] Extract the guest-GPU backend seam (IGuestGpuBackend) The AGC/VideoOut/SystemService export layers now reach the renderer through IGuestGpuBackend via GuestGpu.Current (mirroring HostPlatform), instead of calling VulkanVideoPresenter statics. The Vulkan backend is a thin adapter over the existing presenter, so the extraction stays mechanical; only the adapter and the presenter itself reference the presenter now. The types crossing the seam move to Gpu/GuestGpuTypes.cs and drop their Vulkan prefixes, which an audit showed were misnomers: every field is a neutral primitive or a raw guest value (guest addresses, format and number-type codes, CB_BLEND register bitfields, verbatim sampler descriptor dwords). The one genuine Vulkan value in the old surface — the Silk.NET Format inside VulkanRenderTargetFormat, which callers never read — stops crossing: TryDecodeRenderTargetFormat is replaced at the seam by TryGetRenderTargetOutputKind, which surfaces only the Gen5PixelOutputKind callers actually consume, keeping native formats a backend-internal concern. ToVulkanSampler in AgcExports is renamed ToGuestSampler to match what it always produced. Seam rules are documented on the interface: no host-API value crosses, and submission stays coarse-grained with synchronization internal to backends. Interim exception, resolved next: shader parameters are still SPIR-V blobs. * [Gpu] Move shader compilation behind the guest-GPU backend The seam's interim exception is gone: AgcExports no longer calls Gen5SpirvTranslator or handles SPIR-V bytes. IGuestGpuBackend gains the three TryCompile entry points, which take the backend-neutral (Gen5ShaderState, Gen5ShaderEvaluation) contract plus the flat per-role resource-slot bases a multi-stage draw needs, and return opaque IGuestCompiledShader handles that only the producing backend can submit — the Vulkan backend wraps its SPIR-V in VulkanCompiledGuestShader and rejects foreign handles loudly. Draw and dispatch submissions take handles instead of byte arrays; the shader caches in AgcExports store handles. IGuestCompiledShader.Payload exposes the backend-defined compiled bytes for exactly two callers: the diagnostics dump and the size trace — documented as never-interpret. The unused _pixelSpirvCache is deleted. With this, a Metal or DX12 backend plugs in by implementing IGuestGpuBackend with its own codegen; nothing in the export layers knows which shader format exists. Verified by a clean solution build, the test suite, and a full ShaderDump conformance run under the pinned SDK. * [Gpu] Fix rename collateral from the seam extraction Address review findings: a doc comment picked up the mechanical VulkanVideoPresenter -> GuestGpu.Current rewrite and ended up naming members that do not exist on the interface, and CreateVulkanIndexBuffer kept its Vulkan prefix while every sibling factory was de-Vulkanized — it produces the neutral GuestIndexBuffer, so it is CreateGuestIndexBuffer. * [Gpu] Label diagnostics dumps with the backend's payload extension Address the review's altitude finding on DumpSpirv: the dump helper's IR-disassembly half is backend-neutral and stays put, but writing the opaque payload to a hardcoded .spv interpreted bytes the seam says never to interpret. IGuestCompiledShader now declares its payload's file extension, and the renamed DumpCompiledShader takes the handle and writes honestly-labeled dumps whichever backend produced them. * [Gpu] Make the shader-cache hit path allocation-free and lock-free Every translated draw built its cache key with a LINQ Select feeding string.Join plus one interpolated string per render target — steady per-draw allocation whether or not the shaders were already cached. The output layout is now packed exactly into a ulong (guest slot in 6 bits + output kind in 2 bits per target, host locations being the byte positions, target count in the key beside it), and the Gen5PixelOutputBinding array is only materialized on a cache miss, where compilation dwarfs it. The graphics/compute shader caches switch from Dictionary guarded by _submitTraceGate to ConcurrentDictionary, making the per-draw and per-dispatch hit paths lock-free and decoupling them from the tracing gate they coincidentally shared. And the seam-shaped render-target list is built once when a translated draw is created instead of a Select/ToArray per submission of a cached draw. * [Gpu] Replace LINQ with explicit loops in code this branch introduced Project rule going forward: no LINQ — it allocates enumerators, closures, and delegates, and this codebase is GC-pause-sensitive. The pixel-output and guest-render-target array builds and the ShaderDump store-PC collection become plain loops; pre-existing LINQ elsewhere is left for changes that already touch those lines. * [ShaderCompiler] Suppress CA2255 on the evaluator hook installer The analyzer coverage that arrived with the rebase flags ModuleInitializer in library code; this is the rule's intended advanced scenario — the hook must be installed before any code path can reach the evaluator, and every such path enters through this assembly — so suppress with that justification rather than weaken the guarantee to a static constructor's lazier timing. * [Gpu] Resolve rebase artifacts onto main Dedupe the System.Collections.Concurrent using in AgcExports that the rebase merge duplicated (main and this branch each added it), and regenerate the lock files for the new shader-compiler projects and SharpEmu.Libs against main's current package graph so --locked-mode restore matches at the branch tip. * [CI] Comment per-platform build artifact links on PRs Adds a workflow_run workflow that, after "Build and Release" finishes a pull-request build, posts (and keeps updated in place) a single PR comment linking the Windows, Linux, and macOS artifacts from that run. It runs via workflow_run rather than in the build workflow because PRs from forks build with a read-only token that cannot comment; the follow-on run executes in the base-repo context with write access and without checking out fork code. GitHub only triggers workflow_run from the default branch, so this takes effect once merged to main.	2026-07-15 11:11:24 -06:00