memory: back the free pages of a partially-overlapping fixed mapping (#458)

A SCE_KERNEL_MAP_FIXED request whose window partially overlaps an
existing allocation was failing outright: AllocateAt reserves the whole
range in one all-or-nothing VirtualAlloc, which returns 0 on partial
overlap. The mapping call then returned NOT_FOUND while leaving the free
tail unmapped, so the guest faulted (0xC0000005) writing into it.

Add IGuestAddressSpace.TryBackFixedRange, which walks the range via the
host Query (VirtualQuery reports contiguous same-state runs) and fills
only the free sub-ranges, leaving already-backed pages untouched. This
matches the fixed-mapping contract on hardware. Route the fixed
reservation path through it via a new backPartialOverlap flag.

Co-authored-by: slick-daddy <slick-daddy@users.noreply.github.com>
This commit is contained in:
Slick Daddy
2026-07-20 09:08:29 +03:00
committed by GitHub
parent 184e24fbb6
commit 33be88bdf9
5 changed files with 215 additions and 2 deletions
@@ -4603,7 +4603,8 @@ public static partial class KernelMemoryCompatExports
allowSearch: false,
allowAllocateAtAlternative: false,
"reserve fixed range",
out _);
out _,
backPartialOverlap: true);
}
internal static bool IsGuestRangeBacked(CpuContext ctx, ulong address, ulong length)
@@ -18,7 +18,8 @@ internal static class KernelVirtualRangeAllocator
bool allowSearch,
bool allowAllocateAtAlternative,
string traceName,
out ulong mappedAddress)
out ulong mappedAddress,
bool backPartialOverlap = false)
{
mappedAddress = 0;
if (length == 0)
@@ -42,6 +43,18 @@ internal static class KernelVirtualRangeAllocator
return true;
}
// Fixed mappings must cover the whole requested window even when part of
// it is already backed by another allocation. The single-call AllocateAt
// below is all-or-nothing and fails outright on partial overlap, leaving
// the untouched pages unmapped for the guest to fault into. Fill the free
// pages directly instead.
if (backPartialOverlap &&
addressSpace.TryBackFixedRange(desiredAddress, length, executable))
{
mappedAddress = desiredAddress;
return true;
}
var allocated = addressSpace.AllocateAt(desiredAddress, length, executable, allowAllocateAtAlternative);
if (allocated == 0)
{