🎉 Celebrating 25 Years of GameDev.net! 🎉

Not many can claim 25 years on the Internet! Join us in celebrating this milestone. Learn more about our history, and thank you for being a part of our community!

Trouble with memory mirroring on win32

Started by
4 comments, last by Finalspace 3 years, 4 months ago

Hi there,

i am trying to port over a lock-free ringbuffer which should not require any wrapping code, using a technique called memory mirroring. The idea of this technique is simple: Allocate a block of memory twice the length you need. Release it directly and then map the first block to the first half and map the second block to the second half. This should in “theory” wrap automatically when you write past the first block.

Unfortunatly on win32 this wont work and i have no idea why. I got no error on creating the mirror thing, but when i write past the first block it simply crashes. This is the code i have so far:

typedef struct MirrorMemoryBuffer {
	void* basePtr;
	HANDLE fileHandle;
	uint32_t length;
} MirrorMemoryBuffer;

static bool InitMirrorMemory(MirrorMemoryBuffer* memory, const uint32_t desiredLength) {
	if(memory == NULL || desiredLength == 0) return(false);

	// Get length in multiple of page-sizes
	SYSTEM_INFO sysInfo;
	GetSystemInfo(&sysInfo);
	uint32_t roundedSize = (desiredLength + sysInfo.dwPageSize - 1) / sysInfo.dwPageSize * sysInfo.dwPageSize;

	uint8_t* blockAddress;
	HANDLE fileHandle;

	// Keep trying until we get our buffer, needed to handle race conditions
	int retries = 3;
	while(retries-- > 0) {
		// Create mapped file with the length of the buffer
		fileHandle = CreateFileMappingA(INVALID_HANDLE_VALUE, 0, PAGE_READWRITE, 0, roundedSize, NULL);
		if(fileHandle == INVALID_HANDLE_VALUE) {
			// Failed, we cannot continue
			break;
		}

		// Reserve two memory of twice the length of the buffer
		blockAddress = (uint8_t*)VirtualAlloc(NULL, roundedSize * 2, MEM_RESERVE, PAGE_NOACCESS);
		if(blockAddress == NULL) {
			// Failed, try again
			CloseHandle(fileHandle);
			continue;
		}

		// Release the full range immediately, but retain the address for the re-mapping
		VirtualFree(blockAddress, 0, MEM_RELEASE);

		// Re-map both buffers to both buffers (these may fail, when the OS already used our memory elsewhere)
		if((MapViewOfFileEx(fileHandle, FILE_MAP_ALL_ACCESS, 0, 0, roundedSize, blockAddress) == blockAddress) ||
		   (MapViewOfFileEx(fileHandle, FILE_MAP_ALL_ACCESS, 0, 0, roundedSize, blockAddress + roundedSize)) == blockAddress + roundedSize) {
			 // Success, we can use the blockAddress as our base-ptr
			break;
		}

		// Failed cleanup and try again
		CloseHandle(fileHandle);
		fileHandle = NULL;
		blockAddress = NULL;
	}

	if(blockAddress != NULL) {
		// Test memory writing
		memset(blockAddress, 1, roundedSize); // full fill
		memset(blockAddress + roundedSize - 10, 42, 20); // should wrap
		memset(blockAddress + roundedSize, 1337, 40); // should wrap

		ZeroMemory(memory, sizeof(*memory));
		memory->length = roundedSize;
		memory->basePtr = (void*)blockAddress;
		memory->fileHandle = fileHandle;
		return(true);
	} else {
		return(false);
	}
}

static void CleanupMirrorMemory(MirrorMemoryBuffer* memory) {
	if(memory->basePtr != NULL) {
		UnmapViewOfFile(memory->basePtr);
		UnmapViewOfFile((uint8_t*)memory->basePtr + memory->length);
		VirtualFree(memory->basePtr, memory->length * 2, MEM_RELEASE);
	}
	if(memory->fileHandle != NULL) {
		CloseHandle(memory->fileHandle);
	}
	ZeroMemory(memory, sizeof(*memory));
}

Does someone have a idea why the code does not work? The memory writing at the very end of InitMirrorMemory() function crashes.

The original idea comes from (https://atastypixel.com/blog/a-simple-fast-circular-buffer-implementation-for-audio-processing/comment-page-1/),​ but the code there is for MacOS only.

Advertisement

I had a look into this some time ago as well but unfortunately the memory model of Windows doesn't support this kind of address reuse. It works on MacOS and Linux perhaps because mmap works different on these platforms (which is a UNIX command so it works similar on OSX, Linux and Android).

As far as I udnerstood so far, the difference is that Windows just reserves the file to be mapped but it isn't mapped as long as you don't have a view on it while this step isn't necessary for mmap because it already creates such a view. In Windows you simply run into the virtual memory manager wall

Weirdly enough, yesterday i got it working after changing VirtualFree from MEM_RELEASE to MEM_FREE.

Now today, its not working anymore with the same code O_o. Maybe it was just luck.

Also i searched for similar sources and i found one which looks very similar:

https://github.com/Celtoys/Remotery/blob/main/lib/Remotery.c#L983

Its a GPU/CPU profiler and has similar code for Mac as well. So i assume it should work (the github repo is very active)?

Maybe you need to setup some access privileges or something, who knows…

Finalspace said:
Now today, its not working anymore with the same code

You might have been run into this

Release the range immediately but retain the address for the next sequence of code to try and map to it. In the mean-time some other OS thread may come along and allocate this address range from underneath us so multiple attempts need to be made

And from the issues it seems pretty unstable that this will work. However, as I said, Windows virtual memory manager isn't meant to work this way

@Shaarigan Yep for that reason I gave up and simply written a wrapping buffer. Now everything works just fine ?

Thanks for replying.

This topic is closed to new replies.

Advertisement