Memory Use Not Reduced As Files Are Removed From Ram Disk

[MovieWiz]

Very interesting project you took on here. I got it working on Windows 11 24H2. And I definitely saw that memory allocated dynamically as I added 15GB of files to the ramdisk. However, I did not see the memory usage (in Task Manager) come back down after I deleted the 15GB of files from the RamDisk. I waited several minutes and the memory usage stayed the same. Not sure if this is the expected behaviour or if freeing the the memory was not something you had time to implement. I didn't see any settings that might be relevant to this.

Anyway, if you ever have time to work on the project again, I will definitely test it out again.

[tmcdos]

Hi and thank you for finding my project useful!
I am not so good with C/C++ (that's why I used Object Pascal) so once I got the GUI stable I have not checked the updated CLI tool which accompanies the newer versions of the Arsenal driver. I don't think that there is a way to tell the driver to free the unused memory - at least back then when I reached a stable GUI and switched from ImDisk to this GUI.
It might be possible in the latest driver + CLI tool combination but I do not have the time to dig into C/C++ sources to discover.
In my personal opinion and for my own usage it seems just fine if the RAM-disk uses as much memory as I configure it for, even if it is reserved upfront rather than dynamically. When I had only 8GB of RAM - my RAM-disk was 1GB. When I got 16GB of RAM - my RAM-disk was 2.5 GB. Now, when I have 64GB of RAM - my RAM-disk is 16 GB and I don't care if it uses all 16GB during all day, there is still plenty of RAM for my daily usage.
I totally understand that you or someone else may have different needs and could prefer dynamic allocation. However, I am not aware whether the driver supports this feature at all, and if yes - I currently do not know how to tell the driver to enter such mode.
Maybe I will find some free time some day to explore this but until then - we should wait for someone to scratch his own itch and contribute the solution back :)))

[MovieWiz]

Hello. Wow. Thanks for getting back.

I sort of figured that it was something like that. Thanks for confirming that I wasn't doing something wrong.

This whole refusing to run exe files on IMDisk Ramdisk is a rather annoying wrinkle in Windows 11 24H2.

On my 32 GB desktop, I configure the RamDIsk with 20 GB of dynamic memory. I used it mainly for TEMP files (which were never very large) and temporary storage when editing large video files (mostly from my TV tuner/dvr PCIe card). The dynamic memory allocation in IMDisk Ramdisk was perfect for this since I'd get the memory back when I was done with the large temporary files.

Oh well. I'm not ready to pay the hefty price for Primo RamDisk yet (I'd need at least the pro version). So, I'm keeping the IMDisk Ramdisk and I've just put the TEMP files back on my Drive C SSD for now. Sooner or later, there will be a free or more economical ramdisk alternative that both has dynamic memory and doesn't offend Microsoft.

I almost wonder if Microsoft did this by accident since it doesn't seem to serve any obvious security objective. Maybe this problem will just disappear in one of the upcoming monthly Windows updates.

[tmcdos]

Oh, these newer EXE's are annoying not just on Win11 but on Win7, too!
I have tried to explain the problem in the README but the explanation is probably too technical.
Newer EXE's usually use the WinAPI function GetFinalPathNameByHandle to resolve symlinks (junction points).
Since ImDisk uses Direct-IO and thus bypasses the Volume Manager - ImDisk can not be reached/queried by the Volume Manager when GetFinalPathNameByHandle is called/invoked.
The Arsenal driver is a regular SCSI miniport driver - so it can be queried by the Volume Manager when any EXE calls GetFinalPathNameByHandle WinAPI function.
I am using the RAM-disk as a storage for TEMP folder and for data folders of Chrome and Firefox which like to create a lot of cache files - and I don't want my SDD to wear out because of the unfriendly browsers.
I also use the RAM-disk for my JavaScript projects where NPM or Yarn install hundreds and thousands of packages and tens of thousands of files during development - and I don't want these to wear out my SSD.
If you don't want to pay for another RAM-disk software then perhaps you would like to consider buying another 32GB of RAM?
I do not constantly monitor my RAM usage but I believe it stays below 30% most of the time, often even below 10%.
So the trade-off between ImDisk dynamic RAM allocation and Arsenal fixed allocation (but being able to run Chrome and other problematic EXE's) is worth for me.

[oood]

Great project, looking forward to seeing more progress! @tmcdos

I am not so good with C/C++ (that's why I used Object Pascal) so once I got the GUI stable I have not checked the updated CLI tool which accompanies the newer versions of the Arsenal driver. I don't think that there is a way to tell the driver to free the unused memory - at least back then when I reached a stable GUI and switched from ImDisk to this GUI.

Due to Windows 11 errors and the fact that the ImDisk driver is no longer maintained by the author, the ImDisk Toolkit author w77 has planned to move to Arsenal like you, maybe you can contact him to speed up the development of the project.

https://sourceforge.net/p/imdisk-toolkit/discussion/general/thread/041e10fe01/

This whole refusing to run exe files on IMDisk Ramdisk is a rather annoying wrinkle in Windows 11 24H2.

I made a small fix for the ImDisk error on Windows 11 24H2, but as the ImDisk driver is no longer maintained, it can only be a temporary relief, the real solution is to migrate to Arsenal.

https://github.com/oood/Win11-RAMDisk-Admin-Fix

[tmcdos]

@oood Thank you!
To be honest, I am quite busy at work the last couple of months and did not have any chance to spend some time on my Github projects.
I will try to contact the author of ImDisk but I believe he prefers C/C++ while I am working with Object Pascal so it might not be possible to combine our efforts but at least we can have a discussion.
By the way, there is a VxKex project which aims to allow the running of some of these failing EXE files. I am using it to run Node 18/20/22 on my Win7. Perhaps you can try with ImDisk and see if it helps.

[oood]

@oood Thank you! To be honest, I am quite busy at work the last couple of months and did not have any chance to spend some time on my Github projects. I will try to contact the author of ImDisk but I believe he prefers C/C++ while I am working with Object Pascal so it might not be possible to combine our efforts but at least we can have a discussion. By the way, there is a VxKex project which aims to allow the running of some of these failing EXE files. I am using it to run Node 18/20/22 on my Win7. Perhaps you can try with ImDisk and see if it helps.

Thank you for taking the time to develop this project. I think things are heading in the right direction anyway. Looking forward to seeing the next generation of ImDisk.

VxKex project is also an outstanding personal project. I sympathize with the developer's mother's experience. I have also experienced similar things.

Thank you!

[igowrek]

@oood you should try out Arsenal Image Mount cli tool, just create a dynamic VHD with Windows 11 native tools, assign it a letter, label and filesystem, you can mount the VHD right there as well to change its content, then you can unmount it and mount it using aim_cli.exe. All that's left to do is to add a command to scheduler to mount it on startup, and you'll get your RAM disk with memory being dynamically allocated/deallocated!

[tmcdos]

@oood After looking at the source code of the driver my conclusion is that TRIM-ing is performed only in proxy mode and in image mode. But since my GUI initializes a standard RAM-disk (no proxy and no VHD or ISO image) - once the driver grabs some memory to hold the files, this memory is never released back to the OS. Looks like this is by design - at least in the open-source version of the driver.
My conclusion is that this feature request should go to the author of the Arsenal driver.

[tmcdos]

Mark from Arsenal replied to me and forwarded a response from Olof.
Olof confirms my suspicion that the driver performs TRIM/unmap only for VHD/proxy:

Our basic RAM disk options implemented in the driver do not support unmap/trim. The entire memory size for the RAM disk is allocated as one single memory block when the RAM disk is created. Windows lazy-commits parts of virtual memory, so it might look like it starts with very little memory allocation, but it rather depends on what memory sizes you are looking at (committed, reserved etc).

We have dynamically allocated RAM disks as well in AIM, but they use several other components to work in the expected way. You need to start with a VHD image file as template and then create a RAM disk based on it, either advanced menu in the GUI, or something like:

aim_ll -a -f \\?\vhdaccess\??\awealloc\??\c:\image.vhd

This creates a dynamically allocated RAM disk based on contents of C:\image.vhd. Memory allocation increases as files are added and decreases as files are deleted. This is implemented by not allocating all memory as one single memory block, but instead each block of the VHD image is allocated separately. As VHD blocks are zeroed and moved to physical end of image in response to unmap/trim, the VHD image size shrinks and the last block at the end of underlying memory can be freed as well.

Since I am scratching my own itch with this GUI, I have particular goals which do not align well with VHD image:

1. I want to preload the RAM-disk with the contents of a given folder but once preloaded - I definitely do not want the folder (or a VHD image file in the root filesystem) to be constantly updated by the driver
2. At PC shutdown I want the RAM-disk to be persisted back to the same folder that was used for preloading - but with the ability to exclude some folders (specified in the GUI config) which are considered temporary work and can be safely thrown away
3. I prefer using a bare folder for preloading instead of VHD image not only because it does not require mounting/unmounting but also because I want the preloading/persisting folder to be modified only at shutdown and not at any other time to avoid SSD wearing out. I use this RAM-disk mainly to avoid the SSD wear-out (mostly by Chrome and Firefox but also by the TEMP files), speed is on 2nd place.

So apparently the only option for dynamic RAM allocation is

aim_ll -a -f \\?\vhdaccess\??\awealloc\??\c:\image.vhd

as suggested by @igorbaryshev

[igowrek]

@tmcdos actually, you don't have to update the contents of VHD, mine is only used as a preload template and is not being updated at all. So, you can just mount VHD and preload content from your folder after mounting it, and copy over whatever you need before shutdown, no changes will be written to the VHD.
At least when I mount with --ramdisk --filename=<path_to_vhd>, no changes are saved to it, so no disk writes, which is pretty much what I want, as I use it as my temp dir and for building stuff.

[igowrek]

@tmcdos here's the VHD I use for my RAMDisk, and, as you can see, it hasn't been touched ever since I first created it.

[tmcdos]

@igorbaryshev Got it. Okay, I will add the ability to preload the RAM-disk from a VHD image instead of regular folder.

[igowrek]

@tmcdos also, maybe the neat thing for you would be the fact that you can format the VHD, assign it a label and a letter, save it at that state, keep it and then use that to mount on startup, instead of reassigning the drive letter every time.
You could move the files over to the RAMDisk and back just as you did before, so I don't really see much of a downside from using dynamic VHD as a base versus no-image RAMDisk.

[tmcdos]

@igorbaryshev Oh, don't get me wrong - I am not against the suggestion of using a VHD image. It just was not a requirement for me when I developed this GUI and since grabbing all the RAM at once is not an headache for me - I am just postponing the moment when I will have enough time to sit down and add the missing feature. Especially when the functionality can be achieved with the AIM CLI :)
I am open to PR, though - if it can be compiled under Delphi 7.

[vonguyenkhang]

However, I did not see the memory usage (in Task Manager) come back down after I deleted the 15GB of files from the RamDisk. I waited several minutes and the memory usage stayed the same.

@MovieWiz Did you make sure that "the 15GB of files" is permanently removed, not moved to Recycle Bin?

Sorry if this question sounds dumb, but I think I should ask just in case, because the default behaviour of the RAMDISK created by AIM is to move deleted files to Recycle Bin instead of permanently delete immediately. This is different from Imdisk as explained by @tmcdos here.
And of course the memory usage (in Task Manager) will not come back down if the files are still kept in the Recycle Bin.

EDIT: I've just tried and the RAM usage really didn't come down, sorry for the false info.
Refer to @tmcdos's post below for details.