Raspberry Pi 5 8GB - Out-of-Memory (OOM) Killer Activations

rkorell · March 5, 2025, 8:30pm

Dear @Wheaten
Sounds good!
Maybe you can help me - again …
I’ve just tried to get Volumio running on my Pi5 (my PIMORONI NVME Base and corresponding Samsung NVME M2 SSD 980 was at home as I arrived today from my business trip).
Unfortunately I’m not successful at all
Have set all parameters according first post in this thread and tried twice the “Volumio/System” Option “write to disk” - but this failed twice. Process ran fast to 30%, flickers, the Volumio “waiting clock” appears shortly and background disappears, all stops, copy window disappears and Volumio comes back.
No message, nothing. And: NVME don’t boot.

To double check if I have config/cabling etc. correct I’ve booted a fresh bookworm SD and wrote this SD image via “SD-copier” to NVME.
NVME boots without any issue.
Than I reboot with SD bookworm, put Volumio SD card in an USB-reader and used SD-copier to copy Volumio SD to NVME -(copy) worked - but didn’t boot either.

Last trial: I copied latest version of Volumio .img file to the Raspi5, boot bookworm again from SD and used Raspi Imager to copy Volumio img to NVME.
(Copy) Worked, but didn’t boot.

Any additional idea?
I’m clueless right now…

Thanks for any hint!
Warm regards,
Ralf

EDIT:

Additional Information:

pi@raspi5:~ $ sudo rpi-eeprom-update
BOOTLOADER: up to date
   CURRENT: Mi 12. Feb 10:51:52 UTC 2025 (1739357512)
    LATEST: Mi 12. Feb 10:51:52 UTC 2025 (1739357512)
   RELEASE: latest (/usr/lib/firmware/raspberrypi/bootloader-2712/latest)
            Use raspi-config to change the release.


sudo rpi-eeprom-config
[all]
BOOT_UART=1
POWER_OFF_ON_HALT=0
BOOT_ORDER=0xf461
PCIE_PROBE=1

The update of /boot/config.txt or /boot/firmware/config.txt with overlays:
dtparam=nvme
dtparam=pciex1_gen=2

Is according this thread NOT necessary - so I have done this NOT.

2nd EDIT: Just found a spare USB-NVME adapter so I had a look at the NVME drive from my Laptop - Image is there, Seems absolutely “normal” …

3rd EDIT: I’ve double checked and tried the Volumio/System/Write to disk a third time.
same behaviour - no message nothing, system stops shortly and comes back.
But image IS written.
(as a “view” check allows this statement…). - the NVME disk looks alike the view before - with direct raspi-imager writing…

So my conclusion is, it’s not about the (physical) image - this arrives on disk on all ways - but disk is not willing to boot Volumio…
Any Idea?

THANKS!

Ralf

Wheaten · March 5, 2025, 8:37pm

@nerd

This is your expertise

rkorell · March 5, 2025, 9:03pm

Have edited my above post and added some information

nerd · March 6, 2025, 5:45am

Hey @rkorell,

At 30% of the process, the UUID is not updated yet, causing the system to fail booting from NVMe.

Next Steps:

Boot from MicroSD
Try “Install to Disk” option
When it fails, wait ~2-3 minutes
Create and post a link to debug log:
- Using SSH, run:
```
dmesg > dmesg.txt
```
- Zip the file and attach it to your response.

This will help analyze what’s causing failure.

Kind Regards,

rkorell · March 6, 2025, 8:18am

Dear @nerd ,
thanks!
created NVME at 9:06,
Logfile at 9:10, hope this is sufficient delay:
http://logs.volumio.org/volumio/3njFq4X.html

dmesg.zip (15.1 KB)

Hope you can find something useful.
Warm regards,
Ralf

nerd · March 6, 2025, 8:49am

Hey @rkorell,

Ouch!

Out-of-Memory (OOM) Killer Activations

[ 119.560887] ifplugd invoked oom-killer: gfp_mask=0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), order=0, oom_score_adj=0
[ 119.561280] oom-kill:constraint=CONSTRAINT_CPUSET,nodemask=(null),cpuset=system,mems_allowed=0,global_oom,task_memcg=/,task=node,pid=1272,uid=1000

This is the first time ever seeing this in the wild on Raspberry Pi 5.

Please detail your Raspberry Pi 5 specs from this thread.

Kind Regards,

rkorell · March 6, 2025, 9:24am

Here output information:


             Free Audiophile Linux Music Player - Version 3.0

          © 2015-2021 Michelangelo Guarise - Volumio Team - Volumio.org

Volumio Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Welcome to Volumio for Raspberry Pi (6.6.62-v8+ aarch64)
volumio@volumio:~$ cat /proc/cpuinfo | grep -i revision
CPU revision    : 1
CPU revision    : 1
CPU revision    : 1
CPU revision    : 1
Revision        : d04170
volumio@volumio:~$ cat /proc/cpuinfo | grep -i model
Model           : Raspberry Pi 5 Model B Rev 1.0

complete tree (if necessary):


volumio@volumio:~$ cat /proc/device-tree/compatible | tr '\0' '\n'
raspberrypi,5-model-b
brcm,bcm2712

Thanks!
Ralf

rkorell · March 6, 2025, 9:25am

Do you think it’s helpful to buy another one?
(This part is quite new …)
(recently bought)

nerd · March 6, 2025, 9:53am

@rkorell,

Moved to a new thread as the NVMe preparations are completed.

The OOM should not occur on an 8GB Raspberry Pi 5. This strongly indicates a memory leak or uncontrolled memory allocation during the installation process.

Step-by-Step Debugging for Memory Leak

1. Identify Which Process is Leaking

Use htop or top to monitor memory usage in real-time:

top

Sort by memory usage (`F6 → Select “MEM%”).
Look for Node.js (node), Volumio, systemd, or other services consuming excessive memory.

Alternatively, use:

ps aux --sort=-%mem | head -15

This will list the top 15 processes consuming memory.

2. Log Memory Usage Over Time

Since the leak occurs during installation, track memory usage at intervals.

Run this command in another SSH session while installing to disk:

while true; do free -h >> memory_log.txt; sleep 5; done

This logs memory usage every 5 seconds.
Stop it after failure (Ctrl + C).
Inspect the log:
```
cat memory_log.txt
```
- Look for a steady increase in memory usage.

3. Check for Growing Processes

Monitor live memory growth:

watch -n 1 "ps -eo pid,comm,%mem --sort=-%mem | head -15"

If a process (e.g., node) keeps increasing in memory without releasing it, it is leaking.

4. Capture Detailed Memory Stats

Immediately after the failure, capture:

cat /proc/meminfo > meminfo_before_reboot.txt
cat /proc/slabinfo > slabinfo_before_reboot.txt

These files will help analyze memory allocation trends.

5. Debug Node.js Memory Usage

Since Node.js is a major component of Volumio, check if it is responsible for the leak.

Run:

sudo systemctl restart volumio
ps aux --sort=-%mem | grep node

If memory usage does not reset, Node.js might be leaking.

Enable verbose logging for Node.js:

export NODE_DEBUG=gc
journalctl -fu volumio

This will log garbage collection (GC) activity.
If GC is not reducing memory usage, there is a leak.

6. Debug Kernel Memory Usage

If the leak is in the kernel, check:

cat /proc/buddyinfo

This shows fragmentation in kernel memory pools.
If high-order allocations (order 2+) are failing, the kernel is leaking memory.

7. Final Step: If You Confirm a Leak

If you find a specific process consuming excessive memory:

Restart it manually:
```
sudo systemctl restart <service-name>
```
If Node.js is the cause, try running it with memory limits:
```
node --max-old-space-size=512
```

Summary of Debugging Actions

Action	Command
Monitor live memory usage	`top` or `ps aux --sort=-%mem`
Log memory usage over time	`while true; do free -h >> memory_log.txt; sleep 5; done`
Check for memory growth in processes	`watch -n 1 "ps -eo pid,comm,%mem --sort=-%mem
Check if Node.js is leaking memory	`export NODE_DEBUG=gc` and `journalctl -fu volumio`
Debug kernel memory usage	`cat /proc/buddyinfo`
Check memory fragmentation	`cat /proc/meminfo`

Next Steps

Run these debugging steps while installing to disk.
Post any unusual memory growth or errors you find.
If Node.js is the culprit, we may need to adjust Volumio’s garbage collection settings.

Let me know what you find.

Kind Regards,

rkorell · March 6, 2025, 6:41pm

Dear @nerd ,
First at all and most important: thanks a LOT!

Huge effort from your side to help me - highly appreciated.
Sorry for delay - I had some work

I’ve tried several of your debugging advices, none of the outputs shows anything suspicious …
(at least to my bare eyes…).

While playing around I’ve even tried to “force” Node.js (which is not prominent in any of the debug lists) - because I’ve thought it’s idle and bored because nothing to do …
With this I recognized that Volumio isn’t able to do ANYthing despite dsiplaying the GUI / Menus.
Music playing doesn’t work at all…
(Had not tried before - why? No Speaker, no monitor, just a “test-Pi” for testing NVME…)

So it seems all this is my fault…
I’ve tested with my already running Volumio SD. (thinking this is OK)
But it seems NOT OK.
So now I build up a literally brand new SD card, made the 8 initialization steps and reached finally the System menu where I was able now to write this brand new - unconfigured - image to the NVME.

I did not understand the difference but at some point I missed that it is oviously used to use a NEW image.

So I’ve now successful copied Volumio Image to NVME and Raspi 5 starts from there.

BUT: Playing music doesn’t work either and skipping track or selecting another track from playlist does’t work as well…

Puuuuuhhhhh…
I’m nearby giving up.

Any idea?

Warmest regards,
Ralf

nerd · March 6, 2025, 6:48pm

Hey @rkorell,

Well, at least we are mowing forwards.

Next steps

Another round:

Boot from NVMe
Replicate each problem ~2-3 times
Create and post a link to debug log:
- Using SSH, run:
```
dmesg > dmesg.txt
```
- Zip the file and attach it to your response.

As before, this will help analyze what’s causing failure.

Kind Regards,

rkorell · March 6, 2025, 7:56pm

Thanks again!
I will collect data over the weekend - I’m busy on the road tomorrow and will be back late.

Keep you posted.
Warmest regards,
Ralf

rkorell · March 7, 2025, 9:58am

Dear @nerd ,
was curious and found some minutes inbetween.
http://logs.volumio.org/volumio/Gplla1S.html
dmesg.txt.zip (12.3 KB)
Regards,
Ralf

rkorell · March 7, 2025, 4:05pm

I found some additional information -what’s working, what’s not working…
Just a few minutes ago I’ve started the pi5 (from NVME).
It’s connected to spotify. On my MagicMirror I do have a module “playing now on Spotify”.
Interesting enough the Pi5 send status information to spotify - the songs displayed in the magic mirror module do corrrespond and respond to “skip track” on Volumio - whilst Volumio did this not …
Playing of a song didn’t work, either - no progress shown on Volumio neither on module on mirror …

Regards,
Ralf

nerd · March 7, 2025, 4:36pm

Hey @rkorell,

There is nothing meaningful in the logs, however, I noticed several concerning issues that could be indirectly related to power drain:

Potential Power-Related Issues

1. Continuous CMA (Contiguous Memory Allocation) Failures

[    0.750833] cma: cma_alloc: linux,cma: alloc failed, req-size: 4 pages, ret: -12
[    0.750841] cma: cma_alloc: linux,cma: alloc failed, req-size: 16 pages, ret: -12
...
[   12.125388] cma: cma_alloc: linux,cma: alloc failed, req-size: 3 pages, ret: -12

These failures indicate insufficient memory allocation for devices requiring contiguous memory blocks (e.g., GPU, sound processing, or PCIe NVMe).
Power fluctuations can lead to unstable memory management, especially for NVMe, USB DACs, or onboard audio.

2. NVMe Initialization (High Power Demand)

[    0.377788] nvme nvme0: pci function 0000:01:00.0
[    0.377792] nvme 0000:01:00.0: enabling device (0000 -> 0002)
[    0.381859] nvme nvme0: Shutdown timeout set to 8 seconds
[    0.393002] nvme nvme0: allocated 64 MiB host memory buffer.

NVMe requires high power draw (~2-3W on average, up to 5W during bursts).
If the power supply is insufficient (below 5V/5A), NVMe might not work reliably, leading to storage and audio issues.

3. Time Sync Issue at Boot

[   11.015803] systemd[1]: System time before build time, advancing clock.

Indicates the clock was incorrect at startup.
This suggests that the RTC battery is missing or power loss occurred before boot.
Streaming services may fail if timestamps don’t match authentication servers.

Troubleshooting & Fixes for Power Issues

1. Check for Undervoltage in Real-Time

Run:

vcgencmd get_throttled

If it returns:

0x50000 → Under-voltage detected!
0x20000 → Frequency capped due to low power

Then your power supply is insufficient.

2. Monitor Voltage Stability

Run:

watch -n 1 vcgencmd measure_volts core

Should stay above 4.8V.
If it drops too low, try a different power adapter.

3. Reduce Power Draw

Disconnect USB peripherals that are not needed (Wi-Fi dongles, unused USB DACs).
Test booting from microSD instead of NVMe:
```
sudo blkid
```
If NVMe is causing power instability, try running without it.

4. Use a High-Quality Power Supply

For Raspberry Pi 5 + NVMe, use a 5V/5A power supply.
If using PD (USB Power Delivery), ensure it’s negotiating 5V/5A.
Test with an official Raspberry Pi 5 PSU.

5. Manually Allocate More CMA Memory (If Needed)

If CMA errors persist, increase CMA size in /boot/config.txt:
```
cma=256M
```
Then reboot.

Final Thoughts

No direct undervoltage warnings, but CMA memory allocation failures, NVMe high power draw suggest possible power instability.
Check voltage with vcgencmd get_throttled.
Try a stronger power supply (5V/5A).
Test without NVMe to see if Spotify works.

Kind Regards,

rkorell · March 8, 2025, 11:41am

Dear @nerd ,
thanks - again!

Because I was aware that Pi5 are kind of power-hungry I used the official Pi-Power-Supply from the beginning - this is 5,1V, 27W.
In addition as all is about “testing” right now, the pi is “bare” so no additional equipment added despite NVME-drive.

Today I disconnected the NVME to avoid power consumption and booted from SD card.
Unfortunately the exact same behaviour: No playing, skippping backward works, skipping forward not.

So I ran through your tests - and: Every 1.0s: vcgencmd measure_volts core results in 0,7200V !!!
(According your advise this is way to less?)
For this reason I used a second, identical original PI power supply - with exact same result.

Is Pi broken? I cannot imagine, that two power supplies are defective in parallel (both of them are brand new, as the Pi itself).

Upfront the testing I’ve installed the newest Volumio image (3.795) which was system-proposed today.

Here the output of all your suggested debugging-steps:

volumio@volumio:~$ sudo vcgencmd get_throttled
throttled=0x0

watch -n 1 vcgencmd measure_volts core

Every 1.0s: vcgencmd measure_volts core                                volumio: Sat Mar  8 11:06:05 2025
volt=0.7200V


volumio@volumio:~$ sudo blkid
/dev/mmcblk0p1: LABEL_FATBOOT="boot" LABEL="boot" UUID="51B1-040D" TYPE="vfat" PARTUUID="e8ab1baf-01"
/dev/loop0: TYPE="squashfs"
/dev/mmcblk0p2: LABEL="volumio" UUID="fb80cb44-b4d3-4b36-be35-613cba0a09fb" TYPE="ext4" PARTUUID="e8ab1baf-02"
/dev/mmcblk0p3: LABEL="volumio_data" UUID="2899bccc-8620-4c7a-b56a-43fd2f7a836a" TYPE="ext4" PARTUUID="e8ab1baf-03"
/dev/mmcblk0: PTUUID="e8ab1baf" PTTYPE="dos"



volumio@volumio:~$ sudo nano /boot/userconfig.txt
volumio@volumio:~$ cat /boot/userconfig.txt
# Add your custom config.txt options to this file, which will be preserved during updates
cma=256M

In addition I have created a logfile for the run on SD card, but I guess this becomes obsolete, because of the power findings
http://logs.volumio.org/volumio/tMnAKeP.html

Does it make any sense to try to buy a second Pi5?

Thanks a lot for your kind support!

Ralf

nerd · March 10, 2025, 7:39am

Hey @rkorell,

Given your situation, one thing you might consider is testing with a different Raspberry Pi 5, as sometimes small variations in silicon, often referred to as the “silicon lottery,” can result in subtle differences in performance, particularly with power delivery and other components.

However, it’s highly unlikely that this is the issue here. From my own experience with Raspberry Pi 5, I’ve seen similar voltage readings around 0.72V to 0.75V on my reference Pi 5 2/4/8/16 GB units, and they are functioning perfectly fine. This suggests that the voltage you’re seeing is likely within normal operating parameters for the Pi 5 and not indicative of a hardware fault.

Given this, it seems much more probable that the issue lies in software configuration or system setup rather than a defect with the Pi 5 itself.

Kind Regards,

rkorell · March 10, 2025, 1:23pm

Dear @nerd ,
thanks for this indication.
as you suggested I’ve already played in the mentionend lottery and I have ordered a second Pi5 on Saturday - should arrive today or tomorrow. (Not THAT great hope on that, but I will try at least)

As you stated " software configuration or system setup" errors may occur - I wonder how this could go “wrong” in any way - all I have done is to write the given newest Volumio Image to SD-Card.
Without any changes (and even on a “bare” system: NO additional hardware or add-ons).

ARE there recommended settings/configuration steps for Volumio on Pi5 which I have missed?
(A “normal” bookworm installation works fine from SD-card as well as from NVME-disk)

Last times I’ve downloaded the Volumio Image from Volumio’s website - in the meantime I’ve seen somewhere that Pi-Imager does contain a Volumio Image (not tested yet) - should I give this image a try?

What else I’m missing to avoid “software configuration or system setup” errors/problems?

thanks for any additional hint!
Ralf

rkorell · March 10, 2025, 1:26pm

Yes, Raspi-Imager does offer a Volumio-Image.
Release Date Feb-27-2025 …
(“online-download”)

Wheaten · March 10, 2025, 2:06pm

Never use the image offered by Raspberry Imager. We have only see issues with them and it seems impossible task to get them removed.