With the rise of popularity of the PCIe NVMe storage on Raspberry Pi latest devices starting this thread to share our experiences.
As a starting point please explore excellent overview and write-up with the caveat that the NVMe boards are now readily available and variety is increasing on the daily basis.
We also have a quick guide discussing how to prepare Raspberry Pi5 to interact with PCIe storage boot process.
Volumio implementation
Active-State Power Management (ASPM) saves power in the Peripheral Component Interconnect Express (PCI Express or PCIe) subsystem by setting a lower power state for PCIe links when the devices to which they connect are not in use. ASPM controls the power state at both ends of the link, and saves power in the link even when the device at the end of the link is in a fully powered-on state. When ASPM is enabled, device latency increases because of the time required to transition the link between different power states.
PCI Subsystem Options - pcie tuning will check the device’s MPS against the parent bridge when it is initially added to the pci subsystem, prior to attaching to a driver. If MPS is mismatched, the downstream port is set to the parent bridge’s if capable.
Our cmdline.txt uses pcie_aspm=off pci=pcie_bus_safe to keep NVMe ready at all times.
The official Raspberry Pi documentation reads: The connection is certified for Gen 2.0 speeds (5 GT/sec), but you can force it to Gen 3.0 (10 GT/sec)… Volumio is not aspiring to run quantum level calculations nor run real time processing although it relies on error-free storage I/O.
Our volumioconfig.txt uses:
| Description | Parameter |
|---|---|
| Enable NMVe | dtparam=nvme |
| Use certified Gen 2.0 | dtparam=pciex1_gen=2 |
NVMe SSD Incompatibility List
NVMe SSD drivers equipped with Phison controller (PS5022) are known to cause problems.
NVMe reported as not playing nicely with Raspberry Pi:
| Model | Controller |
|---|---|
| WD Black SN850 series | SanDisk 8-Channel (derivative from E18T) |
| WD Black SN770 series | 20-82-10081-A1 Polaris MP16+ |
| WD SN740 series | 20-82-10081-A1 Polaris MP16+ |
| WD Green SN350 series | 20-82-01008-A2 Polaris MP16 |
| Samsung 990 Pro | Samsung Pascal V7 (FW older than 4B2QJXD7 |
| Samsung 980 Pro | Samsung Elpis (FW older than 5B2QGXA7) |
We will update above list with problematic devices from community feedback.
NVMe SSD Compatibility List
Gen 3: Generations of NVMe SSD drivers lover than 3 are considered safe.
Gen 4: All devices tagged with “Low Power Consumption” and not based on Phison controller (For now, we know that some PS5012S, PS5013 and PS5022 are causing issues).
Gen 5: An overkill. Most of the Gen 5 NVMe are furnished with large heat-sinks rendering them impractical. Power requirements for the Gen 5 NVMe is a separate topic altogether.
Power requirements
I will open with frequent comment “I put NVMe to my PC and it works perfectly. I put the same NVMe to Raspberry Pi 5 and it does not work!”.
I am not surprised. A standard PC PCIe lane delivers a whooping 75W of power. On PC PCIe lanes designed for high performance and throughput delivered power is even higher. Raspberry Pi 5 on the other hand, is spec to provide up to 5W to PCIe lane. Adding FCC unshielded ribbon impedance it can be reduced to as little as 2W. And this is where most of the problems are - a NVMe HAT without additional power source, simply can not provide enough energy to drive demanding NVMe. Such NVMe, the very moment when even proportionally small workload arrives, disconnects, starts throwing I/O errors tantrum and even in some edge cases - a sudden power decay, corrupts other storage devices by simply under-powering device during write. Sound familiar?
Some NVMe HATs producers already observed above problem and added an auxiliary power connector like 2-pin XH2.54 to drive 5V from an additional power supply lane, whilst other producers have exposed pads for soldering.
“But I have original Raspberry Pi power supply which can deliver 27W!”. It is connected to the USB-C port on Raspberry Pi. Power is distributed by Raspberry Pi circuitry down to all connected devices. Connected devices power consumption, however small it seems, adds up. In most of the cases it is sufficient, however add Touch, IR receiver, high power DAC, drive a screen and there comes a surprise. In fact - this is an elementary school level maths.
Usage, community discussion and further steps:
| Description | Community thread |
|---|---|
| Upgrade eeprom | Prepare Raspberry Pi for boot from USB/NVMe |
| Testing Volumio OS with Disk Installer | [ALPHA] Raspberry Pi boot from USB/NVMe |
