LIMITS found in programming

Help and Support
1

Is there a limit in volumio?
After trying this solution on a Raspberry Pi, I wanted to add my windows server share to it.
Setting things correctly, it just kept nagging that I did not know what I was doing, I reject that.
Then I tried to share from my Lenovo NAS, same melody, same false notes.
So, I was browsing and found media servers, I have a Plex server running, so, I could connect to it, no problem there, browsing to the music folder that Plex knows, no problem there either, but when I wanted to pick Pharell Williams, it only went to the letter K, and not further.
Strange, it either looks like a way to either trick me into buying a premium or a simple buffer full programm error. Over 250 GB, 36000 files, 4200 foldersā€¦ must not be that hard to buffer.
If you make something, then expect extremes, some people do have extreme amounts of music and would like to listen to all if needed.

So, whatā€™s the deal here?

2

Welcome,

I run a library on a NAS containing 70.000+ tracks, 6000+ folders over 3TB without issues.
Most of the time it caused due to errors, like faulty files, unsupported archives etc. MPD is not very forgiving.
Please post a log so we can see what is happening after a library update/rescan.

3

Then I donā€™t know what machine you have, but my NAS is a bit older, but still Windows networking is the same since 2004 or so, and the problem of attaching a drive is not working, why does it not have the ability to browse from the RPi, that has my wifi password to access the rest of my network? Many software based RPi solutions are terrible at this.
Going via Plex Media Server should give me access to all my music, but even trying it on my pc with chrome, I get only till the letter K. So a log of trying it to attach a drive is not something I can redo right now, and maybe laterā€¦

4

Very basic Linux IS NOT Windows.
The add drive has some capabilities in Volumio to scan your network, but itā€™s limited.
But if even Chrome struggles with your Plex install, Itā€™s seems not related to Volumio.

You can bypass plex and add the netwrok share in Volumio.

  • Get the UNC to your Music Share
  • Determine if youā€™re going to use NFS or SMB.
  • Since you have an older NAS, find out which vesion of SMB or NFS is supported
  • Add the supported version as option in Add Drive.
5

Expanding on @Wheaten remarks.

Since your NAS functions as a Windows-based file-sharing service, letā€™s address the situation with that context in mind; including file locking, pagination, and network stack details in the context of your NAS MS Windows SMB acting as file-server, and Volumio interaction requires diving deeper into these components. Letā€™s analyze these aspects at very generic level one by one:

1. File Locking in SMB

File locking is a critical mechanism in SMB to prevent data corruption and manage concurrent access to files.

How File Locking Works:

  • Opportunistic Locks (OpLocks): SMB allows clients to lock files for exclusive or shared access to improve performance. The server notifies the client when another client requests access to the locked file.
  • Caching and Coordination: With SMBv3, caching is managed better, but issues can arise if the NAS or client mishandles cache coherence.
  • Lock Conflicts: If a file is locked (e.g., by Plex for metadata or by another process on the NAS), Volumio might fail to access it.

Troubleshooting:

  • Disable OpLocks (if needed): On the NAS, you can disable opportunistic locking for specific shares if file access conflicts occur.
  • Verify File Locks: Use tools like compmgmt.msc (Computer Management) on Windows to view and manage open files and locks under Shared Folders > Open Files.

2. Pagination Challenges

Pagination refers to the ability to manage and display data in chunks when dealing with large libraries. This can be an issue in both the SMB protocol and the applications (Volumio, Plex) interacting with it.

SMB and Pagination:

  • SMB itself does not impose pagination limits but depends on the clientā€™s ability to request and process directory listings.
  • For large directories, SMB may fragment responses into multiple packets. Older NAS systems or clients with limited resources might time out or fail to handle large responses.

Plex and Pagination:

  • Plex uses a database to manage libraries and often paginates results in its user interface.
  • If youā€™re seeing only up to the letter ā€œK,ā€ it might be due to a limit in the clientā€™s buffer size or an issue with pagination in the Plex database query.

Volumio and Pagination:

  • Volumio uses MPD (Music Player Daemon) for library indexing. MPD can struggle with large numbers of files in a single directory, as it loads all metadata into memory.

Solutions:

  • Break Down Large Directories: Organize files into smaller subfolders.
  • Increase Timeouts: For Plex, adjust the serverā€™s timeout settings to allow slower queries. For Volumio, extend CIFS mount timeouts using options like iocharset=utf8,rsize=65536,wsize=65536,timeo=50.

3. Network Stack and SMB/Volumio

The network stack is the backbone for SMB communication. Hereā€™s how it interacts with your setup:

Network Stack in Windows SMB:

  • TCP/IP and Latency: SMB runs over TCP/IP, relying on efficient packet delivery. High latency, dropped packets, or bandwidth limitations can degrade performance.
  • MTU and Packet Size: SMBv3 supports larger packet sizes for faster data transfer, but misconfigured MTU settings can cause fragmentation and retries.
  • Encryption and Signing: SMBv3 encrypts and signs data to improve security. However, this adds CPU overhead, which can impact older NAS devices.

Network Stack in Raspberry Pi:

  • The Raspberry Pi uses Linux-based network stacks, which are robust but resource-constrained.
  • SMB shares are mounted using cifs (Common Internet File System), which supports options like vers (protocol version) and buffer sizes (rsize, wsize) to optimize performance.

Debugging Network Stack Issues:

  1. Check Connectivity:

    • Ping the NAS from the Raspberry Pi: ping <NAS_IP> to check latency.
    • Use smbclient to test SMB connectivity: smbclient -L //<NAS_IP> -U <username>.

  2. Monitor Packet Loss:

    • Use tcpdump or Wireshark to capture traffic between the Raspberry Pi and NAS for debugging.

  3. Tune CIFS Options:

    • Edit the Volumio configuration to include mount options:
      vers=3.0,sec=ntlmssp,iocharset=utf8,rsize=65536,wsize=65536,timeo=50

4. Windows Networking Details

Windows as a NAS is generally robust but has some caveats in its networking stack when interfacing with other systems.

Key Considerations:

  • File Limits: While NTFS can handle millions of files, the Windows Server SMB implementation can slow down when managing very large directories over the network.
  • Concurrent Connections: Check whether the NAS is hitting its connection limit for SMB sessions. Windows limits concurrent connections on non-server editions.
  • Session Timeouts: SMB sessions may timeout if idle for too long or if the client does not request keep-alives.

Diagnostic Tools:

  • Performance Monitor: Use perfmon on the Windows NAS to monitor SMB traffic and resource usage.
  • Event Viewer: Check for SMB-related errors in Event Viewer under Applications and Services Logs > Microsoft > Windows > SMBServer.

5. Practical Steps for Your Setup

  1. File Locking:

    • Use a dedicated user account for Volumio to avoid conflicts with other processes accessing the same files.
    • Monitor and release file locks on the Windows NAS if needed.

  2. Pagination:

    • Divide the library into smaller, more manageable directories.
    • Test Volumioā€™s performance with a subset of the library before scaling up.

  3. Network Stack:

    • Ensure the Raspberry Pi is connected to the network via Ethernet for stability.
    • Optimize CIFS mount options in Volumio.

  4. Plex Alternative:

    • Use DLNA (if Plex supports it) for direct access to your NASā€™s library.

  5. Logs:

    • Enable detailed logging on both the Windows NAS and Volumio to pinpoint issues.

Summary

Your issues seem to be rooted in a combination of SMB file locking conflicts, large directory challenges (pagination), and network stack optimizations.

Kind Regards,

6

So in order to make this Volumio work I need to alter the crap out of my network while I can access it from the other side of the planet without problems. Thanks for enlightning this piece of software with a hanicap, but if normal use requires a whole lot of suggestions, then Volumio is the culprit, not an established working environment. Thank you.
http://logs.volumio.org/volumio/XLLk70N.html

7

I feel that you are missing the point. In the most generic terms allow me to elaborate quite complex technical problem.

Pagination of MPD in Relation to Available RAM and Raspberry Pi Versions (1 to 5)

Understanding MPD and Memory Usage

The Music Player Daemon (MPD) on Linux is central to Volumioā€™s library management. It loads metadata for all files in the library into memory, without using traditional pagination. This approach makes memory a critical factor when dealing with large music collections.

  • No Pagination: MPD does not fetch data incrementally (paginated); instead, it processes and holds the entire library in RAM.
  • Memory-Intensive Operations: Metadata (e.g., artist, album, file paths) for each file is cached in memory, so larger libraries require exponentially more RAM.
  • Impact of Insufficient RAM: When available RAM is insufficient:
    • The system may start swapping (using slower storage as memory), causing performance degradation.
    • Large libraries can cause crashes or out-of-memory (OOM) errors.

Example Memory Usage:

  • Small Library (1,000 songs): ~50ā€“100 MB.
  • Medium Library (10,000 songs): ~500 MB or more.
  • Large Library (36,000 files, 4,200 folders): ~1ā€“2 GB, depending on metadata richness.

Raspberry Pi Versions and RAM Considerations

The Raspberry Piā€™s available RAM, CPU, and storage capabilities directly impact MPDā€™s performance with large libraries.

Model CPU RAM Performance with Large Libraries
Raspberry Pi 1 ARM1176JZF-S @ 700 MHz (single-core) 256ā€“512 MB Insufficient for medium to large libraries. High risk of swapping.
Raspberry Pi Zero ARM11 @ 1 GHz (single-core) 512 MB Insufficient for large libraries. Limited by RAM and shared USB.
Raspberry Pi 2 Cortex-A7 @ 900 MHz (quad-core) 1 GB Handles small to medium libraries. Struggles with large libraries.
Raspberry Pi 3 Cortex-A53 @ 1.2 GHz (quad-core) 1 GB Decent for medium libraries. May lag or swap with larger libraries.
Raspberry Pi 4 Cortex-A72 @ 1.5 GHz (quad-core) 1ā€“8 GB Great for large libraries with 4 GB or more RAM.
Raspberry Pi 5 Cortex-A76 @ 2.4 GHz (quad-core) 4ā€“8 GB LPDDR4X Excellent for very large libraries with smooth MPD performance.

Why the Raspberry Pi 5 Is the Best Choice for Volumio and MPD

The Raspberry Pi 5 introduces significant improvements over its predecessors, making it the ideal choice for large music libraries and demanding Volumio setups:

1. Superior CPU Performance

  • Quad-core Cortex-A76 @ 2.4 GHz offers up to 2ā€“3x better performance than the Pi 4.
  • Faster processing for metadata parsing and MPD library indexing.

2. Increased RAM (4ā€“8 GB)

  • The 8 GB model provides ample memory for MPD to handle large libraries (like 36,000+ files) without swapping.
  • Allows better caching, smoother UI performance, and efficient operation of other processes.

3. Enhanced Networking

  • Gigabit Ethernet: Stable and fast connections to NAS for SMB shares.
  • Wi-Fi 6 (802.11ax): Improved wireless speed and reliability for streaming.

4. PCIe NVMe SSD Support

  • The Pi 5 supports PCIe NVMe SSDs, providing ultra-fast local storage.
  • Ideal for storing libraries locally, bypassing potential NAS-related network issues.

5. USB 3.0 Ports

  • Dual USB 3.0 ports enable high-speed connections to external drives and DACs.

6. Improved Thermals

  • Active cooling support ensures the Pi 5 maintains consistent performance during intensive tasks.

Optimizing Raspberry Pi 5 for MPD and Volumio

Library Management:

  • Store large libraries on an NVMe SSD for fast local access.
  • Organize files into smaller subfolders to optimize MPDā€™s library scan performance.

Network Configuration:

  • Use Gigabit Ethernet for reliable and fast NAS access.
  • Optimize SMB mount options:
    vers=3.0,sec=ntlmssp,iocharset=utf8,rsize=65536,wsize=65536,timeo=50

Monitoring Tools:

  • Use free -h to monitor memory usage during library scans.
  • Check MPD logs (/var/log/mpd/mpd.log) for errors or performance bottlenecks.

Conclusion and Recommendation

For managing a large library of 36,000 files and 4,200 folders with Volumio:

  • The Raspberry Pi 5 (8 GB) is the best choice. It combines a powerful CPU, sufficient RAM, and modern networking and storage capabilities to handle MPDā€™s demands with ease.
  • If youā€™re upgrading, the Pi 5 offers significant performance improvements over earlier models, ensuring smooth playback, fast library scans, and reliable NAS integration.

If you still think that the MPD is the problem, please create relevant issue report with the MPD upstream.

Kind Regards,

8

Nope, if it does the same on my pc, give only till the letter K then with 64 Gb of memory that flyer does not go up. As I said, I leave this alone, I am not willing to do extensive tinkering on whatnot to maybe get it to work, if it does not work right out of the box, it is useless to me whatever you throw at me. Buy a tv in a shop. Do you need to tinker with it to make it show everything available? No! It works right out of the box. Thatā€™s all Iā€™m saying.

Thanks though for trying to help here, I wish programmers would look further into problems and possibilities before putting a pricetag on it!

9

Assisting consumers who can not get their network devices working out of the box keeps two of my employees busy 5 days a week.

Windows networking (WFS) is not the same since 2004. WFS uses the SMB application protocol. There have been 8 different versions (dialects) of SMB since it was first released in 1983. In 2004 there was only SMBv1, which is an insecure plain text dialect. In 2006 the more secure SMBv2 was added to Windows Vista / Server 2008 providing encrypted communication. In 2017 the wannacry virus exploited a serious vulnerability in SMBv1 and the entire network industry has been trying to kill off SMBv1 ever since.

The current state of play is you can not assume any system released in the last 5 years will connect to a WFS share using SMBv1 without additional configuration. Most will not. Even Windows 10 and 11 automatically disable SMBv1 compatibility after it has not been used for 30 days.

Blaming Volumio is similar to blaming Goodyear for no longer making tires that fit a car manufactured in 1924. If you insist on clinging to old hardware and software that is no longer supported and updated by the manufacturer you can expect to jump through hoops to make it work. There are similar incompatibilities that affect old compromised versions of SSH and HTTPS (TLS) amongst others. In the light of global cyber attacks network technology is not standing still.

Volumio has the ability to connect to an older NAS but it does not do it out of the box. The connection parameters must be entered correctly and a NAS that only supports SMBv1 will require the correct options in the Advanced Options field provided by Volumio.
e.g.
Alias: NAS
IP Address: 192.168.1.2
Path: Music
File Share Type: cifs
Username: anasuser
Password: ***********
Options: vers=1.0,sec=ntlm,nodfs

I have provided the most common Options flags for an old NAS. You can find the full list of at:
https://manpages.debian.org/cifs-utils/mount.cifs.8.en.html
You will not need most of them but they all exist for a reason and it will show you network communication is perhaps more complicated than you think.

If tuning the connection options is too much messing around for you there is always the option of taking the hint and buying a new NAS.

Volumio will indicate when the share is successfully mounted but you will not be able to browse files until you update the Volumio index. It will take a very long time (many hours) for Volumio to index 36000 files. I recommend creating a share on the NAS for testing with a just a few music folders. Only index your 36000 files after successfully connecting and indexing the test share.

Plex uses an entirely different set of network protocols (DLNA/UPnP). DLNA/UPnP is much more complicated and generally much more troublesome than WFS. Plex caches extensively and requires a great deal more RAM, CPU cycles and bandwidth at both ends of the connection to build and maintain the index. A partial index is quite typical of an under-resourced Plex setup.

10

Indeed that is, but Iā€™m refferring not to people who have no knowledge of the outlet and hookup to a tv box, I reffered to, take old tv from table, place new one, hook it up and your good to go.

SMBv1 is also not on my device and I mentioned that I have two windows server, an older 2012 and 2019 running 14/7, and plex runs on the first.

What I canā€™t understand is the simple fact that when I use plexamp (the audio app for plex), it sees all the files, and I know, what you said, DLNA etc etc, but plex also reads the same files and folders, yet Volumio can read from plex, but only until the letter K (the artist King in this example) When I add another folder to my list of music with the letter A, the name ā€˜Kingā€™ is pushed out and not visible anymore.
Now others said somethings about the file format, I tend to keep all MP3 because I have a coder for that from last century and works well, so, this disappearing is logical but I do not understand why it just stops at ā€˜Kā€™
Also what was mentioned that the Pi might not have memory enough, totally understandable but why does the chrome page on my 64 GB pc do the exact same thing? That I can also not understand. Is that linked to the amount of memory on the Pi? because that would make sense, but if notā€¦
Then last the username/password thing I tried, I can not see if this works when I add a name and password and cifs/nfs, it would be nice if I directly get an reaction from the software if it connects properly, because I am not sure, I used GUEST without a password and only read acces to my serverā€™s music folder.

I am not mad, and impressed by the software, truely, but the things I need are not totally clear and I will on my part try this one of these coming days on another Rpi, and hopefully I have a Rpi 4 with 8 gig laying around.

Thanks for the help, momentarily Iā€™m using it to play webradio, and that goes rather good, but I really would also like to listen to my own libraryā€¦

Iā€™ll keep in touchā€¦

11

Hey @AvA,

Iā€™ve read through your post, and I wanted to summarize and address some of the points you raised. It seems like your main concerns revolve around perceived limitations in Volumioā€™s programming and technical design. Let me break it down:

  1. Technical Constraints
    There seems to be some confusion about the constraints within different technologies. Concepts like networking protocols, file shares, and system components such as MPD (Music Player Daemon) serve different purposes and have their own inherent trade-offs. These arenā€™t arbitrary limitations but necessary to ensure performance, compatibility, and stability across different environments and devices.

  2. Comparisons to Closed Systems
    Comparing Volumio, which is a modular and open system, to closed ecosystems like TVs isnā€™t entirely valid. Closed systems are tightly controlled by manufacturers, often tailored to very specific hardware and use cases. In contrast, Volumio is designed to be flexible and adaptable to a wide range of hardware and configurations, which naturally introduces additional complexities and constraints.

  3. DLNA and Media Services
    DLNA is a protocol designed for seamless media sharing and playback within closed ecosystems. While useful in specific contexts, it operates under a different paradigm compared to Volumioā€™s open approach. Volumio integrates with a variety of services and protocols, including DLNA, but it also prioritizes flexibility to cater to broader use cases, which might introduce different challenges compared to a dedicated DLNA server or player.

  4. Enhanced Functionality
    Your desire for features like better networking performance or enhanced file-sharing capabilities is valid. However, implementing these features often requires careful balance to maintain compatibility with diverse setups. For instance, file shares rely on proper configuration of SMB or NFS, and MPD integration must respect its design principles to ensure reliable playback.

  5. Trade-Offs in Design
    Every system has to make trade-offs to work efficiently in its environment. Networking and file-sharing protocols, for example, prioritize robustness and reliability, which can sometimes limit performance or flexibility. Similarly, logging is designed to provide useful diagnostic information while minimizing resource use, which is why more detailed logs only surface when problems occur.

If you could focus on specific examples of the challenges youā€™re facingā€”whether itā€™s networking issues, file-sharing errors, or something with MPDā€”we can dig deeper and work toward practical solutions. Letā€™s continue the discussion constructively to improve the experience for everyone.

Kind Regards,

12

Today I listened to a local radiostation, when it was time to switch to another it just stopped reacting.
Good programming! I quit, I stop using this. If I turn on another channel, Iā€™d expect that it works, even wireless, another wireless radio I have works all the time, so, Iā€™m just so tired of troubleshooting all the time.

13

Another thing I have to give you credit for is the fact that Microsoft had in the past a lot of network problems, but those are solved and todayā€™s networking works well, so when Linux came in the picture, I recognized the knowledge they picked up from other networking software and made a rather stable platform, but not every software hunk is trained and educated in this whole history, and small things can go wrong. I can now unplug the Pi, and reconnect it to make it work again. Is this the advertisement that every funny looking piece of software should hold up? or are they not follow strict rules. Cā€™mon, it is webradio, not 8K displayā€¦

I have no idea where to start and know not where it stops so my decision is to try this one thing when I have the time but I will not pursue action to make it play my library.

14

@AvA

Please keep in mind the difference between a DYI and OEM.
A DYI means some tinkeringā€¦

15

Hey @AvA,

I can understand your frustration when things donā€™t work as expected. Technology can sometimes feel overwhelming, especially when multiple issues seem to arise at once. However, mixing different problems together (e.g., network shares, media playback, and system reliability) makes it harder to identify the root cause of each issue.

As a technical enthusiast (or, as you say, a ā€œproper nerdā€), Iā€™m happy to help you troubleshoot these issues step by step. Hereā€™s my suggestion:

1. Focus on One Problem at a Time

Letā€™s start with one specific issue that youā€™re facingā€”whether itā€™s the local radio station not working, network shares not connecting, or something else. Please provide as much detail as possible, such as:

  • What exactly happened?
  • What were you trying to do?
  • Any error messages or unexpected behaviours?

2. Gather Technical Data

Logs are incredibly useful for understanding whatā€™s happening behind the scenes. If you can share Volumioā€™s system logs (e.g., /var/log/mpd.log), link to debug log, we can dive into the technical side of the issue and pinpoint the cause.

3. Addressing Perception

While our community volunteers canā€™t help with perceptions like comparing Volumio to other systems, they can help with the technical side. Volumio is an open, highly customizable system that sometimes requires a bit more tinkering than closed, pre-configured devices. This trade-off allows flexibility but can lead to occasional challenges.

If youā€™re willing to focus on one issue at a time, Iā€™m more than happy to help troubleshoot and find solutions. Letā€™s tackle this together and turn the frustration into a positive outcome.

Kind Regards,

16

First of all, much thanks for trying to help me. I decided to give it another shot, however, I found my only Raspberry Pi4, tried to access the mediaplayer Plex, but it gave the same amount of folders A-K, and missed the other half of my library.

On the mainpage of Volumio I seem to miss something critical, the ability to add a networkdrive etcā€¦ The settings are rather simple for users and no possibility to change or add a network folder, or try to get my musicfiles without the use of half my library through Plex.

Am I wrong? or whatā€¦

Letā€™s focus on that first, and see if you or anybody can make this work for me.

Thanks.

17

@AvA

My music library is a shared folder on a Windows Server 2019 box.
I assume you are doing something similar

To setup.

From the main Volumio window.

Click the top left hamburger button (3 stacked horizontal lines)

Click Sources

click + Add New Drive

Do not worry about scanning for network drive (its never worked properly for me)
Set it up manually.

Alias [call it what ever you want]
NAS IP Address [The IP address of the server with your shared folder]
Path [ only put the name of the shared folder (as it appears when browsing on another windows machine) NOT the complete path]

click Show Advanced Options

File Share Type [CIFS]
Username [ Enter a username that is needed to log into your windows server]
Password [ Enter the password that is needed to log into your windows server]
Option [leave blank]

Click Save and it should mount the drive and start scanning.
Go have a cup of tea and some biccies as this bit takes a while.

Good Luck

This is what mine looks like

Vnetwork
Vnetwork932Ɨ968 47.2 KB

18

I had the layout changed and could not find the hamburgerā€¦ rolled it backā€¦

Nope does not work. I added IP and path, as far as my knowledge tells me Iā€™m right but volumio sees different. It gives 535.?? Mb and that does not even comes close to the files, it is not mounted. I have over 250 GB music files!
Your example with the # and the ~ does not being recognized, invalid path, so why you used that instead of \\192.168.0.10\musicfiles is a mystery to me.

So either the devs make a {browse to server-folder} option or ā€¦ I give it all up, the X-th attempt of indie-a-like software that fails miserably in my eyes.
And I had over 10 kind of software programs for Rpi that failed miserably. Maybe on a PC it works, but I am not using a 18 wheeler to transport my grannie 100 meters further.

19

I looked for about an hour on the website and nowhere I could find the DYI. Even so, if Do Yourself In means that all info is on the internet, it fails to give this what is needed. The website however states that after installing I am able to listen to my own music within 10 minutesā€¦ Still waiting when those 10 minutes start to tickā€¦

20

Hey @AvA,

I have a suspicion that you may be affected by ill-formatted cue files. Can you run library rescan again and whilst the process is ongoing - can you access your player with ssh client and check:

sudo cat /var/log/mpd.log | grep cue

or

sudo su -
tail -f /var/log/mpd.log | grep cue

Kind Regards,