In FRD 0.9u4 YouTube.com plugin v 2.9.5 fails to mux a large .m4v & .m4a

The video is at https://www.youtube.com/watch?v=zJ7hUvU-d2Q, I think the problem may have to do with the large size of the .m4v file (6 550 033 422 bytes on the "Highest available" preferred quality). In any case, download of the .m4v file finishes normally, and so does that of the .m4a file (it seems), but then, as the program creates the 0 bytes .mp4 file and writes something to the HDD, the reported state of the .m4a file changes from "COMPLETED" to "ERROR - Java heap space" and it goes into an endless loop of restarts.



Edited 1 time(s). Last edit at 10/16/2016 02:52PM by naaloh.

For large files, please use the ffmpeg method: [wordrider.net]

What is the max size that the plugin can handle and why will you not fix?

The max size? It depends on java max heap allocation. I think I managed to merge over 1GB video with 512MB max heap, but I forget the details, it was long time ago. See here to increase java max heap allocation.

Why will we not fix it? The youtube plugin depends on mp4parser plugin, it is a useful program to handle mp4. There are some limitations with mp4parser. Some were fixed internally by our team, on some problems we use workarounds (you can read the whole conversations about fixes/workarounds here if you are interested). But there are still some problems which cannot be fixed, the heap memory problem is one of them.

The ideal solution IMO is by using external program (ffmpeg, avconv, etc) to do the merging. Not only it's more stable/battle proven, it also performs better than the internal java solution (mp4parser). Also there is no need to reinvent the wheel, even worse when the resulting wheels are square But there are problems with this solution, mainly: distribution.

My personal solution at this moment is by using a (bash) script to (batch) merge the videos with ffmpeg/avconv. Which then can be used/integrated in file manager.



Edited 1 time(s). Last edit at 10/15/2016 03:41PM by tong2shot.

Thanks for the "max heap" tip, but in what way do the external programs "perform better"? I hope you don't mean that the mp4parser plugin can in some circumstances produce corrupt files...

Producing corrupt files? No, not in my experiences anyway. I have merged hundreds video files (small or medium size, less than 1GB ) with internal mp4parser, and none corrupt file produced.

I was talking about the amount of time to finish the job. For example (with same sample) from here:
- ~2.5 minutes with 512MB heap.
- ~10 minutes with default configuration (160MB heap).
- ~1.5 minutes with ffmpeg/avconv.



Edited 1 time(s). Last edit at 10/15/2016 04:16PM by tong2shot.

After being started with max heap set to 512MB FreeRapid Downloader was able to combine video and audio into a 6 890 899 025 bytes .mp4 file, but the 5:57:50 hours video stops playing at about 3:42:47 hours, which means (as simple calculation shows) when the player accesses the data beyond 4 gigabytes of the video file, which is surely not a coincidence. Obviously not the player's fault (I tried several), so looks like the mp4parser plugin really can produce corrupt files.

I have not tried combining the files with FFmpeg yet, because I (obviously) lost both .m4v and .m4a and don't feel like re-downloading right now.

Thanks for the information. I'm not sure whether it has been fixed in the upstream (https://github.com/sannies/mp4parser) or not, I haven't checked it yet, my google-fu fails me this time. Probably related to this issue: [github.com], but I'm not sure.

To download 6GB file will take ~15 hours with my home connection, I'm not sure I'm going to do that to test and find the bug in the library (mp4parser).

I think we probably should stick to the ffmpeg method for large files.

Thanks again, and sorry for the inconvenience.

I think having FRD doing the job would be much better than relying on third-party software.

By the way, is it really necessary to have the input files to find the bug? I mean, although I know next to nothing about Java programming, a "4GB" issue looks like something related to data type limit (32 bit unsigned)...

We rely many thing on third-party softwares. In case you don't realize, we rely on objectDB for database stuff, we rely on apache httpclient for http stuff, and so on and so forth.. Otherwise we should/need to (re)write the whole things in-house, which is too much, for obvious reason.

FRD is a general purpose downloader, IMO it has none or little businesses on media encoding, decoding, merging, etc, as it should be.

Is it really necessary to have the input files to find the bug? Yes.