Alt.Binz forum
New Alt.Binz versions => Bugs => Topic started by: thebeing on June 06, 2011, 12:56:13 am
-
The following bug has happened in all versions of Alt.Binz I've used (Free version & 3.7.1 - 3.8.5). It has finally gotten annoying enough to report.
Occasionally the Par2 thread will hang and stop adding completed blocks to the Par2 tab. This also causes it to not add any future Par2 files downloaded to the Par2 tab and also fail to join/extract anything downloaded after that point. The problem has a tendency to occur where there are multiple files and Par2 joined into a single collection on the Download Queue tab. I've yet to see it occur when each file has its own entry on the Download Queue, so possibly that would be were to start looking for the bug. This most recent time it happened with a 25GB split file in a 58GB collection, but that is an extreme. The hang may happen more often with massive files, but I believe I've seen it happen with smaller files as well.
This will also cause files to fail to join, extract, and/or repair (including Alt.Binz automatically unpausing and downloading Par2 files needed for repair). In the last few revisions Alt.Binz now even fails to save the Par2 tab on exit when this hang has occured. I've looked at the log and see nothing interesting. Even though detail logging was checked, I see no mention of Par2 activity anywhere in the log.
This is all on WinXP SP3 x86. Other than what I've mentioned, I'm unsure how to reliably reproduce the bug, as it seems to occur at random.
-
As a follow-up of this, I reproduced it again, this time with ~175 split files (~10 parts each) in a single AltBinz collection on the Download Queue. After the 25th download had completed I checked Altbinz and the par2 thread malfunctioned again. On the par2 tab, only the first 10 downloads were shown as 100% blocks, 11-15 were shown with ~0.5% blocks, and no par2 for 16-25 were listed at all.
I paused the downloads, and shortly after AltBinz jumped to 100% CPU and appears to be attempting to recover, but something is still broken. It's been an hour paused now, and it's only joined 4 new files, and added added all the blocks for 3 files to the par2 tab. I'll leave it alone for more hours and see it is able to fully recover. I dread what is going to happen for the remaining 150 files... I may literally need to leave Altbinz eating 100% CPU for days after everything finishes downloading... In the past I've also seen that removing par2 files from the par2 will also hang when Altbiz is in this bugged state.
Rdl, I hope you are able to track down the cause. PAR2 bug? Threading bug? Something else? The computer I'm seeing the bug on has an AMD X2 4800+ (939) dual-core, if that matters. I'm convinced the bug is triggered when downloading many files in a single collection. Unfortunately, it also may be easier to reproduce than I originally thought.
Update1: It did eventually join and add the blocks from the completed files to the par2 tab, but the bug remained when I resumed downloading. No new par2 files were added while downloading. Pausing still resulted in Altbinz using 100% CPU, slowly reading files for a long time, attempting to recover.
Update2: The bug remained after exiting and re-opening Altbiz and resuming downloading. This suggests Altbinz has bugged the entire collection. The par2 files didn't disappear either, possibly since I let Altbinz do its thing until it was idle.
Update3: After further investigation with Process Monitor, it looks like Altbinz is getting stuck in a loop when parsing files. It is doing ~30 loops reading each file part in 640KB chunks (which happens to be the par2 block size) at 1280KB offsets for a total of ~1500 read operations per file part in this case. This may explain the the cause of 100% CPU usage, and why this thread continues to fall way behind the other threads. Now you just need to find out why this abnormal behavior is happening. When this bug isn't happening, I am able to download things in Altbinz with downloading, par2 checking, par2 repair, joining, extracting all happening in succession.
Update4: I tried splitting the uncompleted part of the collection into a new collection, exiting and re-opening Altbinz and resuming download, but I still have the bug.
Update5: This time I tried exporting the collection to an NZB, deleting everything from the Download Queue, and re-adding the NZB. Still bugged... This definitely does seem like a par2 bug. As I mentioned previously these are multiple split files within a single collection, 25MB per part, yENC, and the par2 files being reported by Altbinz as 640000 bytes block size & created by par2cmdline 0.4.
-
Update6: I rebooted, opened Altbinz, downloaded the next file, and paused. The problem got worse somehow. Now the par2 thread seems to be doing ~110 loops for each part in 640KB chunks (~4000 read operations @~11MB/s). At the rate it's going, it looks like it will take around an hour for Altbinz to add all the blocks to the par2 tab and join the split parts for just this single 250MB file...
Update7: I tried splitting a single file's split part into it's own collection, still bugged. Whatever this is, it's carrying over when splitting this collection in the Download Queue, no matter how big or small the split.
-
Clearly it's an issue with a particular post, so you'll need to upload the NZB somehere and link it here so it can be checked out. Try to isolate the problem to just one split file.
-
Update8:By removing all the par2 files from the par2 tab, deleting the next par2 file from the download directory, and re-downloading an NZB (which contains the par2) for just a next file, I was able to workaround the bug. Blocks were added to the par2 and the file was joined as parts were downloaded.
Update9:For the time being, I guess I'll just download the massive collections without any par2 in the par2 tab. Adding the par2 files one by one, seems to add blocks to the par2 tab and join the split files promptly, avoiding the bug.
-
Clearly it's an issue with a particular post, so you'll need to upload the NZB somehere and link it here so it can be checked out. Try to isolate the problem to just one split file.
Do you still think this is the case, taking into account Update 8 & 9? Adding the previously bugged exported NZB, without any par2 in the par2 tab works fine. This seems to confirm it's just a bug with par2 handling. Doesn't it?
-
Par2 works fine in 99% of cases, and there's no way of testing your case without actually trying the download, so an NZB is needed.
Edit: sorry I'm losing track of your updates. It seems that the clearest solution is not to dump everything in one collection (there's really no need to), and make sure each collection is downloaded to its own folder. Put together a test case NZB if you want a better solution for handling multiple par2s in the same folder.
-
I sent you a PM with the NZB created on binsearch which triggered the bug this time.
I do think it is related to some sort of behavior (like multiple par2 files in a folder as you mentioned), rather than a buggy NZB. What triggered the bug in my first post were 12 singular NZBs (of 12 DVD9 ISO files) from a different source, which I joined into a single collection with Altbinz.
When the bug is triggered, the solution appears to be removing all the par2 files from the par2 tab, deleting par2 in your download dir for files not yet downloaded, and re-downloading everything from scratch along with par2.
-
I've prepared the sacrificial virgins and summoned the Kraken to look at your NZB, so let's see. ;)
I doubt very much that you'll see this prob again if you apply a bit of discipline to using Alt.Binz 'cleanly', i.e. always keep one distinct posting in each collection (archive + pars, etc), and always download each collection to a separate folder (Setup > Download > 'Create subfolders based on NZB name') to keep things properly isolated.
-
I doubt very much that you'll see this prob again if you apply a bit of discipline to using Alt.Binz 'cleanly', i.e. always keep one distinct posting in each collection (archive + pars, etc), and always download each collection to a separate folder (Setup > Download > 'Create subfolders based on NZB name') to keep things properly isolated.
Even if that's the case, it would really be nice to see this bug fixed (I've run into it at least 20% of the time for the minimal things I download from usenet). Keeping everything isolated into its own folder means time spent manually moving files around and deleting folders, which has gotten annoying over time, and why I try to avoid doing so when possible... It's so much nicer to just download things which belong in the same folder, directly to said folder.
The bug can't just be multiple par2 files in the download dir, since I still had a lot of par2 in that folder, and dragging them in one by one works. Does Altbinz check par2 files sequentially or in parallel? Does Altbinz do anything when a new par2 file is downloaded and added to the par2 tab? If adding a new par2 to the par2 tab causes in-progress block checking to be interrupted in favor of the new par2, I could see how that could eventually cause things to cascade over time.
As for the endless par2 read loops I saw, it may suggest one of two things. par2 is working correctly, but Altbinz isn't listening when par2 reports the blocks it checked. Or par2 continues to loop because Altbinz is requesting block information and par2 isn't giving it the first time around. Since Altbinz used a modded par2 exe, I wonder if the looping behavior on purpose (aka a retry behavior) when par2 and Altbinz aren't communicating properly?
In any case, I'll try to figure something out. I wonder if just just making the par2 files the last things downloaded would workaround the issue.
I hope someone can reproduce, and this isn't some bug with my CPU or something. These AMD X2 (939) CPUs did have that processor core synchronization bug. Though using the /usepmtimer switch with the updated processor driver as I'm currently using should avoid the issue. Yet if this is indeed a threading bug, rather than a par2 bug, and AltBinz shares very tight timing data between cores/threads, I wonder if Altbinz requires the Dual-Core Optimizer driver. That Dual-Core Opt driver is basically a hack that makes an extra effort to guarantee the cores are always in sync sharing the same global clock, but being a hack, it conflicts with certain programs I use. If Altbinz is one of the rare applications which has the potential to suffer from this processor bug, maybe this would give Rdl an idea of what to fix. Doubtful that this is that issue, but worth mentioning for those not aware. I'll eventually get around to testing this possibility, but I don't see it as a viable long-term solution in my case.
What triggered the bug in my first post were 12 singular NZBs (of 12 DVD9 ISO files) from a different source, which I joined into a single collection with Altbinz.
Oops, I got confused, that was something else. The first post that got bugged were 16 singular NZBs for MKVs (7 were 20-70MB, 6 were 2GB-3GB, 1 was 6GB, 1 was 9GB, 1 was 23GB) joined into a single collection with Altbinz.
-
Well by contrast I would say that I have problems with par2 about once a month (usually a problem with the par2 files themselves). To repeat: there's no benefit at all with merging NZBs like you're doing, use the queue as it's designed to be used.
The standard setup for Alt.Binz is as I've described. Download to separate subfolders on one drive/partition, unrar/join to another, have Alt.Binz automatically remove your empty dirs, use article caching in RAM and be a happy bunny. There's no need to do anything manually, Alt.Binz is designed from the bottom up to be automated.
As for the rest, let's wait and see.
-
About that NZB I PM'ed you, I sorted it alphabetically in Altbinz since that NZB had everything in reverse order. This ended up with the par2 files getting placed after their respective split parts. This isn't particularly noteworthy, since the bug has also happened with par2 files in their normal location (before respective split parts), but if you want everything exactly the same...
I've yet to really fit unrar/join directory into my usage of Altbinz. It still takes some manual work to specify a unique unrar/join folder for a collection, doesn't it? It's not fun to specify folders one by one. I assume it's not currently possible to select multiple collections in the download queue, and set their download and/or unrar folder to the same unique directory, all at once? Or is it?
Unfortunately having everything dumped in a global directory for sorting, doesn't really work well with my current drive/volume organization & free-space strategy.
-
As I mentioned previously, I decided to just download the huge collection without NZBs already in the par2 tab, and just drag in par2 in batches for completed files. Well I just ran into a slightly different, but likely related, variation of the bug.
I dragged in ~15-30 par2 files so Altbinz would join those completed downloads. The first couple went fine, but then one of them only added blocks and joined the first split part without finishing, and then skipped to the next par2. Altbinz only checking blocks for 0-2 parts before skipping the par2 for the next, seems to be a common issue with this bug. If this is related to the cause of the core issue, it may be easier to reproduce the bug by just moving groups of completed files w/ par2 in a directory and dragging in groups of par2 files into Altbinz for checking, over and over again, while other things are downloading.
-
At this stage, you're really adding no new information at all. Who knows what you're piling into the same directory and what the potential collisions might be there.
Continue your unwise decision not to split your large collection into smaller ones and dl to subfolders, by all means, but there's really no need to keep posting about it.
-
Who knows what you're piling into the same directory and what the potential collisions might be there.
Continue your unwise decision not to split your large collection into smaller ones and dl to subfolders, by all means, but there's really no need to keep posting about it.
To answer this question of what I'm piling into the same directory, truthfully not much. I do always place each collection into it's own unique empty directory. The only potential collisions would be when I join multiple collections (usually of 2-15 files) with related items into a single collection and unique empty directory. In case I wasn't clear, this bug happens with very small collections as well. I've seen it occur after only two to three 250MB files (out of around five in a collection) were downloaded to the same empty directory. What this basically means, is I can never join 2 or more NZB collections (downloaded to an empty directory) or I risk experiencing this par2 bug. At least that is how it seems.
I do understand what you're saying, and will try out some of things you suggested so I can continue using Altbinz effectively. At this point I have indeed said all that can be said about this bug, that it really does exist, what may trigger it, and how much I would like to see it fixed. I apologize if all the posts were a bit much. I was only trying to document as much information as possible, no matter how trivial, since for all I know some little piece of info could become useful in tracking down and fixing the issue. Over the years I've done a lot of beta testing for various applications, so I've seen time and time again how difficult issues such as these can be to track down and fix, especially if the developer is unable to reproduce the issue themselves. That this bug has to some extent existed in every version of Altbinz, and if I'm the first to actively complain about it, it makes chances of it getting fixed all the less likely. I can only hope Rdl gets insanely lucky and has a break-through with an idea of how to fix it.
:-X :-X :-X Once again, sorry for all the trivial and semi-repetitive posts. I'll restrain myself and post no more in this thread, unless my feedback is required. :-X :-X :-X
-
Quick, more virgins needed to summon him!
(http://upload.wikimedia.org/wikipedia/commons/9/9d/Colossal_octopus_by_Pierre_Denys_de_Montfort.jpg)
-
2 month update:
The Par2 bug is definitely triggered by joining multiple NZBs (either on index sites or in altbinz) into a single collection. While it was a bit of an inconvenience, I took Hecks suggestion to never join NZBs into a single collection, have each NZB download to its own folder, and make use of the UnRAR directory functionality, I've not run into the bug again *knock on wood*.
-
Found another workaround to this issue which doesn't require avoiding creation of NZB collections, and downloading to separate folders.
If I move all the unpaused PAR2 & SFV for all the files to the very end of the NZB collection, I never get PAR2 hangs, and everything is checked/joined/unrared as normal after downloading completes. This takes trivially longer than checking at the same time as downloading, but never hanging is a major advantage.
Maybe having a "Move all PAR2 & SFV to the end of the queue" option would be useful, if a proper solution for the PAR2 thread hanging on NZB collections is never found.
-
The following bug has happened in all versions of Alt.Binz I've used (Free version & 3.7.1 - 3.8.5). It has finally gotten annoying enough to report.
Using ver 0.39.9 right now. (Win Xp, Intel i7 920, Asus p6t6 ws Revolution)
Yes, this is has been a definite issue for quite a few versions now. I need to find the last "stable" version or just write off Altbinz as useless and find an alternative.
Hard to know what kicks it off, but joining multiple pars of say all the episodes in a season into 1 is one way to do it. Just last night: mkv files of bluray rips 12 episodes +14 episodes +12 episodes. each episode had it's own nzb from teevee group, with each show joined. When I wake up 8 hours later, 1st 12 episodes checked & unrared, the next 14 par check is jung on ep 9 while all 14 are DLed + 3+ episodes of the next 12
Sooner or later the program just starts to eat CPU and go into molasses mode in checking the pars. If you're willing to wait for many hours, it will actually get finished checking the pars and unraring.
Nirsoft's ProcessActivityView show altbinz to be slowly reading through the files that were written to disk hours before. I have a jpg pic of this.
can you please fix this or point me to the last "stable" version of AltBinz
-
So, you're saying you had one collection with 38 par2 sets in it? Ep 9 files it appears to 'hang' on - were they complete?
May I suggest 38 collections each with one par2 set instead.
In any case I'll take a look as soon as I can.
-
What dh2 describes is identical to the issue I originally reported.
100% CPU bug combined with extremely sluggishly reading files of files, which loops ~30 times per par2 set @ KB/s speeds.
Files which normally would take less than a minute to check, suddenly take hours.
The issue never occur when you have only a single par2 set per nzb collection.
The more par2 sets you have in a nzb collection, the more likely it is for the problem to occur.
More often than not, the par2 issue will be triggered when checking a file with 0 missing blocks (100% complete). Likely no relation.
-
One more question. Were all 38 par2 files loaded (pushed in front on import) before download of the main files?
-
So, you're saying you had one collection with 38 par2 sets in it? Ep 9 files it appears to 'hang' on - were they complete?
May I suggest 38 collections each with one par2 set instead.
what I quoted was just the latest.
1 collection of 12 joined nzb you (do have 'join' like joinging all the episodes of a season so that they all end up in the same folder)= 12 pars files, with par being checked right after 1st of the 12 parts is DLed while next is being DLed. Collection 2 14 nzb joined = 14 par files, & collection 3 12 joined nzb = 12 par files
each par file es DLed after each episode is in the 'joined' collection
It could've gone into molasses mode at anytime while I was sleeping. This is just one example of a problem that has been coming up for quite a while. I can't remember at which version that it started. And looking at the bugs reported lately, that it is actually the same problem that people are trying to explain in their own way.
-
In my case, I've seen it occur both ways. I normally have the "move small par2 files to front of queue" option enabled, but occasionally I would join a NZB within Alt.binz. The only way to fully avoid the problem would seem to be having nothing downloading from a particular NZB collection when the PAR2 files from that collection are being checked. Hence my "move small par2 files to end of queue" suggestion as a potentially effective workaround if a fix cannot be found.
I suspect that it may only have a chance of triggering when downloading gets ahead of par2 checking the previous file. In other words, at least 2 par sets which need checking from the same NZB collection, are in the PAR2 tab at the same time. That said, for the past year this issue has caused me to go out of my way to avoid using NZB collections in Alt.Binz, so aside from once again trying an NZB collection and experiencing a problem a few weeks ago, the details aren't as clear in my mind as when I originally reported it.
-
And few more questions:
-Once it starts going slow through files, how many par2 sets are there in par2 tab.
-File that is currently processed (using Nirsoft's ProcessActivityView) by altbinz. Is there such named file in any par2 set loaded by altbinz?
-Once that file is finished - was that file complete 100% or not?
-
I've seen it occur after only two to three 250MB files (out of around five in a collection) were downloaded to the same empty directory.
For how many par2 set in the par2 tab, it would depend how quickly I noticed.
After further investigation with Process Monitor, it looks like Altbinz is getting stuck in a loop when parsing files. It is doing ~30 loops reading each file part in 640KB chunks (which happens to be the par2 block size) at 1280KB offsets for a total of ~1500 read operations per file part in this case.
I've never used Nirsoft's ProcessActivityView, so I don't understand that question. But I did previous use Sysinternals Process Monitor with an AltBinz.exe filter, and the file currently being checked/read by the par2 thread would fill the log. Though I'm unsure if that's what you were asking or not.
If a file happened to be download incomplete with missing parts, and this par2 checking issue occurred, altbinz would not make any attempts to automatically download additional par2 files and repair. If par2 files needed for repair were downloaded manually, they would not be added to the par2 tab under the small par2, nor checked.
-
And few more questions:
-Once it starts going slow through files, how many par2 sets are there in par2 tab.
-File that is currently processed (using Nirsoft's ProcessActivityView) by altbinz. Is there such named file in any par2 set loaded by altbinz?
-Once that file is finished - was that file complete 100% or not?
1. I don't know at what time it went into slow mode this time ( i was asleep) but when I became aware of it around 20 par set had been verifiied and unrared. about 3 par sets were loaded, red in color, with no progress on them. And at least 9 more sets (about 2.2GB of rar w pars file) had been already DLed to HD without showing in the par2 window.
2. No; altbinz is SLOWLY reading thru files (rar) that it had already written to HD hours before. I can manually check the pars of a set already written to HD and all is fine; checking pars of file that Altbinz is currently "slow" reading gives a pars result of "unkown".
I can manually unrar a set that the slow reading hasn't come to yet with no problem, but when manually unraring a set the altbinz is slowly reading through results in winrar asking for that particular rar file as though it is not present. Waitin a few seconds til altbinz is finished with it, saying OK to winrar and it goes on, til it reaches the next file that altbinz is slow reading.
3. None of the files have problems before or after. I'm sure if I had just let altbinz work it's slow ass on, that a lot of hours down the line that it would've been done.
if I had figured out someway to explain the problem in written form I would've reported it long time ago, cause even now I do not do a good job of it. I do have a ProcessActivityView log file that I saved some weeks ago: here is a sample of what is in it when altbinz goes into molasses:
"18:55:34.3436076","altbinz.exe","2896","ReadFile","Z:\Main\00.Daily\Modern.Family.S03E01-24.BDRip.XviD\modern.family.s03e22.bdrip.xvid-demand.r04","SUCCESS","Offset: 0, Length: 768,000"
"18:55:34.3678107","altbinz.exe","2896","ReadFile","Z:\Main\00.Daily\Modern.Family.S03E01-24.BDRip.XviD\modern.family.s03e22.bdrip.xvid-demand.r04","SUCCESS","Offset: 768,000, Length: 384,000"
"18:55:34.3905134","altbinz.exe","2896","ReadFile","Z:\Main\00.Daily\Modern.Family.S03E01-24.BDRip.XviD\modern.family.s03e22.bdrip.xvid-demand.r04","SUCCESS","Offset: 1,152,000, Length: 384,000"
"18:55:34.4131893","altbinz.exe","2896","ReadFile","Z:\Main\00.Daily\Modern.Family.S03E01-24.BDRip.XviD\modern.family.s03e22.bdrip.xvid-demand.r04","SUCCESS","Offset: 1,536,000, Length: 384,000"
-
But that sample is 70ms long. Based on that I would estimate around 1sec for whole file?? It would help if you got whole log for slow processing file. In this one I don't see anything unusual. First 2 blocks read and then block by block. I really need to know if that slow file was complete or not and if there was par set (for that file) loaded before slow checking of that file.
-
But that sample is 70ms long. Based on that I would estimate around 1sec for whole file?? It would help if you got whole log for slow processing file. In this one I don't see anything unusual. First 2 blocks read and then block by block. I really need to know if that slow file was complete or not and if there was par set (for that file) loaded before slow checking of that file.
I can't see how to attach the log file here to my post
The whole log file is 12675 lines long just the processing (slow reading) of 1 15mg par file is 1887 lines with start time "18:55:36.4668250"
and end time of "18:56:21.5671605". That's 45 seconds for a 15MG, which usually takes less than 1 second when all if fine and par files are checked.
-
What dh2 describes is identical to the issue I originally reported.
thebeing,
Thanks for starting this thread and not giving up on it!! I seldom checked the forum, and didn't know how to describe the problem, which has bothered me for some time. It's only thanks to your post, and that I saw what you had written, that I decided to add my 2 cents worth.
I think some of the other 'bug' posts recently have been other users' attempts to describe the same problem, without much success.
-
But that sample is 70ms long. Based on that I would estimate around 1sec for whole file?? It would help if you got whole log for slow processing file. In this one I don't see anything unusual. First 2 blocks read and then block by block. I really need to know if that slow file was complete or not and if there was par set (for that file) loaded before slow checking of that file.
I can't see how to attach the log file here to my post
The whole log file is 12675 lines long just the processing (slow reading) of 1 15mg par file is 1887 lines with start time "18:55:36.4668250"
and end time of "18:56:21.5671605". That's 45 seconds for a 15MG, which usually takes less than 1 second when all if fine and par files are checked.
Please upload log file somewhere and send me the link via PM.