Author Topic: autosave download queue  (Read 4228 times)

Offline DipDancer

  • Contributor
  • ***
  • Posts: 16
autosave download queue
« on: October 27, 2010, 09:44:48 PM »
which time intervall should I use for that?

my problem is that my queue is usually huge and atm it takes between 3 to seconds to save, every 2 minutes. thats quite anoying when using the computer for other stuff during that time.

setting it to every minute should only make things worse, I guess.

is there any timeframe I can set, so that the saving happens faster than those 3-7 seconds?

Offline Hecks

  • Global Moderator
  • *****
  • Posts: 2011
  • naughty cop
Re: autosave download queue
« Reply #1 on: October 27, 2010, 10:16:13 PM »
There really are better ways of managing a large queue than leaving everything on the queue tab (ask yourself why this is even necessary .... *seriously*)  But I've yammered on about that in other threads, so I won't bore you any more with it. 

I say 60 minutes, or any interval for which it wouldn't be too much of a pain to recreate lost queue items in the unlikely event of a crash. Saving time is going to reflect the size of your queue, whenever you save it.
« Last Edit: October 27, 2010, 10:18:01 PM by Hecks »

Offline DipDancer

  • Contributor
  • ***
  • Posts: 16
Re: autosave download queue
« Reply #2 on: October 27, 2010, 11:50:33 PM »
There really are better ways of managing a large queue than leaving everything on the queue tab (ask yourself why this is even necessary .... *seriously*)  But I've yammered on about that in other threads, so I won't bore you any more with it.
havent seen any of those threads, so which are these?

reason for having them all in the queue tab is I want to have an overview sorted by age, for how long the file have already been uploaded.
« Last Edit: October 27, 2010, 11:52:59 PM by DipDancer »

Offline Hecks

  • Global Moderator
  • *****
  • Posts: 2011
  • naughty cop
Re: autosave download queue
« Reply #3 on: October 28, 2010, 12:13:47 AM »
But seriously, why do you need to know that? Or, why do you need to do that for an entire queue all of the time? How long are these items in the queue? If it's more than a day, you may as well save the NZBs to dated folders, and load them up when you actually want to download em.  If it's just for information, searching Binsearch for post date takes 2 seconds.  It's a question really of breaking down what exactly the real use case here is, and asking yourself if it's really worth the overhead.  But hey, it's not really for me to say how anyone here should use Alt.Binz, so you can safely ignore me.

What I *do* know though is that AltBinz saves the whole queue to disk each autosave, serially, in one file plus backup. There's no system for a 'virtual' queue (using disk caches for example), and the queue is not bult on a database structure (though it should be I think), so there's no optimized, partial loading & saving to memory & disk.  It works best when it's cleared as quickly as possible, and not used as a local index.
« Last Edit: October 28, 2010, 12:17:37 AM by Hecks »

Offline Rdl

  • Administrator
  • *****
  • Posts: 3506
Re: autosave download queue
« Reply #4 on: October 28, 2010, 11:44:49 AM »
I already have 90% of the code currently not beeing used that should solve this problem.
BTW Hecks, I'm interested why do you think a queue should be built on a database structure?

Offline Hecks

  • Global Moderator
  • *****
  • Posts: 2011
  • naughty cop
Re: autosave download queue
« Reply #5 on: October 28, 2010, 07:56:02 PM »
Just thinking that the queue list is basically a table-like structure as it is.  Depending on which DB engine is used and how well it might work with the Queue list control, to be able to update the DB via buffers with only the rows that have been added or removed (and with autosave checking integrity only) would be an advantage at I/O level, I would imagine. And DB repair could be an option too (again depending on engine). Also maybe some extra functionality, like extended queue history (so the current 'active' Queue list would represent the results from a query for rows with that status flag, and 'not active' history could be viewed/searched with a different query). Scrolling beyond a certain point could start a new query, or just 'page' the whole list dynamically, etc.

I have no idea what the Delphi DB options are, tbh, but I'm guessing Borland has plenty of stuff there to play with in terms of optimized memory and disk access etc. ;)

In sum: adding to queue would basically become a process of replicating from one DB structure (remote indexer) to another (local) via xml, with all the extra flexibility that this might bring.
« Last Edit: October 28, 2010, 08:10:26 PM by Hecks »

Offline Rdl

  • Administrator
  • *****
  • Posts: 3506
Re: autosave download queue
« Reply #6 on: December 01, 2010, 06:03:13 PM »
I already have 90% of the code currently not beeing used that should solve this problem.

Done

Offline DipDancer

  • Contributor
  • ***
  • Posts: 16
Re: autosave download queue
« Reply #7 on: December 03, 2010, 10:28:20 PM »
thank you  ;D

Offline Rdl

  • Administrator
  • *****
  • Posts: 3506
Re: autosave download queue
« Reply #8 on: December 05, 2010, 07:49:11 PM »
I would like to hear some real expirience with large queues inhere