Your best friend for file transfer.

Fetch application logoFetch

Mirror download order (15 posts)

  • Started 10 years ago by Justin
  • Latest reply 10 years ago from Justin
  • Justin Member

    I need to know is there a way that fetch can be set up to download the most recent files first?

    is there an order that fetch downloads in? as its annoying that it has to download a whole folder when it mirrors and i can set the order for this?

    Posted 10 years ago #

  • Jim Matthews Administrator

    Fetch downloads the contents of a folder in alphabetical order. When you do a mirror download Fetch has to check the modification dates of files first, so it knows which files have changed.

    Thanks,

    Jim Matthews
    Fetch Softworks

    Posted 10 years ago #

  • Justin Member

    Is there any way to sets it so fetch checks for modified folders within a directory and then downloads whatever folders have been modified opposed to files as i have 3000 folders with 11,000 files which takes hours to scan??

    Posted 10 years ago #

  • Jim Matthews Administrator

    No, there isn't a way to do that. I'm not sure it would help, because downloading takes longer than scanning.

    Jim Matthews
    Fetch Softworks

    Posted 10 years ago #

  • Justin Member

    if it only had to check to see when the folders were last modified and not the files in the folders then it would skip a lot of checking and would speed the whole process up if the directory contains lots of folders with files in.

    Posted 10 years ago #

  • Jim Matthews Administrator

    Downloading a folder requires first listing the contents, so unfortunately there's no way to skip that step.

    Posted 10 years ago #

  • Justin Member

    is there a way that i can create a log file that it can check instead of it having to scan everything?

    Posted 10 years ago #

  • Jim Matthews Administrator

    Hi,

    If you know which files need to be transferred, you could write an AppleScript that would only transfer those files. I'd be happy to help with that.

    Thanks,

    Jim Matthews
    Fetch Softworks

    Posted 10 years ago #

  • Justin Member

    Hi Jim,

    It would be useful if there was an applescript which created a log file on the directory which is say called "last90days.log" which fetch checks its modified dats of "last90days.log" if its not been modified in the last 24 hours then leave alone and just check for last 24 hours of files using the log at a point of reference in some way?

    Justin

    Posted 10 years ago #

  • Jim Matthews Administrator

    So the logic would be:

    If "last90days.log" has been modified in the last 24 hours, do nothing
    Otherwise, mirror a folder

    Is that right? What would cause the "last90days.log" file to be modified?

    Thanks,

    Jim Matthews
    Fetch Softworks

    Posted 10 years ago #

  • Justin Member

    Hi Jim,

    I need to run hourly mirrors instead of checking the files in the folders it could just create a log of the folders and then next time check the last90days.log or last30days.log and then skip them folders and only check the modification dates of the most recent files and folders.

    This is how i see it speeding up the process?

    Justin

    Posted 10 years ago #

  • Jim Matthews Administrator

    Hi,

    I'm afraid I still don't understand. Does your system automatically update the last90days.log and last30days.log files? What do the contents of those files look like?

    Thanks,

    Jim Matthews
    Fetch Softworks

    Posted 10 years ago #

  • Justin Member

    Hi Jim,

    So how i see it, applescript checks to see if there is a last90days.log or last30days.log if the modification date is more than 90 or 30 days it skips the log

    The log contains a list of the files and folders which have not been modified for more than 90days so when it checks that it knows which files and folders to skip.

    fetch then updates the log if its been more than 24 hours since the last modification date of the log

    Justin

    Posted 10 years ago #

  • Jim Matthews Administrator

    OK, I think I understand now. Essentially you are saying that if a folder has not been modified in 90 days or 30 days, then there's no need to check it more often than once a day. So when one of those folders does change, it could be 24 hours before the change is noticed.

    That sort of system would definitely speed up hourly mirrors when many or most of the folders don't change very often. I think it would be a fairly complicated AppleScript, however.

    I will think some more about ways that we could speed up mirroring.

    Thanks,

    Jim Matthews
    Fetch Softworks

    Posted 10 years ago #

  • Justin Member

    Thats pretty much it :)

    Well let me know if you can think up another way, as the way its working at the moment works but could be so much faster :)

    Justin

    Posted 10 years ago #

Reply

  • Or nickname, if you prefer.
  • This will be kept confidential.
  • This is to ensure that you’re a person, not a spambot.