Your best friend for file transfer.

Fetch application logoFetch

Generate Fetch Item Report stalls in Automator (3 posts)

  • Started 2 years ago by Garlon
  • Latest reply 2 years ago from Jim Matthews
  • Garlon Member

    I have a workflow created in Automator 2.1.1 on Snow Leopard, that uses the Automator actions from the most recent Fetch 5.7 (Just downloaded it last week). My workflow gets all files and sub-folders from our remote live images directory and outputs that to a txt file. It has worked fine on around 1,300 files and generated a text file in about 1.5mins. We have a very big directory that has over 85k files, and it seems to either take a long time to generate the text file, or get stalled in the "Generate Fetch Item Report" step.

    I've ran it a few times on the whole directory and the same results happen — I wait for about 4-5hours and nothing happens. I even let it run over this last weekend and it still was stuck at "Running - Generate Fetch Item Report". Is that too much of a load for it to handle, or is it just taking its time?

    Workflow process : "Get Selected Fetch Items > Get Fetch Folder Contents (repeat for each sub-folder found) > Generate Fetch Item Report (Date, Name Ext., Item Type, Path) - Tab-delimited w/field names > OS New Text File"

    "Get Fetch Folder contents" took 3287.549 seconds on the last time I ran it. That step always completes.

    Thanks,

    Posted 2 years ago #

  • Garlon Member

    I also wanted to add this from the Fetch transcript. It's an extremely long transcript so I'm only adding the last tidbit where I see it start stalling/looping.

    SSH2_FXP_READDIR 45046
    drwxrwxr-x 2 main web 512 Jan 2 2003 .
    drwxrwsr-x 62 main web 2560 Jan 20 12:45 ..
    -rw-rw-r-- 1 main web 34050 Feb 11 2003 city1.gif
    -rw-rw-r-- 1 main web 33060 Feb 11 2003 city2.gif
    -rw-rw-r-- 1 main web 37712 Feb 11 2003 city3.gif
    -rw-rw-r-- 1 main web 33544 Feb 11 2003 city4.gif
    SSH2_FXP_READDIR 45047
    SSH2_FXP_CLOSE 45048
    SSH2_FX_OK 45048
    SSH2_FXP_REALPATH 45049, /stuff/images
    /stuff/images
    SSH2_FXP_STAT 45050, /stuff/images
    SSH2_FXP_REALPATH 45051, .
    /stuff/home/main
    SSH2_FXP_REALPATH 45052, .
    /stuff/home/main
    SSH2_FXP_REALPATH 45053, .
    /stuff/home/main
    SSH2_FXP_REALPATH 45054, .
    /stuff/home/main
    SSH2_FXP_REALPATH 45055, .
    /stuff/home/main
    SSH2_FXP_REALPATH 45056, .
    /stuff/home/main
    SSH2_FXP_REALPATH 45057, .
    /stuff/home/main
    SSH2_FXP_REALPATH 45058, .
    /stuff/home/main

    " ... (I ran it again over night and its looped all the way to the realpath below and is still going "

    SSH2_FXP_REALPATH 46037, .
    /stuff/home/main

    "... (and it's still going)"

    I've ran the process with passive mode transfer on by default originally, then ran it turned off. I also have "Use 'LIST -al' command to reveal hidden items" off. It was on by default so that option was tried as well.

    Edited 2 years ago #

  • Jim Matthews Administrator

    Thanks for reporting this problem. The Generate Fetch Item Report action is very slow with such long file lists.

    One option is to replace it with the Run AppleScript action and the following script. I ran this script on a 10,000 item folder and it finished in one hour. I don't know that it would handle your 85,000 item folder in 8 1/2 hours, but it might be worth a shot.

    Here's the script:

    on run {input, parameters}
    set outputText to "Name Name Extension Item Type Path" & return

    repeat with oneFetchItem in input
    using terms from application "Fetch"
    set outputLine to (name of oneFetchItem) & tab & (name extension of oneFetchItem) & tab & ItemTypeToString(item type of oneFetchItem) of me & tab & (path of oneFetchItem) & return
    set outputText to outputText & outputLine
    end using terms from
    end repeat

    return outputText
    end run

    on ItemTypeToString(inItemType)
    using terms from application "Fetch"
    if inItemType is isFile then
    return "File"
    else if inItemType is isFolder then
    return "Folder"
    else if inItemType is isLink then
    return "Alias"
    else
    return "Unknown"
    end if
    end using terms from
    end ItemTypeToString

    Thanks,

    Jim Matthews
    Fetch Softworks

    Posted 2 years ago #

Reply

  • Or nickname, if you prefer.
  • This will be kept confidential.
  • This is to ensure that you’re a person, not a spambot.