vendredi 27 mars 2015

scandir() in php far too slow



The target directory has 10 million+ text files. using $a = scandir() in a web page is deadly slow. Need array results in less than two seconds. Filtering does not work (scans the entire list too)


all I can think of is to use a perl or c program to preprocess and stuff x thousand file names from the target directory into a file, tag the filenames in the target dir picked with a .pi at the end (or something) and use php's file() function to get the list from the file instead.


I need to open and work with each file before it gets stuffed into a table. fyi. I can't wait more than 1-2 seconds for the array to work on to be available. Any assistance appreciated. Memory is not an issue. hdd space is not an issue, processor power is not an issue. the issue is getting a list in an array Fast while using a webpage front end. I can't wait because i am tired of waiting.


I tried using a brief fast c program with opendir and readdir but even it takes almost 4 minutes to scan the directory list. at least I could put a governor on it to limit to x files.


It seems the answer is to call the perl or c program which I can limit to x files and I can call this with system() or backticks. Then that list can be opened with file()...OTF... makes sense?




Aucun commentaire:

Enregistrer un commentaire