I've got a screen scraping script in PHP on the GoDaddy shared Light server running via command-line.
The script scrapes, parses and stores the needed information inside a database. It takes approximately 1.5 seconds for the whole process per page, and requires to scrape near to 10,000 pages (as well as for each one of the pages, fetch snacks from two others, which makes it a maximum of 30k pages which are
The whole script will require about 5 hrs to operate. I've done some memory profiling, and memory consumption stays pretty much constant through the run - it doesn't increase.
Basically would run the script overnight, would GoDaddy notice something abnormal about this? CPU consumption shouldn't be an excessive amount of but exactly how bad would the bandwidth use of fetching 3 pages per 1.5 seconds for any amount of 5 hrs be? Enough to boost sensors on GoDaddy's finish?
If so, I guess I possibly could split up the script to operate through 1500 pages, after which halt for just one hour after which resume. Must I do this?
With regard to not departing the question un-answered, I'll publish the solution:
I went the script overnight. It required about 5 hrs to operate also it was neither ended by GoDaddy nor did I get any notice, and so i guess it had been fine together.
Initially I had been getting memory issues in which the script would exhaust the memory allotted in my experience, but apparently which was a pre-PHP 5.3 bug (more particulars on that here). Once fixed, it hovered at 32-34MB RAM usage the whole while. Not a clue about CPU comsumption or bandwidth usage.