FAST Sitemap Generator for site with 1 million records

I have a website with 1 million + records. It's a dynamic website with the overwhelming majority of its pages being dynamic.

I need a sitemap generator that will create full sitemaps for my site in under 24 hours. It must be very well optimized so that it doesn't eat up all of my server's resources. I should also be able to put it on cron jobs in cPanel so I can add the latest entries in the database. I should have the ability to automatically ping Google when new entries are added to the sitemap and select priority for pages. It should also report 404 links to me where I can view them in a log file.

I currently have a sitemap generator but it takes about 2 days for a full run and the last time I ran it, some of sitemap files didn't generate properly. If you want to modify the existing generator, that's fine too.

Just to confirm, I should have the option not only to see the broken links, but of course to exclude them from the sitemap generator so that it contains working URLs only, while at the same time giving me a chance to fix the broken links.

The website is written in PHP and the database is MySQL.

Linux MySQL PHP Python UNIX

Nº du projet : #1076005

À propos du projet