Fermé

Scraping Expert for XML Sitemap Generator

I would like to build an XML sitemap generator in either Python or other language that is preferred by the developer for this type of project.

Here are the features I would like to have for the generator:

- Pattern-matching to check for parallel pages in other languages/locale

- Configurable patterns based on the site URLs

- Ability to configure to crawl only specific directories

- Ability to disallow from crawling parameters

- Cloud-based GUI to add URL and start crawl (I can host on my VPS)

- Menu to export crawl results, configure crawl URLs, disallow parameters, etc. (See Screaming Frog)

- Crawl large sites up to 1,000,000 pages

- Check http status codes and ensure all 200 status

- Only add https URLs to sitemap and if http, check https for a page to add

- Check box for adding video, images, etc. to sitemap either combined or separate sitemaps

- Build sitemap index for large sites and then link off to each associated locale sitemap

- Option to divide sitemap based on URL patterns (Have a certain directory or part of the site in each sitemap)

- No more than 50,000 URLs per sitemap

Compétences : Amazon Web Services, Linux, Python, Architecture Logicielle, Web Scraping

en voir plus : xml feed generator phpnuke, xml file generator, generate xml sitemap script, build script asp will give xml sitemap, sitemapclassphp google xml sitemap, buy sitemap generator php script, google xml sitemap admin cre loaded, xml pdf generator web fop, xml document generator, php web scraping expert needed, google xml sitemap fehler, google xml sitemap, xml sitemap generator, standalone xml sitemap generator page, inurlphp xml sitemap generator, standalone xml sitemap generator, premium xml sitemap generator industrial, xml sitemap wordpress plugin php generator

Concernant l'employeur :
( 0 commentaires ) Novosibirsk, Russian Federation

Nº du projet : #20284625

16 freelance font une offre moyenne de $479 pour ce travail

sky930320

Dear Sir. Glad to meet you. I'm Web developer specializing in web scraping crawling and indexing web pages. Skills: python, scrapy, selenium, requests, beautifulsoup, mechanize, lxml, urllib2, automation, bots, spide Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(24 Commentaires)
6.3
developer2581

We can take your work for fulltime, as we can provide you with the dedicated react native developer for your project. We are ready to start as soon as we discuss the details. We can provide you with the quality remote Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(21 Commentaires)
6.5
adampohp79

Hi. Glad to meet you. I'm Web developer specializing in web scraping crawling and indexing web pages. Skills: python, scrapy, selenium, requests, beautifulsoup, mechanize, lxml, urllib2, automation, bots, spiders I wor Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(66 Commentaires)
6.1
mituld

Hello! I am Scraping Expert for XML Sitemap Generator developing expert and have good experiences with Android, iPhone, Mobile App Development... This project is very interesting for me and perfect fit with my skill Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 15 jours
(41 Commentaires)
6.2
farooq4161

Greetings, I am an experienced professional scrapper and have done similar projects in the past. Same can be verified from my profile. Let me allow to assist you with your requirements. Thanks

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 6 jours
(66 Commentaires)
6.2
zeke

Dear Customer! I have built many scrapers. This is my favorite type of job. I am absolutely confident I can complete your project easily and on time. Ready to begin immediately and finish as soon as possible. Please co Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(59 Commentaires)
6.9
hassanalvi95

Hi sir, I am expert in scrapping data and made scrappers for Amazon, eBay, Adidas, and Nike. I made successful scrappers which are running 24/7 and downloading data for my clients. I need some more information from yo Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(38 Commentaires)
5.8
coolbratan

Hi, I am 10 years experienced scraping and developing expert. I have done hundreds of similar projects. Accuracy is my main goal! Hire me and you will be satisfied for 100%!

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(25 Commentaires)
5.5
smsaurabhv

‌Hi, I have gone through your requirement to scrape lots of websites. I am EXPERT in building scraping tools /scripts. Hence, I can SURELY work on your project. I am having 4 YEARS of EXPERIENCE in developing PHP-PYTHO Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(62 Commentaires)
5.3
shivansh121

Hi Employer, I can also extract any data from any [login to view URL] I have scraped the data from these websites & stored scraped data into CSVs and excels as per my clients requirement & I have made desktop applicatio Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 6 jours
(11 Commentaires)
4.1
athminteam

Hi, This is Vipin.I have 6+ years of experience in customized software,web scrapping, web and mobile app development and has expertise in Android, iOS, Python, Magento, PHP, HTML, Java, Angular and Ionic framework. Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(7 Commentaires)
3.3
micepapai

Hi, I'm a full-stack software developer specialized in Python and PHP for the backend and Javascript for the frontend. Why do you want to use a spider/scraper to generate the sitemap? If it's your site, it should be e Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 3 jours
(1 Évaluation)
2.7
techlinesols6

Dear Prospect Hiring Manager. Thank you for giving me a chance to bid on your project. i am a serious bidder here and i have already worked on a similar project before and can deliver as u have mentioned "I can do th Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(5 Commentaires)
0.0
kevingajjar456

Hi, Thanks for posting !!! I have 14 years of experience in Website and Mobile apps. I have checked the requirement and understand. I have created website SIMILAR to Scraping Expert for XML Sitemap Generator Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 5 jours
(0 Commentaires)
0.0
snaciri81

Hi, I am a data scientist with strong skills in business intelligence, WEB scraping and data analysis tools including Python with Spark and hadoop knowledge. More than 8 years in BI and data analytics and more than 1 Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 7 jours
(0 Commentaires)
0.0
myappsdevelopers

Hello Project manager I have more than 7 years of experience in Android, Iphone, Mobile Applications,Python , PHP, Ionic ,React native ,Angular JS, WORDPRESS, , HTML5,CSS, Ajax, jquery, JavaScript, Mysql, Graphic Des Plus

%bids___i_sum_sub_35% %project_currencyDetails_sign_sub_36% USD en 12 jours
(0 Commentaires)
0.0