We need products crawler into mySQL DB
Crawl the following 7 web sites
[login to view URL]
[login to view URL]
[login to view URL]
[login to view URL]
[login to view URL]
[login to view URL]
[login to view URL]
and save product data into mySQL Database with the following structure:
--
-- Create schema products_crawling
CREATE DATABASE IF NOT EXISTS products_crawling;
USE products_crawling;
-- Definition of table `products`
DROP TABLE IF EXISTS `products`;
CREATE TABLE `products` (
`website_product_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`website` varchar(80) NOT NULL,
`website_product_url` varchar(500) NOT NULL,
`product_category` varchar(250) NOT NULL,
`product_sku` varchar(40) NOT NULL,
`product_model` varchar(40) NOT NULL,
`product_features` text NOT NULL,
`product_name` varchar(200) NOT NULL,
`product_description` text NOT NULL,
`product_overview` text NOT NULL,
`product_specification_text` text NOT NULL,
`product_specification_html` text NOT NULL,
`product_manufacturer_warranty` varchar(40) NOT NULL,
`product_manufacturer_website_url` varchar(500) NOT NULL,
`product_manufacturer_website_support_url` varchar(500) NOT NULL,
`product_manufacturer_website_product_url` varchar(45) NOT NULL,
`product_brochure1` varchar(150) NOT NULL,
`product_brochure2` varchar(150) NOT NULL,
`product_brochure3` varchar(150) NOT NULL,
`product_brochure4` varchar(150) NOT NULL,
`product_video_url` varchar(250) NOT NULL,
`product_image_path1` varchar(150) NOT NULL,
`product_image_path2` varchar(150) NOT NULL,
`product_image_path3` varchar(150) NOT NULL,
`product_image_path4` varchar(150) NOT NULL,
`product_image_path5` varchar(150) NOT NULL,
`product_stock_level` varchar(20) NOT NULL,
`product_price` float NOT NULL,
`product_rrp` float NOT NULL,
`product_created_ts` datetime NOT NULL,
`product_updated_ts` datetime NOT NULL,
`product_status_cd` smallint(5) unsigned NOT NULL,
PRIMARY KEY (`website_product_id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
Product Categories must be saved on MySQL DB in the following format Category > SubCategory > SubSubCategory…
Product images must be downloaded into separate folders for each web site and image paths should be saved on MySQL DB.
Product video URLs must be saved on MySQL DB
Crawler must have 2 modes:
Full – each products details information crawled and stored on DB
List – all categories crawled and only stock level and price stored on DB for all existing products and for new products stored information only available from the List.
Hi,
I have over 12 years of Experience in software design, development and implementation of various commercial applications in Client/Server environment, Web and ERP applications using C# 1.1/2.0/3.5, ASP.Net, VB.Net 1.1/2.0/3.5, AJAX, Visual Foxpro, DOTNETNUKE, VB 6.0, Crystal Reports 8.5, ASP, PHP, JSP tools, PL/SQL, MS SQL Server 2000/2005, My SQL.
Regards,
Sheik Abdul Gafoor
we have a team of 25 members with expertise in their profession. We have made a similar kind of projects. For further information please view your PMB.
Ready to work with you Regards
I have extensive experience using regular expressions to retrieve data from external sources. I can provide you with a modular and maintainable solution that fits your needs.