
In Progress
Posted
Paid on delivery
I want to collect complete product data from a specific e-commerce site—around 50,000-60,000 SKUs divided into each category, with at least 50 items per subcategory, and broken down by gender (male/female) and manufacturer. The script will output the basics in a csv file: product name, full category tree, short and long descriptions, technical specifications, color, size, product manufacturer, price, store name, store URL, and most importantly, one or more image URLs per item. What I still need is a clean, reliable Python 3 script that will contain: All of the above data • Fetch each image URL directly to my server, storing files in a logical folder hierarchy that reflects gender → main category → subcategory. • Keeps the file names unique (product code or other field that already exists). • Records successes and failures, with an easy option to resume if the process is interrupted. • Efficiently handle at least 60,000 products—identity downloads or asynchronous downloads (requests, aiohttp, or similar) are welcome, as long as they remain lightweight. Validate each download (status 200, file not empty) and retry a configurable number of times before marking it as failed. • Generate a final report (CSV or TXT) of skipped or failed data + image, so I can review them later. Feel free to suggest optimal libraries or modifications, but keep dependencies minimal and typical (requests, asyncio, aiohttp, urllib3, pandas, etc.). The script will run on a Linux VPS. Assume Python 3.10 is available. Deliverables 1. The complete, well-commented Python script. The full data results. A separate CSV file with all categories, subcategory URLs, and the total number of products each category and subcategory has in this marketplace. 2. A short README or step-by-step instructions on how to configure paths, credentials (if any), and run the script via CLI or Cron to download the image. 3. Optional: a short test run on a subset of 500 products to demonstrate speed and reliability.
Project ID: 40653302
88 proposals
Remote project
Active 5 days ago
Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
88 freelancers are bidding on average $100 USD for this job

Hello, Combining my 15+ years of experience as a highly skilled developer and my proficiency in Python, your project aligns perfectly with my expertise. I have successfully developed and deployed numerous automation and data scraping solutions in Python that focus on extracting and formatting information, even for large-scale projects like yours. My extensive knowledge in libraries such as requests, aiohttp, asyncio, urllib3 and pandas will ensure that your data is fetched accurately and efficiently. Regarding image downloading and storage, I am proficient at organizing logical folder hierarchies to maintain easy accessibility. I can also guarantee uniqueness in filenames to avoid any potential mix-ups. Furthermore, I understand just how important it is to keep track of successes/failures during a data collection process. Hence, I assure you that the script I deliver will not only produce a report containing skipped or failed data but will also be capable of resuming the process if it's interrupted. In addition to delivering a clean and reliable Python script for Linux VPS, I aim to exceed your expectations by offering you a separate CSV file detailing all the categories, subcategory URLs, and their corresponding number of products in this Greek marketplace. On top of that, you can expect step-by-step instructions on CLI/Cron execution as well as a test run on a subset of 500 products before we go fully functional. Trust my strong Thanks!
$180 USD in 3 days
8.6
8.6

Hi, I reviewed your request to collect complete product data from a Greek e-commerce site for 50,000-60,000 SKUs, outputting CSV with full category tree, descriptions, specs, manufacturer, price, store info, and image URLs. I’ll write a clean, reliable Python 3 script that uses Web Scraping to build the JSON-like product records, then downloads each image to your Linux VPS in a gender → main category → subcategory folder structure. I’ll keep unique filenames using the existing product code, validate status 200 with non-empty files, retry a configurable number of times, and maintain a resumable progress log. You’ll get final CSV/TXT reports for skipped or failed product data and images, with lightweight async fetching for speed. Let’s discuss here now.
$150 USD in 7 days
8.4
8.4

Hello there, I am experienced in web scraping and building scripts or a Windows desktop application using Python. I am also experienced in large data scraping from a given website, bypassing IP, Captcha, and anti-bot or cloud flair protection. Please message me to discuss this project in detail. Best Regards Enamul
$85 USD in 3 days
8.2
8.2

Greetings, I see you're looking to gather detailed product data from a Greek e-commerce site, specifically targeting around 50,000 to 60,000 SKUs. My approach would be to create a robust Python script that efficiently scrapes and organizes this data, ensuring it captures all necessary details like product names, descriptions, specifications, and image URLs. I would structure the image storage logically by gender and category, and implement error handling to ensure reliable downloads and a comprehensive report of any issues. With my experience in Python, web scraping, and data handling, I can develop a clean and effective solution that meets your needs. I will ensure the script is well-documented for easy use and maintenance. Best regards, Saba Ehsan
$85 USD in 4 days
7.5
7.5

Hi, The resume-safe design is the real problem here, not the scraping. For 60k SKUs I'd persist state to a small SQLite index so an interrupted run skips what's already fetched, use aiohttp with a bounded semaphore to keep it lightweight, and validate every image (status 200, non-empty) with a configurable retry count before logging it to the failure report. I've done Python automation and Linux VPS work like this before, including a DevOps integration workflow scored 5 stars on Upwork. Expert DevOps Engineer: Python integration workflow, 5★ One question: does the site require login or throttle by IP, or is the catalog fully public? That changes the fetch strategy. Adil
$100 USD in 7 days
7.5
7.5

Hi, I’ve read your brief carefully, and I can build a clean Python 3 script to collect the full marketplace dataset, export structured CSV files, and download/validate images reliably on your Linux VPS at scale. I’ve handled automation workflows where performance, resume support, and failure logging matter, and here I’d use a lightweight approach with Python, asyncio/aiohttp or requests where appropriate, clear folder hierarchy, unique filenames, retries, and final reporting for skipped or failed items. I will deliver the well-commented script, category/subcategory product counts, README for CLI/Cron setup, and can also run a 500-product test subset first to verify speed and stability before the full run. I can start right away and complete this within 7 days. Do you already have the marketplace category URLs, or should the script first discover the full category tree automatically? Best regards, KANIKA
$100 USD in 7 days
6.9
6.9

Hi! I have extensive experience with large-scale e-commerce scraping, product catalogs, image downloading and VPS automation. I can build a reusable **Node.js + Playwright** scraper that collects the requested product data, categories, manufacturers, prices and image URLs, while downloading images into the required gender/category folder structure. The system will support resume after interruption, duplicate protection, configurable retries, download validation, logging and a final report of failed/skipped records. I can also generate the category/subcategory statistics separately. I suggest starting with the optional 500-product test run to verify the site's structure and processing speed before scaling to the full 50,000–60,000 products.
$300 USD in 7 days
6.7
6.7

Hi there, I understand that you need a reliable Python script to collect extensive product data from a Greek e-commerce site. I can develop a Python 3 script that efficiently gathers the required data, stores it according to your specified hierarchy, validates downloads, and handles failures gracefully. In addition, I will ensure the script is well-commented and comes with a README that provides clear instructions for configuration and execution. I'm confident in delivering a complete solution that meets your requirements within a week. Your satisfaction is my priority and I guarantee that I will deliver you a high-quality result. Regards, Ali
$70 USD in 1 day
6.3
6.3

Hi, This looks like a standard e-commerce scraping job where reliability and data integrity are the main concerns. The main challenge here will be handling 50,000–60,000 items without getting rate-limited or crashing mid-run, while keeping image downloads organized and retriable. I’ve worked on similar crawlers where the site structure was messy, so I’d approach this with a layered design: first scrape the category pages to map all SKUs, then fetch item details in batches, and finally download images with async requests while tracking failures. Pandas will help with the CSV outputs and category breakdowns, and a simple retry mechanism with exponential backoff should handle most transient issues. The folder hierarchy and unique filenames can be derived from product codes or the existing SKU field. The biggest unknown is how the site serves images—whether they’re direct URLs or require session cookies, headers, or even Selenium to bypass bot checks. I’d start with a dry run on a few categories to check the response format and adjust headers if needed. If the site blocks requests, we can fall back to rotating user agents or small delays between batches. I can start working right away. Let's connect and discuss the details. Thanks, Denis.
$100 USD in 3 days
6.3
6.3

Hi, I can build a reliable Python scraper to collect the 50K–60K product records, download and organize images, and support retries/resume with detailed failure reporting. I’ll optimize it for a Linux VPS using lightweight asynchronous processing and provide the complete data, script, and setup guide. Thanks Anshuman
$90 USD in 7 days
6.4
6.4

Hello There! I’m Md Toriqul Islam, an experienced Python scraping and automation developer specializing in large-scale data extraction, asynchronous downloads, CSV processing, image handling, and Linux VPS deployments. I understand you need a reliable Python 3.10 scraper capable of processing 50,000–60,000 products, extracting complete product/category data, downloading images into a gender/category hierarchy, handling retries and failures, and supporting safe resume after interruptions. I’m skilled in Python, requests, aiohttp, asyncio, pandas, web scraping, concurrent processing, data validation, CSV generation, logging, and Linux automation. I can first run a 500-product test to validate extraction accuracy, download speed, folder structure, retry/resume behavior, and reporting before scaling to the full dataset. Looking forward to hearing from you. Best regards, Md Toriqul Islam
$70 USD in 2 days
6.0
6.0

The bottleneck is the need to efficiently and reliably scrape and download large volumes of product data and images from an e-commerce site under 60,000 SKUs. The root cause is the challenge of managing asynchronous downloads and organizing images in a logical folder structure with unique file naming. I will build a Python 3.10 script using asyncio and aiohttp for asynchronous requests, implementing retry logic and validation checks for HTTP status and file integrity. The script will save images by gender, main category, and subcategory, with unique filenames based on product codes, and generate complete CSV reports for all products and failures. This will ensure reliable, resumable operation with validation, verified by a test run on a subset of 500 items Could you clarify if there are any authentication or rate-limiting mechanisms on the e-commerce site that we need to handle in the scraping process?.
$95 USD in 1 day
5.8
5.8

Hello I can build a reliable Python 3.10 scraper for 60,000+ products, including complete product data, category mapping, manufacturer/gender details, and image URLs with efficient downloads, retries, validation, resume support, and failure reporting. I’ll provide the well-documented script, category/product counts, README, and optional 500-product test run optimized for your Linux VPS. Regards Muhammad
$100 USD in 1 day
5.7
5.7

Hi there I can collect and organize the required data from the Greek marketplace and provide it in a clean structured format ready for analysis or import My approach 1 Data Collection Extract the required listings and fields from the marketplace based on your criteria 2 Data Processing Clean and standardize names prices categories locations descriptions and other required information 3 Structured Output Deliver the collected data in Excel CSV or your preferred format 4 Accuracy Check Remove duplicates and validate the collected records before delivery 5 Automation If this is a recurring task I can build a Python scraper so the data can be collected and updated automatically 6 Handover Provide the final dataset or scraper with clear documentation for future use I have worked with Python web scraping data processing APIs and structured data extraction from websites and marketplaces Quick questions to get started 1 Which marketplace and specific categories or listings should be collected 2 Which fields do you need from each listing 3 Approximately how many records need to be collected 4 Do you need a one time dataset or an automated recurring scraper 5 Which output format do you prefer Excel CSV or database Once I have these details I can understand the data structure and start with the collection Let me know and we can jump on a quick call to discuss the requirements Thanks
$95 USD in 7 days
6.0
6.0

Hello there, I have strong experience building scalable Python web scrapers and data extraction pipelines for large e-commerce catalogs, including product categorization, metadata collection, and high-volume image downloading with reliable resume support. I can develop a lightweight Python 3 solution using requests/aiohttp, structured CSV exports, download validation, retry handling, and failure reporting while keeping the system VPS-friendly. I will deliver the complete scraper, image downloader, category analysis CSV, documentation, and a test run with optimized performance for 60,000+ SKUs.
$100 USD in 1 day
5.8
5.8

Hi I have more than 11 years of extensive experience in all areas of data entry, data mining, data extraction, blog posting, mailing list development. I am expert in Data Entry work with 52 WPM of keystroke speed with exception interpersonal communication, management and marketing skills. Product Data Entry in CMS like Shopify, WooCommerce, Squarespace, Magento, Shopify and similar capacities. SERVICE DESCRIPTION Admin Support Services : =================== Product Data Entry for Online E-commerce Website Data Entry Data Conversion Web Research Mailing List Creation Ads Posting Form Filling Product Uploading Data Mining Data Gathering Image Editing Portal Listing etc. Thanks & Regards Rahul Rahangdale
$100 USD in 3 days
5.5
5.5

Hello, I can build a Python 3 script that scrapes the Greek marketplace for 50,000-60,000 SKUs, saves all fields to CSV, and downloads images into gender/category/subcategory folders on your Linux VPS. I will use aiohttp for async downloads, unique filenames from product codes, resume support, retries, and a failure report. I can start today. For the DEPTH, I will validate each image (status 200, non-empty) and log skipped items to a separate CSV for easy review. Questions: 1) Which marketplace URL is the target? 2) Any login or rate limits I should plan around? Looking forward to discussing further. Regards, Shayan.
$77 USD in 3 days
5.5
5.5

I’ve built similar product data pipelines for Greek e-commerce sites before, handling 50K+ SKUs with image downloads and category mapping. This is right in my wheelhouse. I’ll use `aiohttp` for async fetches, `Pillow` for image validation, and `pandas` for CSV output, with retry logic via `tenacity`. The script will scrape product pages first, then download images in gender → category → subcategory folders using product SKUs as filenames, logging failures to a resumable state file. I’ll include a lightweight CLI flag to resume interrupted runs and generate a final error report. All dependencies are standard Linux packages. I can start immediately. Thanks, Andrii.
$175 USD in 2 days
5.1
5.1

Hello, Hope you're doing great! I am a PHP Developer who builds secure, fast, and business-focused web applications. I work with both custom PHP and frameworks, and always ensure that every project is optimized, scalable, and easy to maintain. What I Do 1. Custom web applications & business automation tools 2. API development and integration 3. Secure login, admin panels, and dashboard systems 4. High-speed, mobile-friendly websites 5. Migration, bug fixing, and performance upgrades Why Clients Prefer My Work 1. Clean folder structure & scalable architecture 2. Fully optimized and secure coding practices 3. Excellent communication & professional approach 4. Quick turnaround time with regular updates Ready to Start Share your requirements or preferred reference — I’ll analyze it and provide: 1. Best technical plan 2. Exact timeline 3. Budget estimate Looking forward to building something amazing for you!
$100 USD in 7 days
5.0
5.0

Hi, I got that you are looking for a clean and reliable Python 3 script to collect complete product data from a specific e-commerce site, including around 50,000-60,000 SKUs with specific details such as product name, category tree, descriptions, technical specifications, images, and more. This is what I can help you with, let's chat. My approach is to utilize efficient libraries such as requests, asyncio, and aiohttp to fetch image URLs directly to your server, ensuring unique file names and handling at least 60,000 products seamlessly. I will validate each download, retry if necessary, and generate a final report for review. Additionally, I will provide a well-commented Python script, full data results, a CSV file with category details, and detailed instructions for configuration and execution. As final deliverables, you will receive the complete Python script, data results, a CSV file with category information, and detailed instructions for running the script via CLI or Cron. One thing I'd like to confirm before we start: Do you have any specific preferences regarding the folder hierarchy for storing images? Looking forward to discussing this project further with you. Let's connect soon. Regards, Imran
$70 USD in 1 day
5.1
5.1

Chania Crete, Greece
Payment method verified
Member since Dec 20, 2013
$10-30 USD
$10-20 USD
$10-30 USD
$10-30 USD
$10-15 USD
₹1500-12500 INR
₹1500-12500 INR
₹750-1250 INR / hour
₹100-400 INR / hour
$750-1500 USD
$15-25 USD / hour
₹750-1250 INR / hour
₹1500-2000 INR
$10-30 AUD
₹100-400 INR / hour
$2-8 USD / hour
$15-25 USD / hour
₹100-400 INR / hour
$2-8 USD / hour
$250-750 USD
₹12500-37500 INR
₹750-1250 INR / hour
$8-15 USD / hour
₹1500-12500 INR
₹750-1250 INR / hour