
In Progress
Posted
Paid on delivery
I need a Python developer who can take factory data from a mix of subscription-based industry databases, our own internal lists, plus Alibaba, ImportYeti, and Global Sources, then move it through a fully automated pipeline. Here is what has to happen: • Scraping & Ingestion – Build reliable scrapers or connectors that log in where necessary and pull the latest company profiles, contact details, and verification documents from the sources above. – Normalise and store the results so the same record is never processed twice. • REST-First Data Processing – Pass every clean record to an LLM via a REST API (MiniMax or Volcano Engine for now). – Retrieve the model’s enrichment output and append it to the stored profile. • Outreach Automation – Using Make or n8n, trigger personalised sequences by email and LinkedIn as soon as a profile is marked “ready”. – Track delivery, opens, replies, and LinkedIn connection status in real time. • Lightweight Monitoring Dashboard – A single-page web app (simple Flask or Streamlit is fine) that shows: number of factories scraped, enrichment success rate, outreach status, and error logs. Acceptance criteria 1. Scraper handles captcha or login changes without manual fixes for at least two weeks. 2. API module retries gracefully on rate limits and returns enriched JSON compliant with our schema. 3. Outreach flows send test messages through my sandbox accounts without hitting spam folders. 4. Dashboard updates live and displays no stale counts after refresh. If you have solid experience in Python, web scraping, REST integration, and Make/n8n workflows, let me know how quickly you can deliver a first working version and what libraries or frameworks you intend to use.
Project ID: 40409354
68 proposals
Remote project
Active 27 mins ago
Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
68 freelancers are bidding on average $478 USD for this job

⭐⭐⭐⭐⭐ Create an Automated Pipeline for Factory Data Management ❇️ Hi My Friend, I hope you're doing well. I've reviewed your project requirements and see you're looking for a Python developer to automate factory data processing. Look no further; Zohaib is here to help you! My team has successfully completed 50+ similar projects in data automation. I'll build reliable scrapers to gather data, process it efficiently, and create a monitoring dashboard—all within your budget. ➡️ Why Me? I can easily handle your project as I have 5 years of experience in Python development, specializing in web scraping, data processing, and API integration. My expertise includes building automated pipelines, ensuring data integrity, and creating user-friendly dashboards. Additionally, I have a strong grip on technologies like Flask and n8n, which will enhance the functionality of your project. ➡️ Let's have a quick chat to discuss your project in detail. I can show you samples of my previous work, demonstrating my skills in automation and data management. Looking forward to discussing this with you in our chat! ➡️ Skills & Experience: ✅ Python Development ✅ Web Scraping ✅ Data Processing ✅ REST API Integration ✅ Automation with Make/n8n ✅ Data Normalization ✅ Error Handling ✅ Dashboard Development ✅ Flask Framework ✅ Data Storage Solutions ✅ Real-time Tracking ✅ Project Management Waiting for your response! Best Regards, Zohaib
$350 USD in 2 days
8.1
8.1

⭐⭐⭐⭐⭐ • Proposal for B2B Scraper & Outreach Automation: CnELIndia team will deliver a complete Python pipeline for factory data from subscription databases, internal lists, Alibaba, ImportYeti, and Global Sources. • Scraping & Ingestion: Build Playwright scrapers with login and captcha handling; normalize and deduplicate records using Pandas and PostgreSQL to prevent reprocessing. • LLM Enrichment: Route clean records via REST API to MiniMax or Volcano Engine; append outputs with automatic retries and schema compliance. • Outreach Automation: Configure n8n workflows for personalized email and LinkedIn sequences; track delivery, opens, replies, and connection status in real time. • Monitoring Dashboard: Develop Streamlit single-page app showing live scrape counts, enrichment success, outreach status, and error logs. • Timeline: First fully working version ready in 3 weeks using Playwright, Pandas, Requests, PostgreSQL, and n8n. • How CnELIndia Team Helps You: 1. Kickoff call and credential setup. 2. Phased builds with bi-weekly demos. 3. Sandbox testing for spam-free outreach. 4. Deployment, training, and 30-day post-launch support. This meets all acceptance criteria with proven expertise. Ready to start immediately.
$500 USD in 7 days
7.6
7.6

Greetings! I specialise in Python automation pipelines and web scraping systems, with over 9 years of experience building end-to-end data ingestion, LLM enrichment, and outreach automation workflows that run reliably without constant manual intervention. Here's how I can help: - Build robust scrapers and authenticated connectors for Alibaba, ImportYeti, Global Sources, and your subscription databases — with deduplication logic ensuring no record is ever processed twice and captcha handling that stays functional for weeks without manual fixes - Pass every clean, normalised factory profile to MiniMax or Volcano Engine via REST API with graceful retry logic on rate limits, returning enriched JSON compliant with your schema and appended directly to the stored record - Configure Make or n8n outreach workflows that trigger personalised email and LinkedIn sequences the moment a profile is marked ready, with real-time tracking of delivery, opens, replies, and connection status - Deliver a clean Streamlit monitoring dashboard showing live counts of factories scraped, enrichment success rates, outreach status per record, and a filterable error log that never shows stale data after refresh
$500 USD in 7 days
6.7
6.7

Hi, I will build the full pipeline — scraping with login/session handling across Alibaba, ImportYeti, and Global Sources, LLM enrichment via MiniMax/Volcano Engine REST APIs, and outreach automation through n8n triggered on profile readiness, all feeding a Streamlit monitoring dashboard. For captcha resilience, I will use a rotating fingerprint approach with undetected-chromedriver and fallback to third-party solving services, combined with adaptive request throttling — this keeps scrapers running through site changes far longer than static selectors alone. Questions: 1) For the subscription-based databases, do they offer any API access, or will all ingestion require authenticated browser-based scraping? Looking forward to talking through the details. Kamran
$283 USD in 10 days
7.1
7.1

Hi there, We’ve built similar systems that scrape multiple sources, enrich data with LLMs, and automate outreach through email and LinkedIn. One of our products, Concio, uses this exact approach to convert leads into verified contacts. For your project, we can use libraries like Playwright and BeautifulSoup for scraping, along with FastAPI for a robust REST API. We can also integrate with tools like Zapier and n8n for outreach automation. Let’s schedule a 10-minute call to discuss your project in more detail and see if I’m the right fit. I usually respond within 10 minutes. Best, Adil
$370.26 USD in 7 days
6.1
6.1

Hello Dear! Greetings from Toriqul Global Solutions! We are pleased to introduce our company as a reliable and experienced provider of Web Design & Development services. Founded and led by Engineer Toriqul Islam, a B.Sc. graduate in Computer Science & Engineering from Rajshahi University of Engineering & Technology (RUET), our team brings over 10 years of industry experience. At Toriqul Global Solutions, we specialize in building modern, user-friendly, and high-performance websites that help businesses grow and stand out in the digital world. Our design approach focuses on simplicity, elegance, and functionality to ensure maximum user engagement. Technologies We Use: Custom Websites Development Using ======>Full Stack Development. 1. HTML5 2. CSS3 3. Bootstrap4 4. jQuery 5. JavaScript 6. Angular JS 7. React JS 8. Node JS 9. WordPress 10. PHP 11. Ruby on Rails 12. MYSQL 13. Laravel 14. .Net 15. CodeIgniter 16. React Native 17. SQL / MySQL 18. Mobile app development 19. Python 20. MongoDB What you'll get? • Fully Responsive Website on All Devices • Reusable Components • Quick response • Clean, tested and documented code • Completely met deadlines and requirements • Clear communication We would be honored to discuss your project requirements and help bring your ideas to life. Thank you for your time and consideration. Warm Regards, Toriqul Global Solutions
$250 USD in 7 days
5.7
5.7

Hello, Your need for a robust pipeline that integrates diverse data sources while ensuring reliability is clear, especially given the challenges of handling logins and captchas across various platforms. The complexity of ensuring normalized data storage and efficient outreach automation poses a significant risk if not executed correctly. I would prioritize building the scraping and ingestion components first, as they form the foundation of your entire system. By using libraries like Scrapy or BeautifulSoup for scraping, combined with SQLAlchemy for normalization and storage, we can ensure that no duplicate records are processed. This initial step is crucial to maintain data integrity before moving on to REST API integration and outreach automation. I can deliver a working version within 4-6 weeks. For monitoring the pipeline, I would suggest using Flask due to its lightweight nature; it will allow us to create an intuitive dashboard without overhead. You mentioned interest in libraries or frameworks—I'm considering Scrapy for scraping, SQLAlchemy for ORM, and Flask for the dashboard. Is there a specific timeframe you have in mind for this project’s completion?
$540 USD in 28 days
5.8
5.8

I can build a full Python pipeline—scraping/connectors, deduped storage, LLM enrichment via REST (with retries), and n8n/Make outreach automation—plus a live Flask/Streamlit dashboard for monitoring. I’ve handled large-scale data pipelines with validation, rate-limit handling, and clean JSON schemas, ensuring stable runs and real-time tracking.
$250 USD in 3 days
5.4
5.4

As a seasoned Python developer, I've honed my craft in data automation, scraping and sound API integration over the years. Through this, I have learnt to meet deadlines while ensuring a low error rate. Specifically, I have dealt with complex data manipulation tasks such as login handling and automation on websites for a duration of not less than two weeks without any manual intervention. This makes me a great fit for tackling your diverse range of sources, including subscription-based industry databases, internal lists as well as Alibaba, ImportYeti, and Global Sources. Additionally, my experience in REST-first data processing through REST APIs like MiniMax and Volcano Engine aligns perfectly with your needs. I can comfortably implement an efficient LLM model that enriches and stores records while adhering strictly to your schema. Outreach automation is also within my wheelhouse—I am well-versed in email and LinkedIn sequences using Make or n8n workflows. Moreover, my proficiency in streamlining processes through Flask makes me the perfect candidate to develop your lightweight monitoring dashboard. This will be a real-time gateway to all information about factories scraped, enrichment success rates, outreach statuses and any relevant error logs—all you require displayed concisely on a single page. Hiring me means hiring efficiency; let's get started on speeding up your B2B scraper and outreach process, delivering remarkable results!
$500 USD in 7 days
5.6
5.6

Warm greetings, your pipeline needs scraping, deduping, LLM enrichment, and automated outreach working end-to-end without breaks—we can build this fast and reliably. Here’s how we can help: * Python scrapers (Playwright/Scrapy) with login + captcha resilience * Clean storage + deduping (Postgres/Redis) * REST LLM pipeline with retries + schema-safe JSON * n8n/Make outreach flows with tracking (email + LinkedIn) * Live dashboard via Flask/Streamlit with logs We’re a team of 62 professionals with 9+ years in scraping, REST APIs, and automation systems. We’ve built similar data pipelines and outreach engines. Quick questions: which DB + hosting do you prefer? Any captcha type known? Target daily volume? Sandbox email/LinkedIn ready? We can deliver a working v1 in 5–7 days.
$500 USD in 7 days
5.4
5.4

Hello, I can help you build a clean and automated pipeline that pulls data from mixed sources, enriches it through REST LLM calls, and triggers outreach flows reliably. My approach stays simple and stable, focusing on solid scrapers, clean Python processing, and dependable Make or n8n automation. I’ve worked on similar scraping and enrichment systems, keeping the code lightweight while handling login changes and rate limits smoothly. I can also set up a minimal dashboard that updates live without over‑engineering it. Thanks, Teo
$500 USD in 3 days
5.4
5.4

Your scraper will break the moment Alibaba changes its DOM structure or ImportYeti adds a captcha layer. I've rebuilt three B2B pipelines this year where the original developer hardcoded selectors that failed within 30 days. To ensure stability, I need clarity on two things: What's your current monthly scrape volume per source, and do you already have rotating proxy infrastructure in place? The answer determines whether we build a headless browser pool or use API wrappers where available. Here's the architectural approach: - PYTHON + SCRAPY: Build modular spiders with CSS fallback chains and automatic retry logic that adapts to layout changes without manual intervention for 60+ days. - PLAYWRIGHT + STEALTH: Handle Alibaba and Global Sources login flows with browser fingerprinting rotation to bypass bot detection systems that block standard Selenium setups. - POSTGRESQL + REDIS: Implement content-hash deduplication at ingestion so duplicate records never hit your LLM API, cutting costs by 40-60% based on typical B2B data overlap. - REST API INTEGRATION: Wrap MiniMax calls with exponential backoff and circuit breaker patterns to handle rate limits gracefully - I've processed 2M+ API requests this way without manual restarts. - MAKE/N8N WEBHOOKS: Trigger outreach sequences via webhook endpoints with idempotency keys to prevent duplicate sends when the pipeline retries failed enrichments. - FLASK DASHBOARD: Real-time metrics using Server-Sent Events so your dashboard updates without polling - I'll include error alerting via Slack or email when scrapers hit failure thresholds. I've built similar pipelines for two SaaS companies doing lead enrichment at 50K records per month. Let's schedule a 20-minute technical call to walk through your data sources and confirm API quotas before I scope the timeline - I don't start builds where the requirements around captcha handling aren't crystal clear.
$450 USD in 10 days
5.6
5.6

Hello There, You want a fully automated sourcing and outreach engine that turns raw factory data into verified leads and personalized email and LinkedIn sequences. 1) Do you have active premium accounts for ImportYeti and Global Sources to handle the high volume scraping sessions? 2) Which specific factory metrics should the LLM prioritize when enriching the data for your outreach triggers? 3) Are we using a centralized database like PostgreSQL or a cloud based solution to manage the deduplication of these records? We will transform your manual sourcing into a 24 by 7 lead machine that finds the best suppliers and starts conversations before your competitors even know they exist. You will stop wasting time on duplicate profiles and unverified contacts because the system cleans and validates everything automatically. This gives your procurement team a massive edge with a ready to go pipeline of enriched factory data and live outreach tracking. Best regards, Bharat Joshi
$250 USD in 7 days
5.2
5.2

Hi I already implement scraper lot of time before. As an experienced software engineer, I am confident in my ability to deliver a high-performing, fully automated Python B2B scraper and outreach solution tailored to your unique needs. My extensive experience in Python development, web scraping, REST integration and automation aligns perfectly with your project requirements. Over my 8+ years in the field, I have honed my skills in solving complex problems creatively with efficiency as a top priority. Most notably, my proficiency in integrating AI models like LLMs (specifically LangGraph and LangChain) will be a great asset for the REST-First Data Processing step of this project. This experience will ensure that I can effectively pass all cleaned records to a Linguistic Language Model via a robust REST API, capable of handling high volumes and returning enriched JSON data aligning with your schema. Furthermore, I have delivered multiple successful projects involving outreach automation using the Make/n8n workflows you've mentioned. In line with this experience, I assure you that the personalised sequences by email and LinkedIn will be triggered promptly once a profile is marked "ready", while real-time tracking of delivery, open rates, replies and LinkedIn connection status will allow for immediate course correction as needed. Let's embark upon this B2B scraper & outreach journey together so we can not only meet but exceed your expectations!
$500 USD in 7 days
5.1
5.1

Captcha and login resilience on Alibaba/ImportYeti is where these pipelines usually break — I'd tackle that first. Deduplication and LLM idempotency are the silent costs: reprocessing records wastes calls and bites deliverability. Plan: use Playwright (Python) with residential proxy rotation and a CAPTCHA solver for logged-in scrapers; canonical upserts in Postgres via SQLAlchemy to ensure no duplicate processing; httpx with exponential backoff and idempotency keys when calling MiniMax/Volcano; Redis+RQ for background jobs and Sentry for error capture; n8n to trigger SendGrid + LinkedIn API sequences from webhooks and to track opens/replies; Streamlit for a single-page live dashboard that reads fresh counts and logs. I can deliver a first working version in ~10 business days for $500. Quick question: do you already have proxy access and sandbox SendGrid/LinkedIn test accounts I can use?
$500 USD in 7 days
4.8
4.8

Hi, Your requirement involves building a reliable, end-to-end data pipeline—from scraping to enrichment to outreach—and that’s exactly the kind of system I specialize in. With 10+ years of experience, I’ve developed scalable Python pipelines handling authenticated scraping, deduplication, API integrations, and automation workflows. I can build resilient scrapers (using tools like Playwright/Scrapy) with session handling and fallback strategies to manage login changes and basic anti-bot challenges. For processing, I’ll design a REST-first pipeline that normalizes data, avoids duplicates, and integrates with LLM APIs (MiniMax/Volcano). The system will include retry logic, rate-limit handling, and schema-validated JSON outputs. Outreach automation can be handled via Make or n8n—triggering personalized email/LinkedIn sequences with tracking for delivery, opens, and replies. I’ll ensure flows are sandbox-tested and optimized for deliverability. The dashboard (Flask/Streamlit) will provide real-time metrics—scraped records, enrichment rates, outreach status, and error logs—with clean, live updates. Timeline: MVP in 5–7 days, including scraping, API integration, and basic dashboard; outreach automation shortly after. Tech stack: Python (Scrapy/Playwright), FastAPI, PostgreSQL, Redis (queue/retries), Make/n8n, Streamlit/Flask. If you’re looking for a robust, automation-first system that runs reliably with minimal intervention, I’d be glad to build this. Best regards
$700 USD in 30 days
4.2
4.2

hi, i’m mughira, and i can help you build this end to end pipeline. i’ve worked on similar projects involving scraping from multiple sources, cleaning and deduplicating data, integrating llm apis, and automating outreach with n8n along with simple dashboards. i can handle login based scraping, stable ingestion, api retries, and keep everything structured and reliable so it runs without constant fixes. instead of going back and forth here, let’s jump on a quick call and map the first step. i’m available to start right away. when would be a good time for a short meeting?
$500 USD in 7 days
4.2
4.2

As an experienced and dedicated Python developer, I am confident in my ability to efficiently execute the tasks required for your B2B scraping and automation project. My proficiency in web scraping, REST integration, and workflow automation with tools like Make and n8n will ensure a smooth and reliable implementation of your project plan. In addition, I bring a comprehensive understanding not just of technical requirements but also business objectives. This means that beyond scraping the data from multiple sources as you need, I'll ensure a seamless and efficient flow of these records by normalizing and storing them sensibly. Importantly, with a keen eye for scalable solutions, I will carefully create RESTful data pipelines so that each clean record is passed to an LLM via a REST API in real-time and retrieve enriched results. Lastly, I offer you proficiency in creating lightweight monitoring dashboards using frameworks like Flask or Streamlit. The dashboard would reflect all the desired metrics including factory scraped count, enrichment success rate plus outreach status along with error logs.
$350 USD in 5 days
4.0
4.0

Hi, this is Kris from McKinney, Texas, I've reviewed your project requirements and understand that the key challenges include building reliable scrapers for various sources, passing clean records to an LLM via a REST API, automating outreach sequences, and creating a monitoring dashboard. My approach involves leveraging Python for web scraping, REST API integration, and workflow automation using Make or n8n. I would ensure data normalization, enrichment, and real-time tracking of outreach efforts. A few additional questions: Q1: Are there specific data sources or fields that are of higher priority for scraping? Q2: Do you have any preferences for the design or functionality of the monitoring dashboard? Q3: How do you currently handle data verification and duplication checks in your internal lists? Best regards, Kris Kramer
$250 USD in 1 day
4.3
4.3

Welcome to professional Python development services! Hi there, I'm Alema, a Python expert programmer who strives for clear code in atmospheric, numerical weather prediction, physics, and all other seminal fields. I'm ready to provide you with high-quality services. I have completed 350+ projects with a 100% Positive Rating. If you are looking for Quality work, look no further. Also, we are a team of professional workers, and we are always available 24/7 to help employers without limitations, and delivery is guaranteed on time. Your faithfully. Eng. Alema Akter
$250 USD in 2 days
3.2
3.2

Hanoi, Vietnam
Payment method verified
Member since Apr 30, 2026
₹37500-75000 INR
$1500-3000 USD
€12-18 EUR / hour
$30-250 AUD
$10-30 USD
$10-30 USD
₹750-1250 INR / hour
₹600-1500 INR
₹12500-37500 INR
$30-250 AUD
₹750-1250 INR / hour
₹12500-37500 INR
₹600-1500 INR
$800-3000 HKD
₹12500-37500 INR
₹750-1250 INR / hour
₹750-1250 INR / hour
$30-250 USD
$1500-3000 USD
₹1500-12500 INR