Write a Simple Web Crawler Manager, using Windows Forms / WPF interface.
- Should be possible to create a job, by specifying: the site name, starter urls for crawl (simple text area, one initial url per line), url patterns to download (one per line. if not empty, use CONTAINS to see if page should be download to disk), crawl deep, folder to download the html, and retry count.
- Should be possible to pause/resume any job. Pause should also occur if, internet connection breaks down, or the program is closed. When job is finished, should be possible to run it again. Should be possible to delete a job too.
- Should be possible to specify a proxy list. Turn on/off. If on, every time you make a request you get a different proxy from the list.
- Must be fast.
- Must be Multithread.
- Must be written in C#.
- Could use any open source code, as frameworks like NCrawler or Abot to complete your job.
- Downloaded html files can have a pattern file name, like '[site name]-[date].htm'. You also can suggest me something.
- Application must have a job list, showing what is running, minor status.
- Application must have a simple textarea to show any relevant log/console message. Starting.../Downloading...Stopping.
Our system has .NET 4 and Microsoft SQLExpress.
We need a working sample with clean code including all source files in C#, that is able to index <[url removed, login to view]> with a link-depth of 3 and that can paused and resumed, when that pause and resume on disconnect internet connection and reconnect, or close program and open it again and resume the job.
I can get this job done in around 3 days (maybe less than that)! If anything, then we always can discuss that in PM!
17 freelance font une offre moyenne de $438 pour ce travail
Hi I can provide a Desktop application in C# (without WPF) that will used to crawl any site. Can work on a demo if you like. Thanks
Hi, Please review my portfolio or blog to get most recent samples about ASP.NET,CSS,HTML,C# and other development technologies. [login to view URL]
Hi sir, I am scraping expert, I have did too many similar projects, please check my feedback then you will know. Can you tell me more details? then I will provide demo data for you. Thanks, Kimi
Hello, sir. I am a expert Web app developer. if you give me a chance, I will do best for you. Thank you!
Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to s Plus
I have 3+ years expertise developing software by .NET and SQL SERVER. I can help you finish this project with the best quality.
Hi, I have good experience in developing crawlers, scrapers, large data mining and web automation projects for social media, classified and market place sites with multi-thread support. I can develop this crawler Plus
Hello, my name is Dmitriy. I'm interested in your job offer. One of the projects made on Freelancer was scraper for LinkedIn network, so I have required experience and pieces to boost development. If you interested in Plus
I am suitable for this project. Looking forward for your response. I have 10 years of experience. Thanks.
Last Work Experience: August 2012 – July 2014 Desktop - Web - Mobile Developer. Support - Setup - Deployment Specialist – ERP Developer at ERP DEVELOPMENT AS (Oslo, Norway), developing web applications with Microsof Plus
I have the Working experience in C# and in WPF. Being a part of an IT Company i know almost all the Cross Cutting Concerns of an Application. Crawlers are typically programmed to visit sites that have been submitted by Plus
Hello, Its a great pleasure to place a placeholder bid on your project. I have gone through the requirements posted by you and firmly believe that our team will be able to provide great solution along with ideas Plus