We are creating a large scale web crawler written in Go. We have already started to build the crawler and are looking for strong developer to join the project and work directly with our dev team.
As a test, we want you to write a simplified crawler that is able to do the following:
1) Create a function that can take a list of URLs and fetch them
2) Parse the HTML content and extract the links on the page (bonus if you do this using a regex instead of using a slow library that loads in the entire DOM)
3) Pass the links found to the function in #1
4) Repeat step #2 and now we're done. Output all of the links discovered.
Additional Project Notes:
- crawling speed and performance is essential!
- MUST use Go
- we are not going to accept anything submitted using GoColly, as it is too slow.
- we are looking to see clean code that is well organized.
We are finding developer who can do this test task in 1 day.
If you will do well in 1 day, you can join our team and work as long term.
9 freelance font une offre moyenne de $14/heure pour ce travail
Hi, I have just read and checked your job description carefully. I have much experiences in Golang. I hope to discuss more via chat. Thanks & Regard.