Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

To me, it seems proxycrawl is very expensive! If I may ask, can you talk little about your crawl volume and cost?


I’m crawling around 80-120M per month and the price for me fits my needs. But I suggest that you contact them if you have special needs or requirements.

Also you have to consider the amount of work, time and money that you will save by not maintaining your own system to avoid blocks and bans from the websites you are trying to crawl. With them you just call an API endpoint and you don't have to care about all that


Thanks. One follow up questions, in JS heavy pages, how operations like infinite scrolling etc are exposed/executed?


I can't tell, I haven't scraped any page with infinite scrolling yet




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: