Big News: FounderDating is joining OneVest to build the largest community for entrepreneurs. Details here
Latest Notifications
You have no recent recommendations.
Name
Title
 
MiniBio
FOLLOW
Title
 Followers
FOLLOW TOPIC

Question goes here

1,300 Followers

  • Name
    Entrepreneur
  • Name
    Entrepreneur
  • Name
    Entrepreneur
  • Name
    Entrepreneur
  • Name
    Entrepreneur
  • Name
    Entrepreneur
  • Name
    Entrepreneur
  • Name
    Entrepreneur

What do you think of scraping as a business model?

I am a technical person working on a side project that works by scraping data from various career websites and builds a database of companies and the relative popularity of skills over time in those companies.

The idea is that 40% of jobs aren't advertised and that my tool can be used as a prospecting tool for recruiters or jobseekers to find companies that would hire someone with their skillset.

However I'm in 2 minds whether or not building a foundation on scraping other sites is good idea or not. Google, Indeed and SkyScanner have all built successfull businesses, even though they were (probably) violating the terms of sites they scraped at the start.

I would love to hear the communities feedback.

9 Replies

Bob Graham
0
0
Bob Graham Entrepreneur
Engineering and Software
Do it and see if it gets users and makes money. If it is, then its a good idea.
The only way to know is to try. I like the scraping idea because its fast.
Michael Brill
0
0
Michael Brill Entrepreneur
Technology startup exec focused on AI-driven products
Everyone scrapes everything... all the time... everywhere.

You can do the little ethics dance everyone does or pretends to do.

But the reality is that scraping is the jaywalking of the internet.
Dave Reimherr
0
0
Dave Reimherr Entrepreneur • Advisor
Founder at Magnificent Marketing LLC
I say go for it and don't look back.
Jenn Steele
0
0
Jenn Steele Advisor
Director of Product Marketing at Indix
Step 1: Call it "crawling" rather than "scraping"--it's less creepy that way
Step 2: ??? (or just go for it)
Step 3: Profit (or large crash & burn)
Aaron Konvisser
0
0
Aaron Konvisser Entrepreneur
Strategic Sourcing | Business Development | Contract Negotiation | Supply Chain | Technical Marketing | Product Manager
Sounds interesting. Let us know what you uncover.
Gopi Mattel
0
0
Gopi Mattel Advisor
Director, Chennai Area at The Founder Institute
Well now there is an API for scraping websites, perhaps. And it is popular.
http://techcrunch.com/2016/01/12/import-io-raises-13-million-series-a-for-its-data-extraction-platform/
This could make your technology solution easier to solve.
Chris Leonard
0
0
Chris Leonard Entrepreneur
Scientific Review Manager at SRA International
If you are concerned about violating terms, then adhere to the site's robot exclusion policy (in the robots.txt file). http://www.robotstxt.org/ If they don't have a robots.txt file, then scrape away.
Brad Hall
1
0
Brad Hall Entrepreneur
Founder/President, EntertainmentCareers.Net, Inc.
I know this space well. I would encourage you to shy away from thinking that your business model is"scraping."As you have seen from the comments above, "scaping" has a bit of a negative connotation as it implies to some degree IP theft which is not your goal. It would appear that this project is data mining as you are looking to repurpose data to create a completely different service vs "scraping" and republishing. The mechanics are inconsequential vs what service you are providing and to whom. If you provide a valuable service to employers and/or job seekers the companies may even provide you with xml feeds like they do for indeed, linked in etc so "scraping" is not relevant. I assume you are in the infant stages and you need data to build proof of concept. Don't get muddled in the how, focus on the what first to see if this is a tool worth building and who is the buyer? The company itself, the HRIS/ATS side i.e. Taleo, Worday etc.
Rakesh Andey
0
0
Rakesh Andey Entrepreneur
Got a Solution, Need People to Code, Co-Founder at Ruder App,Co-Founder@Instano(Acquired)
There are couple of players already. webscraper.io & import.io works really good. :)
Join FounderDating to participate in the discussion
Nothing gets posted to LinkedIn and your information will not be shared.

Just a few more details please.

DO: Start a discussion, share a resource, or ask a question related to entrepreneurship.
DON'T: Post about prohibited topics such as recruiting, cofounder wanted, check out my product
or feedback on the FD site (you can send this to us directly info@founderdating.com).
See the Community Code of Conduct for more details.

Title

Give your question or discussion topic a great title, make it catchy and succinct.

Details

Make sure what you're about to say is specific and relevant - you'll get better responses.

Topics

Tag your discussion so you get more relevant responses.

Question goes here

1,300 Followers

  • Name
    Details
  • Name
    Details
  • Name
    Details
  • Name
    Details
  • Name
    Details
  • Name
    Details
  • Name
    Details
  • Name
    Details
Know someone who should answer this question? Enter their email below
Stay current and follow these discussion topics?