Business Insider responds to Marco's complaints about scraping

Business Insider responds to Marco's complaints about scraping(businessinsider.com)

4 points by rradu 14 years ago | 4 comments

smoyer 14 years ago |

Wow ... I think there was a definite misunderstanding on Marco's part, but that's a pretty snarky response veiled as an "apology". And even a little bit of "it was completely accidental" thrown in for good measure.

jnye131 14 years ago | |

The time taken to investigate this really was time spent trying to spin it in the right way. All that talk of stub pages that are not supposed to be read, rubbish. No follow attributes being added to links, well how about no-index meta tags. Only that will have a negative impact with google so they wouldn't want to do that.

Utter BS that whole article.

BenSS 14 years ago | |

What is Marco misunderstanding? He doesn't know how the site operates, or that they were 'trying things out'. I really don't buy the long-winded explanation. The pages are still available, and easily findable. It's not like he went out of his way to locate them.

smoyer 14 years ago | | |

True ... misunderstanding isn't quite what I was trying to convey. When I read Marco's original accusation, it was pretty obvious that they hadn't just scraped the whole article. If you were intent on stealing it, why not take the whole thing? But I was confused by the lack of attribution.

I was more interested in commenting on their reply ... and I agree that it was overly long for what they conveyed. It could have been a lot shorter if they hadn't spent so much time trying to make Marco look like the one who was in the wrong. And I still don't believe he was ... whether or not he knew what he was looking at.

The other thing that struck me was that their technical solution sounded a bit hacked. Perhaps their developers are ham-stringed by poor architectural decisions. If not, then the article made them sound either incompetent or lazy.