Nice work with the write up and thank you for sharing this. The post is interesting, but I think the problem with the YCRank approach currently is that the labelling appears to be subjective opinion, at least if I understand correctly.

Based on the post, you've trained the classifier by labelling a couple of examples of company descriptions you liked better than each other, based on subjective assessments like "harder to execute" or revenue growth that aren't part of the data you're running the classifier against.

If so, you've done a nice job of training a classifier to predict which companies you personally are more likely to be interested in. To improve this, you could use past YC batch company descriptions and success data to have more useful examples and labels for training the classifier based on past data, and which isn't so subjective. That might produce some interesting predictions that are more generalizable (although I think you may need more data points than the description and basic metadata).

If I've misunderstood, it would be interesting to know a little more detail about how the data was labelled.

I've based this on the following: "To investigate this, I made a neural network, YCRank, trained it on a handful of hand-labeled pairwise comparisons, and then used the learned comparator to sort the companies in the most recent W’22 batch."

And then: "I biased my ranking towards what was “harder to execute” on" and "I also tended to rank favorably companies that were already making monthly recurring revenue with double-digit growth rates".

Those may or may not be good criteria.

Based on that, this is essentially what you could call a "DudeRank Classifier" because as The Dude in the Big Lebowski says, "Yeah, well, that's just like, your opinion, man" :)

As I suggested above, it might be more interesting to label the example pairs and train the classifier based on the original company descriptions of known past successful and unsuccessful YC companies.

Possibly there is some signal in the company descriptions and limited metadata from Demo Day alone sufficient to predict successful companies from a batch.

Good luck!

Disclaimer: I am in the W22 batch. Our startup (Andi) ranks pretty well here. And this also is just, like, my opinion :)

[Edit: You could also test the classifier against historical batches to improve it then also!]

I checked the top 6 and 5 of them appear to be vaporware without a real product. To be fair, they're working on hard products, but I wonder if the model is selecting for that. According to the announcement[0] 29% of the batch were accepted with only an idea, so I guess that's not surprising.

Slightly off topic, but I thought it was interesting that 29% of the batch has just an idea while 10% had more than $50k of monthly revenue when accepted. Given they all get the same deal, it would be wild to build out a business that's making a 600k+ a year and get the same terms as two guys with an idea. Seems to go very against everything I read from YC about validation and product market fit, but I guess it's good if they can get away with those terms. That is of course if you're doing a real business rather than MoviePass model (selling $10 bills for $5).

Is there anywhere I can get a breakdown of the companies that fall into these categories?

[0] https://www.ycombinator.com/blog/meet-the-yc-winter-2022-bat...

Thanks for the analysis. Can't say I understood much, but the concept of automating venture investment is an interesting idea. As a side question, what would you recommend a total beginner to learn if they wanted to go from 0 to being competent at neural networks, or machine learning in general. I'm currently dabbling with R in grad school (biology), and I know Python is big within the machine learning world.

Is there a pathway you would recommend (e.g., first learn Python until you're familiar in X, Y, Z, then make sure to learn the required mathematics... etc)? Also, how long would this process of learning roughly take? Been thinking of potentially changing fields after grad school, or maybe working at the intersection of ML and medicine.

Maybe I'm missing something, but wouldn't it make more sense to use data from older companies that had a few years to develop instead of only data from this batch? This model seems to extrapolate based on how you judge the companies rather than on how well they may or may not do.

That is really cool!

What do people think of BayesDB being used for the same thing?:

"BayesDB, which is open source and in use by organizations like the Bill & Melinda Gates Foundation and JPMorgan, lets users who lack statistics training understand the probable implications of data by writing queries in a simple, SQL-like language."[0]

Check out BayesDB if you have not already[1]!

[0]: https://simons.berkeley.edu/talks/vikash-mansinghka-10-06-20... [1]: http://probcomp.csail.mit.edu/software/bayesdb/

Here is an idea:

run it on old batches and see if you can predict which ones went on to raise a Series A, B, C or liquidity event.

Note: you might have to adjust for year since a Seed round in 2020 looks like a Series A in 2010.

Edit: I realize you have to figure out which companies raised out of 100s. As always, the data curation is harder than the algo. Not sure if techcrunch allows scraping.

I didn’t quite understand how the labeling part worked. So you went through a selection of startups and ranked them? And then model took it from there?

Unless we get sales/revenue/profit data, this is not very serious

That's because it uses as only input the description of the company, which is 80% non-informative common words, and 20% buzzwords. It's basically a buzzword detector.

I reckon you could reach the same results with TF-IDF and kmeans.

Investors get yield by investing in unique products. When there is enough* money floating around, anything that can be built in a few weeks will have been built in a few weeks.

This means that in order to get return, investors have to be willing to invest in longer term efforts which are sufficiently difficult to execute that other investors will either refuse to fund the project - or their teams will fail to deliver.

When there is enough* money available, it stops making sense to launch product - you can keep pitching a bigger vision indefinitely, but once you launch you are beholden to real metrics. In fact, launching means that your competition suddenly looks ridiculously capital efficient in comparison.

* for some level of enough.