Show HN: PyTorch search engine

65 points by chris_f 3 years ago | 17 comments

mendeza 3 years ago |

If you could search through github projects (torchvision, mmdetection, etc) and index high quality pytorch snippets, that would be awesome. I learn a lot about how to manipulate tensors and achieving specific deep learning tasks from GitHub projects, that’s how I answer various questions on how to implement key ideas with pytorch.

chris_f 3 years ago | |

Good stuff! Github is one of the sources, but not specific repos. I actually think we can break out Github into individual repo sources pretty easily.

kjkisielewicz 3 years ago |

Niche search is a huge problem with Google and other big players these days, happy someone’s facing this issue head on.

Would be great if you could do this for Web3 / crypto, and other such niches where tons of new info is coming in and poorly indexed

chris_f 3 years ago | |

Any specific sites? Happy to spin one of these up for you focused on Web3. These work the best when the search engine creator has domain expertise to showcase the best sources.

I could try to find some sources, but my guess at the best Web3 sites would probably miss the mark.

herval 3 years ago | |

What kind of info do you need to find on the crypto space that can’t be easily found with Thegraph or similar tools?

chris_f 3 years ago |

Here are some more example queries:

https://you.com/niche/pytorch?q=install+osx&fromSearchBar=tr...

https://you.com/niche/pytorch?q=save+trained+model

https://you.com/niche/pytorch?q=check+if+pytorch+is+using+GP...

learndeeply 3 years ago | |

For the first example the snippet isn't correct, it just says `conda list`. The website (TutorialPoint) it links to is also wrong and useless.

chris_f 3 years ago | | |

Appreciate the feedback, it is helpful! The relevance tuning is a continuous work in progress.

p1esk 3 years ago | |

Sounds like a good idea, but I'd like to see how is it better than googling these phrases together with pytorch keyword. Not saying it's not better, just not clear if it is.

chris_f 3 years ago | | |

In the broadest sense, a highly targeted search engine (like this) can provide better results because Google has to determine user search intent AND return the right results from trillions of webpages. The advantage of this search engine is that all of the users are looking for the same type of information, and the result sources can be curated to ensure high quality and relevant results.

The more targeted the topic, the harder the time Google has to provide quality signal through the noise of SEO and sheer volume of content on the web.

In addition to the above, the UI provides some cool features like allowing horizontal scrolling of sources to provider higher information density (important for discovery), and some source content can be viewed in the side pane without leaving the page.

But ultimately it would be good to hear if this approach does make it easier to find relevant and higher quality Pytorch info.

wanderingmind 3 years ago |

This is great. However how to find other such niche searches. A request for you.com/niche returns a 404 page which means you cannot find other niche searches like this one

chris_f 3 years ago | |

Yeah, that's a good point. As new ones get built, we will need a way to make them discoverable.

Also, just as a heads up, there is a link at the bottom of the search page [0] if anyone would like to build one of these search pages on any topic that might be interesting. It's just a form now bc we are working through the process.

[0] https://form.asana.com/?k=13NZtQkfNVw3CTE1jxngvg&d=119356944...

wanderingmind 3 years ago | | |

Thank you Chris. If I may add, maybe have an approach to get some community feedback on such niche search as well. This can help to narrow down the list. Great work. I'm going to use this search for my work

cariaso 3 years ago |

that seems like a super valuable domain being very under used.

version_five 3 years ago | |

I think (at the risk of sounding like a shill; I don't use them) you.com is an interesting search engine that is not very well known. I'd be much happier seeing it be used for that than some we.com type branding play (although I take your point that in the hands of some branding expert it could have maybe a more clear connotation than anything related to search.