They talk about using query logs to optimize their search results:

>Queries performed by people, if associated to a web page, serve as even cleaner summaries than anchor text. This is because all the logic put in place by the search engine, who resolved the query with a list of web pages, and all human understanding and experience that led one to select the best page from the offered result list end up embedded in the association <query, url>.

This would seem to present a "rich get richer" problem where the oldest links that have the largest click-through tend to float to the top making it difficult for a new result that may be "better" to appear high in the search results. Anyone know how search engines tackle this problem?

> Money and Resources : We have been lucky enough to have fantastic investors, who fund and help us in our journey.

Just as an FYI, this company Cliqz is owned by Hubert Burda Media, a large media conglomerate based in Germany.

That doesn't necessarily inherently mean anything negative, but it's important to understand the potential underlying incentives given their marketing as such a strongly privacy oriented service.

I don't know how many of you tried the engine, but there are 2 features that instantly took my attention:

1) Trackers Stats. Essentially, you can see how many and what trackers there are on the page you are about to visit. Before visiting it.

2) Page previews (I'm not sure about whether I like that)

What I really want, is not another search engine for contextless queries. Except for really basic queries (which Google/etc already do a good job at), I'm trying to answer a question, perhaps open-ended, and it will take multiple queries to resolve. And it's not a linear process of narrowing down with + or - keywords. It's establishing a context: I'm searching for something relevant to "go" the language, not "go" the english verb or "go" the board game.

I want to be able to open a jupyter-like notebook with the start of my search query, and the first step should be to show me the available eigencontexts, from which I can establish the gross context for my entire search. After this first click, none of the results should be about the board game or the english word--unless the relevant search results happen to include an implementation of Go the board game in Go the language.

And then when I'm done, I want to name and archive that notebook so I can return to it at a later date--whether to refresh my memory of the ultimate answer, or to continue the search.

I guess I would call this a 'research engine' instead of a search engine.

The nationalism on the homepage is a little odd... in particular since they're still essentially building on top of Google.

https://cliqz.com/en/

> Europe has failed to build its own digital infrastructure. US companies such as Google have thus been able to secure supremacy. They ruthlessly exploit our data. They skim off all profits. They impose their rules on us. In order not to become completely dependent and end up as a digital colony, we Europeans must now build our own independent infrastructure as the foundation for a sovereign future. A future in which our values apply. A future in which Europe receives a fair share of the added value. A future in which we all have sovereign control over our data and our digital lives. And this is exactly why we at Cliqz are developing digital key technologies made in Germany.

Ah, yes. The old "hiybbprqag" method. Worked great for Bing.

I like Cliqz and am already impressed with the results for several of my test queries, though they certainly have a ways to go.

But what I would really like to see, as has been mentioned in other threads, is an open-source or community-funded search engine. Something that "belongs to the web" itself, so to speak, and not to any particular corporation.

If you are going to make a new search engine, you need to attack a problem that people have, like Duckduckgo solving privacy issues. I don't want to install something that collects a bunch of personal info about me. A better idea is to search bookmarked sites and the cache. And do it locally.

Wow, just wow. I've been working with Odoo for a couple of years now and it's been a frustrating experience because it's documentation suck badly and it's really freakin hard to get relevant answers from DuckDuckGo or Google when stuck. I tried out a search now on Cliqz and can't believe how good and relevant the result was. Could be a lucky shot, but I'm definitly gonna try this out more. Great work guys! :)

Anybody knows Cliqz's database stack? Curious to see what powers a large scale information retrieval index of this sort.

There was another one of these posts a week or so ago. In that one I was one of many that complained the search engine was unusable without javascript enabled.

Now you can search without javascript enabled. Thanks, cliqz devs.

Do you (founders/employees) have any example queries that don’t work well with Google, and correspondingly what pages you think should be top ranked for those queries?

Since the Cliqz devs are here, and this engine is based in Germany, a question: does your search engine have any mechanisms for reporting abusive URLs (doxxing, targeted harassment, revenge porn, etc) beyond right-to-be-forgotten, or are you more a lassiez-faire, everything-goes kind of search company?

I noticed that your engine ranks some of the nastier sites on the internet far higher than any other search engine I've looked into.

I would like to read this but I can't reach the web server. Is it just me?

Had the same issue with another article from this same site a couple of days ago. Looks like everyone else is able to read it but for some reason not me.

Anyone know what's going on?

> The experts, who chose to answer, suggested that we should first start with crawling the whole web. We were told that this would take between 1 and 2 years to complete, and would cost a minimum of $1 billion

Why are costs so high for crawling?

I remember when Cuil launched about 10 years ago they suggested that 1% of the search market was worth a billion dollars so it's big money if you can get inroads. Of course search is probably less important now than it was back then, with discovery happening on social media more and more, but the internet as a whole is much larger than 10 years ago so I wouldn't be surprised if 1% is worth more now days.

We need a smarter search.

One that is based on analyzing the content of a page then on it's page rank.

Self speaking that it has to be open source.

Apache SOLR would be a good starting point.

Cliqz nearly made me stop using firefox a while back https://www.heise.de/-3852129 (german article)

Tldr 1% of german firefox installations automatically uploaded search queries to cliquz. I wont trust a search engine like this with any of my data.

What I miss in this post is a list of references to the huge literature that exists on this topic, and related fields such as NLP.

Also I don't see a clear problem description. What is a search engine, really? How would you compare the quality of two search engines, objectively?

   > Why the second constraint? one might 
   > ask. Besides the obvious potential for 
   > profitability, our mission was

The search engine the world needs is one with independence and non-profitability. If the creators are preoccupied with turning a profit, they’ll introduce the same garbage features as Google. It’s a shame, because a good search engine could shorten the time humanity has to wait for advances (eg: cures for cancers, cheaper energy, etc)

Weird, your domain (cliqz.com) was blocked by my pihole.

There is a dark side to this story. With Burda https://en.wikipedia.org/wiki/Hubert_Burda_Media, the same people who are behind the Cliqz search engine were originally also behind the German Leistungsschutzrecht. https://en.wikipedia.org/wiki/Ancillary_copyright_for_press_... This law, heavily lobbied for by publishers, forces every search engine and everybody else using content from the internet to pay a private tax of 6% of the revenue (not from profit!). https://www.vg-media.de/de/digitale-verlegerische-angebote/f... As the profit of most internet companies is below this margin, it is essentially forcing many companies out of business.

This tax is enforced and collected by VG Media, the German collecting society representing rights of a group of German publishers. https://www.vg-media.de Between 2013 and 2016 Burda was a shareholder of VG Media, which was commissioned to enforce the tax in its name.

The evil thing of this law is, that the publishers are not required to mark their content in machine-readable form as paid content. And a manual selection is infeasible for internet-scale with billions of pages. So a search engine has no means to bypass the paid content and indexing only free content, e.g. like Wikipedia which makes the majority of the internet content. Essentially the "Leistungsschutzrecht" takes the free content hostage to extort money for using the internet, even if you don't use paid content of the publishers (the just 200 publications the VG Media represents).

So while Burda's Cliqz write on their blog "The world needs more search engines" https://www.0x65.dev/blog/2019-12-01/the-world-needs-cliqz-t... they supported a law that made it impossible for many search engines to operate in Germany (and in the EU via the similar EU law "Extra copyright for news sites" (“Link tax”) https://juliareda.eu/eu-copyright-reform/extra-copyright-for... And while today they are not anymore shareholder of the VG Media, they still benefit from the suppressive legal environment they helped to create, as it prevents any new independent competition to enter the search market

[Disclaimer: I work at Cliqz]

I hate to answer this one, becasue it looks too much marketing-speech but this feature exists. Not on beta.cliqz.com but on the drop-down search on Cliqz browser.

Based on the tabs you have opened, different query expansions are selected. For instance as you type "hotel in ma..." probably would show you results for Mallorca, but if you have "Madrid" on a tab, then it will show results for "hotel in madrid".

There will be a blog post about this contextual search because it's our showcase that is possible to do personalization without compromising privacy. All this is done privately, the browser receives results for multiple expansions and can chose which one to display based on local information. We never track or collect sessions of users.

I had very similar idea a few years ago, did some quick numbers and came to conclusion that it was not going to fly commercially.

Interestingly the working name for my idea was also "ResearchEngine", so i guess it summarizes pretty well the unmet need you and me have.

Agreed. tried replicating this with org mode, which required an extensive learning curve to meet the needs akin to what you listed. Although at some level it did the job, I could imagine a tool that does it better.

> I want to be able to open a jupyter-like notebook with the start of my search query, and the first step should be to show me the available eigencontexts

AltaVista used to do this. I miss it terribly.

I believe this is called faceted search.

(Disclaimer: I work at Cliqz) Just read the article. It is from 2017 and very short. For the non-German speakers, I have to translate the relevant part:

> Rund ein Prozent der Firefox-Downloads enthalten künftig das Add-On Cliqz, das bereits beim Eintippen Vorschläge für Webseiten anzeigt. Dafür wertet es die Surf-Aktivitäten aller Nutzer aus.

About 1% of the Firefox downloads will contain the Cliqz Addon, which will show you search suggestions for websites while you type. For that, it uses browsing activities of all users.

---

The last "of all users" is important. Yes, our search is built on data collected from users, but the point is we cannot build profiles of single users; we are only seeing what the whole group of users does. I cannot stress that part enough. We are not Avast.

In fact, we are very open about our data collection system called Human Web:

* https://0x65.dev/blog/2019-12-03/human-web-collecting-data-i...

And this article explains how we provide anonymity while sending:

* https://0x65.dev/blog/2019-12-04/human-web-proxy-network-hpn...

I can understand that you did not like the way that Mozilla rolled it out in 2017. I'm also not glad about how it went (my personal opinion). But from the technical side, I'm more than happy to take any question on that topic (how we collect data in Cliqz).

Would you trust google who does the same thing with chrome? Bing which does the same thing with IE (or whatever it's called now)? Blame firefox for selling out their users not the search engine. I didn't close my account with amazon when Ubuntu started sending searches to them, I just switched distros.

[Disclaimer: I work at Cliqz]

Sorry for taking so long to reply, I was personally trying to dig some information about this. An additional disclaimer: not a lawyer either.

Honestly, I have little idea of how this law affects search engines. What I can say is that we are no paying anything, as AFAIK we do not know anyone who is. Moreover, if some publisher would complain, even one in Burda, we would stop crawling by domain, there is no technical issue here, properties are known by the imprint. We have no say on what the investors do but I can assure you that we have no pressure. For instance, our ad-blocker works everywhere, regardless if the sites are from Burda or not.

On a general level, assuming that what you say is factually correct, I must personally agree that regulation is a bitch. It's typically designed fro big companies to control other big companies, but small ones get negatively affected if only because of the lack of resources. We recently had to suffer all the overhead of GDPR, which consumed a fair amount of our time, relatively we paid a higher price that Google.

Personally, I cannot respond for all the decisions made by the people funding Cliqz, I do not even think I can judge it either. They might be complaining and lobbying, no idea. But they are also putting good money to build a privacy-preserving search engine and a browser, something that no-one else is doing, so on my account they are on the positive side.