You'd like to make a recommendation engine, the idea is that the user could input 1-3 books they liked and it would suggest more books that are similar.

What sort of algorithms should I look into to do that sort of things?

Note that I don't have user profiles with what book they read, only have the database of books, can't do the recommendation engine based on two users liked the same book so they will also like other books either of them liked.

https://www.newstatesman.com/science-tech/social-media/2020/...

For example, searching Old Man and the Sea on isbndb returns (as you’d expect) many isbns:

https://isbndb.com/search/books/Old%2Bman%2Band%2Bthe%2Bsea

Do books have another identifier that logically consolidates editions, foreign language prints, etc.?

https://github.com/KyleBenzle/Good-Reads-Scraper

Except these things began as hobbies before business became involved at all.

It’s a message board for book lovers ffs, pre web people used to build things themselves, at their own expense, just for fun.

I hope it does! I'm banking on it too for my project.

I'm very disappointed to see that they're killing their API. I'll be looking for an alternative.

It is load times are one of the longest of the sites I commonly visit, and it barely had any new functionality added in the last 10 years.

I'm struggling to imagine what they could be; Goodreads doesnt exactly have much data that's really useful.

https://bookshulf.com/

Guess it won't be happening now...

Goodreads is basically the only major social network for bookworms, yet the majority of its users hate it (including me), but are forced to use it to their chagrin.

You would think that would make the market ripe for a disruptor to arrive and topple the incumbent leader, yet each year nothing happens.

I personally have also thought of making a new rival product, but when you do the math on the market potential and the financial benefits, I just don't see a viable way.

People who read books, even if they read them every day, won't use your social network each day since books by themselves are the type of content which is consumed the longest (compared to a movie, tv show, song, or video game).

So you have a social network where users come back on a whim, even if you read like a maniac and try to read one book per week (I tried it one year, it was crazy, you are basically spending all your free time reading), even then users wouldn't use your app each day, but perhaps once or twice a week, and who knows how much time would the average session last?

To make things worse, you could maybe even get away with users using your app once a week if you have a big enough market (user base), but the number of book readers is not that great (especially compared to other media consumption)... The median American reads 4 books a year [1], or simply put, one book in three months.

So you have a social network where users don't need to use it often and there aren't a lot of users, that already spells trouble, but there is another major issue.

You could even succeed with those issues if you had a highly commoditized product to advertise, let's say a social network for yacht lovers, even if you have a small number of users and they do not use it much, you can still manage to succeed with it, since if you advertise yachts to potential yacht owners, you have a very valuable marketing channel which is worth quite an amount to the right people.

You can see where I am going with this, just compare a 5% commission on a yacht, vs a 5% commission on a book... Unfortunately books are not so highly valued (in monetary terms) nor sought after.

To sum up, you have a social network where users don't spend a lot of time, you don't have a lot of users and it is centered around a low profit product... Of course Goodreads has no competition, no sane person would touch that market with a ten foot pole.

Yet, to quote George Bernard Shaw, "all progress depends on the unreasonable man". If someone manages to solve this problem and find a profitable way to survive, I would not be surprised to see Goodreads fall.

I even thought of contacting Scribd to work with them, since I think they might have the best shot currently to position themselves as market leaders. They have an excellent product (Netflix for books) and already have a well sized user base. Would be interesting to see them expand and also became a social network for book lovers.

[1] - https://www.bustle.com/p/how-many-books-did-the-average-amer...

[2] - https://www.goodreads.com/quotes/536961-the-reasonable-man-a...

The FOLIO project aims to facilitate a sustainable, community-driven collaboration around the creation of a modern technology ecosystem that empowers libraries through open source applications to manage library resources and expand library value.

E.g. here's GoodReads work id (in the url): https://www.goodreads.com/work/editions/69741-the-old-man-an...

Here's works/editions in schema.org: https://schema.org/CreativeWork (work) ; https://schema.org/Book (edition)

Here's works/editions in Wikidata: https://www.wikidata.org/wiki/Q47461344 (work) ; https://www.wikidata.org/wiki/Q3331189 (edition) (e.g.: https://www.wikidata.org/wiki/Q19320911 )

All of these are loosely based on the FRBR model in librarian science, where a work is a work, and an edition is called an "Expression": https://en.wikipedia.org/wiki/Functional_Requirements_for_Bi...

And yeah, there's no isbn-analog for series; each site has its own identifier.

Also as a side-note, ISBNs are very inadequate as a unique identifier for an edition; there are frequent cases where an ISBN will be re-used for books with different covers (Dover classics does this a bunch); some modern books published in odd ways are not given an ISBN (e.g. all of these books: https://sorted.club/books/ ); and, most importantly, books published before ISBNs went into effect (~1970) have _no way_ of being accessible when using ISBNs as a unique key.

Where Goodreads has really failed to change is something not easily visible in pithy screenshots: there are a lot of bugs and missing functionality with regard to book metadata, but these flaws are something you don't quite grasp unless you become a Goodreads Librarian.

And even that is a stretch, you can see there are plenty of adjustments in the layout, and the right one looks fine by today's standard.

It turns fun projects into something like a job.

https://github.com/internetarchive/openlibrary/pull/4259

The LT JSON export looks well-formed however, the book records I see are quite vast in content/data/metadata and would probably be a nice python fit. I'm not sure if I could import this with the LT webUI, but it's a nice looking JSON backup nonetheless.

Welp, time to start a better book catalog site with threaded discussions that eBook page turns can be synced with.

Buy it so no one else can have it or buy it so you can shut it down.

I recently started a historical time-series weather data api based on ERA5/GFS and your story really resonated with me. Similarly, I don't come from weather background and initially built it for my work with building energy analysis, with service live but still at an extremely early stage.

Happy holidays. :)

I think that doesn't hold logical water.

If only 1% of people buy things of significant value based on ads, and 99% of people never buy what's advertised, it is still worth advertising while advertising is cheap.

A related phenomenon is the way online games make their money.

The vast majority of players never pay for anything. A few pay a little, and a tiny minority pay so much more than everyone else that it's the tiny minority that the game-maker depends on for their business.

Looking at the strain of our local delivery services right now, that cannot be just 10 per cent of people shopping.

If you ban ads and monetize only via affiliate links, I think you’ll see a lot more poorly written, often algorithmically generated content shilling for affiliate clicks.

Oh, and the old IMDB API you could pay for is gone (or, gone soon), and some of the features it provided are gone permanently.

As a project of the non-profit Internet Archive, having a trusted catalog is pretty paramount to what we're trying to accomplish and Aaron Swartz's original dream for an Open Library.

I've created an issue for this feature (as it's something we've discussed doing for a while). I'll also be adding this to our upcoming Tuesday 2021 roadmap discussion.

https://github.com/internetarchive/openlibrary/issues/4263

Thanks for raising this

Many of our librarian processes & FAQs are detailed here: https://openlibrary.org/librarians

Most data on Open Library is publicly editable by members. In this case, author merging is a capability that only folks who have been added to our Librarians usergroup can access because the process of reverting an accidental merge is quite time consuming (and so we have some training in place).

If you tweet @ our lead community librarian (http://twitter.com/seabelis) she can likely help you make this fix! Also, if you're interested in helping us make changes like this, we can invite you to our slack channel :)

It sounds like perhaps X was in the earlier state of not being recommended for new projects?

I was thinking that with all the data Amazon has about folks (very detailed reading habits from Kindles, for example), they could find people with similar interests to me and recommend the books that those people liked most but that I haven't yet read.

https://en.wikipedia.org/wiki/International_Standard_Book_Nu...

https://en.wikipedia.org/wiki/R._R._Bowker

http://www.bowker.com/products/ISBN-US.html

You're going to trust them to have proper backups, proper disaster recovery, proper resiliency and scalability?

What you're describing works for small-time stuff like blogs, personal projects or other inconsequential things, but anything at the scale of Goodreads, where users are trusting years of data to someone just can't be hosted by random people.

I'm not saying I have the answer but "people should self-host this kind of thing on their Internet connections" is not it.

FWIW, I also share the GP's wishlist 100%. But we're a niche in our own industry these days. I'm not having hopes the market will deliver - on the contrary, all these points are things for unscrupulous vendors to control to extract more profit.

Many of these things already are standard for EU data subjects.

I can confirm it works with at least my import.

Edit: Alas, some errors regarding "Book not in collection" and "No ISBN". But I think I can probably see if there is a way to contribute those books to the repository.

(granted, running on older versions of firefox, but still thought it might be a bug you would like to be aware of.)

> WorldCat is a union catalog that itemizes the collections of 17,900 libraries in 123 countries and territories[4] that participate in the OCLC global cooperative. It is operated by OCLC, Inc.[5] The subscribing member libraries collectively maintain WorldCat's database, the world's largest bibliographic database.[6]

* https://en.wikipedia.org/wiki/WorldCat

* https://www.worldcat.org

Bitcoin has turned out to be expensive to mine. Most other distributed ledger "currencies" have been launched to make money for the creators of the artificial scarcity.

Governments cannot give up control of the money supply but they are the only ones that can establish a form of electronic currency that works the same as cash does now.

Whether they're willing to make it as anonymous as cash or allow it to be exchanged for other government's electronic currency is part of the problem that needs to be solved.

For example Duck Duck Go. I had thought there was no way DDG could possibly be paid at the normal affiliate pricing structure.

Anyone have insight into this

[0] https://familyinequality.wordpress.com/2020/01/15/sci-hub-us...

A government?

...And they even have an API!

There is an explanation under "About the author" section :

This "author" was created to segregate those items which have ISBNs but are not actually books. For more information, see the manual and/or start a thread in the Librarians Group.

When an item which is not a book is imported via ISBN into Goodreads, it does no good to delete it: the item will only be re-imported as long as it remains on the feeder site. (Often these are book-related items which are assigned ISBNs by book publishers so that they can be tracked through their book systems.)

There's very little control. E.g I just published a novel,and Amazon did not in any way validate that the ISBN I have them actually belonged to me - I had not yet registered the book data, so what I told them would not have matched anything they might have looked up.

In this case, based on the reviews on Amazon, it looks like someone changed the description of an existing product, and that the ISBN probably actually belongs to a book.

ImportBot scrapes Amazon.com, matches product id with ISBN10 (which can be converted into ISBN13 without anything else IIRC), imports product as book.

Candle: https://www.amazon.com/gp/product/168298494X

> But that does not absolve the producer. They are still using ethically questionable methods.

Most people don't care. How can it be an ethical issue if the vast majority of people are unconcerned by it?

Sure, for one site, that $3/mo will not break the bank.

But being limited to only $100/$3 = 33 sites a month would be quite constraining.

When I'm reading about a topic or searching for a purchase, I'll typically visit that many sites in one session. Over the course of a month it is normal to visit hundreds of sites (I have counted for auditing a job), sometimes over a thousand. That's without giving it any thought, just following links and reading linked content.

(Heck, when reading HN I sometimes read more than 33 HN stories in a single sitting, but to be fair I don't often follow the links to the articles themselves ;-)

If I had to monitor my usage to keep it down to 33 sites in a month, I could do that but it would be a very different reading experience than I currently have.

Like back in the days when we had to pay per minute of online access (outside the US, using audio modems). The change to a single monthly subscription with the freedom to read as much as you want, as long as you want, was liberating and transformative. It would be a little sad to go back to having to self-police reading in order to keep costs down.

Of course this is totally made up, so it’s impossible to argue about, but my point is that what you described didn’t necessarily seem out of line to me.

AFAIK, this is the "micropayment" problem, that people were hoping bitcoin would solve.

Books pre ~1973 don't have ISBN. And goodreads covers far more material than just modern material.

The current Goodreads importer makes a big/unfortunate tradeoff of trying to have something which works for the majority of material on people's goodreads lists.

It doesn't preclude further efforts -- Open Library knows about millions of books that don't have ISBN. So if anyone wants to help us improve our importer (or at least register their interest by creating an issue which calls out what other identifiers should be considered -- e.g. LOC, etc) that would be very helpful.

# To open an issue:

https://github.com/internetarchive/openlibrary/issues/new/ch...

# Here's the code for the importer

https://github.com/internetarchive/openlibrary/blob/892adebb...

Government requires thousands (literally) of people and political will in order to move any solution forward. In terms of cost and time, it is substantially less efficient than independent entities.

An independent entity could theoretically start immediately, with a small team, and focus explicitly on the stated goals of the team.

> Apple shouldn't be getting plaudits for making privacy a unique selling proposition.

Absolutely agree. The average citizen has no idea how hard they're being exploited.

> Privacy should be a right, the control of my personal information should be mine and consent to have it should be able to be withdrawn at any time.

Also agree, but the government has shown they have an interest in making that not happen. The ties between 3 letter agencies and large tech companies are just the tip of that iceberg.

In the UK the soil association is a private charity which mostly decides which food is "organic" and which food is not and it does a pretty good job. In the US, the USDA decides whether you can call your food organic and from what I understand it's kind of meaningless.

I think a private charity would work better here.

The formation of something new, and independent, means it would also be less susceptible to corruption.

Given that this is one of the primary purposes of governments, what do you suppose has been keeping them from stepping in so far?

To keep the comparison accurate, you can transact many times within an “accounting block”. Bitcoin has the shorter accounting block and tier 2 systems laid on top of it seem to scale Bitcoin well beyond any accounting block maximum transaction cap.

The GP is probably not referring to speed of settlement though. Bitcoin solves KYC and trust and allows frictionless micro-payment.

Nonsense. Your framing is exactly the same. Don't gaslight me on this. I am not naive.

> That's a "motte and bailey" defense. Such a perfect free trade almost doesn't exist, and very few are privileged to partake in it.

Yes the free market doesn't exist because governments stick their noses in.

>Market offerings aren't independent - they're in competition, which means a lot of possible products aren't provided, and those that are face competitive pressure to provide less value for more price

What a load of nonsense. It is because of the free market we have niche products (Amiga accelerators would exist and that is pretty damn niche).

> The most important thing to recognize is that the market, as a dynamic system, optimizes for profitable companies. Not for maximizing value these companies deliver to their customers. Usually, providing value is the most straightforward way for profit. But there are other ways - ways to provide no value, or even negative value while still netting additional profit - and the market takes them as much as it can.

You are speaking out of both sides of your mouth.

> Vendor lock-in and surveillance are just few ways of making money by providing negative value-add.

If vendor lock in a problem if the customer is happy with it? It is up to the individual customer to decide.

> Not illicit. Immoral. Because after all is said and done, the market is still entirely made of people and their decisions, which get to be evaluated through the lens of ethics.

That is splitting hairs. No you want them to be evaluated through the lens of ethics because it benefits to do so in this argument.

> They care and they will use them anyway, because the market doesn't provide any other option.

You don't need yourtube, you don't need facebook, you don't need a lot of this nonsense.

> In the store. Not across stores. And they gave vouchers back, not shoved extra ads in your face. And that's without touching the qualitative difference between a human clerk doing the surveillance and automated systems doing the same.

You have no idea if that data wasn't sold to anyone else. The vouchers are in themselves ads.

These store cards proved two thinds. The first being that people will willingly give up their details for some trickets, and two that tracking customers and optimising via that works. It was a stepping stone.

> I hate to invoke the concept of "victim blaming", because it's usually invoked very unreasonably, but - you can't expect individuals to be able to rationally make market decisions while working their asses off trying to make ends meet, having their attention DDoSed, and facing against compounding improvements in manipulation techniques (courtesy of the market). I'm willing to cut regular folks some slack, and instead focus on the people running these companies, who had a clear choice, and chose to engage in abusive practices. You don't impulse-adopt business models, so you can't excuse it as a moment of weakness either.

Yes I do expect individuals to able to rationally make decisions. People have been told for years and year and years on end what these companies do and they don't care. So it is their fault.

Life is full of risks. Lots of things were approved by bother government and experts in the past that was bad for you.

> That food? Bad. Dead. That tool? Dangerous. Dead. That work on your house? Dangerous. Dead. That car? Unroadworthy. Dead. Those aircraft parts? Counterfeit and not to spec. 300 people dead.

The FDA has stopped people from getting medication in the US that are over the counter medicines because they haven't been approved for use. It is a double edged sword.

>Every single thing humans do is already regulated in some way. Why? Because humans in the end, like all animals, try to achieve the best least effort : highest reward ratio they can.

Unfortunately. What has the current light regulation on the web brought us cookie popups that are irritating that people just click through and a GDPR warnings that don't actually solve the problem of collecting your data. I don't hold out much hope for future regulation, which btw will favour the big tech players that have been collecting our data thus far. BTW you don't know the names of many of them, because they are B2B players and provide services to the companies we do know the name of.

As for "best result for least effort". Well it depends how it manifests itself. It can either be laziness or efficiency. The latter is not a problem.

> In the modern world, these regulations need to be extended to automatically cover modern technologies and prevent inherent harm. They shouldn't be overbearing. They shouldn't be pointlessly excessive. But they are required for all things.

Inviting any sort of regulation will involve government. Government will try to justify itself by demanding more regulation. It will always be overbearing and that will cement these players in place.

At the moment, we have the best chance of these players being toppled. People are looking at alternatives to big tech and are going to smaller players, mainly due to censorship. The trickle has now become a stream, sooner or later it will be a flood. However because of regulation on the horizon (which doesn't address any of the issues we care about)

> Many people think capitalism is greedy because they have literally spent a lifetime experiencing greed fueled capitalism first hand, not because they sit on YouTube watching propaganda videos.

I suspect you are confusing corporatism (which is a form of fascism) with capitalism (which is a party of liberty).

As for propaganda. I never said anything about Youtube. Don't put words in my mouth. I am talking about how hollywood, novelists (since the 19th century), newspapers have framed it since forever. You are soo fermeted in it you don't even realise it is propaganda.

Also, pedantic point, I think you were looking for the phrase en masse, of french origin. See https://en.wiktionary.org/wiki/en_masse

This is VERY different to letting third-parties advertise to you.

Also, consumers on average do a bad job of managing anticompetitive behaviour and harmful externalities, even if they know about them. Convenience and habit are strong motivators. And we need regulation to disincentivize companies from outright lying in the first place.

Companies and people in a fully free market system won’t magically become rational automata that behave ideally. We’re only human. Our superpower is the ability to collectively leverage our individual specialization. Foresight for negative externalities is a specialization that needs regulation to be effective.