How Shopify implemented its secure authentication service(engineering.shopify.com) |
How Shopify implemented its secure authentication service(engineering.shopify.com) |
I think pretty much everyone has acknowledged that it's improved. Where opinions differ is in how much it has improved and whether that's enough to entertain its use (my answers to which are "not enough" and "not even if you paid me", respectively).
So the big one for Rails are Shopify, Github, Cookpad, Gitlab. For Ruby ( Not Rails ) That would be Stripe.
I found hundreds of people complaining about this in the community forums, going back years. If you're dynamically generating images, or on a congested network, 4s is far too short.
Since this is a simple config property, the only justification I can imagine is that they are trying to restrict the amount of time that their (single-threaded, memory-hungry) instances are occupied. Because of Ruby's poor resource management, a core part of their API is barely usable.
I'm pretty disappointed.
Rails is just not meant for heavy transactional load. And e-commerce needs async to handle what can be a huge load.
Taobao is java or php and they handle load far greater without fault.
Shopify is much better than Magento though.
Rails is great. But commerce is heavy and I don’t believe Shopify can keep its existing core code base around much longer without a significant change to aid with performance.
It would be interesting to know the details of how they’re doing authorization. It appears that it’s all or nothing but I might be mistaken.
We started with Doorkeeper and gradually switched to building our own OAuth2/OIDC implementation over time, partially using glued together lower-level libraries like https://github.com/nov/openid_connect
Edit: I forgot, I even have a few small commits to that last project from my time at Shopify: https://github.com/nov/openid_connect/commits?author=meagar
I've used it a bit in the past and it worked fine, but I didn't really push it.
Can you elaborate? Shopify released their GraphQL Admin API in 2018, which has built in documentation, as well as an interactive IDE (Graphiql) that you can run on your shop. As long as you're not doing anything too crazy that exceeds the throttle (e.g., syncing 1000s of products), they're pretty good about listening to developer requests.
https://help.shopify.com/en/api/graphql-admin-api
> Rails is just not meant for heavy transactional load.
Shopify has invested heavily in their sharding setup so that it can be handle high load and scale quickly. e.g., flash sales where baseline traffic will 3x in a matter of seconds. See the discussion of pods:
https://engineering.shopify.com/blogs/engineering/e-commerce...
> And e-commerce needs async to handle what can be a huge load.
Long running processes are async. However, having commerce modeled by a transactional database that provides atomicity is a boon for simplicity. You don't want to deal with eventual consistency when updating inventory or placing orders.
Note: Am ex-Shopify. Ran the API team.
* The Metafields feel both over-engineered and difficult to use. Stripe did a great job - metadata is a set of string key/value pairs that are always fetched when you fetch a thing. Shopify's metafields are typed and need to be fetched explicitly from the REST API. You can't get a list of Products and their associated metafields; you have to fetch each Product's metafields one at a time. It's a critical facility and yet works very poorly.
* The API for metafields is inconsistent. Want to get metafields for a variant? /admin/products/#{id}/variants/#{id}/metafields.json Want to get metafields for a product image? /admin/metafields.json?metafield[owner_id]=#{id}&metafield[owner_resource]=product_image WAT?
* Variants (and images) have unique ids, but you always have to reference them as /products/123/variants/456. Variants should be /variants/456.
* Suffixing all the requests with .json is annoying and not RESTy. We have content negotiation headers for that.
* Yes, I know you can work around some of this with GraphQL. Does Shopify intend to deprecate the REST API? I am not personally a big fan of GraphQL, but if the REST API is getting dustbinned, I'd like to know about it.
Mind you, this is just what I noticed from a few days of working with the API.
PS IMO, 3x isn't much of a flash! 300x, now we're talking :)
Your parent is using Taobao ( Which is like the Amazon in China ) as example. Which means the word "scale" means completely different thing. I am willing to bet Taobao's RPS on singles day is a lot higher than the highest RPM during Shopify's biggest flash sales.
That said there is no reason to believe that pod in shopify cant handle that scale.
I've had a love/hate relationship with the Shopify platform but when it comes to the API it was all love.
If anyone ever brings up the idea of building out oauth or even vaguely user management, I try to point them to at least try a POC (Proof of Concept) with https://www.keycloak.org/ (Apache 2.0 License) or https://www.gluu.org/ (MIT License) before they considering building.
OpenLDAP
├── PrivacyIDEA (TOTP/MFA with LDAP auth backend)
├──---└── SAML iDp (e.g. SimpleSAMLphp or Shibboleth) for SSO: AWS, Google, Github, Atlassian, Snowflake, Azure etc.
├── Dex (https://github.com/dexidp/dex) for anything that wants Oauth flow
├── Native LDAP for apps that support it (e.g. Metabase, Grafana)
├── Any other custom authT that supports LDAP as a backend
I’m happy to be proven wrong about this. I love open standards and protocols.
But honestly, every language has to make some trade-offs. Even if PHP has some things that are bad design choices as opposed to trade-offs, it can still be worthwhile to put up with them if you have an existing codebase written in it or want to take advantage of libraries that don't exist in other languages.
As a result I don't consider any language as bad or old school. They might have downsides but 1) the upsides might outweigh them in certain use-cases and 2) any competent developer should be able to make something good with any Turing-complete language so I don't judge by the language alone, and especially not by the "hype factor" of the language.
(not actual examples–I have luckily forgotten)
That being said, I have yet to see a NodeJS framework beat developer productivity in Symphony or Larvel for CRUD stuff. Django maybe.
PHP hasn't been considered 'bad' for a few years now. It has been battle tested on many large-scale websites.
Now, One of the main issues is that anyone can write a few lines of code and call themselves a developer, so you have horrible code bases still out there...but this has more to do with the developer than the language.
"my answers to which are "not enough" and "not even if you paid me", respectively
I love hearing answers like this. This is why I'm still paid so well to write PHP code after 15 years in the industry.
I don't think lack of support for OAuth is a problem here. OAuth is specifically designed to obtain access to an HTTP service[1], and OpenID Connect is specifically designed for OAuth. LDAP is not an HTTP service.
Crystal just doesn't have the community that ruby has, and as teams reconsider certain aspects of their applications, languages like Elixir and Go make way more sense.
Creating a compiled language with similar syntax to Ruby is great, but there's so much more involved than just that if you're talking about building a serious, commercial product. And unfortunately, I just don't see Crystal getting that kind of traction in the short term.
But Crystal has entirely different semantics to Ruby. They look vaguely similar at a superficial level, but the semantics are not even remotely similar.
I am porting a relatively simple ruby app to Crystal to see how it is and most of it is copy/paste and then some fixing and adding type definitions where needed. The only issues I have had are where I am using a rubygem that doesn't have a crystal counterpart.
For example a pretty simple Gem that connects to a socket and parses incoming data was exceedingly easy to port, including tests.
* https://gitlab.com/overlord-bot/tacview-ruby-client
* https://gitlab.com/overlord-bot/tacview-crystal-client
For a more specific example these two files are almost identical:
* https://gitlab.com/overlord-bot/tacview-ruby-client/blob/mas...
* https://gitlab.com/overlord-bot/tacview-crystal-client/blob/...
The only thing not done was CRC hash calculating for a password because there was no crystal shard for it and it was low priority so I didn't write one.
I merely pointed out that "depending on what you are doing they (semantic differences) are minor considerations." and then gave an example where that was the case with a simple application I was working.
If you want a more complicted library that was ported from Ruby to Crystal then you can also look at Sidekiq ( https://www.mikeperham.com/2016/05/25/sidekiq-for-crystal/ )
The original comment said 'without needing to really think in a whole different programming experience'. Ruby without #send and #eval is a very much a whole different programming experience for a company like Shopify.