The PDF is linked in the article:

https://www.nature.com/articles/s41586-021-03819-2_reference...

I missed an important detail: """an academic team has developed its own protein-prediction tool inspired by AlphaFold 2, which is already gaining popularity with scientists. That system, called RoseTTaFold, performs nearly as well as AlphaFold 2, and is described in a paper in Science paper also published on 15 July"""

One of the things I say about CASP has to be updated. It used to be "2 years after Baker wins CASP, the other advanced teams have duplicated his methods and accuracy, and 4 years after, everything Baker did is now open source and trivially reproducible"

now, it's baker catching up to DeepMind and it took about a year

https://doi.org/10.1126/science.abj8754

Also announced today was RoseTTAFold from UW's Baker Lab, which claims nearly the same accuracy at much higher efficiencies. There's a public server and paper in Science.

More info here and here:

https://www.bakerlab.org/index.php/2021/07/15/accurate-prote...

https://techcrunch.com/2021/07/15/researchers-match-deepmind...

Alphafold 2 is very very cool, but we need a little dose of reality. It's still a bit away from really solving protein folding as it was marketed.

For example, multi-complex proteins are not well predicted yet and these are really important in many biological processes and drug design:

https://occamstypewriter.org/scurry/2020/12/02/no-deepmind-h...

A disturbing thing is that the architecture is much less novel than I originally thought it would be, so this shows perhaps one of the major difficulties was having the resources to try different things on a massive set of multiple alignments. This is something an industrial lab like DeepMind excels at. Whereas universities tend to suck at anything that requires a directed effort of more than a handful of people.

Fantastic, they released the dataset and code to train the model. Science will be able to proceed. edit: not the code to train the model, just the code to run inference.

The underlying sequence datasets include PDB strucrures and sequences, and how those map to large collections of sequences with no known structure (no surprise). Each of those datasets represents decades of thousands of scientists work, along with programmers and admins who kept the databases running for decades with very little grant money (funding long-term databases is something NIH hated to do until recently).

This isn't a criticism - I'm just curious to hear people's thoughts on this. When I look at this code, one of my initial reactions is that it does not seem to be very thoroughly tested. Sure, certain modules have been tested (e.g. `model.quat_affine`) but it's not clear how completely. Meanwhile, other modules, for example `model.folding`, have not been tested at all, despite containing large amounts of complex logic. That kind of code that works with arrays is very easy to get wrong and bugs are difficult to spot.

My experience working with code written by researchers is that it frequently contains a large number of bugs, which brings the whole project into question. I've also found that encouraging them to write tests greatly improves the situation. Additionally, when they get the hang of testing they often come to enjoy it, because it gives them a way to work on the code without running the entire pipeline (which is a very slow feedback loop). It also gives them confidence that a change hasn't lead to a subtle bug somewhere.

Again, I'm not criticising. I am aware that there are many ways to produce high quality software and Google/DeepMind have a good reputation for their standards around code review, testing etc. I am, however, interested to understand how the team that wrote this think about and ensure accuracy.

In general, I hope that testing and code review become a central part of the peer review process for this kind of work. Without it, I don't think we can trust results. We wouldn't accept mathematical proofs that contained errors, so why would we accept programs that are full of bugs?

edit: grammar

> The AlphaFold parameters are made available for non-commercial use only, under the terms of the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license. You can find details at: https://creativecommons.org/licenses/by-nc/4.0/legalcode

Does CC BY-NC actually do this? As far as I can tell it only really talks about sharing/reproducing, not using.

Or is the only thing prohibiting other commercial use the words "available for non-commercial use only"?

I am a structural biologist. This is one of the handful of topics that overlaps with my field here. I'm very excited to play with this, although it might eventually put me out of a job.

Does anyone on HN work in bio or drug discovery?

Could you give an overview of how people can leverage this (or how you might?).

From reading around about it, it sounds like there's often a need to find a certain type of molecule to activate/inhibit another based on shape and the ability to programmatically solve for this makes the searching way easier.

Is this too oversimplified/wrong? How will this be used in practice.

[Edit]: Thanks for the answers!

Honest question: since AlphaFold doesn't really _solve_ the protein folding problem (it's NP-complete after all), but only _approximates_ solutions very well, what are the real impacts of this? Isn't a good approximation of a protein enough to cause unexpected problems? How do we know that an approximate structure will perform the same as the correct solution?

Ok, so biochemists: which bit of the secret sauce are they leaving out?

The model parameters are only available for non-commercial use. That's a shame, as I presume there might be a lot of medical startups that would benefit from having this kind protein-folding tech available.

I'm assuming you can't run this on any consumer computer?

Distribution of this 2 TB file seems like a good use of torrent…

So... is it possible to clone this and turn it into a Folding@Home client? How does it do?

Does anyone know if this can be made to work with rna fold?

edit I was wrong. Please ignore.

fodl

Honest question: since AlphaFold doesn't really _solve_ the protein folding problem (it's NP-complete after all), but only _approximates_ solutions very well, what are the real impacts of this? Isn't a good approximation of a protein enough to cause unexpected problems? How do we know that an approximate structure will perform the same as the correct solution?

>A disturbing thing is that the architecture is much less novel than I originally thought it would be, so this shows perhaps one of the major difficulties was having the resources to try different things on a massive set of multiple alignments.

A similar concern has sparked some worries about "AI overhang" https://www.lesswrong.com/posts/75dnjiD8kv2khe9eQ/measuring-...

Most of the compute in ML research seems to be going into architecture search. Once the architecture is found, training and net finetuning/transfer learning is comparatively cheap, and then inference is cheaper still. This implies we could see 10-100x gains in AI algorithms using today's hardware, or sudden surprising appearance of AI dominance in an unexpected field. (Object grasping in unstructured environments? Art synthesis?) A task could go from totally impossible to trivial in a year. In retrospect, the EfficientNet scaling graph should have alarmed more people than it did: https://learnopencv.com/wp-content/uploads/2019/06/Efficient...

Waymo has been puttering along for years, not announcing much of interest. This may have caused some complacency about self-driving cars, which is a mistake. Algorithms only get better, while humans stay the same. Once Waymo can replace some human drivers some of the time, things will start changing very quickly.

> A disturbing thing is that the architecture is much less novel than I originally thought it would be, so this shows perhaps one of the major difficulties was having the resources to try different things on a massive set of multiple alignments. This is something an industrial lab like DeepMind excels at. Whereas universities tend to suck at anything that requires a directed effort of more than a handful of people.

Yeah, the HN commentary on Alphafold has a high heat-to-light ratio. I'm eager to read the paper because the previous description of the method sounded remarkably similar to methods that have been around for ages, plus a few twists.

The devil is going to be in the details on this one.

I haven't read the full paper but there is certainly some new/exciting developments i'm seeing just from scanning. The “Invariant Point Attention" which is described as a novel, geometry-aware and equivariant attention operation is pretty huge.

Something along these lines was speculated to be to be used by Fabian Fuchs [0] soon after the original CASP competition. Basically, it's a huge win for the geometric deep learning people, and indicates an exciting direction for mainstream academia to move in.

https://fabianfuchsml.github.io/alphafold2/

I'm genuinely curious: could the output of Alphafold be fed into a classical folding algorithm (as a starting point), or is the output of Alphafold too far down the wrong path, in these cases?

Why is it disturbing? Isn't that just a values-neutral outcome, or, are you saying it's disturbing from the perspective of academia?

many of these resources are available, it's mostly that academic scientists don't have the time, money, or expertise to manage large datasets. However, the community has maintained high quality MSA database for decades and that's exactly the work that DM drafted off.

Yes, I skimmed the paper already and it wasn't too surprising. There are details that will take some time to parse out to understand how important they are.

Personally, I've found over decades that academic papers like that are far less useful to me than a github project and downloadable data that I can inspect, run and modify on my own. Other folks I know could read that paper and write the code in a day, I always wish I could do that.

My understanding is that it has been manually tested. I.e. it has produced correct results to previously intractable problems. I'm not sure how much automated testing would add at that point.

Prior to this model, protein folding hadn't seen significant advancements in a decade or more. Worrying about the lack of tests in a first of its kind model is very much akin to complaining about the choice of font in a user manual for the world's first warp drive. I understand you're attempting to frame the problem in terms in things you know, but trying to weigh down pioneering research with professional development ceremony is very much counterproductive. The 'missing' ceremony would not have contributed to the strength of AlphaFold's result, the model's only purpose was to compete within the context of an existing validation framework.

Because it passes the huge number of integration tests.

Research code is highly volatile: the details and architecture changes a lot. It is much more important to invest the time into writing more experimental code and validate it with e2e functional tests that don't need to change, than to constantly having to rewrite the code and the tests.

Here's where I think we need to be going: You go to a doctor's office, sick. 1) They take a blood sample. 2) They find the malignant bacteria and DNA sequence it. 3) If it's a known strain, they know what antibiotics to use on it. 4) If not, they solve protein folding on the genes. 5) From that, they see which existing antibiotics would kill it. 6) If none will, then given the proteins, they have to derive a new antibiotic.

1) is easy. 2) might not be - there can be a lot of things in a blood sample, and finding only the interesting (bad) things might not be simple. The sequencing part is pretty much solved. 3) would take a bit of work, but I think it's possible now. 4) we're getting there. 5) might have a fair amount in common with 3), but it probably takes some additional work. 6) is... probably non-trivial.

That's just one research agenda. There are others. You may have to move to related work, but I doubt you're going to be out of a job in this lifetime.

why would it put you out of job? Wouldn't it just become one of the tools you use?

I've worked in bio and drug discovery for some 25 years. That includes building classifiers using gradient descent in the 90s (when algorithms, computers and data were all much worse). I ported DOCK to Linux in ~96 or 97. Since then I built an academic and then industrial career with some emphasis on using computing to solve problems in drug discovery, but I don't play that role any more.

It doesn't look like the models produced by this would immediately turn the challenging problem of finding, approving, and marketing successful pharmaceuticals (IE, it doesn't eliminate any real bottleneck).

There was a long-term dream of structure-based drug discovery based on docking, but IMO, it has never really proved itself (most of the examples of success are cherry picked from a much larger pile of massive failures).

I haven't worked on the drug-side of things, but here my bio perspective: It's kind of out-of-vogue, but consider the "lock and key" model of proteins and small molecules (drugs). For drug design, what you want to do is get a key that fits just one lock (to pull whatever lever) and not others (to avoid side-effects). It's relatively easy to find a molecule that fits a protein, because that protein is what you might spend years researching and probing, but it's tricky to check if it does anything against ~100,000 others in humans. If you could do an in silico computational survey to be like, oh, maybe it'll target this accidentally, you could spot-check those in vitro, and/or stick on some other atoms to your small-molecule to make it not fit that off-target.

Holy grail, IMO, though is being able to design de novo protein sequences (to make "biologics", aka engineered protein drugs) that can a) target (bind/block/enhance) or do (chemical reactions) what you want and only that, b) are easily synthesizeable by bacteria/yeast (cheap to make), and c) are stable (easy to transport/store).

> Could you give an overview of how people can leverage this (or how you might?).

Short answer: nobody knows. Traditionally, protein folding is a solution in search of a problem, but that's largely because the predictions were...unusably bad. This was always more of a super-difficult validation problem for the force fields and simulation methods, which could then be used for other problems of greater value (such as rational protein design, or simulation of the motion of proteins with known structures).

These predictions are better, but still pretty far from the level of precision that you'd want for any kind of rational drug design, where the exact locations of protein side-chains (for example) matter a lot. You'll note that AlphaFold returns structures that are "relaxed" using one of the oldest simulation systems for proteins: AMBER. So it's not exactly a clean-room solution to the problem, and you can't assume that the details (which matter to drug design) are going to be any better than for the older methods.

But that said, if you have a method that can reliably give you a blurry view of the overall shape of a protein, even that could be useful for things like target discovery or inference of biological networks. But this is still a lot closer to pure research than "revolutionizing drug discovery", as is frequently batted around on reddit, HN and the press.

You can't do intelligent drug design if you don't know what the target protein looks like. We've gotten great at solving protein structures with things like crystallography and cryo-EM microscopy. Unfortunately, many interesting drug targets reside in the membrane of a cell, which means you can't easily work with them in a lab because they aren't soluble in anything but a plasma membrane. For instance, this is an issue with the 5HT2A protein, a g coupled protein receptor that is implicated in many serotonin related pathways.

Being able to predict what it would look like would be a huge deal because then you can go about intelligently designing drugs for it.

It can be an aid in drug development, and can perhaps assist a bit in tuning small molecule drugs for more stable binding.

Though I think the major impacts will be two-fold:

(1) The field of structural biology is going to see a change, with much more data available. Some structures of difficult to crystallize proteins will be solved, which may lead to much greater biological understanding. We may enter a time, where once you have a primary sequence, you also have a likely 3d-structure, which will probably change the daily work of quite a few biologists a bit.

(2) Industrial protein design. A tool such as this can potentially have great utility in optimizing proteins as chemical catalysts for various processes in different industries. This includes expanding the conditions under which a protein is active and also making their conformation more stable and so the protein more long-lived in solution.

I work in cancer research with a drug discovery focus in a lab with some structure biologists. My understanding is that if we identified proteins targets suitable for therapeutics then understand its structure to identify secondary binding sites could be crucial for drug discovery. Drugs can then be designed to modulate its biological functions.

> (it's NP-complete after all)

Protein folding is a physical/biological phenomenon. AFAIK we don't currently have a proper exact mathematical formulation of the problem that would let one determine its complexity.

You may be referring to this paper [1]. It only claims that one particular optimization problem, believed to give a solution to protein folding problems, is NP-hard. So, even if a suitable exact formulation exists, it is not yet proven that protein folding is hard, although it for sure seems plausible.

By the way, it is perfectly possible today to solve some very large-scale NP-hard problems (think millions of variables and constraints) in reasonable amounts of time (think minutes or hours). Examples are knapsack problems, SAT problems [2], the Traveling Salesman Problem [3] or more generally Mixed Integer Programming [4].

[1] "Complexity of protein folding", 1993, by Aviezri S. Fraenkel

[2] http://www.satcompetition.org

[3] http://www.math.uwaterloo.ca/tsp/

[4] http://plato.asu.edu/bench.html

The protein folding problem is not NP complete. The "formal" protein folding problem, as posed (find the set of dihedral angles whose resulting structure has the lowest energy) might be, but that bears only a distant resemblance to how people "solve" the problem today. At the very least, the statement is incorrect because many proteins don't actually fold to their energy minimum, they get stuck in kinetic traps, and the formal PF defintion never accomodated that idea.

Not really an answer to your question, but is the problem really NP-complete, or just combinatorially difficult? For example how is this condition of NP-completeness satisfied?

> it is a problem for which the correctness of each solution can be verified quickly [0]

[0] https://en.wikipedia.org/wiki/NP-completeness

I dont know much about protein folding, but for most things in life,exact solutions to NPC problems usually aren't needed for non-contrived problems. In many cases, approximations are good enough.

Besides, this is real life - if predictions and real life match, that's great. If they don't, well you know you went wrong somewhere.

A very-non-expert opinion, if an approach approximates it pretty well and can be improved upon, then it could end up being quite useful. Given that biology exists on a real, tangible scale then perfection in the fold prediction isn't necessary, instead just an approximation that is sufficiently good to be functionally useful.

^ That sounds like word-salad BS but I think there's some truth to it. I know protein folding has been postulated to be useful in terms of understanding basic biology, understanding disease pathology, and drug prediction. While a wide range of approximations are functionally useless, perhaps the Alphafold approach or some improved version of it surpasses the functionally useful threshold.

At least I hope so

Yes, it is still useful. Even structures obtained through traditional means (eg. x-ray crystallography) are approximations to an extent since there are limits to the resolution that you can obtain and oftentimes regions of proteins are "disordered". Additionally, these structures are only snapshots of a protein in a particular state, which may not completely reflect the dynamics of the protein in its native environment.

You want to find a protein that has X structure (since structure determines function to a degree).

If AlphaFold is substantially more accurate at solving proteins, it can mean that drug discovery is faster, assays are faster, etc. etc.

The "unexpected problems" would be caught in the assay stage.

I would expect that once AlphaFold has helped you identify a potential protein (e.g. as a drug) out of a bigger set of potential proteins, there will still be a manual step of traditional cryoEM, NMR, etc. to get an accurate high-resolution structure.

To me, the interesting thing is not the specific results but rather that you can accurately predict crystal structures from sequence alone. This begets the question: what other physical biological properties can we predict?

Is it really np complete? If so we could map other np complete problems onto it and let biology solve it for us.

NP completeness tells you about the hardest cases, not the most useful cases.

AlphaFold is not about solving any kind of NP-complete problem.

Proteins consist of chains of amino acids which spontaneously fold up to form a structure. Understanding how the amino acid chain determines the protein structure is highly challenging, and this is called the "protein folding problem".

People use mathematical models to predict how proteins fold in nature. Many such mathematical models are stated in terms such as "proteins fold into a configuration that minimizes a certain energy function". Even the simplest such models [1] give rise to NP-hard decision problems, which are also known (somewhat confusingly) as "protein folding problems". To make this a bit less confusing, I will call the mathematical decision problems PFPs.

Like all mathematical models, our protein folding models don't correspond exactly to reality. Even if you are somehow able to determine the exact mathematical solution to a mathematical PFP, that _still_ doesn't guarantee that the real protein that you were trying to model behaves like the mathematical solution would indicate. E.g. the protein may fold in such a way that it gets stuck in a local optimum of the energy function you were using.

How do we detect this? We make inferences about how the protein should behave, given the mathematical solution to the Protein Folding Problem, and then we perform experiments, and find out (empirically) that the protein behaves in a manner that is inconsistent with the inferences drawn from the mathematical model. Scientists _do_ do this. And they would have to do it even if they had a fast, exact way to solve NP-complete problems, because the NP-complete problems are still just part of a mathematical model, and need not correspond to reality in any way.

The success of AlphaFold is not measured by how well it solves (or approximates) mathematical PFPs. The success of AlphaFold is measured by making successful predictions about how certain proteins will fold. And this is exactly how it was tested [2]: they threw it at a bunch of problems for which scientists have empirically determined how certain amino acid chains fold, but didn't release the results. And then they compared the solutions predicted by AlphaFold, and found that most of the predictions were consistent with what they knew to be the case.*

[1] https://en.wikipedia.org/wiki/Lattice_protein

[2] https://predictioncenter.org/casp14/index.cgi

* That's an understatement. The solutions were really very good, much better than those produced by any other submission to CASP14.