Category Theory ∩ Machine Learning(github.com) |
Category Theory ∩ Machine Learning(github.com) |
Hopf algebras (which are really just tensors with recurrence relations built in) subsume convnets, transformers and diffusion model and also provide a theoretically better autodiff that operates within single layers as opposed to across entire graphs.
Furthermore, there is a correspondence between Hopf algebra and cyclical linear logic and Hopf algebras are related to zonotopes, which are polyhedra that have been used in verified numerical computation. I'm strongly convinced the LL connection can provide proofs over zonotopes which paves the way towards interpretable AI and will be central for XAI.
I know this sounds too good to be true but Persi Diaconis has also written a paper that shows how useful Hopf algebras are in the context of Markov chains https://arxiv.org/abs/1206.3620
I'm working on a next gen Hopf algebra based machine learning framework.
Join my discord if you want to discuss this further https://discord.cofunctional.ai.
====
My account is currently rate limited so I will use this comment to respond to comments below.
red_trumped: What about Hopf algebras do I not understand?
gaze: Haha, it's been a while since I have commented about QC. What do I not understand about it? And what comment are you referring to?
You are right, there's a missing sentence fragment, "standard tensor product that satisfies the property...".
Read the Diaconis paper. "
--- This isn't a sock puppet and I hope this isn't against site rules. I just wanted to try and help facilitate good discussion. I think trumpet brought up some interesting criticisms and felt Adam had a legitimate interest in responding ASAP.
https://arxiv.org/abs/2302.01834 appears to have a typo in section 4.5
S(hg) = S(g)S(g)
looks like it should be S(hg) = S(h)S(g) or S(hg) = S(g)S(h)
My mind is blown.
Is the Hopf Algebra based ML framework you are working on on your github? I took a glance, but you have 1500 repositories and it wasn't on the first few of them.
What's the most compelling application so far?
The exponential growth in CS publication is much faster. This repository is simply a testament that CT is slowly ramping up.
It's meant to show what kind of expressive power and breadth current CT models have, which to my knowledge isn't something that's well-known outside of our niche community.
So ... yes?
The OP GitHub site doesn't promote any material that introduces the concepts at all. The "survey" paper at the top is nigh-impenetrable. I'm sure the category theorists are having fun modelling machine learning, but it doesn't show how machine learning benefits from the category theory.
I'm still shocked no one has developed language learning software along these lines. I had a prototype in the works for thai years ago but never got time to get it off the ground. using statistical models trained on web corpus for a language learning app seems like a no brainer.
think of it like navigating a word as a point in a graph connected to every example context it is in, with associated words being clickable into similar context bundles. then make it differential between host and target language given a translation so you can see which contexts the translation fails and succeeds in.
this being math, I wonder to which extent can category theory be re-expressed in terms of sets.
perhaps a better question is if category theory can be re-expressed (or founded on) functions?
lastly, I wonder if category theory can be expressed in terms of functions (i think maybe it can, without sets?) why shouldn't it be expressible in terms of sets (for some reason I don't think just sets are sufficient, may have to define functions (which possible in terms of sets) before 'expressing' categories starting with set theory)?
Alternative foundations of mathematics (Set Theory, Category Theory, Type Theory, and all their variations) can all mutually interpret the other by just postulating sufficiently large universes. You don't pick or advocate one based off its ability to encode mathematics, but instead based on its ability to express your intention and ideas.
Really its no different from programming language preference in my book.
Since you are concerned with sets and functions, the following analogy is helpful to build your intuition for the subject.
Dimension 0: sets
Dimension 1: functions
Dimension 2: commutative squares
Dimension 3: commutative cubes
The majority of categories, expressed in terms of structured sets and functions, never touch on the second dimension. That is the domain of 2-categories, which although they have three types of elements are nonetheless considered to be two dimensional because we count from zero. That is also where commutative squares come in to play, because as you can imagine squares are quite obviously two dimensional.
Functions are 1-dimensional lines or arrows from one place to another. Sets are more analogous to points then anything else, and so naive set theory is zero dimensional. But I think you have the wrong question. You should ask the opposite question: what if functions can be re-expressed or founded on higher dimensional category theory?
The reason you're looking for is that the category of sets is not a set.
Objects, not categories.
yoneda
You are right, it can be more polished but also it covers quite a lot and it was surprisingly hard to fit it all in.
The main response I have been getting though is "I won't believe this until I see the implementation" so I have been concentrating on that.
And AGI is totally irrelavent here.
That doesn’t seem to make any sense.
But even then, you cannot define "C" as "C \otimes C", because the right hand side only makes sense if "C" is already defined. And in math you cannot define something twice. As soon as you defined something, it stays the same in the given context.
Think of it as a signature in say Rust
fn coproduct(in: C) -> (C, C)
Also you were talking about my knowledge of Hopf algebras, this is not knowledge of Hopf algebra but quibbling about things that are pretty clear from context.
For example, if someone works with a probability space (Ω, F, P), they would state so very clearly in their paper even though it is quite obvious from the notation that it is supposed to be a probability space.
Similarly, if someone writes "Given a symmetric monoidal category C with tensor product ⊗...", it is understandable what is going on, or at least it is understandable what I should look up. But in that paper I have no idea how I am supposed to interpret "C", "⊗" and "=" in such a way that the formulas make sense.
> It's a type signature, not a formula. C is not values.
I am not sure what distinction you are trying to draw, but surely any sequence of symbols that adheres to a given formal grammar is a (well-formed) formula; and surely if by value you mean a mathematical object, types and type signatures are values.
Don't read it, it's apparently too stressful for you. You are right, I assume some knowledge but I don't know if this discourse is productive, you are too hung up on things that don't matter.
C is general. Coproduct is a general construct.
Read the Diaconis paper first, you might get what I'm getting at.
- programming languages with semantics that are better adapted to specific problems (e.g. Rust's ownership);
- better compilers (see e.g. Haskell's supercompiler, which puts to shame `constexpr`-style features);
- better static analyzers (e.g. better type systems, abstract interpretation, model checkers).
In the case of Machine Learning, it might some day help us create Machine Learning that we can understand and trust better. Or it might fail. Or it might help us invent something different entirely, in 30 years.
But it is my understanding that Rust's ownership traces its roots back to both linear logic (well, affine logic) and region-based resource management, both of which have formal semantics.
I haven't followed what got merged into GHC, but I remember seeing demos (and a paper) of a Haskell supercompiler during a conference many years ago, so it is something.
In my sleep-deprived brain, supercompilation is directly related to algebraic effects (although probably not in Haskell itself), which are themselves related to category theory. I could be wrong.
I suspect that Apple is also using model-checking at various layers, but I have no proof :)
You're completely right that the repo doesn't promote any introductory material. CT is notoriously difficult to get into, and this repository wasn't meant to be a pedagogical one, but rather a list of all the relevant papers.
Though, I'll see about remedying this. I have a different repository that curates a list of (what I consider to be, as a CS major) relatively approachable CT introductory materials https://github.com/bgavran/Category_Theory_Resources
and it might be a good idea to add a pointer to it.
A lot of computer programmers who never studied Abstract Algebra or Type Theory before think that all of that is Category Theory because Haskell is their first exposure to all that, and Category Theory is a big meme in Haskell, thanks to Monad.
\s