C4 is a standardized identification system that separates file description from file content so they can be stored or transmitted independently. You can send a small text-based filesystem description by email while the file content is fulfilled by content ID from any source. Running c4 on any path produces a c4m file and stores the content: C4 solves a general problem in distributed computing, inspired by digital media production — where projects span millions of files across terabytes of content, bulk transfers move on shuttle drives, and work is distributed across organizations and data centers. File paths are unreliable identification in this environment. C4 IDs remove the dependency on location — any system that can return content given an ID becomes a valid source: object storage, production SANs, shuttle drives, another workstation. Because the c4m travels independently of content, you can do a remarkable amount of work before a single byte arrives. Pre-organize an incoming delivery into your pipeline's directory structure. Diff two project states to plan what needs to transfer. Deduplicate across vendors to skip terabytes you already have. Merge deliveries from multiple sources into a unified view. The content catches up later from whatever source is fastest — the c4m already tells it where to go. c4sh adds shell utilities that make c4m files behave like mounted filesystems in your existing terminal. cd project.c4m and you're inside — ls, mv, rm, cat all work on the description. cp ./footage/ project.c4m: captures files in; cp project.c4m:shots/ ./workspace/ extracts them out. pool bundles a c4m with its content for transport; ingest absorbs it on the other end. c4: https://github.com/Avalanche-io/c4 c4sh: https://github.com/Avalanche-io/c4sh Also: Python, TypeScript, C, Swift implementations Install: brew install mrjoshuak/tap/c4 — or — pip install c4py — or — npm install @avalanche-io/c4 All Apache 2.0. Solo developer. Happy to answer questions. |