Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename web3.storage to Storacha in Documentation #98

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15,137 changes: 15,137 additions & 0 deletions package-lock.json

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@
"test": "echo 'no tests yet'"
},
"dependencies": {
"next": "14.0.1",
"next": "12.0.0",
"nextra": "^2.13.2",
"nextra-theme-docs": "^2.13.2",
"react": "^18",
"react-dom": "^18",
"react": "^17.0.2",
"react-dom": "^17.0.2",
"shiki": "^0.14.5"
},
"devDependencies": {
Expand Down
4 changes: 2 additions & 2 deletions src/pages/docs/concepts/_meta.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"ucans-and-web3storage": "UCANs and web3.storage",
"ucans-and-storacha": "UCANs and Storacha",
"content-addressing": "Content-addressing",
"upload-vs-store": "Upload vs. Store",
"architecture-options": "Architecture options",
Expand All @@ -13,4 +13,4 @@
"ucan": {
"display": "hidden"
}
}
}
20 changes: 10 additions & 10 deletions src/pages/docs/concepts/architecture-options.mdx
Original file line number Diff line number Diff line change
@@ -1,25 +1,25 @@
import { Callout } from 'nextra/components'

# Possible architectures using web3.storage to upload
# Possible architectures using Storacha to upload

UCAN opens up a number of options in how to integrate with w3up: Should you, the developer, own the Space? Should you delegate permissions to your users? Or should your user own their own Space? Broadly, there are three ways to integrate:

- **Client-server**: You (the developer) own the space, your users upload data to your application server, and you upload it to web3.storage from there.
- **Client-server**: You (the developer) own the space, your users upload data to your application server, and you upload it to Storacha from there.
- **Delegated**: You own the Space, but you provide your user with a delegation that allows them to upload directly to the service. This avoids proxying the upload through your backend (no egress from your infrastructure).
- **User-owned**: Your user owns the Space and registers it and they use it to upload directly with the service; if you want to instrument visibility into what they're uploading, you'll have to write separate code in your app for it.

In the How-tos section of the docs, we focused on the first two options, as they are the most typical today. However, you can implement each of these in a number of ways, but we talk through some considerations when implementing a given option.

## Client-server

In this set up, you (the developer) own the space, your users upload data to your application server, and you upload it to web3.storage from there.
In this set up, you (the developer) own the space, your users upload data to your application server, and you upload it to Storacha from there.

```mermaid
sequenceDiagram
participant User
w3up-client in backend-\>\>w3up-client in backend: Client set with Agent with delegation from Space
User-\>\>w3up-client in backend: Upload data
w3up-client in backend-\>\>web3.storage w3up service: Upload data
w3up-client in backend-\>\>storacha w3up service: Upload data
```

You'll need a registered Space, and your client in the backend to have a delegation to use the Space.
Expand All @@ -40,12 +40,12 @@ In this set up, you own the Space, but you provide your user with a delegation t
sequenceDiagram
participant w3up-client in user
participant w3up-client in backend
participant web3.storage w3up service
participant storacha w3up service
w3up-client in backend-\>\>w3up-client in backend: Client created with Agent and delegation from Space
w3up-client in user-\>\>w3up-client in user: Client instantiated with default Agent
w3up-client in user-\>\>w3up-client in backend: Request delegation with user's Agent DID
w3up-client in backend-\>\>w3up-client in user: Send delegation from Space to user's Agent DID
w3up-client in user-\>\>web3.storage w3up service: Upload data
w3up-client in user-\>\>storacha w3up service: Upload data
```

Typically `w3up-client` will be running in your end-user's client code, as well as backend code that's able to generate UCANs that delegate the ability to upload and pass them to your users (e.g., `w3up-client` running in a serverless worker).
Expand Down Expand Up @@ -144,11 +144,11 @@ In this set up your user owns the Space and registers it and they use it to uplo
sequenceDiagram
participant User
participant Application backend
participant web3.storage w3up service
participant storacha w3up service
Application backend-\>\>User: Front end code that includes w3up-client
User-\>\>web3.storage w3up service: (If needed) Create Space and register it
User-\>\>web3.storage w3up service: (If needed) Use Agent email verification to "log in" to Space
User-\>\>web3.storage w3up service: Upload data using w3up-client
User-\>\>storacha w3up service: (If needed) Create Space and register it
User-\>\>storacha w3up service: (If needed) Use Agent email verification to "log in" to Space
User-\>\>storacha w3up service: Upload data using w3up-client
```

If you want your user to own their own Space, you'll typically be relying on the `w3up-client` methods to create a Space, authorize the Space, and authorize the Agent on the user-side; afterwards they can run any of the upload methods.
Expand Down
24 changes: 12 additions & 12 deletions src/pages/docs/concepts/car.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,28 @@
# Working with Content Archive (CAR) files

When you upload files to web3.storage using the client library, your data is converted into a graph of data structures, which are then packed into a format called a Content Archive (CAR) before being sent to the web3.storage service.
When you upload files to Storacha using the client library, your data is converted into a graph of data structures, which are then packed into a format called a Content Archive (CAR) before being sent to the Storacha service.

For most use cases, you never need to know about this process, as the conversion happens behind the scenes when using the client library. However, you might want to know more about how to manipulate CAR files directly, especially if you're using `upload` and `store` separately in web3.storage.
For most use cases, you never need to know about this process, as the conversion happens behind the scenes when using the client library. However, you might want to know more about how to manipulate CAR files directly, especially if you're using `upload` and `store` separately in Storacha.

## What is a Content Archive?

The [Content Archive format](https://ipld.io/specs/transport/car/) is a way of packaging up [content addressed data](https://web3.storage/docs/concepts/content-addressing/) into archive files that can be easily stored and transferred. You can think of them like [TAR files](https://en.wikipedia.org/wiki/Tar_\(computing\)) that are designed for storing collections of content addressed data.
The [Content Archive format](https://ipld.io/specs/transport/car/) is a way of packaging up [content addressed data](https://storacha.network/docs/concepts/content-addressing/) into archive files that can be easily stored and transferred. You can think of them like [TAR files](https://en.wikipedia.org/wiki/Tar_\(computing\)) that are designed for storing collections of content addressed data.

The type of data stored in CARs is defined by [IPLD](https://ipld.io/), or InterPlanetary Linked Data. IPLD is a specification and set of implementations for structured data types that can link to each other using a hash-based Content Identifier (CID). Data linked in this way forms a Directed Acyclic Graph, or DAG, and you'll likely see a few references to DAGs in the documentation for IPLD and IPFS.

IPFS files are one example of IPLD data, but IPLD can also be used to access data from Ethereum, Git, and other hash-addressed systems. You can also use IPLD as a general purpose format for your structured data, sort of like a Web3-flavored JSON. See Advanced IPLD formats below for more information.

## CARs and web3.storage
## CARs and Storacha

When the web3.storage client and CLI pack up regular files into a CAR to store on IPFS, the CAR contains data encoded in the same format used by IPFS when importing files using the command line or other IPFS APIs.
When the Storacha client and CLI pack up regular files into a CAR to store on IPFS, the CAR contains data encoded in the same format used by IPFS when importing files using the command line or other IPFS APIs.

This format uses an IPLD "codec" called [dag-pb](https://ipld.io/docs/codecs/known/dag-pb/), which uses [Protocol Buffers](https://developers.google.com/protocol-buffers) to encode an object graph. Inside the graph are [UnixFS objects](https://docs.ipfs.io/concepts/file-systems/#unix-file-system-unixfs) that describe the files and their contents. We do this client-side for a few reasons.

First, formatting everything on the client allows us to calculate the root Content Identifier for the data you're uploading before we send any data to the remote service. This means that you can compare the CID returned by the web3.storage service to the one you calculated locally, and you don't have to trust the service to do the right thing.
First, formatting everything on the client allows us to calculate the root Content Identifier for the data you're uploading before we send any data to the remote service. This means that you can compare the CID returned by the Storacha service to the one you calculated locally, and you don't have to trust the service to do the right thing.

Another reason to use CARs is to support large files, which would otherwise hit size limits on the web3.storage backend platform. The data in a CAR is already chunked into small blocks, which makes CARs easy to shard into small pieces that can be uploaded in batches. This also enables the web3.storage platform to get larger content into Filecoin deals.
Another reason to use CARs is to support large files, which would otherwise hit size limits on the Storacha backend platform. The data in a CAR is already chunked into small blocks, which makes CARs easy to shard into small pieces that can be uploaded in batches. This also enables the Storacha platform to get larger content into Filecoin deals.

CAR files are a format that pretty much any IPFS tool or implementation can interact with. You can export data from your personal IPFS node into a CAR file and upload it to web3.storage using `w3 up --car` or `client.uploadCAR`. As a result, we dive into the various ways you might interact with CAR files.
CAR files are a format that pretty much any IPFS tool or implementation can interact with. You can export data from your personal IPFS node into a CAR file and upload it to Storacha using `w3 up --car` or `client.uploadCAR`. As a result, we dive into the various ways you might interact with CAR files.

## Command line tools

Expand Down Expand Up @@ -85,17 +85,17 @@ ipfs dag import path/to/input.car

### `ipfs-car`

The [`ipfs-car`](https://github.com/web3-storage/ipfs-car) package includes library functions for packing and unpacking files into CARs, using the IPFS UnixFs data model. The library includes the same functionality as the ipfs-car command line utility [described above](https://web3.storage/docs/how-tos/work-with-car-files/#ipfs-car).
The [`ipfs-car`](https://github.com/web3-storage/ipfs-car) package includes library functions for packing and unpacking files into CARs, using the IPFS UnixFs data model. The library includes the same functionality as the ipfs-car command line utility [described above](https://storacha.network/docs/how-tos/work-with-car-files/#ipfs-car).

See the `ipfs-car` [README](https://github.com/web3-storage/ipfs-car#api) for API documentation and usage examples.

### `@ipld/car`

The [`@ipld/car`](https://github.com/ipld/js-car) package contains the main JavaScript implementation of the CAR specification and is used by ipfs-car under the hood. If you want to store non-file data using [advanced IPLD formats](https://web3.storage/docs/how-tos/work-with-car-files/#advanced-ipld-formats), you should use @ipld/car directly.
The [`@ipld/car`](https://github.com/ipld/js-car) package contains the main JavaScript implementation of the CAR specification and is used by ipfs-car under the hood. If you want to store non-file data using [advanced IPLD formats](https://storacha.network/docs/how-tos/work-with-car-files/#advanced-ipld-formats), you should use @ipld/car directly.

@ipld/car also provides the CarReader interface used by the web3.storage client's [putCar](https://web3.storage/docs/reference/js-client-library/#store-car-files)[method](https://web3.storage/docs/reference/js-client-library/#store-car-files).
@ipld/car also provides the CarReader interface used by the Storacha client's [putCar](https://storacha.network/docs/reference/js-client-library/#store-car-files)[method](https://storacha.network/docs/reference/js-client-library/#store-car-files).

Here's a simple example of loading a CAR file from a Node.js stream and storing it with web3.storage:
Here's a simple example of loading a CAR file from a Node.js stream and storing it with Storacha:

```js
import fs from 'node:fs'
Expand Down
Loading
Loading