Trouble getting a grip on where to get started

[SeeIfIDont]

(First of all, I realize that this is an early, alpha-quality piece of software. I'm not trying to run this in production or expect fully sweated-over technical documentation, I am just confused and thought I would document this confusion for posterity.)

I read the introductory post (Stop syncing everything) with great interest, and I see that both it and the front page readme bills libgraft as the easiest way to start using graft. But I am somewhat puzzled by what I see there.

The post mentions blob/object storage a lot, and rightly points out that being able to rely on the guarantees of blob storage solves a lot of issues. The libgraft page mentions volumes, and I read that understanding that at some point there will be configuration. There is configuration, but it mostly talks about the metastore, pagestore and API tokens for those. None of them are explained, although I find a technical details page which is more of a "what needs to be built" design document (and very interesting reading!) than instructions to what to actually run to take advantage of libgraft.

I'm assuming from some of the other configuration options that libgraft falls back to writing blocks locally, which is fine. But as an interested user, I basically don't see a road to using it in the way it was described. Where do I plug in the details to the blob storage, how do I run these ancillary services that probably mediate the access to blob storage, and so on.

I assume that all of this is forthcoming. Or am I missing something?

[carlsverre]

Hey @SeeIfIDont! Thanks for opening this discussion. A couple people have mentioned it and it's at the very top of my todo list (#54). I'm also thinking of building graft-in-a-box (#28) to make it easy to run the whole environment.

In the mean time, you can spin up a local Graft environment connected to Minio using this docker-compose file: https://github.com/orbitinghail/graft/blob/main/tests/antithesis/docker-compose.yaml

The dockerfile for building the metastore/pagestore are at the root of the repo, and you can use just to run all the scripts in the Justfile.

Alternatively here are some commands to run the metastore/pagestore from source. By default they will just write to the local filesystem, but you can change that by editing metastore.toml and pagestore.toml at the root of the repo:

cargo run --bin pagestore
cargo run --bin metastore

Once both are running you can use just run sqlite shell to drop into a SQLite shell that will connect to them by default.

Again, still very rough - will be documenting and packaging everything so it's easy asap.

[SeeIfIDont]

Thanks for the quick answer. Apologies for not finding the filed issues.

In light of the cost concern mentioned in #54 I definitely see wanting to hold off a bit so I appreciate this, but the S3 interface being what it is, it's neat that people who are interested can use a self-hosted implementation like minio as you do in the Docker Compose example. (It sounds like it's also being tested with deterministic simulation testing, which is very exciting!)

I look forward to seeing where this project is headed in the future.

[carlsverre]

Hey @SeeIfIDont - thanks for your patience! I've closed the cost concern earlier today (#69) and will finish writing up hosting docs tomorrow (initial pass: #73). Also will be releasing a blog post on DST very soon.

[carlsverre]

I've just released a self-hosting guide on Graft! You can find it here.

https://github.com/orbitinghail/graft/blob/main/docs/hosting.md