Skip to content

Commit 86bd8db

Browse files
authored
Revise README with updated description and remove roadmap
Removed roadmap section and updated project description.
1 parent 978770a commit 86bd8db

1 file changed

Lines changed: 1 addition & 13 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Doubleword Inference Stack
22

3-
A comprehensive Helm chart for deploying production grade LLMs. This stack is a lightweight, transparent framework to allow you to deploy any model on any inference engine with minimal configuration. It is designed to be flexible, allowing you to easily switch between different models and inference engines without changing your client code.
3+
A Helm chart for deploying LLMs. This stack is a lightweight, transparent framework to allow you to deploy any model on any inference engine with minimal configuration. It is designed to be flexible, allowing you to easily switch between different models and inference engines without changing your client code.
44

55
The spirit of this project is to produce a framework that solves distributed serving of LLMs in a non-specific way to the inference engine running the weights.
66

@@ -20,15 +20,6 @@ The stack consists of:
2020
- **Model Groups**: A grouping of kubernetes resources that represent a deployment of an inference engine. Each model group can only have one active model at a time, but can have custom numbers of replicas.
2121
- **Inference Engines**: Backend services like vLLM, SGLang, TensorRT-LLM that handle the actual inference requests.
2222

23-
## Roadmap
24-
25-
We are actively developing this stack to support sophisticated LLM deployments. See our [issues](https://github.com/doublewordai/inference-stack/issues) for the latest features and improvements. Some key features we are working on include:
26-
27-
- **Model Group Prefix Aware Routing**: <https://github.com/doublewordai/inference-stack/issues/4>
28-
- **More complicated model group configurations**, such as dynamo: <https://github.com/doublewordai/inference-stack/issues/5>
29-
- **AutoScaling**: dynamic: <https://github.com/doublewordai/inference-stack/issues/6> and scale to zero: <https://github.com/doublewordai/inference-stack/issues/7>
30-
- **Inference Engine Support**: Adding example and support for more inference engines, please [open an issue](https://github.com/doublewordai/inference-stack/issues/new) to request specific engines.
31-
3223
## Getting Started
3324

3425
### Install from OCI Registry
@@ -307,7 +298,4 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
307298
## Support
308299

309300
Reach out at: <hello@doubleword.ai>!
310-
311-
- 📚 [Documentation](https://github.com/doublewordai/inference-stack/wiki)
312-
- 🐛 [Issue Tracker](https://github.com/doublewordai/inference-stack/issues)
313301
- 💬 [Discussions](https://github.com/doublewordai/inference-stack/discussions)

0 commit comments

Comments
 (0)