ModelGen

CLI AI Minecraft 3D model generator. Describe a model in plain language, get a .bbmodel file you can open directly in Blockbench.

ModelGen is a standalone CLI — not a Blockbench plugin. Blockbench is only the final editor.

prompt -> LLM -> Universal Model JSON -> validation -> .bbmodel -> Blockbench

Install

npm install
npm run build
npm link          # makes `modelgen` available globally

Requires Node.js 20 or newer.

Configure

Copy the example environment file and fill in your key:

copy .env.example .env

API_BASE_URL=https://ai.mangso.net/v1
API_KEY=your-api-key-here

ASTRA_MODEL=cx/gpt-6-astra
SOL_MODEL=cx/gpt-5.6-sol
DEFAULT_MODEL=cx/gpt-6-astra

Reference images are on by default; VISION=0, MAX_IMAGES and MAX_IMAGE_BYTES tune them.

.env is git-ignored and the key is never printed — debug logs show Authorization: Bearer ********, and an attached image as its name, size and pixel dimensions, never its contents.

Configuration resolves in this order, highest priority first:

CLI arguments -> .modelgenrc.json -> ~/.modelgen/config.json -> .env -> defaults

Use

Run with no arguments for the interactive terminal UI — the primary experience:

modelgen

╭──────────────────────────────────────────────────────╮
│                       MODELGEN                       │
│            AI Minecraft 3D Model Generator           │
╰──────────────────────────────────────────────────────╯

? Action
  > Create new model
    Refine existing model
    Open project
    Settings
    Exit

Pick a model type, edition, texture resolution and AI model, then describe what you want. Multi-line input ends on a blank line.

Design from a picture

Any prompt may carry reference images. The model reads them for silhouette, proportion, structure and palette, then builds the equivalent in cuboids — the written prompt still wins wherever the two disagree.

Interactive mode asks for images right before the description, and an image path written anywhere in a prompt is picked up on its own:

? Reference images (optional, up to 4) refs/knight_front.png refs/knight_side.png

╭──────────────────────────────────────────────────────╮
│                  Describe your model                 │
│  Type or paste, or leave this blank to design from   │
│                  the images alone.                   │
╰──────────────────────────────────────────────────────╯

> make the cape longer than in the reference

From the command line:

# One reference
modelgen generate "a knight" --image ./refs/knight.png

# Several views of one subject — front, side, back
modelgen generate "a knight" -i ./refs/front.png -i ./refs/side.png -i ./refs/back.png

# A path inside the prompt works too, quoted when it has spaces
modelgen generate "match this armor ./refs/armor.png"

# Refine against a reference
modelgen refine ./output/knight/model.json "make the helmet like this" -i ./refs/helm.png

# Ignore images for one run
modelgen generate "a knight ./refs/knight.png" --no-vision

PNG, JPEG, WebP and GIF are accepted, as local paths or https:// URLs, up to 4 images and 10 MB each by default (MAX_IMAGES, MAX_IMAGE_BYTES). The images are sent with the first request only — the repair loop works on the JSON, so it never re-uploads them. The references used are recorded in generation.json.

Multiple images are treated as views of ONE subject unless the prompt says otherwise. The model that answers must support image input; when it does not, the CLI says so and suggests --model.

Direct commands

Every subcommand goes through the same application services as interactive mode.

# Generate
modelgen generate "Create a giant Minecraft dragon"

# Use Sol instead of Astra
modelgen generate "Create a giant Minecraft dragon" --model cx/gpt-5.6-sol

# Options
modelgen generate "An armored skeleton boss" --type character --edition java --resolution 64x64

# Refine an existing model
modelgen refine ./output/dragon/model.json "Make the wings larger"

# Validate without calling the LLM
modelgen validate ./output/dragon/model.json

# Convert without calling the LLM
modelgen convert ./output/dragon/model.json

# Summary of an existing model
modelgen inspect ./output/dragon/model.json

Global flags: --model, --api-base-url, --output, --repair-attempts, --temperature, --no-vision, --debug. generate and refine also take -i, --image <path>, repeatable.

Web UI

modelgen web --open          # or: npm run web

Serves a local page at http://127.0.0.1:4321 with generate and refine forms (reference images by picker, drag-and-drop or paste), live progress, a project list, and a 3D viewer that renders the painted texture, plays the model's animations and downloads the .bbmodel. --port and --host change where it listens. The page talks only to this local process; the API key never reaches the browser. The viewer loads three.js from a CDN, so it needs a network connection.

The header has two tabs: BBModel for Minecraft models and Pixel Art for 2D game assets. Pixel Art includes drawing tools, selection, layers, animation frames, undo/redo, local autosave, JSON import/export, transparent PNG and sprite sheet export. Both studios share the chat/vision model configuration. Pixel Art can also use a dedicated image API or a local NexusSprite/FLUX engine, with an optional reference image, shared frame palettes and PNG-based visual review. Create new assets in the left sidebar; write frame sizes, counts and directions in the prompt. The right sidebar edits the open asset or selected region and shows its properties, layers, palette and exports. AI settings are collapsed. The Skill prompt box stores reusable art direction (style, proportions, color theory, lighting and pixel techniques). Click Import thành Markdown to save it as pixel-skills/drawing-skill.md under the configured output directory. Saved guidance is automatically supplied to generation, edits and visual review, including image/Nexus engines. The current prompt takes priority over conflicting skill guidance; drafts apply only after Import. Download the Markdown with Tải .md, or stop applying it with Gỡ skill. The default image model is Mangso cx/gpt-image-2.5, using the existing API_BASE_URL and API_KEY. Select Tự động or AI tạo ảnh in Pixel Studio. Override it with PIXEL_IMAGE_MODEL; PIXEL_IMAGE_BASE_URL and PIXEL_IMAGE_API_KEY optionally select a separate image service. The model selector below the engine controls visual review and experimental geometry drawing. Mangso cx/ models receive reference images as a JSON images array on /images/generations; other OpenAI-compatible services use multipart /images/edits. Auto requires a configured image API or NexusSprite; it no longer silently uses expensive chat-generated geometry. Valid previews appear immediately and are archived as PNG/JSON in pixel-generations/ under the output directory. Visual critique returns editable drafts with warnings and never triggers a paid redraw. Partial results survive later frame errors. Geometry drawing is available only as an explicit experiment, with bounded tokens and deadlines. Cloudflare Workers AI is supported through its native multipart image API: set PIXEL_IMAGE_PROVIDER=cloudflare, PIXEL_IMAGE_MODEL=@cf/black-forest-labs/flux-2-klein-4b, CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_API_TOKEN in .env, then restart the server. The token needs Workers AI Read/Edit. References are resized below 512px as required by this model. Account/token values stay server-side. Pixel projects live in this browser's IndexedDB; export project JSON for backup or transfer. See PIXEL_ART_ARCHITECTURE.md for the implemented architecture. See PIXEL_ART_RESEARCH.md for the source survey, engine configuration, quality limits and verification commands.

Output

Each model gets its own directory:

output/
└── ancient-warrior/
    ├── model.json                  Universal Model JSON (current)
    ├── ancient-warrior.bbmodel     open this in Blockbench
    ├── prompt.txt                  original prompt and refine history
    ├── generation.json             provider, model id, timestamp, token usage
    └── versions/
        ├── v001.json
        └── v002.json

Nothing is overwritten silently — every write snapshots the previous model into versions/ first.

How it works

The LLM never writes .bbmodel. It produces a Universal Model JSON document that ModelGen owns and controls:

{
  "version": "1.0",
  "model": {
    "name": "ancient_warrior",
    "type": "character",
    "minecraft": { "edition": "java", "texture_resolution": [64, 64] }
  },
  "bones": [
    {
      "name": "body",
      "parent": null,
      "pivot": [0, 12, 0],
      "cubes": [
        {
          "from": [-5, 5, -3],
          "to": [5, 15, 3],
          "origin": [0, 10, 0],
          "color": "#8a7f6d",
          "uv": { "offset": [0, 0] }
        }
      ]
    }
  ]
}

The full schema lives in schemas/universal-model.schema.json.

Textures

An LLM cannot emit PNG bytes, so it assigns a colour per cube instead — optionally per face — and ModelGen rasterizes those into the box-UV layout itself. The result is a real texture sheet, shaded by face orientation (tops lighter, undersides darker) with a little per-pixel grain, written into the .bbmodel and ready to paint over in Blockbench.

{
  "color": "#52694b",
  "material": "stone",
  "faces": { "north": "#607650", "down": "#45543d" }
}

material is what keeps the result from looking like Lego. A flat fill reads as plastic; each material paints a real surface instead — metal gets a specular band down its length, wood gets grain, gem gets facets, glow gets a bright core that ignores directional shading. Every face also gets a dark outline and a lit bevel, which is what gives a box its edges.

Available: metal, wood, stone, cloth, leather, gem, glow, bone, scale, flat.

A model-wide "texture_style": { "noise": 0.05, "shading": true, "bevel": true } tunes the sheet. Cubes with no colour come out transparent, and a model with no colours at all falls back to a blank sheet at the right resolution.

Glowing and animated effects

An aura, flame or magic trail is light, not geometry. Scattering loose cubes around a model to suggest an effect produces floating debris — so ModelGen rejects disconnected geometry: a cube touching nothing else is a validation error, and the repair loop is asked to attach it. A cube that genuinely must float says "detached": true.

Effects are built with the glow material instead — on the part itself, on an inflated shell around it, or on thin planes that intersect it. Adding

"texture_style": { "animation": { "frames": 8, "frame_time": 2 } }

renders the glowing surfaces across 8 frames as a vertical texture strip with Blockbench's frame_time set, so the aura pulses. Frames are only produced when the model actually has a glow surface.

The strip is emitted for Bedrock models only. Java models open in Blockbench's Modded Entity format, which maps the whole image into the model's UV space — an 8-frame strip would appear eight times over on every face, so the sheet is flattened to a single frame. The glow is still painted; it just does not pulse.

Animations

When the description asks for movement, the model carries an animations array of keyframes per bone on the position, rotation and scale channels. These convert straight into Blockbench animations, playable on the timeline.

{
  "name": "idle",
  "loop": true,
  "length": 2.0,
  "bones": [
    {
      "bone": "head",
      "rotation": [
        { "time": 0, "value": [0, 0, 0] },
        { "time": 1, "value": [0, 12, 0] },
        { "time": 2, "value": [0, 0, 0] }
      ]
    }
  ]
}

Requests are streamed, so a long generation never sits idle long enough for an API gateway to cut the connection (a Cloudflare 524), and the CLI can show output arriving as the model writes.

That document is validated in three stages — JSON schema, then geometry (bone hierarchy, parent references, coordinates, rotations, UV bounds), then project constraints (cube and bone counts, texture and model size). When validation fails, the errors are sent back to the LLM as a repair request, up to MAX_REPAIR_ATTEMPTS times. Only a valid model reaches the converter, which is fully deterministic and never calls the LLM.

Architecture

src/
├── cli/            interactive UI and commands
├── application/    generate / refine / validate / convert / inspect services
├── llm/            provider abstraction, Astra, Sol, parser, repair loop
├── prompts/        system and per-type prompt builders
├── model/          Universal Model types, schema, validators, geometry
├── converters/     Universal Model -> .bbmodel (deterministic)
├── storage/        project layout, output, version history
└── config/         layered configuration

The design principle:

LLM      = intelligence / creativity
ModelGen = validation / orchestration / conversion
Blockbench = final editor

See MODELGEN_ARCHITECTURE.md for the full design.

Development

npm run dev -- generate "a test dragon"   # run from source via tsx
npm run typecheck
npm run build