vLLM/Recipes
Search
AutoTrust

autotrust/JEV-27B-VL

Dense 27B vision-language decision model on a Qwen3.8-27B backbone, with a rank-32 LoRA (jev-decision) for calibrated choices over text and images.

Qwen3.8-27B VL backbone; System 1 decisions through the jev-decision LoRA

View on HuggingFace
dense27B262,144 ctxvLLM 0.32.0+multimodaltext
Guide

Overview

JEV-27B-VL is AutoTrust's vision-language decision model. The checkpoint is a dense BF16 Qwen3_5ForConditionalGeneration model (about 27.8B parameters, vision tower included) continued from Qwen/Qwen3.8-27B. System 2 is the served checkpoint, named autotrust/JEV-27B-VL. System 1 is the rank-32 LoRA jev-decision (adapter_vllm/, r 32, lora_alpha 64), used for calibrated choice probabilities over text or images.

This recipe has no verified hardware entry. The model card's quick start asks for one GPU with 80 GB or more, and the card's published latency figures are from a B200. Those runs are the author's, not a badge on this page.

Prerequisites

  • vLLM: >= 0.32.0. Earlier releases are not supported for this checkpoint.
  • Weights: hf download autotrust/JEV-27B-VL --local-dir JEV-27B-VL so the LoRA path below exists. The same adapter_vllm directory is in the ModelScope checkout; if that folder is not named JEV-27B-VL, point the module at <checkout>/adapter_vllm (the HF snapshot path ends in snapshots/<rev>/adapter_vllm).
  • Docker: the image only bind-mounts the Hugging Face cache. Mount the checkout into the container and keep jev-decision pointed at that adapter_vllm directory.
  • Context: text_config.max_position_embeddings is 262144. The validated command uses --max-model-len 32768 (the serve.sh default). The card says a full 256K prompt needs about 17 GB of KV cache on top of the weights, and suggests --max-model-len 131072 on an 80 GB GPU.

Decision LoRA and serve flags

These flags are the model card's validated vllm serve line. They are base arguments, not optional toggles.

  • jev-decision — --enable-lora, --max-lora-rank 32, and --lora-modules jev-decision=JEV-27B-VL/adapter_vllm. The module name stays jev-decision. The path is the local checkout JEV-27B-VL/adapter_vllm, or the adapter_vllm directory inside an HF or ModelScope checkout.
  • --mamba-cache-mode align — the Qwen3.8 hybrid backbone mixes full attention with Mamba-style layers; the card serves it in align mode.
  • --limit-mm-per-prompt '{"image": 8}' — at most 8 images per request.
  • --max-num-seqs 8 — required. The card says that with more than 8 sequences in one batch, vLLM's LoRA path for this multimodal class returns wrong System 1 probabilities. Extra requests queue.
  • --logprobs-mode processed_logprobs — System 1 reads post-processor logprobs. On /v1/chat/completions or /v1/completions for jev-decision, the card also says to send top_k: 0 and top_p: 1.0: generation_config.json sets top_k 20 and top_p 0.95, and those defaults would truncate the returned distribution.
  • --trust-request-chat-template — image decisions send a raw decision chat template on the request. System 2 keeps the checkpoint's own template.
  • --served-model-name autotrust/JEV-27B-VL — public name for System 2. --enable-prefix-caching matches the card's serve line.

POST /v1/decide is not part of stock vllm serve. The checkpoint ships serve_decide.py, which starts the OpenAI server with these same flags and adds that route. Stock vllm serve still exposes System 2 as autotrust/JEV-27B-VL and System 1 as the LoRA model jev-decision.

Client usage

System 2 (the base checkpoint), with an image:

client.chat.completions.create(
    model="autotrust/JEV-27B-VL",
    messages=[{
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": image_url}},
            {"type": "text", "text": "What does this image show?"},
        ],
    }],
)

System 1 through the checkpoint's /v1/decide route (serve_decide.py):

curl localhost:8000/v1/decide -H 'Content-Type: application/json' -d '{
  "kind": "choice",
  "state": "Customer: my card was charged twice for one coffee.",
  "question": "Which team should handle this?",
  "options": ["billing", "shipping", "tech support"]}'

Text-only System 1 calls the LoRA directly (model jev-decision). The card says those text decisions match autotrust/JEV-27B.

References