Backend-Dependent Behavior on Legacy HBM GPUs

• In ROCm 7.2, gfx900 is not listed in the official support matrix; however, this local environment retains residual build/runtime paths and shipped artifacts.
• For students and small labs, the key question is whether execution is possible, stable, and reproducible under specific configurations.
• We built a matched dual-backend testbed on the same Vega device to separate support policy, backend choice, and runtime behavior.
• Goal: examine whether "not officially supported" and "not executable in any configuration" are the same statement — within this tested setup.

• Legacy Vega GPUs are often treated as categorically unsuitable for modern ROCm workloads.
• Success or failure may be decided at different layers: distribution presets, build targets, init validation, backend macros, or runtime compute paths.
• We ask: in these runs, is the observed failure caused by Vega hardware itself, or by a specific backend compute path under matched conditions?

Source: work_log/investigations/2026-03-07_gfx900_gate_matrix.md | ollama/CMakeLists.txt:127 | ggml-cuda/common.cuh:65

Source: ollama/CMakeLists.txt:121-128

check_language(HIP)
if(CMAKE_HIP_COMPILER)
    if(NOT AMDGPU_TARGETS)
        find_package(hip REQUIRED)
        list(FILTER AMDGPU_TARGETS INCLUDE REGEX
            "^gfx(94[012]|101[02]|1030|110[012]|120[01])$")
    endif()
    if(AMDGPU_TARGETS)
        add_subdirectory(.../ggml-hip)

dp4a dispatch in ggml_cuda_dp4a() (common.cuh:666–704):

V_DOT2_F32_F16_AVAILABLE (common.cuh:715) also excludes gfx900 — no hardware fp16 dot product instruction.

• gfx900 is not listed in the ROCm 7.2 official support matrix, yet residual build/runtime paths remain in this environment — distinguishing policy from capability requires layered analysis.
• num_gpu is often misread as device count, causing wrong failure attribution when num_gpu>=2 is tested.
• Under matched qwen3.5:2b conditions in these runs, the same Vega hardware shows different outcomes across backends — hardware alone does not determine the result.
• ROCm results in this setup depend on HSA_OVERRIDE_GFX_VERSION=9.0.0; this is an override-assisted configuration, not a standard deployment.
• Diagnosis requires layered evidence (source, logs, run artifacts), not single-run anecdotes.

Principle: falsifiable diagnosis over optimistic one-off success.

Matched rerun on 2026-03-08: ROCm run_20260308_201305 (0.17.6, override-assisted) again passed all tested values; Vulkan run_20260308_201641 again failed for num_gpu=1/2/-1 with nearly identical crash times (2.70 / 7.64 / 9.81s).

Observation (in these matched qwen3.5:2b runs): Vulkan failure is conditional on num_gpu >= 1, not universal. ROCm/HIP passed all tested values under override-assisted configuration on both days.

Data source: vega_work_log/*/result.json | 16 matched-record rows extracted from the full log set.

Source: vega_work_log/run_20260307_013050/ollama_journal_since_start.txt

From: vega_work_log/run_20260307_013050/ollama_journal_since_start.txt — immediately after runner reported ready

SIGSEGV: segmentation violation
PC=0x7efbc5b801b5 m=41 sigcode=1 addr=0x3a8000
signal arrived during cgo execution

goroutine 853 [syscall]:
runtime.cgocall(0x55e54a0e2710, 0xc0000bdaa0)
    runtime/cgocall.go:167
github.com/ollama/ollama/ml/backend/ggml._Cfunc_ggml_backend_sched_graph_compute_async(...)
    _cgo_gotypes.go:979
github.com/ollama/ollama/ml/backend/ggml.(*Context).ComputeWithNotify(...)
    ml/backend/ggml/ggml.go:825 +0x1b2
github.com/ollama/ollama/runner/ollamarunner.(*Server).computeBatch(...)
    runner/ollamarunner/runner.go:716 +0x862
github.com/ollama/ollama/runner/ollamarunner.(*Server).run.gowrap1()
    runner/ollamarunner/runner.go:459

Crash point: C function ggml_backend_sched_graph_compute_async called via cgo. Go runtime receives SIGSEGV and terminates the runner. The exact Vulkan API call within that C function is not resolved from this trace alone.

Source: run_20260308_212345, run_20260308_212254, and analysis_summary.md derived from 15 runs / 119 records in vega_work_log/.

Journal evidence (01:31:37):

ggml_vulkan: 0 = AMD Radeon RX Vega (RADV VEGA10) (radv) |
  uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 |
  shared memory: 65536 | int dot: 0 | matrix cores: none

int dot: 0 = VK_KHR_shader_integer_dot_product not hardware-accelerated. matrix cores: none = no cooperative matrix. Both are consistent with source-level AMD_GCN code paths.

Four-Tier Verdict (Based on Collected Evidence)

Replay command: OLLAMA_HOST=http://127.0.0.1:11435|11434 EPOCHS=1 NUM_PREDICT=512 NUM_GPU=0,1,2,-1 python vega-loop_qwen_rocm.py

Stability check: OLLAMA_HOST=http://127.0.0.1:11434 EPOCHS=5 NUM_GPU=0 NUM_PREDICT=512 python vega-loop_qwen.py

Limitations:

• Matched comparison is still centered on qwen3.5:2b. However, the one-epoch matched run was reproduced on two days, Vulkan num_gpu=0 was additionally stable for 5 epochs, and phi4-mini did not reproduce the num_gpu=1 crash.
• Ollama versions differ between endpoints (0.17.4 Vulkan vs 0.17.5/0.17.6 ROCm) — a potential confound remains.
• ROCm result depends on HSA_OVERRIDE_GFX_VERSION=9.0.0; outcome without the override is not tested here, and the override-off check was deferred because it requires service restart and recovery handling.
• Root cause within ggml_backend_sched_graph_compute_async (C/Vulkan side) is not resolved from the Go stack trace alone.
• The new phi4-mini result indicates that gfx900 Vulkan stability is model- or workload-dependent; the exact trigger in qwen3.5:2b Q8_0 remains unresolved.

Future work: Additional models and quantization formats; matched Ollama version pair; same-version override-off vs override-on comparison; deeper Vulkan trace (RADV validation layers); broader multi-epoch stress under GPU-offload settings.

Based on the collected evidence: "not officially supported" and "not executable in any configuration" are distinct statements. The shipped-artifacts finding adds a fourth dimension: vendor packaging actively includes gfx900 in ways that surpass some supported architectures.