---
title: "Standalone Bayesian BTL with CmdStan"
output:
  rmarkdown::html_vignette:
    toc: true
    toc_depth: 2
vignette: >
  %\VignetteIndexEntry{Standalone Bayesian BTL with CmdStan}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
```

`fit_bayes_btl_mcmc()` fits a Bayesian Bradley--Terry--Luce (BTL) model to an existing fixed set of
pairwise outcomes. It reuses the adaptive fit and summary contracts but does not perform adaptive
pair selection.

The [integrated CJ workflow](adaptive-cj-workflow.html) explains how Bayesian EAP
reliability differs from frequentist SSR, empirical score recovery, and held-out
prediction. It also introduces the experimental frequentist lapse/position
estimator that matches the `btl_e_b` likelihood without Bayesian priors.

## When to use this guide

Use this workflow when you already have pairwise winners and want Bayesian
score estimates, uncertainty, and convergence diagnostics. It is also a route
to the completed Bayesian input required for rubric calibration. To let the
package choose new comparisons during collection, use the adaptive guide instead.

**You need:** recorded comparisons, their item IDs, and CmdStan. **You get:**
one score summary per item plus fit diagnostics. The code below uses synthetic
outcomes; the short sampling settings demonstrate the interface, not a validated
assessment or a sufficient sampling budget for your study.

## Prerequisites

The fit requires the suggested `cmdstanr` and `parallelly` packages, CmdStan, and a working C++ toolchain. Installation
is a machine-level setup and does not require provider credentials. Fitting chunks are shown but not evaluated during document builds.
Run them interactively after setup; no provider requests are needed for this example.

```{r install, eval=FALSE}
install.packages("parallelly")
install.packages(
  "cmdstanr",
  repos = c("https://stan-dev.r-universe.dev", getOption("repos"))
)
cmdstanr::check_cmdstan_toolchain(fix = TRUE)
cmdstanr::install_cmdstan()
```

```{r availability}
cmdstan_available <- requireNamespace("cmdstanr", quietly = TRUE) &&
  tryCatch(!is.null(cmdstanr::cmdstan_version()), error = function(e) FALSE)
cmdstan_available
```

A `TRUE` value means CmdStan was found. Compilation and model fitting are
separate checks performed when you run the fit.

## Prepare modeling input

“Canonical” means the standard column layout expected by this workflow.
The public builder accepts `ID1`, `ID2`, and `better_id`, validates that each winner belongs to its
pair, and adds deterministic keys, iteration values, timestamps, and provenance columns.

```{r input}
library(pairwiseLLM)

data("example_writing_pairs", package = "pairwiseLLM")
observed <- example_writing_pairs[seq_len(min(40L, nrow(example_writing_pairs))), ]
results_tbl <- build_btl_results_data(
  observed,
  backend = "offline_fixture",
  model = "deterministic_observations"
)
ids <- sort(unique(c(results_tbl$A_id, results_tbl$B_id)))

results_tbl[, c("pair_uid", "A_id", "B_id", "better_id", "winner_pos", "phase")]
```

The four likelihood variants are:

- `"btl"`: item parameters only;
- `"btl_e"`: item parameters plus a lapse/error component;
- `"btl_b"`: item parameters plus a presentation-position effect;
- `"btl_e_b"`: both lapse and position components (the default).

Choose the variant before inspecting results, based on the study design and estimands. A more
complex variant is not automatically preferable, especially with sparse data.

## Fit and summarize

The following seeded sampling example is not evaluated during documentation builds. Run it after the
availability check returns `TRUE`. The small iteration count keeps the walkthrough practical and is
for workflow demonstration, not production inference.

```{r fit, eval=FALSE, purl=TRUE}
fit <- fit_bayes_btl_mcmc(
  results = results_tbl,
  ids = ids,
  model_variant = "btl_e_b",
  cmdstan = list(
    chains = 2L,
    parallel_chains = 2L,
    iter_warmup = 250L,
    iter_sampling = 250L,
    seed = 7007L,
    core_fraction = 1
  )
)

refits <- summarize_refits(fit)
items <- summarize_items(fit)

refits[, c(
  "round_id", "total_pairs", "diagnostics_pass",
  "divergences", "max_rhat", "min_ess_bulk", "reliability_EAP"
)]
items[, c("ID", "theta_mean", "theta_sd", "rank_mean", "deg")]
```

The item scale is relative and identified by the model constraints; its absolute origin is not an
external score. Posterior SDs and intervals are conditional on the selected model and observed
judgments. Check divergences, R-hat, effective sample size, and sampling warnings before interpreting
ranks. A failed diagnostic is not repaired by hiding the warning or reporting only posterior means.

### What the output means

| Output | Interpretation |
| --- | --- |
| `theta_mean`, `theta_sd` | Posterior mean relative quality and posterior uncertainty for each item. |
| `rank_mean` | Posterior mean rank; uncertainty can make it non-integer. |
| `deg` | Number of committed comparisons involving that item. |
| `diagnostics_pass` | Whether the fit meets the implemented diagnostic checks; inspect individual diagnostics too. |
| `reliability_EAP` | Posterior mean score variance divided by that variance plus average posterior variance; an internal model-based summary, not empirical recovery. |
| `max_rhat`, `min_ess_bulk`, `divergences` | Sampling checks for chain agreement, effective information, and numerical problems. |

Higher quality locations indicate stronger writing, but adjacent ranks can be
uncertain. Keep both the item summaries and diagnostic table when saving or
reporting an analysis.

### Chains and CPU allocation

`chains` sets the total number of chains; `parallel_chains` sets how many run at
once. Four chains can therefore run two at a time. `threads_per_chain` defaults
to one; the bundled models do not use within-chain parallel likelihoods, so
increasing it does not accelerate likelihood evaluation.

When `parallel_chains` is omitted, scheduling uses at most two available CPU
slots and applies `core_fraction = 0.8`, retaining at least one slot. On a
two-slot allocation this schedules one chain at a time. Detection respects
scheduler, container, and check constraints without changing the total chains.
The historical default chain count remains `min(8, detected physical cores)`,
with logical-core and one-core fallbacks when hardware detection is unavailable.
Set `chains` explicitly when you need the same count across machines.

An explicit `parallel_chains` request bypasses `core_fraction` and is capped at
the total chains. Its product with `threads_per_chain` must fit the available
allocation; otherwise fitting stops with guidance. For example,
`cmdstan = list(chains = 4L, parallel_chains = 4L)` requests four CPU slots and
prints a resource message when sampling. Active check limits still cap the
allocation at two. Examples and package checks stay within that limit.

If resource detection fails, fitting warns and uses one CPU slot. Independent
concurrent fits each see the same allocation, so divide resources among them
yourself. `summarize_refits(fit)` reports `mcmc_cores_available`,
`mcmc_parallel_chains_requested`, `mcmc_concurrency_budget`, and
`mcmc_concurrency_used`. An automatic request is `NA`; historical fits without
this metadata also report `NA` for the new fields.

## Cumulative refits

`pair_counts` fits increasing subsets of the same data and records one round/item-log view per
subset. `subset_method = "first"` follows row order. `"sample"` creates one seeded permutation and
uses nested prefixes, so provide `seed` for reproducibility.

```{r cumulative, eval=FALSE, purl=TRUE}
cumulative <- fit_bayes_btl_mcmc(
  results_tbl,
  ids = ids,
  pair_counts = c(20L, 40L),
  subset_method = "sample",
  seed = 7007L,
  cmdstan = list(chains = 2L, parallel_chains = 2L, seed = 7007L)
)

summarize_refits(cumulative)
summarize_items(cumulative, refit_id = 2L)
```

## Files, failures, and reproducibility

CmdStan writes generated C++/executables and sampling CSV files to its cache or configured
`cmdstan$output_dir`. Preserve the seed, model variant, CmdStan configuration, package version, and
diagnostics with reported results. Missing CmdStan, invalid schemas, unknown IDs, invalid winners,
empty data, invalid subset sizes, and unsupported variants abort explicitly. Sampling failures and
poor diagnostics should be investigated rather than converted to partial rankings.

For adaptive selection and stopping, use
[Guide: Adaptive Pairing](https://shmercer.github.io/pairwiseLLM/articles/adaptive-pairing.html). For
fixed-pair frequentist alternatives, see `fit_bt_model()` and `fit_elo_model()`.

## Predictive warm-start priors

`fit_bayes_btl_mcmc()` accepts `warm_start_prior`, constructed with
`make_warm_start_prior()` from calibrated model predictions or explicit expert scores.
The four active variants use centered predictive locations and a default raw-theta
SD of 0.5. Omitting the argument preserves raw means zero and SDs one. Ensemble
prediction SD is diagnostic disagreement, not an automatic prior SD.
See [Guide: Adaptive Warm Start](adaptive-warm-start.html) for the offline workflow.

For downstream performance levels or calibration to human rubric scores, see
[Guide: Rubric Calibration](rubric-calibration.html). It consumes the completed
fit directly and supports all four Bayesian BTL variants.

## Citation

> Mercer, S. H. (2026). *Standalone Bayesian BTL with CmdStan* [R package vignette]. Comprehensive
> R Archive Network. https://doi.org/10.32614/CRAN.package.pairwiseLLM