---
title: "Data Schemas and Prompt Management"
output:
  rmarkdown::html_vignette:
    toc: true
    toc_depth: 2
vignette: >
  %\VignetteIndexEntry{Data Schemas and Prompt Management}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
```

This article follows data from source samples to modeling inputs and shows how prompt templates are
managed. All examples are deterministic and run without provider credentials.

A **schema** is the required set of column names and types. You need a data
frame of texts; you will get pair rows, prompts, and modeling inputs. This page
explains those transitions. Start with [Getting Started](getting-started.html)
if you want to see the complete analysis before customizing it.

## Bring a spreadsheet or data frame

Use one row per writing sample. Keep a unique, non-missing character ID for each
sample across collection, saved results, and scoring; do not use row position as
identity. A CSV import can map your existing column names:

```{r personal-csv, eval=FALSE}
library(pairwiseLLM)
writing <- read.csv("writing.csv", colClasses = "character")
samples <- read_samples_df(writing, id_col = "essay_id", text_col = "response")
```

For example, `essay_id = "E001"` and `response = "The essay text..."` become
`ID = "E001"` and `text = "The essay text..."`. Remove blank or missing IDs and
texts before continuing. The reader checks duplicate IDs; it is not a substitute
for inspecting the quality of your input data.

## Samples and pairs

`read_samples_df()` normalizes a data frame and `read_samples_dir()` reads one text file per item.
Both return an `ID`/`text` table suitable for the fixed-pair workflow. Existing data frames can be
used directly when they already have those columns.

```{r samples-and-pairs}
library(pairwiseLLM)

data("example_writing_samples", package = "pairwiseLLM")
samples <- read_samples_df(example_writing_samples[, c("ID", "text")])

pairs <- samples |>
  make_pairs() |>
  sample_pairs(n_pairs = 4L, seed = 10L) |>
  randomize_pair_order(seed = 11L)

pairs[, c("ID1", "text1", "ID2", "text2")]
```

The fixed-pair schema uses `ID1`, `text1`, `ID2`, and `text2`. Reversal helpers preserve the
unordered pair while changing presentation order. Keep the identifiers: they are needed to map a
provider's `SAMPLE_1` or `SAMPLE_2` decision back to `better_id`.

## Traits and prompt templates

Generative templates require all four placeholders: `{TRAIT_NAME}`, `{TRAIT_DESCRIPTION}`, `{SAMPLE_1}`, and
`{SAMPLE_2}`. `build_prompt()` substitutes them but does not submit anything to a provider.

```{r build-prompt}
trait <- trait_description("overall_quality")
template <- set_prompt_template()

prompt <- build_prompt(
  template = template,
  trait_name = trait$name,
  trait_desc = trait$description,
  text1 = pairs$text1[[1]],
  text2 = pairs$text2[[1]]
)

substr(prompt, 1L, 180L)
```

The named registry lasts only for the current R session. Registration validates placeholders;
`overwrite = FALSE` protects an existing name. Persist a custom template by storing its text in
your project and registering it from a startup script, not by modifying package files.

```{r template-registry}
example_name <- "task07_example"
register_prompt_template(example_name, template = template)
example_name %in% list_prompt_templates()
identical(get_prompt_template(example_name), template)

register_prompt_template(example_name, template = template, overwrite = TRUE)
remove_prompt_template(example_name)
```

`set_prompt_template()` returns the built-in default or validates an inline/file template.
Save its return value and pass it as `prompt_template`; it does not change a
global active prompt.
`get_prompt_template()` resolves a user registration before a built-in template of the same name.
`remove_prompt_template()` removes only session registrations; it cannot delete built-in files.

## Decision instructions and probabilities

OpenAI Decisions and TypeSafe Jev ask a binary choice question instead of asking
the model to generate a tagged answer. Omitting `prompt_template` selects the
same built-in `decision_cj_v1` instructions for both. This default is separate
from `set_prompt_template()` and its session registry. Pass your complete
`trait_name` and `trait_description`; the description is preserved in full.
For a custom trait, include its boundaries and what the assessor should ignore.

You can prepare the settings entirely offline:

```{r decision-settings}
decision_settings <- list(
  trait_name = "Organization",
  trait_description = paste(
    "Judge logical sequencing and connections between ideas.",
    "Exclude spelling and grammar except when they obscure organization."
  )
)
# Optional replacement instructions; omit to use decision_cj_v1.
custom_instructions <- paste(
  "Assess {TRAIT_NAME} using the full guidance below.",
  "{TRAIT_DESCRIPTION}",
  "Select the sample with stronger performance on this trait.", sep = "\n"
)
```

Supply custom instructions as `decision_template`, retaining both trait
placeholders. Sample text is supplied separately in presentation order. Do not
put sample placeholders or the generative `<BETTER_SAMPLE>` tag into decision
instructions. See [`llm_compare_pair()`](../reference/llm_compare_pair.html) for
supported controls and validation.

This **invented result** illustrates the output without making a provider call:

```{r decision-result}
synthetic_decision <- data.frame(
  ID1 = "A", ID2 = "B", better_id = "B",
  p_sample_1 = 0.3, p_sample_2 = 0.7, p_chosen = 0.7,
  decision_confidence = 0.4
)
synthetic_decision
build_bt_data(synthetic_decision)
```

`p_sample_1` refers to `ID1`, and `p_sample_2` to `ID2`, as presented. Compare the
probability assigned to the **same ID** when auditing reversals; do not compare
position columns without remapping IDs. Equal probabilities still require an
explicit selected winner. `decision_confidence` is a separate provider field,
not a substitute for either probability. These values have no demonstrated
calibration for educational judgments and do not become BT weights or extra
comparisons. The [compatibility record](model-compatibility.html#decision-model-live-qualification)
describes the observed live results and their limits.

## Provider and result schemas

The live wrappers accept the fixed-pair rows directly. Batch request builders convert the same rows
to provider request records, and their matching parsers convert downloaded output back to package
results. Do not send a request table built for one provider to another provider's submission
function.

The normalized result bundle retains pair identifiers and adds provider metadata, visible content,
optional `thoughts`, `better_sample`, `better_id`, and token counts. Row-wise live calls return a
list whose valid `results`, unresolved `failed_pairs`, and attempt-level `failed_attempts` components
must be inspected separately. Batch parsers may omit unsuccessful rows from their successful result
table, so preserve the provider output/error file and job registry for audit and recovery.

```text
samples (ID, text)
  -> pairs (ID1, text1, ID2, text2)
     -> live call ---------------------> normalized results
     -> provider request + batch output -> provider parser -> normalized results
```

## Modeling and adaptive branches

For frequentist Bradley--Terry or Elo models, start from rows with `ID1`, `ID2`, and a valid
`better_id` and call `build_bt_data()` or `build_elo_data()`. Invalid or missing winners are not
valid outcomes; inspect failures before modeling rather than silently treating them as ties.

```{r modeling-schemas}
data("example_writing_pairs", package = "pairwiseLLM")

bt_data <- build_bt_data(example_writing_pairs)
elo_data <- build_elo_data(example_writing_pairs)
bayes_data <- build_btl_results_data(example_writing_pairs)

names(bt_data)
names(elo_data)
names(bayes_data)
```

Standalone Bayesian BTL uses the stricter canonical schema returned by
`build_btl_results_data()`; see
[Standalone Bayesian BTL with CmdStan](https://shmercer.github.io/pairwiseLLM/articles/bayesian-btl.html).

Adaptive ranking is a separate branch. `adaptive_rank()` starts from raw items and records each
attempt in an adaptive state with canonical step, round, link-stage, and item logs. A normalized
fixed-pair result table is not a drop-in replacement for an adaptive session. Use
`adaptive_results_history()` only when you need the committed adaptive outcomes in
`build_bt_data()` format.

## Failure checks at schema boundaries

- Reject missing, blank, or duplicate sample IDs before pairing.
- Preserve the original pair IDs and provider `custom_id` through batch submission and parsing.
- Model only outcomes whose `better_id` matches one member of the pair.
- Treat `thoughts` and raw responses as sensitive submitted/returned text when deciding retention.
- Validate persisted adaptive sessions with `validate_session_dir()` rather than editing `.rds`
  artifacts or coercing their schemas by hand.

## Related documentation

See [Getting Started with pairwiseLLM](https://shmercer.github.io/pairwiseLLM/articles/getting-started.html)
for the basic fixed-pair workflow,
[Provider Controls and Recovery](https://shmercer.github.io/pairwiseLLM/articles/provider-controls-and-recovery.html)
for live and batch failures, and
[Guide: Adaptive Pairing](https://shmercer.github.io/pairwiseLLM/articles/adaptive-pairing.html)
for adaptive state and logs.

## Citation

> Mercer, S. H. (2026). *Data schemas and prompt management* [R package vignette]. Comprehensive R
> Archive Network. https://doi.org/10.32614/CRAN.package.pairwiseLLM