Background | Choose a model | Installing writeAlizer | Optional model dependencies | Development version | Quick start | Try the workflow without downloads | Importing data | How text IDs are handled | Predicting writing quality | ReaderBench and Coh-Metrix results | GAMET results | Comparing scores across groups of texts | Save your results | Available models at a glance (with references) | ReaderBench | Coh-Metrix | GAMET and the offline demonstration | Working with the model download cache | Troubleshooting | Setting up the analysis programs | Preparing your analysis files | File format and Windows encodings | Detecting File Encoding | Converting to UTF-8 | Processing Files in ReaderBench | Setting the Java Path in Windows | Processing Files in Coh-Metrix | Processing Files in GAMET | References

Overview | Setup and API Keys | Example Data and Prompt Template | First batch: submit once, then resume | OpenAI storage, output limits, and error files | OpenAI Batch prompt caching and costs | Designing the Batch Grid | Submitting Many Batches with the Multi‑Batch Helpers | Create a run plan and output directory | Submit all runs (no polling) | Polling, Downloading, and Parsing (Resumable) | Building a single merged results table (all runs) | Resuming After Interruption | Next Steps | Citation

Choose a backend and model | What the package supports | Dated compatibility record | Optional future checks | Decision-model live qualification | Reproduce the design without sharing private data | Citation

Bring a spreadsheet or data frame | Samples and pairs | Traits and prompt templates | Decision instructions and probabilities | Provider and result schemas | Modeling and adaptive branches | Failure checks at schema boundaries | Related documentation | Citation

What you will do | Install the tools for this example | Your first result: scores from bundled comparisons | Read the scores | Read the reliability summary | An option for comparisons chosen in advance | An option for comparisons chosen adaptively | An experimental comparison with the Bayesian likelihood | Save an analysis | Prepare your own writing samples | Prepare a judging prompt | Collect a small set of new judgments | Configure one provider | Submit the pairs | Check what succeeded | Choose the next workflow | Common first-run problems | Citation

Start with the symptom | Storage, sampling, and service tiers | Controls by backend | Live calls: retain successes and failures | Decision models: save, resume, and inspect | Batch recovery | Adaptive recovery | Ollama setup and resources | Privacy and retention | Estimate costs before a larger run | Deterministic calculation example | Reuse pilot results | Citation

Start with the samples and their comparisons | Choose a method and fit the first comparison | Save your work and add the next comparison | Use saved uncertainty with E2 | Return to the original comparisons with E3 | Link a prepared standalone reference | Predict a comparison and check the result | Use saved Phase A results or several spokes | Dependency upgrades and saved hashes | Citation

What the correction estimates | A fixed random schedule | An adaptive p50 schedule | Reading the results and managing cost | Reusing an external worker pool | Citation

Before you start | Choose where starting scores are used | Opt in to predictive uncertainty | Choose the initial graph independently | Count evidence consistently across arms | A complete example with three algorithms | Load example texts and inspect the features | Extract features when you have Python configured | Share the same training and validation splits | Fit the three algorithms | Inspect validation before combining models | Combine algorithms trained on this assessment | Predict new items and initialize an assessment | Save a model and resume a session | Combining models from separate assessments | Inspect calibrated standardized coefficients | Understand saved models | Citation

Reading guide | Foundational concepts | Glossary and units | Algorithm overview | Predictive initialization and connected bootstrap | Online TrueSkill update and selection utility | Post-bootstrap pairing strategies | Hybrid rounds, anchors, and rank strata | Hybrid candidate filters, coverage, and fallback | Conceptual example of one adaptive step | Coverage and exploration | Duplicate and degree controls | Fallback ladder | Global identification and hybrid long-link tapering | Bayesian BTL refits | Code-derived defaults | Diagnostics, reliability, and stopping | Audit trail and reproducibility | Citation

How to use this reference | Recommended Models for Use | Scoring Model Development | Predictive Algorithms and R Packages Used | Steps | 1. Import Data | 2. Pre-Process Data | 3. Determine Optimal Tuning Parameters | 4. Final/Optimal Model for each Algorithm | 5. Estimate an Ensemble Model to Combine the Algorithms | 6. Generate Predicted Quality Scores from each Ensemble | 7. Average Scores to get Final Predicted Quality Scores | ReaderBench Model 1 | General Description | ReaderBench Model 1a | Algorithm Weightings in Ensemble | Metric Importance in Each Algorithm and Ensemble | ReaderBench Model 1b | ReaderBench Model 1c | ReaderBench Model 1d | Proportion of Variance by Varimax Rotated Component (RC) | Varimax Rotated Loadings | ReaderBench Model 1e | ReaderBench Model 1f | ReaderBench Model 2 | ReaderBench Model 2a | ReaderBench Model 2b | ReaderBench Model 2c | ReaderBench Model 3 | ReaderBench Model 3narr | ReaderBench Model 3exp | ReaderBench Model 3per | Coh-Metrix Model 1 | Coh-Metrix Model 1a | Coh-Metrix Model 1b | Coh-Metrix Model 1c | Coh-Metrix Model 1d | Coh-Metrix Model 1e | Coh-Metrix Model 1f | Coh-Metrix Model 2 | Coh-Metrix Model 2a | Coh-Metrix Model 2b | Coh-Metrix Model 2c | Coh-Metrix Model 3 | CohMetrix Model 3narr | Coh-Metrix Model 3exp | Coh-Metrix Model 3per | Automated Written Expression CBM (aWE-CBM) Model 1 | Correct Word Sequences Model | Correct Minus Incorrect Word Sequences Model | References

Five related questions | Start with comparisons chosen in advance | Conventional fitting and estimator provenance | What the adjustments change | Internal reliability: SSR and Bayesian EAP | Scale Separation Reliability (SSR) | Bayesian BTL and EAP reliability | Collect an adaptive schedule | Hybrid recovery stays within hybrid | A small provider-free p50 run | Empirical recovery and scale dispersion | Held-out predictive evaluation | Schedule-aware parametric bootstrap | Fixed versus adaptive scheduling | What the Monte Carlo summaries mean | Experimental lapse and positional-bias model matching | Reporting the complete workflow | Citation

What you will do | Choose a comparison workflow | Install and check CmdStan | A complete offline run with adaptive_rank() | Inspect the ranking and stopping state | Inspect comparisons and audit logs | What happens during a run? | Choose how later pairs are selected | Lower-level lifecycle and offline judges | Replay a sparse reservoir | Replay frozen directed judgments | Persistence, interruption, and resume | Invalid judgments and failed calls | A detailed live-LLM workflow | Inspect the live run | Continue the same study | Troubleshooting | Related documentation | Predictive warm-start priors | Citation

When to use this guide | Prerequisites | Prepare modeling input | Fit and summarize | What the output means | Chains and CPU allocation | Cumulative refits | Files, failures, and reproducibility | Predictive warm-start priors | Citation

Choose the workflow | What the research shows | Start from completed CJ results | Define the ordered rubric and labels | Same-set linear calibration | Read probabilities and hard scores | Evaluate held-out labels and inspect diagnostics | Explicit monotone alternative | Percentile levels without human labels | Reuse a historical rubric reference for a new set | Prepare a reference from a standalone ranking | Link the reference and new samples | Separate analytic traits | Uncertainty and reporting | Citation

Coordinates, identification, and the judge model | E1: fixed-shape offset | E2: full-covariance Gaussian posterior bridge | E3: single-use joint-offset model | MAP/Laplace calculation | E3-MCMC audit engine | Evidence, prediction, and uncertainty contracts | Selection, probes, and stopping | Persistence, multiple spokes, and rubric transport | Frozen standalone rubric-reference identity | Why old Phase B sessions cannot resume | Citation

Choose what to retain from Phase A | Prepare your comparisons | Fit and inspect the link | Save, add comparisons, and reuse a reference | Restart a session from an older version | Citation

How to read this guide | Motivation | Testing Process Summary | Trait descriptions and custom traits | Built-in traits | Setting a different built-in trait | Creating a custom trait | Why traits matter for positional bias testing | Example data used in tests | Built-in prompt templates | Forward and reverse pairs | Thinking / Reasoning Configurations Used in Testing | Anthropic (Claude 4.5 models) | thinking = "no_thinking" | thinking = "with_thinking" | Gemini 3 Pro Preview | Only thinking = "with_thinking" was used | OpenAI (gpt-4.1, gpt-4o, gpt-5.1) | thinking = "with_thinking" (gpt-5.1 only) | TogetherAI (Deepseek-R1, Deepseek-V3, Kimi-K2, Qwen3) | Summary of historical request configurations | Loading summary results | Column definitions | Interpreting the statistics | Summary results by prompt | Template test1 | Template text | Summary table | Template test2 | Template test3 | Template test4 | Template test5 | Per-backend summary | Anthropic models | Gemini models | OpenAI models | TogetherAI-hosted models | Conclusion | Citation