FictionRefine

A two-LLM workflow for iterative story generation and improvement using a writer model and a critic model.

Overview

FictionRefine orchestrates a collaborative workflow between two language models:

• Writer/Reviser (Model A): Generates initial stories and applies revisions
• Critic/Verifier (Model B): Reviews stories and provides detailed feedback

The system iteratively improves stories through cycles of writing, reviewing, revising, and verification until the story meets quality thresholds.

Features

• Comprehensive Critique: Multi-dimensional story evaluation (plot, character, setting, dialogue, mechanics)
• Iterative Improvement: Automated revision cycles with measurable quality improvements
• Flexible Configuration: YAML-based story configuration and model parameters
• Robust Pipeline: Handles model failures, validates outputs, and prevents story collapse
• Rich Output: Detailed artifacts for every step, enabling debugging and analysis

Installation

From PyPI (when published)

pip install fictionrefine

Development Installation

git clone https://github.com/davealaw/fictionrefine
cd fictionrefine
pip install -e ".[dev]"

Requirements

• Python 3.9+
• LM Studio
• Two language models (writer and critic)

Quick Start

1. Start LM Studio and load two models:

   • Model A (writer): e.g., qwen/qwen3-30b-a3b-2507
   • Model B (critic): e.g., gpt-oss-120b

2. Run a basic story generation:

fictionrefine \\
  --model-a "qwen/qwen3-30b-a3b-2507" \\
  --model-b "gpt-oss-120b" \\
  --story-config examples/story-config.yaml \\
  --outdir runs/my_first_story

3. Check the results:

cat runs/my_first_story/story_published_clean.txt

Configuration

Story Configuration

Create YAML files to define story parameters:

story:
  target_words: 1500
  setting: "a haunted lighthouse during a fierce storm"
  protagonist: "Mara, an elderly keeper who fears the sea"
  conflict: "the lighthouse lamp fails as ships approach"
  theme: "confronting one's deepest fears"

See examples/ directory for more examples.

Command Line Options

fictionrefine --help

Key parameters:

• --model-a, --model-b: Model names in LM Studio
• --story-config: Story configuration file
• --max-cycles: Number of revision cycles (default: 1)
• --critic-temperature: Creativity level for critic model (default: 0.5)
• --context-length: Context window size (default: 32768)

How It Works

1. WRITE: Model A generates a tagged story based on constraints
2. REVIEW: Model B provides comprehensive critique with specific suggestions
3. REVISE: Model A applies fixes and improvements
4. VERIFY: Model B validates changes and scores improvements
5. GATE: Accept if quality threshold met, otherwise iterate
6. PUBLISH: Output clean, human-readable story

Output Structure

Each run creates a timestamped directory with:

• story_published_clean.txt: Final story
• step1_write_raw.txt: Initial story draft
• step2_review.json: Detailed critique
• step3_revise.json: Applied revisions
• step4_verify.json: Quality verification
• run_summary.json: Overall metrics

Model Recommendations

Writer Models (Model A)

• Creative models: Qwen, Llama, Mistral variants
• Temperature: 0.6-0.8 for creativity
• Best for: Narrative generation, creative writing

Critic Models (Model B)

• Analytical models: GPT variants, Claude alternatives
• Temperature: 0.3-0.6 for consistency
• Best for: Detailed analysis, objective critique

Context Length

• Short stories (600-800 words): 8192 tokens
• Medium stories (1200-2000 words): 32768 tokens
• Long stories (3000+ words): 65536 tokens

Set context length in LM Studio when loading models.

Development

Setup Development Environment

git clone https://github.com/davealaw/fictionrefine
cd fictionrefine
pip install -e ".[dev]"
pre-commit install

Run Tests

pytest

Code Quality

black src/ tests/
ruff src/ tests/
mypy src/

Contributing

1. Fork the repository
2. Create a feature branch
3. Make changes with tests
4. Run quality checks
5. Submit a pull request

License

MIT License - see LICENSE file.

Citation

If you use FictionRefine in research or production, please cite:

@software{fictionrefine,
  title = {FictionRefine: Two-LLM Collaborative Story Generation},
  author = {Dave Law},
  year = {2025},
  url = {https://github.com/davealaw/fictionrefine}
}