diff --git a/docs/asklits/building.md b/docs/asklits/building.md new file mode 100644 index 000000000..59697e419 --- /dev/null +++ b/docs/asklits/building.md @@ -0,0 +1,215 @@ +--- +id: building +title: Build an AskLIT +sidebar_label: Build an AskLIT +description: Use the AskLIT Project Scaffolder to add sources, write a prompt, and try the chat. +--- + +# Build an AskLIT + +The [AskLIT basics](/docs/asklits/overview) page explains the three building +blocks: a system prompt, an optional knowledge base, and a chat. This page shows +where those pieces appear in the Project Scaffolder. + +The first three screens configure the assistant: + +1. **Knowledge:** What information may the assistant use? +2. **Prompt:** What should the assistant do with that information? +3. **Chat:** Does it behave the way you intended? + +The Scaffolder also has Evaluate and Export screens, covered in the +[evaluation](/docs/asklits/evaluating) and +[deployment](/docs/asklits/deploying) pages. + +## Open the Project Scaffolder + +Open the Scaffolder supplied by your instructor or organization. The sidebar +contains the five steps and a **Save or resume** panel. + +Some installations require a shared password before Chat or Evaluate can make +model calls. Uploading documents and editing prompts do not require a +password. The password gate protects a shared model budget. + +For a first project, follow this order: + +1. Add a small source packet, if your task needs one. +2. Write one focused prompt. +3. Try several questions in Chat. +4. Save any surprising questions for evaluation. + +## 1. Knowledge: add source documents + +![The current AskLIT Knowledge screen, showing the prompt profile selector and document uploader.](../assets/asklit/01-knowledge.png) + +### Choose a prompt profile + +Use **Attach uploaded files to** to choose a prompt profile. A new workspace +starts with one `Default` profile and a `default` knowledge-base name. If you +create multiple profiles, this selector connects uploaded documents +to the intended assistant. + +### Upload and index + +AskLIT accepts PDF, DOCX, TXT, and Markdown files. + +1. Select one or more files under **Upload Documents**. +2. Choose **Process & Index Documents**. +3. Wait for the confirmation message. +4. Confirm that the files appear under the selected knowledge base. + +Indexing extracts text, splits it into passages, and creates the searchable +representation used by Chat and Evaluate. + +AskLIT reports files that were indexed, skipped, or rejected. A duplicate file +is skipped instead of indexed twice. + +### Choose good source material + +Use documents that are: + +- Current and approved +- Text-based rather than image-only scans +- Organized with clear headings +- Explicit about dates, exceptions, and definitions +- Focused so relevant passages are easy to retrieve + +For a clinical exercise, a source packet might contain a fictional intake sheet, +a short statute or regulation excerpt, a clinic protocol, a referral list, and +an assignment rubric. Never upload confidential client information; see +[Ethical guidelines and data privacy](/docs/asklits/ethics) for data safety rules. +Give files clear names such as `fictional-client-packet.pdf`. + +### Why this step matters + +The knowledge base limits what the app can retrieve. It does not make the model +an expert, and it does not guarantee that every relevant passage will be found. +Keep the collection small and verify the sources yourself. + +:::warning + +**A source packet is not a source-of-truth guarantee.** Retrieval can miss a +relevant passage or surface a distracting one. In clinical exercises, teach +students to open **Sources used**, verify the passage, and note when the packet +does not answer the question. + +::: + +If the Prompt screen reports that the knowledge base is empty, verify that you +uploaded the files to the same profile and that the knowledge-base name matches. + +## 2. Prompt: give the app a job + +![The current AskLIT Prompt screen, showing the prompt name, knowledge-base name, and system-prompt editor.](../assets/asklit/02-prompt.png) + +The Prompt screen controls the app’s behavior. A useful prompt defines the role, +intended audience, specific task, communication style, and boundaries. + +### Main fields + +| Field | What it means | +| --- | --- | +| **Prompt to edit** | Switch between prompt profiles. | +| **Prompt name** | The label users will see, such as `Interviewing coach`. | +| **Knowledge-base name** | The source collection this prompt may search. | +| **System prompt** | The standing instructions for the assistant. | +| **Conversation starters** | Example questions displayed as quick-start buttons. | + +### A simple prompt recipe + +Keep initial prompts short and direct. A useful structure is: + +1. Role +2. Audience +3. Task +4. Source boundary +5. Answer style +6. Safety limits + +Make instructions concrete so behavior is easy to verify: + +| Vague instruction | Concrete instruction | +| --- | --- | +| "Be helpful." | "Give the user two practical next steps in plain language." | +| "Know the lease." | "Use only the uploaded fictional lease and checklist. Name the section supporting each point." | +| "Be careful with legal questions." | "Do not decide whether a real lease is valid. Identify terms to review and state what facts are missing." | +| "Sound like a client." | "Reveal facts only when asked an appropriate question. Stay in role until the user requests a debrief." | + +Good prompts also tell the app what to do when information is missing: +"If the source packet does not answer the question, state that clearly and suggest what professional help to seek." + +Do not paste entire source documents into the prompt text. Upload those as +knowledge-base files instead. Prompts supply instructions; knowledge bases supply facts. + +Example simulation prompt: + +```text +You are a simulated client in a law-school interviewing exercise. + +Use only the fictional client packet in the knowledge base. Reveal facts +naturally and only when the student asks an appropriate question. Do not +volunteer legal analysis or tell the student what question to ask next. + +Stay in role unless the student asks for a debrief. During a debrief, identify +missed facts, unanswered safety questions, and judgmental language. +Give specific, respectful feedback tied to the exercise rubric. + +This is a fictional educational simulation. Do not provide legal advice about +real people or real cases. +``` + +### Add another prompt + +Choose **Add another prompt** when a single published app should offer multiple +modes. For example: + +- `Simulated client` +- `Research coach` +- `Supervisor feedback` + +Each profile maintains its own prompt text, knowledge-base name, and conversation +starters. + +### Advanced pairing controls + +Open **Advanced: deployment details for this prompt** to see the YAML key and +the **Connected files** selector. + +Connected files narrow a prompt to specific files within its knowledge base. If +all files are selected, the profile searches the entire knowledge base, +including files added later. + +### Why this step matters + +The same underlying model can act as an intake client, a research coach, or a +supervisor. The prompt defines that persona and gives you an inspectable text +block to refine when output needs correction. + +## 3. Chat: try the app + +![The current AskLIT Chat screen, showing prompt and model selectors, the preview limit, and the question box.](../assets/asklit/03-chat.png) + +Chat provides a session-only preview for exploratory testing: + +- Use **Prompt to try** to switch profiles. +- Use **Model to try** to compare approved models. +- Try available conversation starters. +- Open **Sources used** under responses to inspect filenames, page numbers, and + retrieved excerpts. +- If no passages appear, check indexing status, connected files, and search terms. +- Use **Clear preview chat** before testing a new scenario. + +The preview allows twelve questions per conversation by default to protect shared class model budgets. + +### Questions to try + +During preview chat, test how the assistant handles direct answers, missing facts, and safety boundaries. Note any surprising or failing responses, as these make ideal test cases when building formal scenarios in [Evaluate an AskLIT](/docs/asklits/evaluating). + +## Save and resume + +Open **Save or resume** in the sidebar to download a workspace YAML file. It stores +settings, prompts, conversation starters, scenarios, and shared rubrics. It does +not store API keys, document text, vector indexes, branding images, or +generated answers. + +When importing a workspace, AskLIT lists any documents or branding files that +must be re-uploaded before running Chat or Evaluate. diff --git a/docs/asklits/deploying.md b/docs/asklits/deploying.md new file mode 100644 index 000000000..5f5afcba6 --- /dev/null +++ b/docs/asklits/deploying.md @@ -0,0 +1,98 @@ +--- +id: deploying +title: Deploy an AskLIT with GitHub and Streamlit +sidebar_label: Deploy with GitHub and Streamlit +description: Review Export, publish a project to GitHub, and deploy it with Streamlit Community Cloud. +--- + +# Deploy an AskLIT with GitHub and Streamlit + +The Scaffolder saves deployment settings for the final step. First build and test the +assistant, then use Export to package the project for GitHub and Streamlit Community Cloud. + +## Export is a final review + +![The current AskLIT Export screen, showing App settings and AI model panels before the deployment summaries and publish options.](../assets/asklit/05-export.png) + +The Export screen configures the published app. You can return to earlier screens to adjust prompts or documents before exporting. + +### App settings + +Set the public **App title** and **Welcome message**. Clarify what the tool does and does not do. For a clinic simulation, specify that it is an educational simulation. + +### AI model + +Select the provider and default model. The Scaffolder’s administrative credentials are +never exported. When deploying the generated app, add your API key to Streamlit's private secrets. + +If your institution provides an API gateway or approved model list, use those values. + +### Access and branding + +Choose **Public** or **Password Protected** access, and decide whether to enable the admin backend. You can also configure a custom logo, favicon, homepage URL, and footer text. + +When you set an access or administrator password, AskLIT hashes it before storing. Use distinct passwords for visitors and administrators. + +### Review the project + +The Export screen summarizes: + +- Prompt profiles and conversation starters +- Knowledge bases, document counts, and connected files +- Evaluation scenarios, rubrics, and recent pass rates + +Evaluation scenarios and test runs are not bundled into the exported chatbot. +Save the workspace YAML or download the results CSV to preserve your test history. + +### Deployment settings and secrets + +The **Deployment settings & secrets** panel generates a pre-formatted TOML block for Streamlit Cloud. + +Never put real API keys in: + +- A system prompt +- An uploaded document +- A public code repository +- A class assignment submission + +## Publish the project + +### Option A: Download a ZIP + +Choose **Prepare ZIP Download** to download the complete runtime package. You can manually upload the files to GitHub or host them on your own server. + +### Option B: Push to GitHub + +Choose **Connect to GitHub**. AskLIT uses GitHub's OAuth device code flow: + +1. Keep the AskLIT window open. +2. Open the GitHub authorization link provided by the Scaffolder. +3. Enter the one-time code. +4. Approve repository permissions. +5. Return to AskLIT and confirm authorization. + +Name the repository and select public or private visibility. If public, all indexed knowledge-base documents will be publicly accessible. + +## Deploy with Streamlit Community Cloud + +Streamlit Community Cloud hosts the generated app directly from GitHub: + +1. Open [Streamlit Community Cloud](https://share.streamlit.io/) and sign in with the GitHub account that owns the repository. +2. Choose **Create app**. +3. Select your generated repository. +4. Set the branch to `main`. +5. Set **Main file path** to `app.py`. +6. Choose your desired app URL. +7. Open **Advanced settings**. +8. Paste your configuration into the private **Secrets** box. +9. Choose **Deploy**. + +Initial dependency installation may take a few minutes. Once live, test a question requiring knowledge-base retrieval to verify that sources are functioning properly. + +For a working reference, explore the [AskLIT Clinical Skills Gallery](https://asklit-clinical-skills-gallery-5hokdmbr8cbcyztuvkeaty.streamlit.app/) and select Haiku helper, Lease lens, or Interviewing client from the sidebar. The [gallery page](./gallery) explains the design of each profile. + +## Ownership and privacy + +The published GitHub repository and deployed Streamlit app belong to your accounts. Keep secrets private and rotate API keys if accidentally committed. + +For guidance on API provider data policies, enterprise Zero Data Retention (ZDR), and HIPAA compliance, see [Ethical guidelines and data privacy](./ethics). diff --git a/docs/asklits/ethics.md b/docs/asklits/ethics.md new file mode 100644 index 000000000..7517eebe3 --- /dev/null +++ b/docs/asklits/ethics.md @@ -0,0 +1,109 @@ +--- +id: ethics +title: Ethical guidelines and data privacy +sidebar_label: Ethical guidelines +slug: ethics +description: Data privacy, API training policies, ABA Formal Opinion 512, HIPAA compliance, and Zero Data Retention (ZDR) for AskLITs. +--- + +# Ethical guidelines and data privacy + +When building AskLITs for legal education, legal aid, or clinic workflows, protecting confidential information and understanding provider data policies are fundamental requirements. + +This guide explains how commercial AI providers handle data, how legal ethics rules (including ABA Formal Opinion 512) apply, and when specialized regulatory frameworks like HIPAA come into play. + +--- + +## Data privacy and API training policies + +A common concern with AI applications is whether uploaded documents and user questions will be used to train future models. + +### Consumer web chats vs. developer APIs + +There is a critical distinction between consumer chat interfaces and developer APIs: + +- **Consumer web interfaces** (such as standard consumer web versions of ChatGPT, Claude, or Gemini) often enable model training on user conversations by default unless the user opts out. +- **Commercial developer APIs** (such as the OpenAI API, Anthropic API, Google Cloud Vertex AI, and Microsoft Azure OpenAI) **do not train on your API inputs or outputs out of the box**. + +As long as your AskLIT connects to standard commercial APIs using an API key or enterprise endpoint, your data is not used to train future models unless you explicitly opt in to data sharing or custom fine-tuning programs. + +### Data retention and abuse monitoring + +Although standard commercial developer APIs do not train on your data, most providers retain API logs temporarily (typically for 15 to 30 days) on secure servers strictly for abuse and misuse monitoring, after which logs are deleted. + +This transient logging is standard across cloud software and does **not** involve model training or third-party exposure. + +--- + +## Legal ethics and ABA Formal Opinion 512 + +The American Bar Association's [Formal Opinion 512: Generative Artificial Intelligence Tools](https://www.americanbar.org/content/dam/aba/administrative/professional_responsibility/ethics-opinions/aba-formal-opinion-512.pdf) (see also the [ABA Ethics Opinions archive](https://www.americanbar.org/groups/professional_responsibility/publications/ethics_opinions/)) addresses a lawyer's duty of confidentiality under Model Rule 1.6 when using generative AI. + +The primary confidentiality concern under Opinion 512 is whether an AI tool "learns" from client information, meaning the system retains or uses confidential data to train future models or could expose that information to other users. + +Because commercial developer APIs do not train on API inputs or outputs by default: + +- **Using a standard commercial developer API generally satisfies the core confidentiality concern regarding "learning" AI systems**, because the model does not train on or assimilate your data. +- **Transient abuse logging (15 to 30 days) does not violate ethical rules under Opinion 512**, provided the attorney exercises reasonable due diligence regarding the vendor's security, privacy terms, and reliability. + +```mermaid +flowchart TD + A["User Input / Document Upload"] --> B{"Contains Real Client Data?"} + B -- "No (Fictional / Public Data)" --> C["Standard Commercial API\n(No training, 30-day transient logging)"] + B -- "Yes" --> D{"Contains Protected Health Info (HIPAA)?"} + D -- "No (General Legal Matter)" --> E["Standard Commercial Developer API\n(Complies with ABA 512: no model training)"] + D -- "Yes (Medical Records / PHI)" --> F["Enterprise Cloud Provider\n(Signed BAA + Zero Data Retention)"] +``` + +--- + +## When is Zero Data Retention (ZDR) needed? (HIPAA & PHI) + +**Zero Data Retention (ZDR)** is an enterprise configuration where the cloud provider disables even temporary abuse logging, processing prompts strictly in memory and discarding them immediately. + +ZDR is **not** required for standard legal ethics compliance under ABA Opinion 512. In practice, ZDR is almost exclusively offered for specialized regulatory regimes, particularly **HIPAA** (the Health Insurance Portability and Accountability Act): + +### HIPAA and Business Associate Agreements (BAAs) + +If an AskLIT processes Protected Health Information (PHI), such as medical records in a disability clinic, domestic violence triage with health disclosures, or personal injury matters, HIPAA regulations apply: + +- Standard pay-as-you-go commercial developer API accounts are **not HIPAA-compliant by default**. +- You must use an enterprise cloud provider (such as Microsoft Azure OpenAI, AWS Bedrock, or Google Cloud Vertex AI) and execute a formal **Business Associate Agreement (BAA)** before transmitting PHI. +- Providers typically bundle or require **Zero Data Retention (ZDR)** as part of their enterprise HIPAA/BAA compliance package. +- All other components in the hosting stack (such as Streamlit hosting and application databases) must also satisfy HIPAA physical, technical, and administrative safeguards. + +--- + +## Safe starting rules for law school clinics + +For law-school courses, clinical simulations, and student projects, adopting a "safe by design" approach simplifies development and avoids unnecessary risk: + +### 1. Use fictional or public source material +- Build knowledge bases from public statutes, court forms, government handbooks, published case opinions, or intentionally fictional case files. +- Avoid uploading unredacted client names, addresses, dates of birth, Social Security numbers, or privileged client communications to student or public projects. + +### 2. Guard secrets and credentials +- Never paste API keys into system prompts, uploaded documents, or public code repositories. +- In AskLIT, export credentials into Streamlit Cloud's private **Secrets** panel only. +- If a GitHub repository is public, all documents bundled into its knowledge base are publicly viewable. + +### 3. Maintain human supervision +- An AskLIT is an educational and workflow tool, not a licensed attorney. +- In student simulations, prompts should instruct users to consult a licensed supervising attorney or instructor for real matters. +- Design single-turn tools to produce issue checklists, practical questions, and guidance rather than unsupervised legal conclusions. + +### 4. Verify citations and retrieval +- LLMs can generate persuasive text that misstates facts or cites non-existent authorities. +- Teach students to open **Sources used** in AskLIT Chat, confirm retrieved passages against primary sources, and note when the source packet does not contain the answer. + +--- + +## Summary checklist + +| Context | Data Handling & Compliance | +| :--- | :--- | +| **Model Training** | Standard commercial developer APIs **do not train** on your data out of the box. | +| **Legal Ethics (ABA 512)** | Using non-training commercial APIs satisfies the duty of confidentiality regarding "learning" AI systems. Standard 15 to 30 day transient abuse logging is permissible with reasonable vendor due diligence. | +| **Protected Health Info (HIPAA)** | Requires an enterprise cloud provider, a signed **Business Associate Agreement (BAA)**, and a **Zero Data Retention (ZDR)** agreement. | +| **Classroom & Clinic Projects** | Best practice is to use fictional case packets, public authorities, and de-identified materials. | +| **Supervision** | Maintain human-in-the-loop oversight from a supervising attorney or instructor for all legal work. | diff --git a/docs/asklits/evaluating.md b/docs/asklits/evaluating.md new file mode 100644 index 000000000..975682155 --- /dev/null +++ b/docs/asklits/evaluating.md @@ -0,0 +1,303 @@ +--- +id: evaluating +title: Evaluate an AskLIT +sidebar_label: Evaluate an AskLIT +description: Build scenarios, use gold labels and rubrics, compare prompts and models, and read results. +--- + +# Evaluate an AskLIT + +Chat helps you explore. Evaluation helps you verify the same expectations +systematically whenever you update a prompt, add documents, or switch models. + +An evaluation scenario is a saved test case. It contains an input, an expected +condition, and an optional description. A set of scenarios makes your quality +standards transparent and repeatable. + +:::info + +**Evaluation is evidence, not a guarantee.** A passing score means the app +satisfied your test conditions for these specific scenarios. It does not guarantee +perfection on unseen questions or different document sets. + +::: + +## Why evaluate? + +An AI answer can sound plausible while still being factually incorrect or ungrounded. +Repeatable evaluation tests whether the app: + +- Grounds answers in the source packet +- Acknowledges missing information +- Adheres to the assigned role +- Maintains desired tone and formatting +- Avoids unsupported or unsafe claims + +## 1. Build scenarios + +![The current AskLIT Evaluate screen, showing provider controls, the gold-labeled scenario editor, and the beginning of Run settings.](../assets/asklit/04-evaluate.png) + +The scenario table has three primary columns: + +| Column | What to enter | +| --- | --- | +| **Input** | The question, instruction, or user prompt. | +| **Gold label / `__expected`** | The condition that determines whether the response passes. | +| **Description** | A short label such as `Safety screening` or `Missing fact`. | + +Add rows directly in the editor or upload a UTF-8 CSV. AskLIT accepts standard +headers such as `input`, `question`, `query`, `gold_label`, `expected`, and +`reference_answer`. Use **Download scenarios as CSV** to save test suites. + +### Write useful scenarios + +A balanced test suite includes: + +1. A direct question answered explicitly in a source +2. A question combining multiple passages +3. A question using paraphrased terminology +4. An exception or edge case +5. A question unanswerable from the source packet +6. A query testing safety and ethical boundaries + +The last two categories are essential for verifying that the model acknowledges uncertainty rather than fabricating answers. + +## 2. Choose how an answer passes + +AskLIT separates evaluation into two complementary tools: **row-specific gold labels** in the scenario table and **shared LLM rubrics** in the advanced settings panel. + +### Row-specific checks: Gold labels in the table + +The **Gold label / `__expected`** column in the scenario table tests whether a specific question receives its required factual answer. Use gold labels for decisive terms, dates, statutory deadlines, or classifications: + +- Classifications (such as `eligible`, `not eligible`, or `refer`) +- Specific dates, deadlines, or citations from the source packet +- Required keyword combinations + +Effective gold labels are: + +- **Source-backed:** Supported directly by your uploaded materials. +- **Decisive:** Focused on the key factual condition for that specific question. +- **Concise:** Checking key phrases rather than requiring full-sentence matches. +- **Flexible:** Using phrase checks so valid alternative phrasing still passes. + +#### Exact and phrase checks + +Plain text performs an exact match after whitespace normalization. Prefix checks allow flexible matching: + +```text +contains:next court date +icontains:fourteen days +contains-any:eviction,notice +icontains-any:tenant,renter,lessee +contains-all:deadline,appeal +icontains-all:summons,complaint,entry date +``` + +`contains` is case-sensitive, while `icontains` ignores case. + +- Use `contains-any` or `icontains-any` when any one of several equivalent terms is sufficient. +- Use `contains-all` or `icontains-all` when every listed term must appear. + +These string checks verify text presence. To evaluate overall reasoning, tone, and groundedness, use a shared LLM rubric. + +### Overall quality criteria: Shared LLM rubrics + +General quality standards (such as staying grounded in sources, writing in plain language, acknowledging uncertainty, and avoiding conclusive legal advice) should apply consistently across your entire test suite. It rarely makes sense to write a different LLM rubric for each individual row. + +To set standing quality criteria, click **Advanced: shared rules for every scenario** and add your rules (one per line). For example: + +```text +Stays grounded in the retrieved passages and does not invent facts. +Uses plain language suitable for a client without legal training. +Identifies the next practical step and states when human supervision is needed. +``` + +When evaluating with rubrics: + +1. A separate judge model scores responses from 0 to 1, where 0.70 or higher passes. +2. The judge reviews the user question, retrieved passages, model answer, and all shared rules. +3. Focus rubric rules on observable criteria (using active verbs such as `identifies`, `explains`, `avoids`). +4. Explicitly state boundaries (such as avoiding definitive legal conclusions). + +*(Note: While AskLIT supports row-level `llm-rubric:` prefixes in the scenario table for compatibility with Promptfoo CSV imports, defining shared rules in the Advanced panel is the standard, cleaner design pattern).* + +### How gold labels and shared rubrics work together + +When a scenario includes a gold label in the table and shared rules in the Advanced panel: + +- The **Gold label** checks that the specific factual requirement for that question was satisfied. +- The **Shared rubric** checks that the overall response was communicated clearly, safely, and accurately. +- **Both checks must pass** for the scenario row to pass. + +:::tip + +**Calibrate rubrics on sample answers.** Test new rubrics against one clearly strong answer, one failing answer, and one borderline response to ensure the judge grades consistently before running large matrices. + +::: + +## 3. Generate and refine draft scenarios + +The **Generate gold-labeled scenarios** feature can draft starter test cases from your prompt and indexed documents. + +Always review generated scenarios before testing. Remove questions that rely on facts outside your packet, adjust imprecise gold labels, and add scenarios testing omissions and safety boundaries. The instructor or domain expert, not the model, defines the standard. + +## 4. Run settings + +Choose a provider and approved model. + +### Single model + +Runs each scenario once using one prompt, knowledge base, and model. Ideal while refining test cases. + +### Prompt × model matrix + +Runs every scenario across all selected prompts, knowledge bases, and models. + +Total answer completions equal: + +```text +scenarios × prompts × knowledge bases × models +``` + +For example, 8 scenarios × 2 prompts × 2 models = 32 answer calls. If LLM rubrics are enabled, each answer generates a separate judge call. AskLIT caps matrix runs at 60 calls to protect shared budgets. + +**Retrieved passages per run** sets how many passages accompany each prompt (5 is a practical default). + +## Case study: evaluating the Haiku helper + +The Haiku helper provides a clear baseline because formatting and prompt adherence can be evaluated without retrieval variables. + +We evaluated the three prompt versions from [AskLIT basics](./overview): + +| Prompt | Instruction | +| --- | --- | +| **Haiku Minimal** | Write a haiku about the user's topic. | +| **Haiku Specific** | Write exactly three English lines; add no title or explanation. | +| **Haiku Detailed** | Add format rules, approximate 5-7-5 syllable rhythm, and concrete imagery. | + +Test inputs: + +```text +Write a haiku about a lease. +Write a haiku about a rainy courthouse. +Write a haiku about a missed deadline. +``` + +Because many valid poems exist, we used shared rubric rules instead of fixed text labels: + +```text +Writes exactly three non-empty lines about the requested topic. +Uses haiku-like imagery rather than explaining the topic in prose. +Adds no title or explanation. +``` + +The matrix evaluated three response models (`gpt-5.4-mini`, `gpt-5.4-nano`, and `gpt-5.6-sol`) using `gpt-4.1-mini` as an independent judge. This generated 27 answer calls and 27 judge calls. + +![Haiku evaluation setup with three prompts, three response models, three shared rubric rules, and an independent judge model.](../assets/asklit/06-haiku-evaluation-setup.png) + +### Results + +All three models passed all 9 test combinations (three prompts × three scenarios): + +| Response model | Passed | Pass rate | Takeaway | +| --- | ---: | ---: | --- | +| `gpt-5.4-mini` | 9/9 | 100% | Satisfied formatting rules across all prompts. | +| `gpt-5.4-nano` | 9/9 | 100% | Successfully met criteria at lower computational cost. | +| `gpt-5.6-sol` | 9/9 | 100% | Met criteria with minor stylistic variations in poems. | + +Total token usage was approximately 10,079 tokens across all answer and judge calls. + +![AskLIT evaluation results showing 27 runs, 27 graded rows, a 100% pass rate, and approximately 10,079 tokens.](../assets/asklit/07-haiku-evaluation-results.png) + +![The detailed AskLIT results table expanded in full-screen view.](../assets/asklit/08-haiku-evaluation-results-fullscreen.png) + +### Choosing a model and managing bias + +When selecting models and judges: + +1. **Use independent judges:** When feasible, use a different model family for evaluation to reduce self-preference bias. +2. **Review judge rationales:** Check why a judge passed or failed borderline cases. +3. **Calibrate with human review:** Compare LLM judge grades against instructor assessments. +4. **Balance speed and cost:** Smaller models often perform well on straightforward tasks, while larger models provide more nuanced feedback on complex reasoning. + +## Case study: when a knowledge base changes the answer + +To measure the impact of document retrieval, we tested a question about Massachusetts summary process rules: + +> When must a Massachusetts tenant file an answer in an eviction case, and is the trial date the deadline? + +*(Educational exercise based on Massachusetts Trial Court rules; not legal advice.)* + +### The legal rules in the source packet + +The source packet established that: + +- The answer deadline is on or before the Monday following the Monday entry date. +- The official summons identifies the exact case deadline. +- The answer must be filed and served by that deadline, not held until trial. +- Under Uniform Summary Process Rule 10, appearing without an answer does not cause immediate default, but postpones trial by one week unless the plaintiff agrees in writing to proceed immediately. + +We evaluated three prompts (General, Rule Checklist, and Uncertainty First) across three models (`gpt-5.4-mini`, `gpt-4.1-nano`, and `gpt-5.4-nano`) with `gpt-5.6-sol` as the judge. + +### Baseline without knowledge base + +Running the matrix with no uploaded documents produced a **33% pass rate** (9 of 27 passed). Without access to local procedural rules, models frequently gave inaccurate or vague deadline guidance. + +![Evaluation setup for the Massachusetts eviction deadline matrix without an indexed knowledge base.](../assets/asklit/09-ma-eviction-no-kb-setup.png) + +![Massachusetts eviction deadline evaluation without a knowledge base: 27 graded rows and a 33% pass rate.](../assets/asklit/10-ma-eviction-no-kb-results.png) + +### Results with source packet + +Adding the Massachusetts procedural packet increased the pass rate to **100%** (27 of 27 passed). + +![Evaluation setup for the Massachusetts eviction deadline matrix with the indexed source packet.](../assets/asklit/11-ma-eviction-kb-setup.png) + +![Massachusetts eviction deadline evaluation with the source packet: 27 graded rows and a 100% pass rate.](../assets/asklit/12-ma-eviction-kb-results.png) + +| Run | Knowledge base | Graded rows | Pass rate | Approx. tokens | +| --- | --- | ---: | ---: | ---: | +| **Baseline** | None | 27 | 33% | 37,492 | +| **Source-grounded** | Massachusetts answer-deadline packet | 27 | 100% | 63,580 | + +### What this demonstrates + +- **General models struggle with local procedural specifics:** RAG is essential when answers depend on local court rules, agency guidelines, or internal policies. +- **Prompts guide how sources are applied:** System instructions tell the model how to explain exceptions and when to direct users to human counsel. +- **Gold labels and shared rubrics work together:** Gold labels verify that the exact deadline or Rule 10 distinction is identified for that specific question, while shared rubrics ensure groundedness and plain-language explanation across all questions. + +## 5. Read the results + +The results table displays: + +- **Runs:** Total combinations executed +- **Graded:** Rows evaluated by string match or rubric +- **Pass rate:** Percentage of graded rows passing criteria +- **Approx. tokens:** Estimated total token consumption +- **Detailed breakdown:** Input, prompt, model, gold label, answer, score, rationale, latency, and retrieved sources + +You can filter outcomes and download the complete data as a CSV. + +If a clear winner emerges, **Use this prompt and model for the exported app** automatically applies that configuration to the Export step. + +## Clinical scenario examples + +For a fictional initial-interview coach, add standing quality standards to **Advanced: shared rules for every scenario** (such as staying grounded in the source packet, maintaining an empathetic tone, and avoiding unverified assumptions). Then use the **Gold label** column for question-specific factual checks: + +| Input | Gold label in scenario table | What it tests | +| --- | --- | --- | +| "Which deadline appears in the packet?" | `icontains:Monday` | Checks that the specific procedural deadline is identified. | +| "Does the client need immediate safety help?" | `icontains:safety` | Verifies that emergency safety screening is triggered. | +| "Ask for the facts needed to assess service of process." | `contains-all:summons,complaint,date` | Ensures all essential procedural elements are requested. | +| "What does the packet say about an unmentioned issue?" | `icontains:not mentioned` | Verifies acknowledgment of missing information. | + +## Evaluation workflow summary + +1. Explore behavior in **Chat**. +2. Identify edge cases or weaknesses. +3. Add scenarios with question-specific gold labels in **Evaluate**. +4. Set standing quality criteria in **Advanced: shared rules for every scenario**. +5. Run a single-model test to calibrate rules and labels. +6. Refine the prompt or source documents. +7. Run a prompt × model matrix to verify improvements across configurations. diff --git a/docs/asklits/gallery.md b/docs/asklits/gallery.md new file mode 100644 index 000000000..4dfa191bd --- /dev/null +++ b/docs/asklits/gallery.md @@ -0,0 +1,183 @@ +--- +id: gallery +title: "AskLIT gallery: three toy projects" +sidebar_label: AskLIT gallery +description: Three small AskLIT examples, packaged as one deployable app, that show different uses for prompts, knowledge bases, and chat. +--- + +# AskLIT gallery: three toy projects + +These examples demonstrate different use cases for system prompts, knowledge bases, and chat interactions. + +## One app, three focused prompts + +A single AskLIT deployment can offer multiple prompt profiles in one interface. +The [AskLIT Clinical Skills Gallery](https://asklit-clinical-skills-gallery-5hokdmbr8cbcyztuvkeaty.streamlit.app/) deployable app includes three exercises: + +| Prompt profile | Student activity | Shared source material | +| --- | --- | --- | +| **Haiku helper** | Practice following a constrained output format. | None needed. | +| **Lease lens** | Review a fictional lease and identify questions for supervision. | Fictional lease excerpt and review checklist. | +| **Interviewing client** | Practice open-ended, non-leading intake questions. | Fictional client profile. | + +The source files are fictional. Replace them with approved materials before teaching, and never upload confidential client information (see [Ethical guidelines and data privacy](./ethics)). + +:::tip + +**Try the deployed gallery:** [Open AskLIT Clinical Skills Gallery](https://asklit-clinical-skills-gallery-5hokdmbr8cbcyztuvkeaty.streamlit.app/). +Choose a prompt profile from the sidebar and try the quick-start buttons. + +::: + +## 1. Haiku helper + +### What it does + +Converts any user topic into a three-line haiku. + +### Knowledge base + +None. The exercise evaluates prompt constraints and formatting without document retrieval. + +### Prompt + +```text +You write short haiku in English. + +When the user gives you a topic, write exactly three lines about that topic. +Aim for a 5-7-5 syllable pattern. Do not explain the poem or add a title. +Use simple, concrete images. +``` + +### Try in Chat + +```text +Write a haiku about a rainy courthouse. +``` + +### What it teaches + +Prompts control role, style, and formatting. A knowledge base is optional. + +![The live AskLIT Clinical Skills Gallery with the Haiku helper selected and its conversation starter visible.](../assets/asklit/14-gallery-live-haiku-start.png) + +![The live Haiku helper responding to the rainy-courthouse starter.](../assets/asklit/15-gallery-live-haiku-response.png) + +## 2. Lease lens + +### What it does + +Examines a **fictional** lease excerpt and generates a checklist of clauses to review. It does not determine whether the lease is legally enforceable. + +This provides a pattern for single-turn inquiries: converting a user question into structured review questions and recommending professional counsel. + +### Knowledge base + +Upload a fictional lease excerpt and review checklist to a shared knowledge base (such as `gallery-sources`). + +### Prompt + +```text +You are a plain-language lease review helper for a legal-education exercise. + +Use only the fictional lease and review checklist in the knowledge base. +Point out terms that deserve closer review and quote or name the relevant +section when possible. Ask for missing information instead of guessing. + +Do not say that a lease is legally valid or invalid. Do not give legal advice +about a real person’s lease. End with a short list of questions the reader can +bring to a qualified lawyer or housing counselor. +``` + +### Try in Chat + +```text +Is this lease acceptable? I am worried about the late fee and the entry clause. +``` + +### How to evaluate + +- **Gold label in scenario table:** `contains-all:late fee,entry` +- **Shared rules in Advanced panel:** + ```text + Identifies the relevant terms in the fictional excerpt and explains what information is missing. + Avoids definitive legal conclusions and provides a practical next step. + ``` + +### What it teaches + +AskLITs can provide structured issue spotting for single-turn questions while maintaining explicit safety limits and directing users to human supervision. + +![The live Lease lens response identifying fictional lease sections, missing facts, and questions for a lawyer or housing counselor.](../assets/asklit/16-gallery-live-lease-response.png) + +## 3. Interviewing client + +### What it does + +Simulates an initial intake client. It reveals facts only when asked appropriate questions. A separate debrief profile can evaluate communication choices after the simulation. + +### Knowledge base + +Upload a fictional client profile to the shared `gallery-sources` knowledge base. + +### Prompt + +```text +You are a fictional client in a law-school interviewing exercise. + +Stay in role. Reveal facts naturally and only when the student asks. Do not +volunteer legal analysis or tell the student what to ask next. Do not invent +facts outside the client profile. + +If the student asks for a debrief, give respectful feedback about missed facts, +safety questions, and judgmental language. This is a fictional exercise, not +legal advice. +``` + +### Try in Chat + +```text +Hi, thanks for meeting with me. Can you tell me what brought you here today? +``` + +### How to evaluate + +Configure standing quality rules in **Advanced: shared rules for every scenario**: + +```text +Stays in character as a fictional intake client and reveals facts only when asked appropriate questions. +Does not volunteer legal analysis or invent facts outside the fictional client profile. +``` + +Then test specific student inputs in the scenario table: + +- An open-ended question testing whether the client shares their basic situation +- A leading question testing whether the client clarifies or pushes back naturally +- A safety question testing whether the client discloses urgent concerns +- A question about an unlisted fact testing whether the client states they do not know + +### What it teaches + +An AskLIT can serve as an interactive practice partner. Multiple prompt profiles can share one knowledge base: one for roleplay and one for debriefing. + +![The live Interviewing client profile responding in role to an open-ended intake question.](../assets/asklit/17-gallery-live-interview-response.png) + +## How to package the gallery + +To package multiple profiles in one deployment: + +- Configure three prompt profiles, each with a clear conversation starter +- Connect profiles to the shared knowledge base (`gallery-sources`) +- Write a welcome message explaining the exercises +- Choose public or class-protected access +- Keep API keys and confidential materials out of the repository + +Reference repository: [nonprofittechy/asklit-clinical-skills-gallery](https://github.com/nonprofittechy/asklit-clinical-skills-gallery). Deploy via Streamlit Community Cloud with private secrets as detailed in [Deploy with GitHub and Streamlit](./deploying). + +## Choosing an example + +To define an AskLIT, start with one sentence: + +> "This AskLIT helps [audience] practice or complete [task] using [source documents]." + +If that sentence requires multiple unrelated tasks, split the project into separate AskLIT profiles. diff --git a/docs/asklits/overview.md b/docs/asklits/overview.md new file mode 100644 index 000000000..ece23632d --- /dev/null +++ b/docs/asklits/overview.md @@ -0,0 +1,210 @@ +--- +id: asklits_overview +title: AskLIT basics +sidebar_label: AskLIT basics +slug: overview +description: What an AskLIT is, what it is for, and how a prompt, knowledge base, and chat work together. +--- + +# AskLIT basics + +An **AskLIT is a simple, focused AI application built to do one task well**. +You give it instructions. You can also give it a small collection of documents. +People then use it through a chat-like screen. + +The name describes the idea: an AskLIT is an AI application designed for a +particular question, task, audience, or learning exercise. The AskLit Project +Scaffolder helps you build one without writing a complete application from +scratch. + +## What an AskLIT is (and is not) + +An AskLIT is not a new AI model. It is a configured application that +uses an existing model, adding light structure: + +1. A user asks a question or gives an instruction. +2. AskLIT finds relevant passages in the documents you uploaded, if any. +3. AskLIT sends the user’s message, your instructions, and those passages to + the model. +4. The model writes a response in the role and format you requested. + +The documents are optional. A Haiku writer does not need a knowledge base. A +housing-rights helper probably does. In both cases, the **system prompt** keeps +the app focused. + +### Similar tools + +| Tool or approach | Main job | What an AskLIT adds | +| --- | --- | --- | +| **General-purpose chat assistant** | Talk about almost anything. | A narrower role that is easier to explain, test, and teach. | +| **Document or search chatbot** | Find answers in a collection of documents. | The option to use documents while also simulating a person, coaching a skill, or following a special format. | +| **Form or workflow app** | Ask fixed questions and apply fixed rules. | Open-ended conversation that can respond to each user turn. | +| **Simulation tool** | Let someone practice a conversation. | A fictional role, a case packet, and a separate debrief or feedback role. | +| **AI evaluation tool** | Test whether an AI response meets expectations. | Evaluation built into the same project you are building. | + +AskLITs work well for: + +- Interactive Q&A about one source collection +- Single-turn questions (such as "Is this lease acceptable?") followed by a checklist of things to verify +- Simulated clients, opposing parties, mediators, or supervisors +- Practice coaches that ask follow-up questions instead of giving answers directly +- Tutors that explain a specific topic in a required format +- Rubric-based feedback on fictional student work product + +They are not suitable for emergency response, confidential client files, unsupervised +legal advice, or tasks requiring strict deterministic calculations. Always involve a human +supervisor when an outcome affects real legal rights or safety. + +## The three building blocks + +### 1. A system prompt + +A system prompt is the app’s standing set of instructions. It tells the model: + +- What role to play +- Who it is helping +- What task to perform +- What sources to trust +- How to format the response +- What boundaries to respect + +The prompt does not magically add facts. Telling a model "You know landlord-tenant law" is not +a substitute for uploading an approved source packet. The prompt controls +behavior; the knowledge base supplies selected facts. + +#### What makes a system prompt good? + +A good system prompt is: + +- **Specific:** Gives the app one clear job. +- **Observable:** Describes behavior you can see and test. +- **Ordered:** Tells the app what to do before listing exceptions. +- **Bounded:** Explains what to do when a source does not contain the answer. +- **Concise:** Avoids conflicting or redundant rules. + +Avoid starting with only "Be helpful." Instead, name the role, audience, task, response format, and key limits. +The [building page](/docs/asklits/building) provides a prompt checklist and clinical examples. + +### 2. A knowledge base + +A knowledge base is a searchable collection of uploaded documents. AskLIT supports +PDF, DOCX, TXT, and Markdown files. When you index a file, AskLIT extracts +its text and breaks it into searchable passages. + +Use a knowledge base when answers should come from a particular handbook, +policy, fictional case packet, statute excerpt, or referral guide. Keep source +collections small and approved. Never upload confidential client information. + +If a prompt points to an empty knowledge base, the app will answer using the +model’s general background knowledge. A fluent answer is not proof that the model +grounded its response in your documents. + +### 3. A chat + +Chat is how users interact with an AskLIT. The user asks a question, the app applies +the prompt, retrieves relevant passages if available, and the model +generates a response. + +Chat is ideal for exploratory testing. It helps you notice if a prompt is too vague, +if a source is missing, or if the assistant adopts an unwanted tone. The [evaluation page](/docs/asklits/evaluating) explains how to turn those observations into repeatable tests. + +## A small example: the Haiku AskLIT + +Start with a simple example that demonstrates the core workflow without requiring legal source documents. + +### Goal + +Build an AskLIT called **Haiku helper** whose sole task is to turn a user’s topic +into a three-line haiku. + +This is an effective first exercise because success is easy to evaluate visually. If the answer is +not a haiku, the issue lies in the prompt or model response rather than a missing document or complex legal rule. + +### Try three prompt versions + +In the **Prompt** screen, set **Prompt name** to `Haiku helper`. Then try each version: + +**Version A: minimal** + +```text +Write a haiku about the user's topic. +``` + +**Version B: more specific** + +```text +You write short haiku in English. When the user gives you a topic, write +exactly three lines about it. Do not add a title or explanation. +``` + +**Version C: detailed** + +```text +You write short haiku in English. + +When the user gives you a topic, write exactly three lines about that topic. +Aim for a 5-7-5 syllable pattern. Do not explain the poem or add a title. +Use simple, concrete images. +``` + +Test each version with the same three inputs: + +```text +Write a haiku about a lease. +Write a haiku about a rainy courthouse. +Write a haiku about a missed deadline. +``` + +Compare the responses: + +- Exactly three lines +- Direct connection to the topic +- No extraneous introductory or concluding text +- Concrete imagery +- Approximate 5-7-5 syllable structure + +The detailed version generally produces more consistent structure. However, longer prompts are not automatically better. The goal is the shortest prompt that reliably produces the desired behavior. + +:::tip + +**Turn prompt testing into an experiment.** Have students predict which prompt version will work best before testing. Keep the inputs, model, and evaluation criteria constant while changing only the prompt text. + +::: + +Leave the knowledge base empty. Add a conversation starter such as `Write a haiku about a rainy courthouse.` + +### Example chat + +Ask: + +```text +Write a haiku about a lease. +``` + +Example response: + +```text +Lease terms hold your home +Read the words before you sign +Ask what you don't know +``` + +The exact phrasing will vary. The key requirement is that the response contains three lines on topic without unrequested prose. + +### Key takeaways + +The Haiku AskLIT illustrates fundamentals used in complex applications: + +- **The prompt controls behavior:** It establishes the role and response format. +- **Prompt iterations are experiments:** Add instructions to address observed flaws rather than adding unnecessary length. +- **Knowledge bases are optional:** Attach documents only when specific factual grounding is needed. +- **Chat is a preview tool:** Use it to discover edge cases. +- **Narrow tasks are easier to manage:** Single-purpose tools are easier to build, test, and evaluate. + +Next, you can replace the haiku prompt with a simulated client, a housing handbook, or a supervisor review checklist. The [building page](/docs/asklits/building) walks through each Scaffolder step. + +## Safe starting rules + +For law school courses, clinical simulations, and public legal information tools, always use fictional case files, public authorities, or de-identified training materials. An AskLIT is an educational and workflow tool, not a substitute for a licensed supervising attorney. + +See [Ethical guidelines and data privacy](./ethics) for details on API training policies, zero data retention (ZDR), HIPAA compliance, and data safety boundaries. diff --git a/docs/asklits/teaching-clinical-skills.md b/docs/asklits/teaching-clinical-skills.md new file mode 100644 index 000000000..7cf8525a1 --- /dev/null +++ b/docs/asklits/teaching-clinical-skills.md @@ -0,0 +1,85 @@ +--- +id: teaching-clinical-skills +title: AskLITs for law-school clinical teaching +sidebar_label: Clinical teaching ideas +description: Ideas for using AskLITs to teach interviewing, counseling, research, ethics, and other clinical skills. +--- + +# AskLITs for law-school clinical teaching + +An AskLIT is most useful in a clinic course when it gives students a place to +practice, make mistakes, and reflect. It should support a clear learning objective +without replacing instructor or supervisor feedback. + +An effective design has four parts: + +1. Learning objective +2. Curated fictional or approved source packet +3. System prompt establishing role boundaries and safety limits +4. Evaluation scenarios testing specific skills and failure modes + +## Ideas by clinical skill + +| Skill | AskLIT design | What to look for | +| --- | --- | --- | +| **Client interviewing** | A simulated client reveals facts only when asked. | Open-ended questions, chronology, clarification, listening, and missed safety facts. | +| **Trauma-informed communication** | A role-play client responds to tone and pacing; a debrief coach identifies assumptions. | Permission before sensitive questions, respectful language, and attention to client priorities. | +| **Intake and triage** | An intake coach organizes facts into issue, urgency, safety, eligibility, and referral categories. | Missing facts, urgent risks, and appropriate escalation to a supervisor. | +| **Issue spotting** | A fictional fact pattern is paired with a short statute or case packet. | Issues tied to facts, counterarguments, and unsupported assumptions. | +| **Legal research** | A research coach works only from a limited authority packet. | Accurate source use, distinction between holding and background, and honest treatment of gaps. | +| **Client counseling** | A counselor translates options into plain language without choosing for the client. | Options, risks, benefits, uncertainty, client goals, and a check for understanding. | +| **Case theory** | A coach sorts facts into known, disputed, missing, helpful, harmful, and evidence-needed. | Fact-to-element mapping and recognition of facts requiring verification. | +| **Drafting and review** | A drafting coach reviews a fictional letter, motion outline, or form instructions. | Completeness, audience-appropriate language, procedural accuracy, and ambiguities. | +| **Negotiation and mediation** | Separate profiles simulate a client, opposing party, or mediator. | Preparation, interests versus positions, ethical boundaries, and accurate summaries. | +| **Professional responsibility** | An ethics coach presents conflicts, confidentiality, supervision, scope, and unauthorized-practice scenarios. | Issue identification, rule-to-fact reasoning, and escalation to a supervisor. | +| **Supervisor feedback** | A feedback coach applies a rubric to a fictional work product. | Specific evidence, actionable feedback, balanced strengths and weaknesses, and appropriate confidence. | + +## Teaching patterns + +### Practice first, debrief second + +Create two prompt profiles that share a fictional packet: + +- The first plays the simulated client or other role +- The second provides coaching feedback when the student requests a debrief + +Keep evaluation rubrics out of the role-play prompt until the student finishes the exercise. Have the debrief coach reference the student's conversation turns to separate missed questions from legal reasoning errors. + +### Separate information retrieval from counseling judgment + +Use one profile to help students locate and explain authorities, and a separate profile to practice client counseling. This isolates whether difficulties stem from source retrieval, legal analysis, or communication style. + +### Make evaluation part of the assignment + +Have students submit: + +1. Learning objective and target audience +2. Inventory of approved source documents +3. System prompt including safety limits +4. Five to ten evaluation scenarios, including at least one question the packet cannot answer +5. Evaluation results CSV showing pass/fail performance +6. Short reflection on prompt iterations and observed failure modes + +This transforms building an AskLIT into an inspectable design and testing exercise. + +### Compare prompt designs systematically + +Keep the model fixed when testing prompts, and keep prompts fixed when comparing models. + +Useful prompt comparisons: + +- Concise direct assistant vs. Socratic coach +- General intake helper vs. trauma-informed intake coach +- Direct-answer format vs. structured checklist with referral guidance + +## Responsible use in clinic settings + +Before deploying an AskLIT with students, clients, or the public, verify that: + +- Source packets are current, approved, and free of confidential information (see [Ethical guidelines and data privacy](./ethics)) +- An instructor or supervising attorney oversees the learning objective and scenario rubrics +- Prompts state explicit limits and escalation rules +- Test scenarios evaluate missing information, uncertainty, and safety boundaries +- Retrieved source excerpts are inspected for actual relevance +- Students understand the distinction between practice simulations and real legal casework +- Live legal work is verified against primary authorities under attorney supervision diff --git a/docs/assets/asklit/01-knowledge.png b/docs/assets/asklit/01-knowledge.png new file mode 100644 index 000000000..2178da033 Binary files /dev/null and b/docs/assets/asklit/01-knowledge.png differ diff --git a/docs/assets/asklit/02-prompt.png b/docs/assets/asklit/02-prompt.png new file mode 100644 index 000000000..25dffcff7 Binary files /dev/null and b/docs/assets/asklit/02-prompt.png differ diff --git a/docs/assets/asklit/03-chat.png b/docs/assets/asklit/03-chat.png new file mode 100644 index 000000000..100e2ccf3 Binary files /dev/null and b/docs/assets/asklit/03-chat.png differ diff --git a/docs/assets/asklit/04-evaluate.png b/docs/assets/asklit/04-evaluate.png new file mode 100644 index 000000000..9d793a0cc Binary files /dev/null and b/docs/assets/asklit/04-evaluate.png differ diff --git a/docs/assets/asklit/05-export.png b/docs/assets/asklit/05-export.png new file mode 100644 index 000000000..e8f998499 Binary files /dev/null and b/docs/assets/asklit/05-export.png differ diff --git a/docs/assets/asklit/06-haiku-evaluation-setup.png b/docs/assets/asklit/06-haiku-evaluation-setup.png new file mode 100644 index 000000000..799622352 Binary files /dev/null and b/docs/assets/asklit/06-haiku-evaluation-setup.png differ diff --git a/docs/assets/asklit/07-haiku-evaluation-results.png b/docs/assets/asklit/07-haiku-evaluation-results.png new file mode 100644 index 000000000..77ddd6778 Binary files /dev/null and b/docs/assets/asklit/07-haiku-evaluation-results.png differ diff --git a/docs/assets/asklit/08-haiku-evaluation-results-fullscreen.png b/docs/assets/asklit/08-haiku-evaluation-results-fullscreen.png new file mode 100644 index 000000000..c83ebcb1c Binary files /dev/null and b/docs/assets/asklit/08-haiku-evaluation-results-fullscreen.png differ diff --git a/docs/assets/asklit/09-ma-eviction-no-kb-setup.png b/docs/assets/asklit/09-ma-eviction-no-kb-setup.png new file mode 100644 index 000000000..dd2843835 Binary files /dev/null and b/docs/assets/asklit/09-ma-eviction-no-kb-setup.png differ diff --git a/docs/assets/asklit/10-ma-eviction-no-kb-results.png b/docs/assets/asklit/10-ma-eviction-no-kb-results.png new file mode 100644 index 000000000..a60d14a05 Binary files /dev/null and b/docs/assets/asklit/10-ma-eviction-no-kb-results.png differ diff --git a/docs/assets/asklit/11-ma-eviction-kb-setup.png b/docs/assets/asklit/11-ma-eviction-kb-setup.png new file mode 100644 index 000000000..3c16ea814 Binary files /dev/null and b/docs/assets/asklit/11-ma-eviction-kb-setup.png differ diff --git a/docs/assets/asklit/12-ma-eviction-kb-results.png b/docs/assets/asklit/12-ma-eviction-kb-results.png new file mode 100644 index 000000000..2ca8015c9 Binary files /dev/null and b/docs/assets/asklit/12-ma-eviction-kb-results.png differ diff --git a/docs/assets/asklit/13-ma-eviction-kb-results-fullscreen.png b/docs/assets/asklit/13-ma-eviction-kb-results-fullscreen.png new file mode 100644 index 000000000..a06367e54 Binary files /dev/null and b/docs/assets/asklit/13-ma-eviction-kb-results-fullscreen.png differ diff --git a/docs/assets/asklit/14-gallery-live-haiku-start.png b/docs/assets/asklit/14-gallery-live-haiku-start.png new file mode 100644 index 000000000..0ff47cdc3 Binary files /dev/null and b/docs/assets/asklit/14-gallery-live-haiku-start.png differ diff --git a/docs/assets/asklit/15-gallery-live-haiku-response.png b/docs/assets/asklit/15-gallery-live-haiku-response.png new file mode 100644 index 000000000..6fde6d6b8 Binary files /dev/null and b/docs/assets/asklit/15-gallery-live-haiku-response.png differ diff --git a/docs/assets/asklit/16-gallery-live-lease-response.png b/docs/assets/asklit/16-gallery-live-lease-response.png new file mode 100644 index 000000000..da16378da Binary files /dev/null and b/docs/assets/asklit/16-gallery-live-lease-response.png differ diff --git a/docs/assets/asklit/17-gallery-live-interview-response.png b/docs/assets/asklit/17-gallery-live-interview-response.png new file mode 100644 index 000000000..fac8c3b8d Binary files /dev/null and b/docs/assets/asklit/17-gallery-live-interview-response.png differ diff --git a/docs/assets/asklit/ma-eviction-answer-deadline.md b/docs/assets/asklit/ma-eviction-answer-deadline.md new file mode 100644 index 000000000..d58f1a95f --- /dev/null +++ b/docs/assets/asklit/ma-eviction-answer-deadline.md @@ -0,0 +1,47 @@ +# Massachusetts summary process answer deadlines + +This teaching file was prepared on August 25, 2026 from current Massachusetts +Trial Court public guidance. It is an educational source packet, not legal +advice. Always check the current summons, court notice, court rules, and local +clerk's office for the case. + +## Core timing rule + +Massachusetts summary process cases use several dates: service, entry, answer, +and trial. The answer deadline is not automatically the trial date. The +Massachusetts Trial Court's current public guidance says that a tenant's written +answer must be filed and served on or before the first Monday after the Monday +entry date stated in the summons and complaint. The summons and complaint or a +court notice should identify the actual deadline for that case. The answer must +be received by the court clerk and by the plaintiff or plaintiff's attorney. + +Official sources: + +- File an eviction case: https://www.mass.gov/how-to/file-an-eviction-case +- Respond to an eviction against you: https://www.mass.gov/info-details/respond-to-an-eviction-against-you +- Uniform Summary Process Rule 3: Answer: https://www.mass.gov/trial-court-rules/uniform-summary-process-rule-3-answer +- Current sample Summary Process Summons and Complaint: https://www.mass.gov/doc/summary-process-summons-and-complaint-sample/download + +## Worked example + +If the Monday entry date is June 1, 2026, the first Monday after the entry +date is June 8, 2026. If the court notice gives a different deadline, follow +the deadline in the official court paperwork and contact the clerk's office +with questions. Do not wait for the trial date to file an answer. + +## If the tenant misses the answer deadline + +Uniform Summary Process Rule 10 says that if the defendant appears but has not +filed a timely answer, no default enters automatically. The court generally +postpones the trial one week from the original trial date unless the plaintiff +consents in writing to an immediate trial. This does not make missing the +deadline safe: the defendant should appear and ask the clerk or judge what to +do. + +## Limits + +Court department standing orders and later amendments may change procedures. +This packet does not answer every question about emergency rental assistance, +jury trials, discovery, extensions, service, or accommodations. The assistant +should identify missing facts and recommend checking the current court notice +or speaking with a qualified Massachusetts housing advocate or attorney. diff --git a/sidebars.js b/sidebars.js index 2fbf0a57b..e3c637591 100644 --- a/sidebars.js +++ b/sidebars.js @@ -142,6 +142,20 @@ module.exports = { }, ], }, + { + label: 'AskLITs', + type: 'category', + collapsed: false, + items: [ + 'asklits/asklits_overview', + 'asklits/building', + 'asklits/evaluating', + 'asklits/deploying', + 'asklits/gallery', + 'asklits/teaching-clinical-skills', + 'asklits/ethics', + ], + }, { label: 'Writing good questions', type: 'category',