classroomio/prd/study-from-source/implementation-plan.md

Study from Source Implementation Plan

Engineering Position

Study from Source should be built as a durable source-ingestion and generation pipeline, not as a direct "send file to AI" feature.

The system owns:

1. source intake
2. storage
3. deterministic extraction
4. normalization
5. segmentation
6. AI generation from bounded source segments
7. draft persistence
8. teacher review
9. publish into existing course primitives
10. learner progress and completion evidence

The AI should not fetch arbitrary URLs, read private files directly, or decide what gets persisted. The AI receives normalized source segments and returns structured drafts that are validated before storage and again before publish.

Task Separation

Deterministic System Tasks

These tasks should be regular backend services and workers:

Task	Owner	Notes
Auth and permission checks	API routes/middleware	Organization-scoped, course-team-scoped where relevant
File upload and asset registration	API service	Reuse asset pipeline where possible
Webpage fetch	API worker	Server-side fetch with strict SSRF protections
Text extraction	Extraction service	PDF/DOCX/PPTX/TXT/Markdown/HTML
Text normalization	Extraction service	Remove boilerplate, normalize whitespace, preserve headings
Source segmentation	Segmentation service	Deterministic first, AI-assisted only for weak structure
Source snapshot persistence	DB/query layer	Source must remain stable after publish
Job state, retries, cancellation	Worker/job service	Durable and resumable
Draft validation	Utils validation + API service	Zod schema validation before persistence
Publishing	Publish service	Creates sections, lessons, lesson language content, exercises
Learner events	Tracking endpoint/service	Idempotent events and progress aggregation
Completion evaluation	Completion service	Central rule evaluator, not UI-driven
Reporting/export	Reporting service	Reads progress aggregates and submissions

AI Tasks

The AI should perform bounded transformations only after source extraction:

AI Task	Input	Output
Structure inference	Source segment outline and selected segment text	Proposed lessons, optional sections, learning objectives
Lesson drafting	Segment text for one planned lesson	Lesson body content, summary, key concepts
Quiz generation	Segment text and learning objectives	Auto-gradable questions, options, correct answers, explanations
Activity quiz generation	One source/activity segment set	Key subject matter, quiz, optional reflection prompt
Coverage mapping assist	Generated item + source segment IDs	Segment references and confidence
Regeneration	Existing draft item + source segments + teacher instruction	Replacement draft item

The AI must not:

• publish directly
• change permissions
• create records without service validation
• infer facts outside the provided source unless explicitly configured
• generate questions without source segment references
• fetch URLs itself
• grade learner submissions in v1 unless the question type already has a supported grading path

High-Level Architecture

flowchart TB
    Teacher["Teacher UI"]
    API["Study Source API"]
    Worker["Source Worker"]
    Extractor["Extraction Service"]
    Segmenter["Segmentation Service"]
    AI["AI Generation Service"]
    Draft["Generation Draft"]
    Publish["Publish Service"]
    Course["Course / Lesson / Exercise Tables"]
    Learner["Learner UI"]
    Tracking["Tracking Service"]
    Reports["Reports"]

    Teacher --> API
    API --> Worker
    Worker --> Extractor
    Extractor --> Segmenter
    Segmenter --> AI
    AI --> Draft
    Teacher --> Draft
    Draft --> Publish
    Publish --> Course
    Learner --> Tracking
    Tracking --> Reports
    Course --> Learner

Pipeline Overview

Course Type Pipeline

create source
  -> extract source
  -> normalize text
  -> segment source
  -> generate course outline draft
  -> teacher reviews outline
  -> generate lesson and quiz drafts per lesson
  -> teacher approves
  -> publish into course/sections/lessons/exercises
  -> learner completes course
  -> reports show progress, score, and source coverage

Activity Type Pipeline

create source inside existing course
  -> extract source
  -> normalize text
  -> segment source
  -> generate source-study activity draft
  -> teacher reviews quiz and requirements
  -> publish as lesson variant + generated exercise
  -> learner studies source and submits quiz
  -> completion service evaluates activity completion

Source Intake and Extraction

Intake Sources

File Upload

Supported v1:

• PDF
• DOCX
• PPTX
• TXT
• Markdown

Implementation:

1. API receives file metadata and creates a study_source record with status draft.
2. File is uploaded to object storage and registered as an asset.
3. API creates a study_source_job with type extract_source.
4. Worker claims job and extracts text.
5. Worker updates source status to ready or failed.

Use the current AI chat document extraction as a reference, but do not reuse its temporary conversation assumptions. The existing service stores extracted text against ai_chat_document; Study from Source needs durable source records and source segments.

Webpage URL

Supported v1:

• one public URL per source
• server-side fetch
• readable article/body extraction
• stored snapshot

Implementation:

1. Validate URL scheme is https or http.
2. Block localhost, private IP ranges, link-local ranges, metadata service IPs, and internal hostnames.
3. Resolve DNS before fetch and validate resolved IPs.
4. Enforce redirect limit, response size limit, and timeout.
5. Reject unsupported content types unless the response is parseable HTML/text.
6. Store:
   • original URL
   • final URL
   • HTTP status
   • content type
   • fetch timestamp
   • extracted readable text
   • page title
   • canonical URL if present

The learner should see the stored snapshot/readable text by default, with the original URL shown as a citation/link.

Pasted Text

Supported v1:

• plain text
• Markdown

Implementation:

1. API stores text directly in study_source.rawText.
2. Worker normalizes and segments it like other source types.
3. No external asset is required.

Extraction Libraries

Candidate extraction approach:

Source	Extractor
PDF	Existing pdf-parse initially; consider page-aware extractor if page locators are required
DOCX	Existing mammoth
PPTX	Existing ZIP/XML extraction, later replace with a stronger parser if needed
HTML	readability-style extraction using DOM parsing
TXT/Markdown	Direct decode and normalization

Extraction Output Contract

Extraction should produce a structured intermediate result:

type ExtractedSource = {
  title: string | null;
  rawText: string;
  normalizedText: string;
  wordCount: number;
  pageCount?: number | null;
  language?: string | null;
  locators: Array<{
    kind: 'page' | 'heading' | 'slide' | 'paragraph' | 'url_fragment';
    label: string;
    startOffset: number;
    endOffset: number;
    metadata?: Record<string, unknown>;
  }>;
  warnings: Array<{
    code: string;
    severity: 'info' | 'warning' | 'error';
    message: string;
  }>;
};

The AI should consume the normalized text and segment records, not the uploaded file.

Normalization

Normalization should be deterministic and conservative.

Steps:

1. decode text as UTF-8 where possible
2. normalize line endings
3. collapse repeated whitespace
4. preserve headings, page breaks, slide breaks, and URL headings
5. remove obvious webpage boilerplate such as nav/footer/cookie banners where the parser identifies them
6. detect very low text density and mark extraction as weak
7. compute word count and character count
8. detect likely language

Do not rewrite content during normalization. Rewriting belongs to AI generation, not extraction.

Segmentation

Segmentation is the contract between source extraction and AI generation.

Deterministic Segmentation First

Use deterministic rules before AI:

• PDF: pages, detected headings, paragraph blocks
• DOCX: heading styles if available, then paragraphs
• PPTX: slide boundaries, slide titles
• HTML: h1/h2/h3 sections, article paragraphs
• Markdown: heading hierarchy
• TXT: heading-like lines and paragraph grouping

AI-Assisted Segmentation

Use AI-assisted segmentation only when deterministic segmentation is weak:

• no headings
• long flat text
• inconsistent formatting
• extracted text has unclear boundaries

AI-assisted segmentation should receive chunked text and return only segment boundaries and titles, not rewritten lesson content.

Segment Size Rules

Default target:

• minimum useful segment: 150 words
• target segment: 500-1,200 words
• maximum segment: 2,000 words before further splitting

Very short segments should be merged with neighbors unless they are headings, definitions, or policy clauses that need to remain separate.

Segment Output

Each study_source_segment should include:

• stable order
• title where available
• kind
• text
• parent segment if hierarchical
• source locator
• extraction confidence or warnings

AI Generation Responsibilities

Course Outline Generation

Input:

• source metadata
• segment list with IDs, titles, sizes, and short previews
• teacher instructions
• generation settings

Output:

• title
• description
• usesSections
• optional sections
• lessons
• lesson-to-segment mapping
• learning objectives
• recommended quiz placement
• warnings

Rules:

• for small sources, generate a flat lesson list
• for large/naturally structured sources, generate sections
• do not create a lesson without source segment IDs
• do not include ungrounded content
• include warnings for uncovered or low-confidence segments

Lesson Draft Generation

Input:

• approved lesson plan item
• full text of mapped source segments
• course style settings
• teacher instructions

Output:

• lesson body HTML
• summary
• key concepts
• source segment references
• warnings

Rules:

• lesson body starts at h3 or lower to match existing lesson content rules
• keep source-grounded; do not add external facts by default
• explain and teach, but do not invent new policy requirements
• include source coverage metadata

Quiz Generation

Input:

• source segment text
• learning objectives
• desired question count
• allowed question types
• passing score

Output:

• questions
• options
• correct answers
• answer explanations
• source segment references
• difficulty

Rules:

• v1 should prefer auto-gradable question types
• every question must map to at least one source segment
• avoid trick questions
• avoid questions whose answer depends on ambiguous wording
• avoid questions about insignificant details unless the teacher asks for strict recall
• generated options must be mutually exclusive for single-answer questions
• checkbox questions must have at least two correct options only when the source supports that clearly

Activity Generation

Input:

• source segments for one activity
• teacher settings

Output:

• activity title
• source summary
• key subject matter list
• quiz
• completion settings
• source coverage

Rules:

• do not split activity into multiple lessons
• focus quiz on key subject matter, not every paragraph
• activity summary is optional learner support, not proof of study

Data Model Additions

Enums

Add:

• COURSE_TYPE: STUDY_FROM_SOURCE
• STUDY_SOURCE_TYPE: file, webpage, text
• STUDY_SOURCE_STATUS: draft, extracting, ready, failed, archived
• STUDY_SOURCE_JOB_TYPE: extract_source, segment_source, generate_course_outline, generate_activity, publish_draft
• STUDY_SOURCE_JOB_STATUS: pending, running, completed, failed, canceled
• STUDY_SOURCE_SEGMENT_KIND: chapter, section, paragraph, page, slide, inferred_topic
• STUDY_SOURCE_DRAFT_KIND: course_outline, activity_quiz, lesson_regeneration, quiz_regeneration
• STUDY_SOURCE_ACTIVITY_STATUS: draft, published, archived

Tables

Use the PRD data model plus add an explicit job table:

study_source_job

Field	Type	Notes
id	uuid	Job ID
organizationId	uuid	Scope
sourceId	uuid nullable	Source
draftId	uuid nullable	Draft
type	enum	Job type
status	enum	Job status
attempt	integer	Current attempt
maxAttempts	integer	Retry cap
leasedBy	text nullable	Worker lease
leaseExpiresAt	timestamp nullable	Worker lease expiry
progressPercent	integer	UI progress
stage	text	Human-readable stage
checkpoint	jsonb	Resume data
input	jsonb	Job input
output	jsonb	Job result
warnings	jsonb	Non-fatal warnings
error	jsonb	Terminal error
createdAt / updatedAt	timestamp	Audit fields

The job system should use DB-backed leasing with FOR UPDATE SKIP LOCKED, matching the pattern proposed in the course importer plan.

API Boundaries

Routes

Keep routes thin:

• validate input
• enforce auth/middleware
• call service
• return one response shape
• use handleError

Services

Suggested service modules:

• apps/api/src/services/study-source/source.ts
• apps/api/src/services/study-source/extraction.ts
• apps/api/src/services/study-source/segmentation.ts
• apps/api/src/services/study-source/generation.ts
• apps/api/src/services/study-source/publish.ts
• apps/api/src/services/study-source/progress.ts
• apps/api/src/services/study-source/reporting.ts

Queries

Suggested query module:

• packages/db/src/queries/study-source/study-source.ts

Queries stay pure and do not know HTTP concerns.

Validation

Suggested validation module:

• packages/utils/src/validation/study-source/study-source.ts

Schemas should cover:

• source create
• file finalize
• URL create
• pasted text create
• generation settings
• draft update
• publish
• activity config
• learner event
• report query

Async Job Design

Extraction and AI generation should be async jobs.

Do not hold a request open while:

• fetching a webpage
• parsing a large PDF
• segmenting large text
• generating a full course outline
• generating all lesson content
• publishing a large generated course

Job Lifecycle

pending -> running -> completed
                  -> failed
                  -> canceled

Retry Policy

• extraction failures caused by invalid input should not retry
• network timeouts for webpage fetch can retry with backoff
• AI rate limits can retry with backoff
• schema validation failures from AI output should retry once with a repair prompt
• publish failures should be resumable with idempotency keys

Idempotency

Use idempotency keys for:

• source creation from URL
• draft generation
• publish
• learner event ingestion

Publish must be safe to retry without duplicate sections, lessons, exercises, or questions.

Publish Strategy

Course Type Publish

Publishing a course draft should:

1. create or update course shell with type STUDY_FROM_SOURCE
2. set isContentGroupingEnabled based on approved outline
3. create sections if usesSections is true
4. create lessons in order
5. create lesson language content
6. create exercises and questions
7. create source coverage records
8. mark draft published

Activity Publish

Publishing an activity draft should:

1. create a lesson in the existing course
2. create study_source_activity
3. attach source metadata to the activity
4. create generated exercise linked to the lesson
5. create source coverage records for quiz questions
6. mark activity/draft published

Partial Publish Failure

Publish should store checkpoints after each created entity:

• course created
• section IDs created
• lesson IDs created
• lesson content written
• exercise IDs created
• questions created
• coverage written

On retry, the service should read the checkpoint and continue instead of creating duplicates.

Learner Tracking

Events

Events should be append-only where possible:

• opened
• segment_viewed
• progress_updated
• active_time_recorded
• quiz_started
• quiz_submitted
• attestation_checked
• completed

Event Validation

Reject or normalize:

• progress below previous progress unless explicitly resetting
• active time heartbeats that are too frequent
• active time while tab is hidden
• events for unpublished activities
• events from users not enrolled/authorized for the course

Progress Aggregation

Progress should be aggregated server-side from events and quiz submissions.

Completion should be evaluated by a single service:

type StudySourceCompletionInput = {
  requiredProgressPercent: number;
  minimumActiveTimeSeconds: number | null;
  passingScorePercent: number;
  requiresAttestation: boolean;
  progressPercent: number;
  activeTimeSeconds: number;
  quizScorePercent: number | null;
  attestedAt: string | null;
};

The UI should never mark an activity complete by itself.

Frontend Implementation

Course Creation

Add a Study from Source option to course creation.

Screens:

1. source input
2. extraction status and preview
3. generation settings
4. outline review
5. publish progress
6. generated course preview

Activity Creation

Add Study from Source to the content create modal.

Screens:

1. source input
2. extraction preview
3. quiz settings
4. generated quiz review
5. publish

Learner Reader

The reader should support:

• normalized text display
• source metadata
• original source link for webpages
• progress tracking
• generated quiz
• attestation if enabled

Do not render raw untrusted HTML from webpages without sanitization.

Edge Cases

File Edge Cases

Edge Case	Expected Behavior
Unsupported MIME type	Reject with clear error
MIME type spoofing	Verify extension and content signature where practical
File exceeds limit	Reject before extraction
Empty file	Mark extraction failed
PDF has no text layer	Mark extraction failed with scanned-PDF warning
Password-protected PDF	Reject with unsupported/protected file error
Corrupt DOCX/PPTX	Mark extraction failed and keep source in failed state
Huge extracted text	Truncate or require teacher to split source; show warning
Tables/formulas extracted poorly	Store warning; teacher can review extracted preview
Images contain important text	Mark as not extracted in v1; OCR out of scope
Duplicate upload	Allow, but optionally detect hash and warn

Webpage Edge Cases

Edge Case	Expected Behavior
URL is localhost/private IP	Reject to prevent SSRF
URL redirects to private IP	Reject after redirect validation
URL returns non-HTML binary	Reject unless supported document type
URL blocks fetch	Mark failed with fetch error
URL times out	Retry with backoff, then fail
Page is mostly nav/ads	Extract best readable content; warn if text density is low
Page requires login	Fail with "content not accessible"
Paywall or cookie wall	Extract available text; warn that content may be incomplete
Page content changes later	Existing course/activity keeps stored snapshot
Canonical URL differs	Store both original and canonical/final URL
Robots/meta restrictions	Store warning and respect product policy decided before launch

Text Edge Cases

Edge Case	Expected Behavior
Text is too short for quiz	Allow source, block quiz generation or generate fewer questions
Text is too long	Enforce character limit and show warning
Text has no structure	Use AI-assisted segmentation
Text is not in supported locale	Detect language and warn if generation quality may be lower

AI Edge Cases

Edge Case	Expected Behavior
AI returns invalid JSON	Retry once with repair prompt; then fail draft generation
AI omits source references	Reject output and retry
AI creates unsupported question type	Reject output and retry or downgrade to supported type
AI invents facts outside source	Hard to detect automatically; require teacher review and source references
AI makes every lesson too long	Enforce max output lengths per lesson
AI creates too many lessons	Enforce generation settings and max lesson count
AI creates trivial quiz questions	Flag via quality checks where possible; teacher can regenerate
AI rate limit	Retry job with backoff
AI token limit	Chunk by segments and generate per lesson/activity

Publish Edge Cases

Edge Case	Expected Behavior
Teacher edits draft while publish runs	Lock draft during publish
Publish interrupted after sections created	Resume from checkpoint
Existing course has content	Activity publish appends; course-type publish creates new course unless explicit merge is supported
Duplicate publish click	Idempotency prevents duplicates
Question validation fails	Fail publish before partial question creation where possible
Lesson content validation fails	Keep draft unpublished with actionable error

Learner Tracking Edge Cases

Edge Case	Expected Behavior
Learner opens source then immediately submits quiz	Completion depends on configured progress/score rules
Learner switches tabs	Active time heartbeat pauses
Learner loses network	Events queue client-side and replay with idempotency keys
Learner retakes quiz	Progress uses best or latest score based on activity settings
Teacher changes passing score after attempts	Recompute completion according to product policy; default should apply new rules only to future attempts
Source/activity unpublished	Reject learner events

Testing Plan

Unit Tests

• URL validation and SSRF blocking
• file type validation
• text normalization
• deterministic segmentation
• source completion evaluator
• generated draft schema validation
• publish mapping functions

Integration Tests

• create source from pasted text
• upload file and create source record
• webpage fetch success/failure
• extraction job state transitions
• generation draft persistence
• publish draft to course
• publish activity into existing course
• learner event ingestion and progress aggregation

E2E Tests

• teacher creates Study from Source course from pasted text and publishes
• teacher adds source-study activity to existing course
• learner studies activity, passes quiz, completion appears in report
• failed extraction shows actionable UI state
• retry publish does not duplicate content

Delivery Order

1. Data model and validation schemas
2. Source creation and pasted text ingestion
3. Async job foundation
4. File extraction using existing extraction libraries
5. Webpage fetch and readable extraction with SSRF protections
6. Segmentation service
7. Course outline draft generation
8. Activity quiz draft generation
9. Teacher review UI
10. Publish service for activity path
11. Publish service for course type path
12. Learner reader and tracking
13. Reports and CSV export

Build the activity path before the full course path if delivery needs to be staged. It exercises the same source pipeline with lower publish complexity.