Commit Graph

10 Commits

Author SHA1 Message Date
Nils-Johan Gynther d9f992ca9a feat(ai): enhance AI trace warnings and reason codes system
Test Suite / backend-pr-quick (push) Has been skipped
Test Suite / quick-import-pr-quick (push) Has been skipped
Test Suite / backend-full (push) Successful in 4m21s
Test Suite / flutter-quality (push) Failing after 1m38s
- Added structured warning system with `AdminAiWarning` type in backend and Flutter
- Implemented detailed reason descriptors with `FlyerReasonDescriptor` for parse and match operations
- Added `legacyWarnings` field to maintain backward compatibility
- Enhanced AI trace service to collect and format warnings with item-level context
- Updated flyer import services to include detailed reason descriptions in responses
- Added Swedish diacritic preservation for cheese variants (Prästost, Herrgårdsost, Grevéost)
- Implemented UTF-8 content validation for AI responses
- Added new reason code definitions in `reason-codes.ts`
- Updated Flutter UI to display structured warnings with severity indicators
- Added error report generation and copy functionality in admin panel
- Added comprehensive test coverage for new warning system and cheese normalization

BREAKING CHANGE: AI trace warnings are now structured objects instead of simple strings
2026-05-23 21:11:46 +02:00
Nils-Johan Gynther 026323b72a refactor(ai): enhance AI trace integration and OCR normalization
Test Suite / backend-pr-quick (push) Has been skipped
Test Suite / quick-import-pr-quick (push) Has been skipped
Test Suite / backend-full (push) Successful in 3m54s
Test Suite / flutter-quality (push) Failing after 1m29s
- Add FlyerTraceSupplement type for AI trace metadata
- Implement getFlyerTraceSupplements method to fetch trace supplements
- Update AiTraceService to include prompt/rawOutput and counters in flyer traces
- Add persistFlyerTrace method to FlyerImportService for trace persistence
- Enhance AiFlyerParserService to return structured trace data with prompts and retries
- Update FlyerNormalizerService with OCR typo fixes for cheese variants and spröd bakad firre
- Improve Flutter admin panel with selectable text, warnings display, and tooltips
- Add comprehensive tests for AI trace supplements and normalization rules
2026-05-21 19:11:54 +02:00
Nils-Johan Gynther 2d94a83e73 feat(flyer-import): add cheese variant splitting and normalization rules
Test Suite / backend-pr-quick (push) Has been skipped
Test Suite / quick-import-pr-quick (push) Has been skipped
Test Suite / backend-full (push) Successful in 5m48s
Test Suite / flutter-quality (push) Failing after 1m9s
- Add logic to split Swedish cheese variants (Präst, Herrgård, Grevé) into separate products
- Implement brand normalization for "Arla Ko" and category assignment to "Hårdost"
- Update flyer parser with detailed rules for bundle/group announcements
- Add unit tests for variant splitting and brand/category normalization
- Replace single-item return with flatMap for expanded product lists
2026-05-21 14:44:37 +02:00
Nils-Johan Gynther 67c3170067 feat(flyer-import): add bundle support and new product fields
Test Suite / backend-pr-quick (push) Has been skipped
Test Suite / quick-import-pr-quick (push) Has been skipped
Test Suite / backend-full (push) Successful in 3m43s
Test Suite / flutter-quality (push) Failing after 1m51s
- Add bundle support with isBundle, bundleWeight, and bundleItems fields
- Add brand, weight, and comparisonUnit fields to FlyerItem model
- Update AI flyer parser to extract bundle information
- Add sanitization for bundle items in FlyerNormalizerService
- Update DTOs and interfaces to include new fields
- Add migration for new database fields
- Update tests to cover bundle item handling
2026-05-21 13:26:50 +02:00
Nils-Johan Gynther 7bbb5a63b5 changed model to: ministral-8b-2512
Test Suite / backend-pr-quick (push) Has been skipped
Test Suite / quick-import-pr-quick (push) Has been skipped
Test Suite / backend-full (push) Successful in 2m47s
Test Suite / flutter-quality (push) Failing after 1m27s
2026-05-21 09:56:10 +02:00
Nils-Johan Gynther 505339aa33 Changed Mistral LLM to mistral-8b-2512. Changed timeout for LMM in .env
Test Suite / backend-pr-quick (push) Has been skipped
Test Suite / quick-import-pr-quick (push) Has been skipped
Test Suite / backend-full (push) Successful in 3m4s
Test Suite / flutter-quality (push) Failing after 1m24s
2026-05-21 09:45:39 +02:00
Nils-Johan Gynther 8b8f8b7b6f Update flyerimport. flutter timeout 300 sek
Test Suite / backend-pr-quick (push) Has been skipped
Test Suite / quick-import-pr-quick (push) Has been skipped
Test Suite / backend-full (push) Successful in 4m58s
Test Suite / flutter-quality (push) Failing after 1m41s
2026-05-19 20:53:39 +02:00
Nils-Johan Gynther 33190bd8e0 refactor(ai-parser): implement chunk-based text processing for flyer parsing
Test Suite / backend-pr-quick (push) Has been skipped
Test Suite / quick-import-pr-quick (push) Has been skipped
Test Suite / backend-full (push) Successful in 2m28s
Test Suite / flutter-quality (push) Failing after 1m14s
- Replace fixed text window retry strategy with dynamic chunk-based approach
- Add configurable chunk size, overlap, and maximum chunks via environment variables
- Implement chunk splitting with overlap handling for context preservation
- Add chunk processing with retry logic per chunk
- Include deduplication of items across chunks
- Update logging to reflect chunk-based processing

BREAKING CHANGE: Changes the retry strategy from fixed text windows to dynamic chunk-based processing. Existing configurations may need adjustment for FLYER_AI_CHUNK_SIZE_CHARS, FLYER_AI_CHUNK_OVERLAP_CHARS, and FLYER_AI_MAX_CHUNKS.
2026-05-19 20:24:03 +02:00
Nils-Johan Gynther 4d2942a8e5 chore(infra): add AI flyer parsing configuration and retry logic
Test Suite / backend-pr-quick (push) Has been skipped
Test Suite / quick-import-pr-quick (push) Has been skipped
Test Suite / backend-full (push) Successful in 2m9s
Test Suite / flutter-quality (push) Failing after 1m19s
- Add FLYER_AI_TIMEOUT_MS and FLYER_AI_RETRIES environment variables
- Configure timeout and retry settings in compose.yml
- Update AiFlyerParserService with configurable timeout and retry logic
- Add text window reduction strategy for retry attempts
- Update documentation in TEKNISK_BESKRIVNING.md
- Fix ESLint configuration in app.security.spec.ts
2026-05-19 20:13:59 +02:00
Nils-Johan Gynther 187d0283a5 feat(flyer-import): integrate AI-based flyer parsing with image support
Test Suite / quick-import-pr-quick (push) Has been skipped
Test Suite / backend-full (push) Successful in 2m31s
Test Suite / flutter-quality (push) Failing after 3m48s
Test Suite / backend-pr-quick (push) Failing after 13m57s
- Add support for PNG, JPEG, and WebP image formats in flyer import
- Replace external importer service with internal AI-based parsing pipeline
- Add new services: TextExtractorService, AiFlyerParserService, FlyerNormalizerService
- Integrate Mistral AI, pdf-parse, and tesseract.js dependencies
- Add quality confidence indicators and warning panels in Flutter UI
- Update package.json with new dependencies and transform ignore patterns
- Add documentation for flyer importer system
- Add Kilo AI planning file for Happy Island project

BREAKING CHANGE: Flyer import now uses internal AI parsing instead of external importer service
2026-05-19 19:57:54 +02:00