feat(flyer-import): add detailed product signals and display names
Test Suite / backend-pr-quick (push) Has been skipped
Test Suite / quick-import-pr-quick (push) Has been skipped
Test Suite / backend-full (push) Successful in 5m12s
Test Suite / flutter-quality (push) Failing after 2m8s

- Added `signals` and `displayNameDetailed` fields to FlyerItem model in Prisma schema
- Introduced `FlyerImportSignals` type with origin countries, labels, quality flags, variant, and packaging
- Added `displayNameDetailed` field to FlyerImportItem DTO and Flutter model
- Implemented utility functions for signal extraction and display name building
- Updated flyer import service to persist and return signals/category data
- Enhanced Flutter UI to display detailed product information including badges for signals
- Added new test coverage for signals persistence and display name generation
- Added new import-common module for shared import utilities
- Created database migration for new fields
- Added Kilo plan for feature development
This commit is contained in:
Nils-Johan Gynther
2026-05-24 19:32:13 +02:00
parent d9f992ca9a
commit b04d157915
16 changed files with 1124 additions and 107 deletions
+198
View File
@@ -0,0 +1,198 @@
# Plan: Harmonisera flyer-import och kvitto-import
## Mål
Implementera en gemensam importmodell och matchningspipeline så att flyer-import och kvitto-import beter sig så likt som möjligt, med fokus på:
- Automatisk strukturering av namn/brand/vikt samt bundle-detaljer
- Automatiskt kategoriupplösning (`categoryHint -> categoryId`)
- Matchning mot befintliga produkter via normaliserade namn + signaler
- Ingen automatisk skapning av produkter
- Förberedelse för framtida automation via strukturerade signaler (`signals` JSON)
## Icke-mål (denna implementation)
- Ingen auto-create av produkter i produktkatalog
- Ingen ändring av övergripande UI-flöde (manuell import/validering kvar)
- Ingen full omskrivning av receipt-import; vi extraherar och återanvänder delar stegvis
## Nuvarande gap (från kodbasen)
1. `FlyerItem.categoryId` sätts till `null` i parse-flödet trots `categoryHint`.
2. Flyer-matchning använder enklare strategi än receipt-import (färre regler/signalvikter).
3. Ingen strukturerad lagring av ursprung/etiketter (t.ex. Sverige, Eko) i flyer.
4. Bundleinformation finns men exponeras inte tydligt som detaljnamn i payload.
5. Receipt och flyer använder olika “kontrakt” för mellanrepresentation.
## Övergripande design
Inför en gemensam intern domänmodell för importerade rader (backend), och låt både flyer- och kvittoflöde mappa till den innan kategori/matchning.
### Gemensam intern modell (ny)
`ImportedItemCandidate` (internt, ej API-brytande initialt):
- `rawName`, `normalizedName`, `brand`
- `weight`, `bundleWeight`, `isBundle`, `bundleItems`
- `price`, `priceUnit`, `comparisonPrice`, `comparisonUnit`
- `categoryHint`, `categoryId`
- `matchedProductId`, `matchedProductName`, `matchedVia`, `matchConfidence`, `matchReasons`
- `signals` (JSON):
- `originCountries: string[]`
- `labels: string[]` (ekologisk, laktosfri, etc)
- `qualityFlags: string[]` (normaliserade flaggor, ex `eco`)
- `variant: string | null`
- `packaging: string | null`
- `displayNameDetailed` (beräknat fält, kan persistas eller beräknas vid response)
## Faser och implementation
## Fas 1: Datamodell och migration
1. Uppdatera `backend/prisma/schema.prisma`:
- Lägg till `signals Json?``FlyerItem`
- Lägg till `displayNameDetailed String?``FlyerItem`
2. Skapa Prisma-migration.
3. Säkerställ bakåtkompatibilitet:
- Nullabla fält
- Ingen ändring av befintliga constraints/index som bryter drift
4. (Valfritt i samma fas) indexera vanligt använda JSON-signaler senare först efter verifierad nytta.
### Acceptanskriterier fas 1
- Migration appliceras lokalt utan dataförlust.
- Befintliga endpoints fungerar med gamla rader (`signals = null`).
## Fas 2: Gemensamma normaliserings-/signalverktyg
1. Skapa gemensam utility-modul i backend, exempel:
- `backend/src/import-common/import-item.types.ts`
- `backend/src/import-common/import-signals.util.ts`
- `backend/src/import-common/import-display-name.util.ts`
2. Implementera signal-extraktion från textfält (`rawName`, `brand`, `offerText`):
- Ursprungsländer till `originCountries`
- Etiketter/märkningar till `labels`/`qualityFlags`
- Pack-format till `packaging`
3. Normalisera utan att förlora information:
- Ta bort signalord från primär matchsträng men spara i `signals`
- Ex: `Fläskytterfilé (Sverige)` -> matchsträng `flaskytterfile`, `signals.originCountries=["Sverige"]`
4. Implementera `displayNameDetailed`:
- Bundle: inkludera `bundleItems` i visningsnamn
- Ex: `Kaptenens Favoriter (Chumlax 3x100g + Alaska pollock 3x100g)`
### Acceptanskriterier fas 2
- Signals extraheras deterministiskt för kända mönster (Sverige/Tyskland/Eko/Ekologiskt).
- `displayNameDetailed` genereras för bundles.
## Fas 3: Kategoriupplösning i flyer (paritet med kvitto)
1. Extrahera/återanvänd kategori-regelmotorn från receipt-import till gemensam tjänst:
- Ex: `backend/src/import-common/category-resolver.service.ts`
2. Använd den i flyer-import efter normalisering:
- `categoryHint` + signaltext + regler -> `categoryId`
3. Prioritet:
- Produktmatchad kategori (om säkert matchad produkt har kategori) kan väga högst
- Annars regelbaserad kategori
- Annars behåll `categoryHint` utan `categoryId`
4. Specifika regler för kött/fläskytterfilé verifieras.
### Acceptanskriterier fas 3
- `Fläskytterfilé` får korrekt `categoryId` i flyer-session.
- `categoryId` sätts automatiskt för en betydande andel rader med tydlig signal.
## Fas 4: Matchningsparitet flyer <-> kvitto
1. Bryt ut matchning till gemensam matcher (eller harmonisera algoritm):
- alias exact
- canonical/normalized exact
- token/fuzzy
- bonus för brand/weight/signalträffar
2. Matchning ska använda signalrensad namnsträng + metadata:
- Länder och eco-etiketter ska inte sabotera namnmatch
3. Standardisera reason codes mellan flöden (så långt möjligt utan brytande API):
- `alias_exact`, `normalized_exact`, `token_overlap:*`, `no_match`
4. Behåll strikt policy: ingen auto-create produkt.
### Acceptanskriterier fas 4
- Färre `no_match` på samma flyer-input jämfört med baseline.
- Matchningsorsaker blir mer förklarbara och konsekventa.
## Fas 5: API/DTO och persistens
1. Uppdatera flyer DTO:
- `backend/src/flyer-import/dto/flyer-import.response.ts`
- Lägg till `signals` och `displayNameDetailed`.
2. Uppdatera persistens i `flyer-import.service.ts`:
- Spara `signals`, `displayNameDetailed`, `categoryId`.
3. Säkerställ att `getSession`, `getLatestSession`, `updateSessionItem` returnerar nya fält.
4. Behåll kompatibilitet mot klient:
- Nya fält adderas utan att ta bort befintliga.
### Acceptanskriterier fas 5
- Response innehåller tydlig bundle-info och signaler per rad.
- Inga regressions i existerande frontend-parsing.
## Fas 6: Frontend (flyer import-tab)
1. Uppdatera domänmodeller i Flutter:
- `flutter/lib/features/import/domain/flyer_import_item.dart`
- ev. session/result-objekt
2. Visa `displayNameDetailed` där tillgängligt, annars fallback `rawName`.
3. Visa `bundleItems` tydligt i list-/detaljrad.
4. Visa badge/metadata för signaler (`Sverige`, `Ekologisk`) utan att skriva över produktnamn.
5. Säkerställ att manuellt urval till inköpslista fortsätter fungera.
### Acceptanskriterier fas 6
- Bundle-rader är tydligare i UI.
- Ursprung/eko syns som metadata.
## Fas 7: Teststrategi
### Backend enhetstester
- `flyer-normalizer.service.spec.ts`
- extraktion av `signals` (origin/labels)
- bundle-detaljnamn
- Ny kategori-resolver-spec
- `Fläskytterfilé` -> köttkategori
- `flyer-import.service.spec.ts`
- `categoryId` sätts vid tydlig signal
- `signals` och `displayNameDetailed` persisteras/returneras
- Matchningstester
- namn med land/eko matchar korrekt produkt
### Integrationstester
- End-to-end parseAndMatch med representativ flyer-fixture.
- Verifiera att inga produkter auto-skaps.
- Verifiera att shopping-list insertion fungerar med/utan `matchedProductId`.
### Frontendtester
- Serialisering av nya fält i import-session.
- Rendering av `displayNameDetailed` + `bundleItems`.
## Fas 8: Mätning och rollout
1. Lägg till enkel före/efter-mätning i logg/trace:
- andel `no_match`
- andel med satt `categoryId`
2. Soft rollout via feature flag (om möjligt), annars stegvis release.
3. Utvärdera verkliga flyer-sessioner innan vidare automatisering.
## Konkreta filer att ändra (planerad)
- `backend/prisma/schema.prisma`
- `backend/src/flyer-import/flyer-import.service.ts`
- `backend/src/flyer-import/services/flyer-normalizer.service.ts`
- `backend/src/flyer-import/dto/flyer-import.response.ts`
- `backend/src/receipt-import/receipt-import.service.ts` (endast för extraktion/återanvändning av gemensamma delar)
- Nya gemensamma filer under `backend/src/import-common/*`
- `flutter/lib/features/import/domain/flyer_import_item.dart`
- `flutter/lib/features/import/data/flyer_import_session.dart`
- `flutter/lib/features/import/presentation/flyer_import_tab.dart`
- Relevanta spec/test-filer i backend + flutter
## Risker och mitigering
- Risk: API-kontraktsändringar bryter klient.
- Mitigering: endast additive fält, fallback på gamla fält.
- Risk: Felkategori vid aggressiva regler.
- Mitigering: regelprioritet + reason-codes + tester för edge cases.
- Risk: Övermatchning av produkter.
- Mitigering: tröskelvärden + konservativ confidence för fuzzy.
## Leveransordning (rekommenderad)
1. Fas 12 (schema + signals + utilities)
2. Fas 3 (kategoriupplösning flyer)
3. Fas 4 (matchningsparitet)
4. Fas 5 (DTO/persistens)
5. Fas 6 (frontend)
6. Fas 78 (tester + mätning/rollout)
## Definition of Done
- Flyer och kvitto använder samma centrala regler för kategorisering/matchning där möjligt.
- Flyer-rader innehåller `signals` och tydligare produktrepresentation (`displayNameDetailed`, bundle-innehåll).
- `categoryId` sätts automatiskt i flyer när tillräcklig signal finns (inkl. fläskytterfilé-fall).
- Ingen automatisk produktskapning sker.
- Tester uppdaterade och gröna.
@@ -0,0 +1,4 @@
-- AlterTable
ALTER TABLE `FlyerItem`
ADD COLUMN `signals` JSON NULL,
ADD COLUMN `displayNameDetailed` VARCHAR(191) NULL;
+2
View File
@@ -325,6 +325,8 @@ model FlyerItem {
bundleWeight String? bundleWeight String?
isBundle Boolean @default(false) isBundle Boolean @default(false)
bundleItems Json? bundleItems Json?
signals Json?
displayNameDetailed String?
offerText String? offerText String?
parseConfidence Float parseConfidence Float
parseReasons Json? parseReasons Json?
@@ -1,5 +1,13 @@
export type FlyerImportMatchVia = 'alias' | 'exact' | 'token' | 'none'; export type FlyerImportMatchVia = 'alias' | 'exact' | 'token' | 'none';
export type FlyerImportSignals = {
originCountries: string[];
labels: string[];
qualityFlags: string[];
variant: string | null;
packaging: string | null;
};
export type FlyerReasonDescriptor = { export type FlyerReasonDescriptor = {
code: string; code: string;
kind: 'parse' | 'match'; kind: 'parse' | 'match';
@@ -24,6 +32,8 @@ export type FlyerImportItem = {
bundleWeight: string | null; bundleWeight: string | null;
isBundle: boolean; isBundle: boolean;
bundleItems: string[]; bundleItems: string[];
displayNameDetailed: string | null;
signals: FlyerImportSignals | null;
offerText: string | null; offerText: string | null;
isOffer: boolean; isOffer: boolean;
offerLimitText: string | null; offerLimitText: string | null;
@@ -5,15 +5,18 @@ import { FlyerImportService } from './flyer-import.service';
import { TextExtractorService } from './services/text-extractor.service'; import { TextExtractorService } from './services/text-extractor.service';
import { AiFlyerParserService } from './services/ai-flyer-parser.service'; import { AiFlyerParserService } from './services/ai-flyer-parser.service';
import { FlyerNormalizerService } from './services/flyer-normalizer.service'; import { FlyerNormalizerService } from './services/flyer-normalizer.service';
import { CategoriesModule } from '../categories/categories.module';
import { CategoryResolverService } from '../import-common/category-resolver.service';
@Module({ @Module({
imports: [PrismaModule], imports: [PrismaModule, CategoriesModule],
controllers: [FlyerImportController], controllers: [FlyerImportController],
providers: [ providers: [
FlyerImportService, FlyerImportService,
TextExtractorService, TextExtractorService,
AiFlyerParserService, AiFlyerParserService,
FlyerNormalizerService, FlyerNormalizerService,
CategoryResolverService,
], ],
}) })
export class FlyerImportModule {} export class FlyerImportModule {}
@@ -3,25 +3,44 @@ import { FlyerImportService } from './flyer-import.service';
describe('FlyerImportService', () => { describe('FlyerImportService', () => {
const prismaMock = { const prismaMock = {
product: {
findMany: jest.fn(),
},
receiptAlias: {
findMany: jest.fn(),
},
flyerSession: { flyerSession: {
findFirst: jest.fn(), findFirst: jest.fn(),
findUnique: jest.fn(), findUnique: jest.fn(),
create: jest.fn(),
}, },
flyerItem: { flyerItem: {
findUnique: jest.fn(), findUnique: jest.fn(),
update: jest.fn(), update: jest.fn(),
create: jest.fn(),
},
aiTrace: {
create: jest.fn(),
}, },
category: { category: {
findUnique: jest.fn(), findUnique: jest.fn(),
}, },
}; };
const createService = () => const createService = (overrides?: {
categoriesService?: any;
categoryResolver?: any;
textExtractor?: any;
aiParser?: any;
normalizer?: any;
}) =>
new FlyerImportService( new FlyerImportService(
prismaMock as any, prismaMock as any,
{} as any, overrides?.categoriesService ?? { findFlattened: jest.fn().mockResolvedValue([]) },
{} as any, overrides?.categoryResolver ?? { resolveForFlyer: jest.fn().mockReturnValue(null) },
{} as any, overrides?.textExtractor ?? {},
overrides?.aiParser ?? {},
overrides?.normalizer ?? {},
); );
beforeEach(() => { beforeEach(() => {
@@ -78,6 +97,8 @@ describe('FlyerImportService', () => {
bundleWeight: null, bundleWeight: null,
isBundle: false, isBundle: false,
bundleItems: [], bundleItems: [],
displayNameDetailed: 'Tomat',
signals: { originCountries: ['Sverige'], labels: [], qualityFlags: [], variant: null, packaging: null },
offerText: 'Max 2 kop/hushall', offerText: 'Max 2 kop/hushall',
parseConfidence: 0.9, parseConfidence: 0.9,
parseReasons: ['ai_parsed'], parseReasons: ['ai_parsed'],
@@ -97,6 +118,8 @@ describe('FlyerImportService', () => {
expect(result.items).toHaveLength(1); expect(result.items).toHaveLength(1);
expect(result.items[0].flyerItemId).toBe(99); expect(result.items[0].flyerItemId).toBe(99);
expect(result.items[0].matchedVia).toBe('exact'); expect(result.items[0].matchedVia).toBe('exact');
expect(result.items[0].displayNameDetailed).toBe('Tomat');
expect(result.items[0].signals?.originCountries).toEqual(['Sverige']);
expect(result.items[0].parseReasonsDetailed[0].title).toBe('AI-tolkad rad'); expect(result.items[0].parseReasonsDetailed[0].title).toBe('AI-tolkad rad');
expect(result.items[0].matchReasonsDetailed[0].title).toBe('Exakt normaliserad matchning'); expect(result.items[0].matchReasonsDetailed[0].title).toBe('Exakt normaliserad matchning');
expect(result.sourceAvailable).toBe(false); expect(result.sourceAvailable).toBe(false);
@@ -140,6 +163,162 @@ describe('FlyerImportService', () => {
}); });
}); });
describe('parseAndMatch', () => {
it('persists and returns signals/displayNameDetailed/categoryId in parse pipeline', async () => {
prismaMock.product.findMany.mockResolvedValue([
{ id: 11, name: 'Fläskytterfilé', canonicalName: 'Fläskytterfilé', categoryId: 7 },
]);
prismaMock.receiptAlias.findMany.mockResolvedValue([]);
prismaMock.flyerSession.create.mockResolvedValue({ id: 200 });
prismaMock.flyerItem.create
.mockResolvedValueOnce({ id: 1001 });
prismaMock.aiTrace.create.mockResolvedValue({ id: 1 });
const categoriesService = { findFlattened: jest.fn().mockResolvedValue([]) };
const categoryResolver = { resolveForFlyer: jest.fn().mockReturnValue(7) };
const textExtractor = { extractText: jest.fn().mockResolvedValue('raw flyer text') };
const aiParser = {
parseWithAI: jest.fn().mockResolvedValue({
items: [
{
rawName: 'Fläskytterfilé (Sverige) EKO',
normalizedName: 'flaskytterfile sverige eko',
brand: 'Garant',
category: 'Kött',
price: 99.9,
unit: 'kg',
comparisonPrice: null,
comparisonUnit: null,
weight: '900g',
bundleWeight: null,
isBundle: true,
bundleItems: ['Del 1', 'Del 2'],
offer: 'ekologiskt från Sverige',
confidence: 0.93,
reasonCodes: ['ai_parsed'],
},
],
trace: { prompt: null, rawOutput: null, chunkCount: 1, retryCount: 0 },
}),
};
const normalizer = {
normalize: jest.fn().mockReturnValue([
{
rawName: 'Fläskytterfilé (Sverige) EKO',
normalizedName: 'flaskytterfile sverige eko',
brand: 'Garant',
categoryHint: 'Kött',
price: 99.9,
priceUnit: 'kg',
comparisonPrice: null,
comparisonUnit: null,
weight: '900g',
bundleWeight: null,
isBundle: true,
bundleItems: ['Del 1', 'Del 2'],
offerText: 'ekologiskt från Sverige',
parseConfidence: 0.93,
parseReasons: ['ai_parsed'],
},
]),
};
const service = createService({
categoriesService,
categoryResolver,
textExtractor,
aiParser,
normalizer,
});
const result = await service.parseAndMatch(
{
originalname: 'flyer.pdf',
mimetype: 'application/pdf',
size: 10,
buffer: Buffer.from('pdf'),
} as any,
1,
);
expect(result.items).toHaveLength(1);
expect(result.items[0].displayNameDetailed).toBe('Fläskytterfilé (Sverige) EKO (Del 1 + Del 2)');
expect(result.items[0].signals?.originCountries).toEqual(['Sverige']);
expect(result.items[0].signals?.qualityFlags).toContain('eco');
expect(result.items[0].categoryId).toBe(7);
expect(result.items[0].normalizedName).toBe('flaskytterfile');
expect(prismaMock.flyerItem.create).toHaveBeenCalledWith(
expect.objectContaining({
data: expect.objectContaining({
displayNameDetailed: 'Fläskytterfilé (Sverige) EKO (Del 1 + Del 2)',
categoryId: 7,
signals: expect.objectContaining({ originCountries: ['Sverige'] }),
}),
}),
);
});
it('logs warning when categories fallback is used', async () => {
prismaMock.product.findMany.mockResolvedValue([]);
prismaMock.receiptAlias.findMany.mockResolvedValue([]);
prismaMock.flyerSession.create.mockResolvedValue({ id: 201 });
prismaMock.flyerItem.create.mockResolvedValue({ id: 1002 });
prismaMock.aiTrace.create.mockResolvedValue({ id: 2 });
const categoriesService = { findFlattened: jest.fn().mockRejectedValue(new Error('db down')) };
const textExtractor = { extractText: jest.fn().mockResolvedValue('raw') };
const aiParser = {
parseWithAI: jest.fn().mockResolvedValue({
items: [{ rawName: 'Tomat' }],
trace: { prompt: null, rawOutput: null, chunkCount: 1, retryCount: 0 },
}),
};
const normalizer = {
normalize: jest.fn().mockReturnValue([
{
rawName: 'Tomat',
normalizedName: 'tomat',
brand: null,
categoryHint: null,
price: null,
priceUnit: null,
comparisonPrice: null,
comparisonUnit: null,
weight: null,
bundleWeight: null,
isBundle: false,
bundleItems: [],
offerText: null,
parseConfidence: 0.9,
parseReasons: ['ai_parsed'],
},
]),
};
const service = createService({
categoriesService,
textExtractor,
aiParser,
normalizer,
});
const warnSpy = jest.spyOn((service as any).logger, 'warn');
await service.parseAndMatch(
{
originalname: 'flyer.pdf',
mimetype: 'application/pdf',
size: 10,
buffer: Buffer.from('pdf'),
} as any,
1,
);
expect(warnSpy).toHaveBeenCalledWith(
expect.stringContaining('Could not load categories for flyer import'),
);
});
});
describe('getLatestSession', () => { describe('getLatestSession', () => {
it('returns empty response when no sessions exist', async () => { it('returns empty response when no sessions exist', async () => {
prismaMock.flyerSession.findFirst.mockResolvedValue(null); prismaMock.flyerSession.findFirst.mockResolvedValue(null);
+206 -83
View File
@@ -6,9 +6,10 @@ import {
NotFoundException, NotFoundException,
ServiceUnavailableException, ServiceUnavailableException,
} from '@nestjs/common'; } from '@nestjs/common';
import { Prisma } from '@prisma/client'; import { Prisma } from '@prisma/client';
import { PrismaService } from '../prisma/prisma.service'; import { PrismaService } from '../prisma/prisma.service';
import { normalizeName } from '../common/utils/normalize-name'; import { normalizeName } from '../common/utils/normalize-name';
import { CategoriesService } from '../categories/categories.service';
import { import {
FlyerImportItem, FlyerImportItem,
FlyerImportMatchVia, FlyerImportMatchVia,
@@ -18,6 +19,10 @@ import { TextExtractorService } from './services/text-extractor.service';
import { AiFlyerParserService } from './services/ai-flyer-parser.service'; import { AiFlyerParserService } from './services/ai-flyer-parser.service';
import { FlyerNormalizerService } from './services/flyer-normalizer.service'; import { FlyerNormalizerService } from './services/flyer-normalizer.service';
import { describeMatchReason, describeParseReason } from './services/reason-codes'; import { describeMatchReason, describeParseReason } from './services/reason-codes';
import { CategoryResolverService } from '../import-common/category-resolver.service';
import { buildDisplayNameDetailed } from '../import-common/import-display-name.util';
import { extractImportSignals } from '../import-common/import-signals.util';
import { ImportedItemSignals } from '../import-common/import-item.types';
type FlyerParseItem = { type FlyerParseItem = {
rawName: string; rawName: string;
@@ -58,11 +63,12 @@ type ExtractedOfferSignals = {
hasCampaignPattern: boolean; hasCampaignPattern: boolean;
}; };
type ProductLite = { type ProductLite = {
id: number; id: number;
name: string; name: string;
canonicalName: string | null; canonicalName: string | null;
}; categoryId: number | null;
};
@Injectable() @Injectable()
export class FlyerImportService { export class FlyerImportService {
@@ -70,29 +76,37 @@ export class FlyerImportService {
private readonly MAX_BUNDLE_ITEMS = 20; private readonly MAX_BUNDLE_ITEMS = 20;
private readonly MAX_BUNDLE_ITEM_LENGTH = 120; private readonly MAX_BUNDLE_ITEM_LENGTH = 120;
constructor( constructor(
private readonly prisma: PrismaService, private readonly prisma: PrismaService,
private readonly textExtractor: TextExtractorService, private readonly categoriesService: CategoriesService,
private readonly aiParser: AiFlyerParserService, private readonly categoryResolver: CategoryResolverService,
private readonly normalizer: FlyerNormalizerService, private readonly textExtractor: TextExtractorService,
) {} private readonly aiParser: AiFlyerParserService,
private readonly normalizer: FlyerNormalizerService,
) {}
async parseAndMatch(file: Express.Multer.File, userId: number): Promise<FlyerImportResponse> { async parseAndMatch(file: Express.Multer.File, userId: number): Promise<FlyerImportResponse> {
const startedAt = Date.now(); const startedAt = Date.now();
const parsed = await this.parseViaInternal(file); const parsed = await this.parseViaInternal(file);
const [products, aliases] = await Promise.all([ const [products, aliases, categories] = await Promise.all([
this.prisma.product.findMany({ this.prisma.product.findMany({
where: { ownerId: userId, isActive: true }, where: { ownerId: userId, isActive: true },
select: { id: true, name: true, canonicalName: true }, select: { id: true, name: true, canonicalName: true, categoryId: true },
}), }),
this.prisma.receiptAlias.findMany({ this.prisma.receiptAlias.findMany({
where: { where: {
OR: [{ ownerId: userId, isGlobal: false }, { isGlobal: true }], OR: [{ ownerId: userId, isGlobal: false }, { isGlobal: true }],
}, },
select: { receiptName: true, productId: true }, select: { receiptName: true, productId: true },
}), }),
]); this.categoriesService.findFlattened().catch((error) => {
this.logger.warn(
`Could not load categories for flyer import, proceeding without rule categories: ${error instanceof Error ? error.message : String(error)}`,
);
return [];
}),
]);
const aliasToProduct = new Map<string, number>(); const aliasToProduct = new Map<string, number>();
for (const alias of aliases) { for (const alias of aliases) {
@@ -109,20 +123,39 @@ export class FlyerImportService {
} }
const items: FlyerImportItem[] = parsed.items.map((item) => { const items: FlyerImportItem[] = parsed.items.map((item) => {
const match = this.matchItem(item, products, aliasToProduct, productById); const signalData = extractImportSignals({
rawName: item.rawName,
brand: item.brand,
offerText: item.offerText,
});
const match = this.matchItem(item, signalData.normalizedMatchName, signalData.signals, products, aliasToProduct, productById);
const signals = this.extractOfferSignals(item.offerText); const signals = this.extractOfferSignals(item.offerText);
const price = item.price ?? signals.price; const price = item.price ?? signals.price;
const priceUnit = this.normalizeUnit(item.priceUnit) ?? signals.priceUnit; const priceUnit = this.normalizeUnit(item.priceUnit) ?? signals.priceUnit;
const comparisonPrice = item.comparisonPrice ?? signals.comparisonPrice; const comparisonPrice = item.comparisonPrice ?? signals.comparisonPrice;
const comparisonUnit = this.normalizeUnit(item.comparisonUnit) ?? signals.comparisonUnit; const comparisonUnit = this.normalizeUnit(item.comparisonUnit) ?? signals.comparisonUnit;
const offerLimitText = this.extractOfferLimitText(item.offerText); const offerLimitText = this.extractOfferLimitText(item.offerText);
const displayNameDetailed = buildDisplayNameDetailed({
rawName: item.rawName,
isBundle: item.isBundle,
bundleItems: this.sanitizeBundleItems(item.bundleItems),
});
const categoryId = this.categoryResolver.resolveForFlyer({
categories,
signalText: [item.rawName, item.brand ?? '', item.offerText ?? ''].join(' ').trim(),
categoryHint: item.category,
matchedProductCategoryId: match.product?.categoryId ?? null,
matchConfidence: match.confidence,
});
return { return {
flyerItemId: null, flyerItemId: null,
rawName: item.rawName, rawName: item.rawName,
normalizedName: item.normalizedName, normalizedName: signalData.normalizedMatchName || item.normalizedName,
brand: item.brand, brand: item.brand,
category: item.category, category: item.category,
categoryId: null, categoryId,
price, price,
priceUnit, priceUnit,
comparisonPrice, comparisonPrice,
@@ -131,6 +164,8 @@ export class FlyerImportService {
bundleWeight: item.bundleWeight, bundleWeight: item.bundleWeight,
isBundle: item.isBundle, isBundle: item.isBundle,
bundleItems: this.sanitizeBundleItems(item.bundleItems), bundleItems: this.sanitizeBundleItems(item.bundleItems),
displayNameDetailed,
signals: signalData.signals,
offerText: item.offerText, offerText: item.offerText,
isOffer: this.isOfferItem(item, signals.hasCampaignPattern), isOffer: this.isOfferItem(item, signals.hasCampaignPattern),
offerLimitText, offerLimitText,
@@ -145,6 +180,8 @@ export class FlyerImportService {
matchReasonsDetailed: this.describeMatchReasons(match.reasons), matchReasonsDetailed: this.describeMatchReasons(match.reasons),
}; };
}); });
this.logImportMetrics(items);
const persistedItems = await this.persistSessionWithItems(userId, parsed.retailer, items, file); const persistedItems = await this.persistSessionWithItems(userId, parsed.retailer, items, file);
@@ -385,36 +422,40 @@ export class FlyerImportService {
select: { id: true }, select: { id: true },
}); });
const savedItems: FlyerImportItem[] = []; const savedItems: FlyerImportItem[] = [];
for (const item of items) { for (const item of items) {
const created = await this.prisma.flyerItem.create({ const createData: Prisma.FlyerItemUncheckedCreateInput = {
data: { sessionId: session.id,
sessionId: session.id, rawName: item.rawName,
rawName: item.rawName, normalizedName: item.normalizedName,
normalizedName: item.normalizedName, brand: item.brand,
brand: item.brand, categoryHint: item.category,
categoryHint: item.category, categoryId: item.categoryId,
categoryId: item.categoryId, price: item.price != null ? new Prisma.Decimal(item.price) : null,
price: item.price != null ? new Prisma.Decimal(item.price) : null, priceUnit: item.priceUnit,
priceUnit: item.priceUnit, comparisonPrice:
comparisonPrice: item.comparisonPrice != null ? new Prisma.Decimal(item.comparisonPrice) : null,
item.comparisonPrice != null ? new Prisma.Decimal(item.comparisonPrice) : null, comparisonUnit: item.comparisonUnit,
comparisonUnit: item.comparisonUnit, weight: item.weight,
weight: item.weight, bundleWeight: item.bundleWeight,
bundleWeight: item.bundleWeight, isBundle: item.isBundle,
isBundle: item.isBundle, bundleItems: item.bundleItems,
bundleItems: item.bundleItems, displayNameDetailed: item.displayNameDetailed,
offerText: item.offerText, signals: item.signals as Prisma.InputJsonValue,
parseConfidence: item.parseConfidence, offerText: item.offerText,
parseReasons: item.parseReasons, parseConfidence: item.parseConfidence,
matchedProductId: item.matchedProductId, parseReasons: item.parseReasons,
matchedProductName: item.matchedProductName, matchedProductId: item.matchedProductId,
matchedVia: item.matchedVia, matchedProductName: item.matchedProductName,
matchConfidence: item.matchConfidence, matchedVia: item.matchedVia,
matchReasons: item.matchReasons, matchConfidence: item.matchConfidence,
}, matchReasons: item.matchReasons,
select: { id: true }, };
});
const created = await this.prisma.flyerItem.create({
data: createData,
select: { id: true },
});
savedItems.push({ ...item, flyerItemId: created.id }); savedItems.push({ ...item, flyerItemId: created.id });
} }
@@ -431,21 +472,23 @@ export class FlyerImportService {
return `${d.getUTCFullYear()}-W${String(weekNo).padStart(2, '0')}`; return `${d.getUTCFullYear()}-W${String(weekNo).padStart(2, '0')}`;
} }
private matchItem( private matchItem(
item: FlyerParseItem, item: FlyerParseItem,
products: ProductLite[], normalizedMatchName: string,
aliasToProduct: Map<string, number>, itemSignals: ImportedItemSignals,
productById: Map<number, ProductLite>, products: ProductLite[],
): { aliasToProduct: Map<string, number>,
productById: Map<number, ProductLite>,
): {
product: ProductLite | null; product: ProductLite | null;
via: FlyerImportMatchVia; via: FlyerImportMatchVia;
confidence: number; confidence: number;
reasons: string[]; reasons: string[];
} { } {
const normalized = normalizeName(item.rawName || item.normalizedName); const normalized = normalizedMatchName || normalizeName(item.normalizedName || item.rawName);
if (!normalized) { if (!normalized) {
return { product: null, via: 'none', confidence: 0, reasons: ['empty_name'] }; return { product: null, via: 'none', confidence: 0, reasons: ['empty_name'] };
} }
const aliasedProductId = aliasToProduct.get(normalized); const aliasedProductId = aliasToProduct.get(normalized);
if (aliasedProductId) { if (aliasedProductId) {
@@ -471,17 +514,30 @@ export class FlyerImportService {
} }
} }
let best: { product: ProductLite; confidence: number; overlap: number } | null = null; let best: { product: ProductLite; confidence: number; overlap: number } | null = null;
const itemTokens = this.tokenize(item.rawName); const itemTokens = this.tokenize(normalized);
for (const product of products) { for (const product of products) {
const productTokens = this.tokenize(product.canonicalName ?? product.name); const productTokens = this.tokenize(product.canonicalName ?? product.name);
const overlap = this.tokenOverlap(itemTokens, productTokens); const overlap = this.tokenOverlap(itemTokens, productTokens);
if (overlap <= 0) continue; if (overlap <= 0) continue;
const confidence = Math.min(0.92, 0.5 + overlap * 0.4);
if (!best || confidence > best.confidence) { let confidence = Math.min(0.93, 0.48 + overlap * 0.42);
best = { product, confidence, overlap }; if (this.hasBrandSignal(item.brand, product)) {
} confidence += 0.04;
} }
if (this.hasWeightSignal(item.weight, product)) {
confidence += 0.03;
}
if (this.hasQualitySignal(itemSignals, product)) {
confidence += 0.03;
}
confidence = Math.min(0.95, confidence);
if (!best || confidence > best.confidence) {
best = { product, confidence, overlap };
}
}
if (best && best.confidence >= 0.66) { if (best && best.confidence >= 0.66) {
return { return {
@@ -508,7 +564,7 @@ export class FlyerImportService {
.filter((part) => part.length >= 3); .filter((part) => part.length >= 3);
} }
private tokenOverlap(a: string[], b: string[]): number { private tokenOverlap(a: string[], b: string[]): number {
if (a.length === 0 || b.length === 0) return 0; if (a.length === 0 || b.length === 0) return 0;
const as = new Set(a); const as = new Set(a);
const bs = new Set(b); const bs = new Set(b);
@@ -518,8 +574,32 @@ export class FlyerImportService {
} }
const union = new Set([...as, ...bs]).size; const union = new Set([...as, ...bs]).size;
if (union === 0) return 0; if (union === 0) return 0;
return intersection / union; return intersection / union;
} }
private hasBrandSignal(brand: string | null, product: ProductLite): boolean {
if (!brand) return false;
const normalizedBrand = normalizeName(brand);
if (!normalizedBrand) return false;
const normalizedProduct = normalizeName(`${product.name} ${product.canonicalName ?? ''}`);
return normalizedProduct.includes(normalizedBrand);
}
private hasWeightSignal(weight: string | null, product: ProductLite): boolean {
if (!weight) return false;
const normalizedWeight = normalizeName(weight);
if (!normalizedWeight) return false;
const normalizedProduct = normalizeName(`${product.name} ${product.canonicalName ?? ''}`);
return normalizedProduct.includes(normalizedWeight);
}
private hasQualitySignal(signals: ImportedItemSignals, product: ProductLite): boolean {
if (!signals.qualityFlags.includes('eco')) return false;
const normalizedProduct = normalizeName(`${product.name} ${product.canonicalName ?? ''}`);
return /\beko\b|\bekolog/i.test(normalizedProduct);
}
private isOfferItem(item: FlyerParseItem, hasCampaignPattern: boolean): boolean { private isOfferItem(item: FlyerParseItem, hasCampaignPattern: boolean): boolean {
return ( return (
@@ -745,6 +825,8 @@ export class FlyerImportService {
bundleWeight: string | null; bundleWeight: string | null;
isBundle: boolean; isBundle: boolean;
bundleItems: Prisma.JsonValue | null; bundleItems: Prisma.JsonValue | null;
displayNameDetailed?: string | null;
signals?: Prisma.JsonValue | null;
offerText: string | null; offerText: string | null;
parseConfidence: number; parseConfidence: number;
parseReasons: Prisma.JsonValue | null; parseReasons: Prisma.JsonValue | null;
@@ -759,6 +841,24 @@ export class FlyerImportService {
return value.map((entry) => String(entry)); return value.map((entry) => String(entry));
}; };
const toSignals = (value: Prisma.JsonValue | null | undefined): ImportedItemSignals | null => {
if (!value || typeof value !== 'object' || Array.isArray(value)) return null;
const record = value as Record<string, unknown>;
const toArray = (key: string): string[] => {
const maybeArray = record[key];
if (!Array.isArray(maybeArray)) return [];
return maybeArray.map((entry) => String(entry));
};
return {
originCountries: toArray('originCountries'),
labels: toArray('labels'),
qualityFlags: toArray('qualityFlags'),
variant: typeof record.variant === 'string' ? record.variant : null,
packaging: typeof record.packaging === 'string' ? record.packaging : null,
};
};
const normalizedMatchVia = const normalizedMatchVia =
item.matchedVia === 'alias' || item.matchedVia === 'exact' || item.matchedVia === 'token' item.matchedVia === 'alias' || item.matchedVia === 'exact' || item.matchedVia === 'token'
? item.matchedVia ? item.matchedVia
@@ -784,6 +884,14 @@ export class FlyerImportService {
bundleWeight: item.bundleWeight, bundleWeight: item.bundleWeight,
isBundle: item.isBundle, isBundle: item.isBundle,
bundleItems: this.sanitizeBundleItems(toStringArray(item.bundleItems)), bundleItems: this.sanitizeBundleItems(toStringArray(item.bundleItems)),
displayNameDetailed:
item.displayNameDetailed ??
buildDisplayNameDetailed({
rawName: item.rawName,
isBundle: item.isBundle,
bundleItems: this.sanitizeBundleItems(toStringArray(item.bundleItems)),
}),
signals: toSignals(item.signals),
offerText: item.offerText, offerText: item.offerText,
isOffer: isOffer:
item.price != null item.price != null
@@ -858,6 +966,8 @@ export class FlyerImportService {
bundleWeight: string | null; bundleWeight: string | null;
isBundle: boolean; isBundle: boolean;
bundleItems: Prisma.JsonValue | null; bundleItems: Prisma.JsonValue | null;
displayNameDetailed?: string | null;
signals?: Prisma.JsonValue | null;
offerText: string | null; offerText: string | null;
parseConfidence: number; parseConfidence: number;
parseReasons: Prisma.JsonValue | null; parseReasons: Prisma.JsonValue | null;
@@ -918,4 +1028,17 @@ export class FlyerImportService {
.slice(0, this.MAX_BUNDLE_ITEMS) .slice(0, this.MAX_BUNDLE_ITEMS)
.map((entry) => entry.slice(0, this.MAX_BUNDLE_ITEM_LENGTH)); .map((entry) => entry.slice(0, this.MAX_BUNDLE_ITEM_LENGTH));
} }
private logImportMetrics(items: FlyerImportItem[]): void {
if (items.length === 0) return;
const noMatchCount = items.filter((item) => item.matchReasons.includes('no_match')).length;
const categoryAssignedCount = items.filter((item) => item.categoryId != null).length;
const noMatchRatio = (noMatchCount / items.length) * 100;
const categoryAssignedRatio = (categoryAssignedCount / items.length) * 100;
this.logger.log(
`Flyer import metrics: no_match=${noMatchCount}/${items.length} (${noMatchRatio.toFixed(1)}%), category_id=${categoryAssignedCount}/${items.length} (${categoryAssignedRatio.toFixed(1)}%)`,
);
}
} }
@@ -0,0 +1,36 @@
import { CategoryResolverService } from './category-resolver.service';
describe('CategoryResolverService', () => {
const service = new CategoryResolverService();
const categories = [
{ id: 1, name: 'Kött, chark & fågel', path: 'Kött, chark & fågel' },
{ id: 2, name: 'Kött', path: 'Kött, chark & fågel > Kött' },
{ id: 3, name: 'Fläsk', path: 'Kött, chark & fågel > Kött > Fläsk' },
{ id: 4, name: 'Bröd', path: 'Bröd & kakor > Bröd' },
];
it('resolves Fläskytterfilé to pork category', () => {
const categoryId = service.resolveForFlyer({
categories,
signalText: 'Fläskytterfilé Sverige',
categoryHint: null,
matchedProductCategoryId: null,
matchConfidence: 0,
});
expect(categoryId).toBe(3);
});
it('prefers matched product category when confidence is high', () => {
const categoryId = service.resolveForFlyer({
categories,
signalText: 'Något annat',
categoryHint: 'Bröd',
matchedProductCategoryId: 99,
matchConfidence: 0.95,
});
expect(categoryId).toBe(99);
});
});
@@ -0,0 +1,114 @@
import { Injectable } from '@nestjs/common';
import { FlatCategory } from '../categories/categories.service';
type ResolveFlyerCategoryParams = {
categories: FlatCategory[];
signalText: string;
categoryHint: string | null;
matchedProductCategoryId: number | null;
matchConfidence: number;
};
@Injectable()
export class CategoryResolverService {
resolveForFlyer(params: ResolveFlyerCategoryParams): number | null {
if (params.matchedProductCategoryId != null && params.matchConfidence >= 0.9) {
return params.matchedProductCategoryId;
}
const normalizedSignal = normalizeForRules(params.signalText);
if (hasPorkLikeSignal(normalizedSignal)) {
const pork = this.resolvePorkCategory(params.categories);
if (pork) return pork.id;
}
if (hasBreadLikeSignal(normalizedSignal)) {
const bread = this.resolveBreadCategory(params.categories);
if (bread) return bread.id;
}
if (!params.categoryHint) return null;
return this.resolveByHint(params.categories, params.categoryHint)?.id ?? null;
}
private resolveByHint(categories: FlatCategory[], categoryHint: string): FlatCategory | undefined {
const normalizedHint = normalizeForRules(categoryHint);
return categories.find((category) => {
const normalizedName = normalizeForRules(category.name);
const normalizedPath = normalizeForRules(category.path);
return normalizedName === normalizedHint || normalizedPath === normalizedHint;
});
}
private resolvePorkCategory(categories: FlatCategory[]): FlatCategory | undefined {
return (
categories.find(
(category) =>
category.name.toLowerCase() === 'fläsk' &&
category.path.toLowerCase().startsWith('kött, chark & fågel > kött > '),
) ||
categories.find(
(category) =>
category.name.toLowerCase() === 'kött' &&
category.path.toLowerCase() === 'kött, chark & fågel > kött',
) ||
categories.find((category) => category.path.toLowerCase() === 'kött, chark & fågel')
);
}
private resolveBreadCategory(categories: FlatCategory[]): FlatCategory | undefined {
return (
categories.find(
(category) =>
category.name.toLowerCase() === 'rostbröd' &&
category.path.toLowerCase().startsWith('bröd & kakor > bröd > '),
) ||
categories.find(
(category) =>
category.name.toLowerCase() === 'bröd' &&
category.path.toLowerCase() === 'bröd & kakor > bröd',
) ||
categories.find((category) => category.path.toLowerCase() === 'bröd & kakor')
);
}
}
function normalizeForRules(value: string): string {
return value
.toLowerCase()
.normalize('NFD')
.replace(/[\u0300-\u036f]/g, '')
.replace(/[^a-z0-9]+/g, ' ')
.trim();
}
function hasPorkLikeSignal(normalized: string): boolean {
return (
normalized.includes('bacon') ||
normalized.includes('sidflask') ||
normalized.includes('pancetta') ||
normalized.includes('flask') ||
normalized.includes('flaskytterfile') ||
normalized.includes('ytterfile') ||
normalized.includes('karre') ||
normalized.includes('kotlett')
);
}
function hasBreadLikeSignal(normalized: string): boolean {
return (
/\brostbrod\b/.test(normalized) ||
/\brost\s*n\s*toast\b/.test(normalized) ||
/\broast\s*n\s*toast\b/.test(normalized) ||
/\btoastbrod\b/.test(normalized) ||
/\bformbrod\b/.test(normalized) ||
/\blantbrod\b/.test(normalized) ||
/\bfullkornsbrod\b/.test(normalized) ||
/\bfranska\b/.test(normalized) ||
/\blimpa\b/.test(normalized) ||
/\bbrod\b/.test(normalized) ||
/\btoast\b/.test(normalized)
);
}
@@ -0,0 +1,15 @@
export function buildDisplayNameDetailed(params: {
rawName: string;
isBundle: boolean;
bundleItems: string[];
}): string {
const rawName = params.rawName.trim();
if (!params.isBundle) return rawName;
const items = params.bundleItems
.map((item) => item.trim())
.filter((item) => item.length > 0);
if (items.length === 0) return rawName;
return `${rawName} (${items.join(' + ')})`;
}
@@ -0,0 +1,38 @@
export type ImportedItemSignals = {
originCountries: string[];
labels: string[];
qualityFlags: string[];
variant: string | null;
packaging: string | null;
};
export type ImportedItemCandidate = {
rawName: string;
normalizedName: string;
brand: string | null;
weight: string | null;
bundleWeight: string | null;
isBundle: boolean;
bundleItems: string[];
price: number | null;
priceUnit: string | null;
comparisonPrice: number | null;
comparisonUnit: string | null;
categoryHint: string | null;
categoryId: number | null;
matchedProductId: number | null;
matchedProductName: string | null;
matchedVia: string;
matchConfidence: number;
matchReasons: string[];
signals: ImportedItemSignals | null;
displayNameDetailed: string | null;
};
export const EMPTY_IMPORTED_SIGNALS: ImportedItemSignals = {
originCountries: [],
labels: [],
qualityFlags: [],
variant: null,
packaging: null,
};
@@ -0,0 +1,45 @@
import { buildDisplayNameDetailed } from './import-display-name.util';
import { extractImportSignals } from './import-signals.util';
describe('import signals utilities', () => {
it('extracts deterministic origin and eco labels', () => {
const result = extractImportSignals({
rawName: 'Fläskytterfilé (Sverige) EKO',
brand: 'Garant',
offerText: 'Ekologiskt kött från Sverige',
});
expect(result.signals.originCountries).toEqual(['Sverige']);
expect(result.signals.labels).toContain('Ekologisk');
expect(result.signals.qualityFlags).toContain('eco');
expect(result.normalizedMatchName).toBe('flaskytterfile');
});
it('extracts Germany and keeps labels deterministic', () => {
const result = extractImportSignals({
rawName: 'Korv från Tyskland',
offerText: 'Tysk kvalitet',
});
expect(result.signals.originCountries).toEqual(['Tyskland']);
expect(result.signals.labels).toEqual([]);
});
it('builds detailed display name for bundle rows', () => {
expect(
buildDisplayNameDetailed({
rawName: 'Kaptenens Favoriter',
isBundle: true,
bundleItems: ['Chumlax 3x100g', 'Alaska pollock 3x100g'],
}),
).toBe('Kaptenens Favoriter (Chumlax 3x100g + Alaska pollock 3x100g)');
});
it('extracts storpack packaging signal', () => {
const result = extractImportSignals({
rawName: 'Kycklingfilé storpack',
});
expect(result.signals.packaging).toBe('storpack');
});
});
@@ -0,0 +1,103 @@
import { normalizeName } from '../common/utils/normalize-name';
import {
EMPTY_IMPORTED_SIGNALS,
ImportedItemSignals,
} from './import-item.types';
type SignalExtractionInput = {
rawName: string;
brand?: string | null;
offerText?: string | null;
};
const ORIGIN_COUNTRY_PATTERNS: Array<{ label: string; regex: RegExp }> = [
{ label: 'Sverige', regex: /\b(sverige|svensk(t|a)?|sweden)\b/i },
{ label: 'Tyskland', regex: /\b(tyskland|tysk(t|a)?|germany|deutschland)\b/i },
{ label: 'Norge', regex: /\b(norge|norsk(t|a)?)\b/i },
{ label: 'Danmark', regex: /\b(danmark|dansk(t|a)?)\b/i },
{ label: 'Finland', regex: /\b(finland|finsk(t|a)?)\b/i },
];
const LABEL_PATTERNS: Array<{ label: string; qualityFlag: string | null; regex: RegExp }> = [
{ label: 'Ekologisk', qualityFlag: 'eco', regex: /\b(eko|ekologisk(t|a)?|organic)\b/i },
{ label: 'Laktosfri', qualityFlag: 'lactose_free', regex: /\b(laktosfri(tt|a)?|lactose\s*free)\b/i },
{ label: 'Glutenfri', qualityFlag: 'gluten_free', regex: /\b(glutenfri(tt|a)?|gluten\s*free)\b/i },
{ label: 'Vegansk', qualityFlag: 'vegan', regex: /\b(vegansk(t|a)?|vegan)\b/i },
{ label: 'Vegetarisk', qualityFlag: 'vegetarian', regex: /\b(vegetarisk(t|a)?|vegetarian)\b/i },
];
export type SignalExtractionResult = {
signals: ImportedItemSignals;
normalizedMatchName: string;
};
export function extractImportSignals(input: SignalExtractionInput): SignalExtractionResult {
const text = [input.rawName, input.brand ?? '', input.offerText ?? '']
.filter((part) => part.trim().length > 0)
.join(' ');
const origins = ORIGIN_COUNTRY_PATTERNS
.filter((pattern) => pattern.regex.test(text))
.map((pattern) => pattern.label);
const labels = LABEL_PATTERNS
.filter((pattern) => pattern.regex.test(text))
.map((pattern) => pattern.label);
const qualityFlags = LABEL_PATTERNS
.filter((pattern) => pattern.qualityFlag && pattern.regex.test(text))
.map((pattern) => pattern.qualityFlag as string);
const packaging = resolvePackaging(text);
const variant = extractVariant(input.rawName);
const signals: ImportedItemSignals = {
...EMPTY_IMPORTED_SIGNALS,
originCountries: Array.from(new Set(origins)),
labels: Array.from(new Set(labels)),
qualityFlags: Array.from(new Set(qualityFlags)),
variant,
packaging,
};
const normalizedMatchName = normalizeForMatching(input.rawName);
return { signals, normalizedMatchName };
}
export function normalizeForMatching(rawName: string): string {
let cleaned = rawName;
for (const pattern of ORIGIN_COUNTRY_PATTERNS) {
cleaned = cleaned.replace(pattern.regex, ' ');
}
for (const pattern of LABEL_PATTERNS) {
cleaned = cleaned.replace(pattern.regex, ' ');
}
cleaned = cleaned.replace(/[()\[\]]/g, ' ');
cleaned = cleaned.replace(/\s+/g, ' ').trim();
return normalizeName(cleaned) || normalizeName(rawName);
}
function resolvePackaging(text: string): string | null {
const normalized = text.toLowerCase();
if (/\b\d+\s*[x×]\s*\d+\s*(g|kg|ml|cl|dl|l)\b/.test(normalized)) {
return 'multipack';
}
if (/\bstorpack\b/.test(normalized)) {
return 'storpack';
}
if (/\b(2-pack|3-pack|4-pack|5-pack|6-pack|pack)\b/.test(normalized)) {
return 'pack';
}
return null;
}
function extractVariant(rawName: string): string | null {
const variantMatch = rawName.match(/\(([^)]+)\)/);
if (!variantMatch) return null;
const value = variantMatch[1].trim();
return value.length > 0 ? value : null;
}
@@ -1,15 +1,60 @@
import 'flyer_reason_descriptor.dart'; import 'flyer_reason_descriptor.dart';
class FlyerImportSignals {
final List<String> originCountries;
final List<String> labels;
final List<String> qualityFlags;
final String? variant;
final String? packaging;
const FlyerImportSignals({
this.originCountries = const [],
this.labels = const [],
this.qualityFlags = const [],
this.variant,
this.packaging,
});
factory FlyerImportSignals.fromJson(Map<String, dynamic> json) {
return FlyerImportSignals(
originCountries:
(json['originCountries'] as List?)?.map((e) => e.toString()).toList() ?? const [],
labels: (json['labels'] as List?)?.map((e) => e.toString()).toList() ?? const [],
qualityFlags:
(json['qualityFlags'] as List?)?.map((e) => e.toString()).toList() ?? const [],
variant: json['variant'] as String?,
packaging: json['packaging'] as String?,
);
}
Map<String, dynamic> toJson() {
return {
'originCountries': originCountries,
'labels': labels,
'qualityFlags': qualityFlags,
'variant': variant,
'packaging': packaging,
};
}
}
class FlyerImportItem { class FlyerImportItem {
final int? flyerItemId; final int? flyerItemId;
final String rawName; final String rawName;
final String normalizedName; final String? displayNameDetailed;
final String normalizedName;
final String? brand;
final String? category; final String? category;
final int? categoryId; final int? categoryId;
final double? price; final double? price;
final String? priceUnit; final String? priceUnit;
final String? offerText; final String? weight;
final bool isOffer; final String? bundleWeight;
final bool isBundle;
final List<String> bundleItems;
final FlyerImportSignals? signals;
final String? offerText;
final bool isOffer;
final String? offerLimitText; final String? offerLimitText;
final double? comparisonPrice; final double? comparisonPrice;
final String? comparisonUnit; final String? comparisonUnit;
@@ -24,14 +69,21 @@ class FlyerImportItem {
final List<FlyerReasonDescriptor> matchReasonsDetailed; final List<FlyerReasonDescriptor> matchReasonsDetailed;
FlyerImportItem({ FlyerImportItem({
required this.flyerItemId, required this.flyerItemId,
required this.rawName, required this.rawName,
required this.normalizedName, this.displayNameDetailed,
required this.normalizedName,
this.brand,
this.category, this.category,
this.categoryId, this.categoryId,
this.price, this.price,
this.priceUnit, this.priceUnit,
this.offerText, this.weight,
this.bundleWeight,
this.isBundle = false,
this.bundleItems = const [],
this.signals,
this.offerText,
this.isOffer = false, this.isOffer = false,
this.offerLimitText, this.offerLimitText,
this.comparisonPrice, this.comparisonPrice,
@@ -49,14 +101,23 @@ class FlyerImportItem {
factory FlyerImportItem.fromJson(Map<String, dynamic> json) { factory FlyerImportItem.fromJson(Map<String, dynamic> json) {
return FlyerImportItem( return FlyerImportItem(
flyerItemId: (json['flyerItemId'] as num?)?.toInt(), flyerItemId: (json['flyerItemId'] as num?)?.toInt(),
rawName: json['rawName'] as String? ?? '', rawName: json['rawName'] as String? ?? '',
normalizedName: json['normalizedName'] as String? ?? '', displayNameDetailed: json['displayNameDetailed'] as String?,
normalizedName: json['normalizedName'] as String? ?? '',
brand: json['brand'] as String?,
category: json['category'] as String?, category: json['category'] as String?,
categoryId: (json['categoryId'] as num?)?.toInt(), categoryId: (json['categoryId'] as num?)?.toInt(),
price: (json['price'] as num?)?.toDouble(), price: (json['price'] as num?)?.toDouble(),
priceUnit: json['priceUnit'] as String?, priceUnit: json['priceUnit'] as String?,
offerText: json['offerText'] as String?, weight: json['weight'] as String?,
bundleWeight: json['bundleWeight'] as String?,
isBundle: json['isBundle'] == true,
bundleItems: (json['bundleItems'] as List?)?.map((e) => e.toString()).toList() ?? const [],
signals: json['signals'] is Map
? FlyerImportSignals.fromJson(Map<String, dynamic>.from(json['signals'] as Map))
: null,
offerText: json['offerText'] as String?,
isOffer: json['isOffer'] == true, isOffer: json['isOffer'] == true,
offerLimitText: json['offerLimitText'] as String?, offerLimitText: json['offerLimitText'] as String?,
comparisonPrice: (json['comparisonPrice'] as num?)?.toDouble(), comparisonPrice: (json['comparisonPrice'] as num?)?.toDouble(),
@@ -87,11 +148,18 @@ class FlyerImportItem {
return { return {
'flyerItemId': flyerItemId, 'flyerItemId': flyerItemId,
'rawName': rawName, 'rawName': rawName,
'displayNameDetailed': displayNameDetailed,
'normalizedName': normalizedName, 'normalizedName': normalizedName,
'brand': brand,
'category': category, 'category': category,
'categoryId': categoryId, 'categoryId': categoryId,
'price': price, 'price': price,
'priceUnit': priceUnit, 'priceUnit': priceUnit,
'weight': weight,
'bundleWeight': bundleWeight,
'isBundle': isBundle,
'bundleItems': bundleItems,
'signals': signals?.toJson(),
'offerText': offerText, 'offerText': offerText,
'isOffer': isOffer, 'isOffer': isOffer,
'offerLimitText': offerLimitText, 'offerLimitText': offerLimitText,
@@ -119,11 +187,18 @@ class FlyerImportItem {
return FlyerImportItem( return FlyerImportItem(
flyerItemId: flyerItemId, flyerItemId: flyerItemId,
rawName: rawName ?? this.rawName, rawName: rawName ?? this.rawName,
displayNameDetailed: displayNameDetailed,
normalizedName: normalizedName, normalizedName: normalizedName,
brand: brand,
category: category ?? this.category, category: category ?? this.category,
categoryId: categoryId ?? this.categoryId, categoryId: categoryId ?? this.categoryId,
price: price, price: price,
priceUnit: priceUnit, priceUnit: priceUnit,
weight: weight,
bundleWeight: bundleWeight,
isBundle: isBundle,
bundleItems: bundleItems,
signals: signals,
offerText: offerText, offerText: offerText,
isOffer: isOffer, isOffer: isOffer,
offerLimitText: offerLimitText, offerLimitText: offerLimitText,
@@ -501,6 +501,40 @@ class _FlyerImportTabState extends ConsumerState<FlyerImportTab> {
); );
} }
Widget _buildMetadataBadges(FlyerImportItem item, ThemeData theme) {
final values = <String>[
...(item.signals?.originCountries ?? const <String>[]),
...(item.signals?.labels ?? const <String>[]),
];
if (values.isEmpty) return const SizedBox.shrink();
return Wrap(
spacing: 6,
runSpacing: 6,
children: values
.map(
(value) => Container(
padding: const EdgeInsets.symmetric(horizontal: 8, vertical: 3),
decoration: BoxDecoration(
color: theme.colorScheme.primary.withValues(alpha: 0.08),
borderRadius: BorderRadius.circular(999),
border: Border.all(
color: theme.colorScheme.primary.withValues(alpha: 0.25),
),
),
child: Text(
value,
style: theme.textTheme.labelSmall?.copyWith(
color: theme.colorScheme.primary,
fontWeight: FontWeight.w600,
),
),
),
)
.toList(),
);
}
Future<void> _copyText(String value, String label) async { Future<void> _copyText(String value, String label) async {
await Clipboard.setData(ClipboardData(text: value)); await Clipboard.setData(ClipboardData(text: value));
if (!mounted) return; if (!mounted) return;
@@ -744,7 +778,7 @@ class _FlyerImportTabState extends ConsumerState<FlyerImportTab> {
}, },
title: Row( title: Row(
children: [ children: [
Expanded(child: Text(item.rawName)), Expanded(child: Text(item.displayNameDetailed ?? item.rawName)),
IconButton( IconButton(
tooltip: 'Redigera', tooltip: 'Redigera',
visualDensity: VisualDensity.compact, visualDensity: VisualDensity.compact,
@@ -765,6 +799,10 @@ class _FlyerImportTabState extends ConsumerState<FlyerImportTab> {
crossAxisAlignment: CrossAxisAlignment.start, crossAxisAlignment: CrossAxisAlignment.start,
children: [ children: [
if (priceText.isNotEmpty) Text('Pris: $priceText'), if (priceText.isNotEmpty) Text('Pris: $priceText'),
if (item.isBundle && item.bundleItems.isNotEmpty)
Text('Paketinnehåll: ${item.bundleItems.join(' + ')}'),
if (item.brand != null && item.brand!.trim().isNotEmpty)
Text('Varumärke: ${item.brand}'),
if ((item.category ?? '').trim().isNotEmpty) if ((item.category ?? '').trim().isNotEmpty)
Text('Kategori: ${item.category}'), Text('Kategori: ${item.category}'),
if (comparisonText.isNotEmpty) if (comparisonText.isNotEmpty)
@@ -778,6 +816,11 @@ class _FlyerImportTabState extends ConsumerState<FlyerImportTab> {
), ),
), ),
if (sanitizedOfferText.isNotEmpty) Text(sanitizedOfferText), if (sanitizedOfferText.isNotEmpty) Text(sanitizedOfferText),
if ((item.signals?.originCountries.isNotEmpty ?? false) ||
(item.signals?.labels.isNotEmpty ?? false)) ...[
const SizedBox(height: 6),
_buildMetadataBadges(item, theme),
],
if (item.matchedProductName != null) if (item.matchedProductName != null)
Text('Match: ${item.matchedProductName}'), Text('Match: ${item.matchedProductName}'),
if (detailedReasons.isNotEmpty) ...[ if (detailedReasons.isNotEmpty) ...[
@@ -0,0 +1,29 @@
import 'package:flutter_test/flutter_test.dart';
import 'package:recipe_flutter/features/import/domain/flyer_import_item.dart';
void main() {
test('parses signals and detailed bundle name', () {
final item = FlyerImportItem.fromJson({
'flyerItemId': 7,
'rawName': 'Kaptenens Favoriter',
'displayNameDetailed':
'Kaptenens Favoriter (Chumlax 3x100g + Alaska pollock 3x100g)',
'normalizedName': 'kaptenens favoriter',
'isBundle': true,
'bundleItems': ['Chumlax 3x100g', 'Alaska pollock 3x100g'],
'signals': {
'originCountries': ['Sverige'],
'labels': ['Ekologisk'],
'qualityFlags': ['eco'],
'variant': null,
'packaging': 'multipack',
},
});
expect(item.isBundle, isTrue);
expect(item.displayNameDetailed, contains('Chumlax 3x100g'));
expect(item.bundleItems, hasLength(2));
expect(item.signals?.originCountries, ['Sverige']);
expect(item.toJson()['signals'], isA<Map<String, dynamic>>());
});
}