Retail competitor price monitoring system
The retail chain sees competitor prices and its own market position — without manual exports or false matches.
Project
The system collects competitor prices, checks them against our own assortment, and turns the observations into a verifiable view of market position.
- Context
- The same product is named differently by different retail chains, and price and availability depend on the region and the specific store — matching without human review can produce misleading analytics.
- Business problem
- Bring data about our own and competitor assortments into one verifiable model without losing the accuracy of product matching.
Chosen solution boundary
A pipeline from data collection to manual verification of product pairs and reporting — an automated match is never treated as a final fact.
Contribution
From problem boundary to production
- 01 Data model and integration design
- 02 Competitor data collection adapters
- 03 Product matching and manual verification workflow
- 04 Analytics, reporting, and operational monitoring
Business problem and context
Problem
- The same product is named differently across retail chains and cannot be matched by simple text similarity.
- Price and availability depend on the region and the specific store, not just the retail chain as a whole.
- Automated product matching can produce misleading analytics if the result is not verified by a person.
- A competitor data source can become unavailable or change format, and promotional and regular prices carry different meaning.
Users and context
- A business user works with the dashboard, category and competitor analytics, and a specific product view.
- A data operator manages sources and stores, runs collection and matching, and reviews candidate product pairs.
- The client's external system sends its own assortment, prices, and stock through a protected API in idempotent batches.
Solution
- Own-catalog import of prices and stock through a token-protected API with idempotent batch operations.
- Separate adapters for several competing retail chains with region and store-level detail.
- Verified matching product attribute normalization and a matching algorithm that requires manual confirmation before a pair reaches business reports.
- Price-position analytics price history, a per-product view, and Excel export.
- One shared pricing engine the dashboard, analytics workspace, reports, and product view all calculate market position through the same module, not separate independent calculations.
How it works
- Collection & normalization
Adapters collect competitor prices, availability, and promotions per store and region, and bring them into a common format without re-saving duplicate observations.
- Matching
The algorithm proposes candidate matches between our product and a competitor product based on text and structural similarity.
- Manual review
An operator confirms or rejects the proposed match; only confirmed pairs are used in reports.
- Analytics & reporting
Confirmed comparisons form the price position and history, available in the interface and exportable to Excel.
Product surfaces
Live fragment
Try it yourself: manual matching review
The same screen as the screenshot above, made interactive. Confirm or reject the pairs — the state lives only on this page, nothing is sent or stored anywhere.
- Product #4471, 0.5 vol.Listing "Product 0.5 standard"96
- Product #4472, 0.7 vol.Listing "Product 0.7 line A"88
- Product #4473, 1.0 vol.Listing "Product 1.0 version B"74
- Product #4474, 0.33 vol.Listing "Product 0.33 pack"69
Architecture and reliability
A web application, a background worker, and a scheduler run as separate processes around a shared database; competitor adapters plug into one common contract, while the own assortment arrives through a separate protected import.
- Web application and server-rendered interface
- Background worker and scheduler
- Competitor data collection adapters
- Own-assortment import
Reliability and security
- Separate web, worker, and scheduler processes with recovery of unfinished jobs after a restart.
- User roles, an audit log, and brute-force login protection.
- A defined backup and update procedure before any schema change.
- Security headers and a content security policy are enforced at the application level, not left to web-server configuration.
Engineering decisions
Only a manually confirmed match is used in working reports.
Challenge Automated product matching can be wrong and distort business analytics.
Why it matters Automation speed is not traded for the accuracy of the data decisions rely on.
Current state and price history are stored separately.
Challenge Current prices need to be shown quickly while price history must still be preserved.
Why it matters Analytics stays fast without losing the change history.
Own-price import is idempotent by batch identifier.
Challenge A client system may resend the same price batch after a network error.
Why it matters A repeated request cannot create duplicate or conflicting data.
Collection concurrency is controlled separately per source.
Challenge Several competitor sources cannot be collected the same way without risking overload or blocking.
Why it matters Data collection does not put unnecessary load on competitor sources.
Automatic confirmation is disabled entirely: every candidate match, regardless of its score, waits for an operator decision.
Challenge A high similarity score from the algorithm does not guarantee it is genuinely the same product.
Why it matters No automated matching result reaches the reports without a human decision.
A confirmed match applies at the competitor level as a whole, not per individual store.
Challenge Verifying a match separately for every competitor store and region would make manual review unworkable as the network grows.
Why it matters Manual review scales across many store locations without a proportional increase in operator workload.
Technologies
- Python
- Flask
- PostgreSQL
- SQLAlchemy / Alembic
- RapidFuzz
- Chart.js
- Gunicorn / Nginx / systemd