NLP retrieval and open data
Valencia Open Data Navigator
Combines lexical and latent-semantic signals to make multilingual municipal metadata easier to inspect without hiding the ranking trade-offs.
Measured project results
- datasets in versioned snapshot
- 296
- manual relevance queries
- 20
- best MRR@10 (BM25)
- 0.793
- best Recall@5 (hybrid)
- 0.975
Metrics come from the committed reproducible run in the linked repository. Their interpretation and limits are documented below.
Architecture
System flow and reviewable outputs.
- 01
Versioned CKAN snapshot
- 02
Metadata quality controls
- 03
Spanish and Valencian normalization
- 04
BM25, LSA and rank fusion
- 05
Versioned retrieval benchmark
- 06
FastAPI and Streamlit
Reviewable artifacts
- Catalog snapshot
- BM25 and LSA
- RRF and MMR
- Benchmark suite
- FastAPI
- Tests and CI
Technology stack
- Python
- scikit-learn
- FastAPI
- Streamlit
- TF-IDF
- pytest
Question
What the project investigates.
A municipal catalog is often multilingual, unevenly tagged and difficult to navigate with a short information need. The project asks how to improve metadata discovery while keeping the ranking logic and its limits visible.
Approach
Method, in order.
- 01
Fetch all public packages from the official CKAN endpoint and keep a compact metadata snapshot with a SHA-256 manifest.
- 02
Validate required fields, duplicate slugs and declared resources before indexing the corpus.
- 03
Normalize Spanish and Valencian text, then compare BM25 with word-and-character TF-IDF reduced through latent semantic analysis.
- 04
Fuse lexical and semantic rank positions with reciprocal-rank fusion, and apply MMR only when a diverse top result set is useful.
- 05
Evaluate all strategies against 20 versioned, manually curated discovery queries before serving results through Streamlit and FastAPI.
Evidence
Results with context.
- The included 2026-07-13 snapshot contains 296 datasets, 1,184 declared resources and no duplicate slugs or missing titles.
- BM25 has the strongest MRR@10 (0.793) and nDCG@10 (0.832) on the small benchmark; the hybrid strategy achieves the highest Recall@5 (0.975).
- Hybrid MMR preserves the top-five recall but slightly reduces nDCG, an explicit trade-off for diversity rather than a claim of universal improvement.
Quality controls
How the work can be reviewed.
- The pipeline validates labels against the snapshot before evaluation and can rebuild reports without a network request.
- Each API result includes BM25, LSA and RRF components plus matched normalized terms.
- Pytest, Ruff, dependency checks and GitHub Actions cover retrieval, labels, API contracts and offline reconstruction.
Limits
What the output is not.
- The corpus is a snapshot, not a live or official search index.
- Twenty manual labels support regression testing but do not establish universal user-search quality.
- Rank scores do not validate the contents, freshness or reuse conditions of linked data resources.