About

How this project works and where the data comes from.

Mission

Pellucid Labs builds civic transparency tools that make public data searchable and understandable. This project focuses on the Baltimore City Office of Inspector General, making years of oversight reports, investigations, and annual summaries available through full-text search and AI-powered question answering.

Data Sources

All documents are sourced from public pages on baltimorecity.gov and crawled automatically:

OIG Reports — Investigation synopses, full reports, and agency responses
Archived OIG Reports — Historical investigations from prior years
Annual Reports — Yearly summaries of OIG activity and findings

Document Types

SynopsisReportResponseExhibitAnnual ReportReferralFollow-up

Case numbers follow the format YY-NNNN-L (e.g., 25-0046-I), where YY is the fiscal year, NNNN is the sequence number, and L indicates the case type.

How It Works

Crawl

Public OIG report pages on baltimorecity.gov are automatically scraped for new PDF links.

S3 Storage

Downloaded PDFs are synced to an S3 bucket, deduplicated using a tracking manifest.

Text Extraction

Apache Tika extracts full text from each PDF, handling scanned and digital documents.

Named Entity Recognition

spaCy NER identifies people, organizations, dates, and case numbers within extracted text.

Knowledge Base

Extracted content is ingested into an Amazon Bedrock Knowledge Base for semantic search.

RAG Q&A

Users ask natural-language questions and receive answers grounded in the indexed documents.

Key Roles

Inspector General

Appointed to independently investigate fraud, waste, and abuse in Baltimore City government operations.

City Departments

Agencies and departments that are the subject of OIG investigations, including DPW, DHCD, Finance, and others.

Pellucid Labs is an independent project and is not affiliated with, endorsed by, or connected to the Baltimore City government or the Office of Inspector General. All data is sourced from publicly available documents.