How Civic Meeting Recordings Became Searchable Public Records

TL;DR: A resident should be able to follow a council decision from its meeting notice to the relevant agenda item, recorded discussion, and supporting documents through a stable page. While working on the coding project, I ran into the difficulty of connecting very long agendas, hours-long recordings, and official links that can change; the approach was to give each meeting a stable identity and link transcript chapters to agenda items and timestamps.

Start by checking what the meeting actually provides

The earliest milestone came from checking what a reader could reach from a calendar entry. The Grey County inventory contained 84 retained meetings: 83 had agenda PDFs and 74 had video URL fields. Those fields captured listed addresses; the project tracked downloads, transcripts, and publications separately. An April 27, 2026 Owen Sound Council package shows the scale of the material: a 134-page agenda paired with more than eight hours of meeting audio. The first design decision gave each meeting a stable identity and an agenda-preview path for records that started with documents.

Related project: Owen Sound Reporter shares work documented in this history.

Git history: agenda preview and subreport preparation.

Project repository: grey-county-reporter.

When a meeting had a package but no recording, the system did not fail; it instead generated a structured HTML summary from the agenda PDF alone. The March 30 Grey agenda-only example demonstrated this capability with a 24,612,196-byte PDF and structured HTML output where recorded meeting metadata had empty video and audio arrays. This path allowed the pipeline to function when official sources were incomplete. The documentation checked during this phase included an operator checklist that defined exact command order for daily use, ensuring that teams could handle meetings with varying levels of completeness.

The source audit process established that a meeting notice, agenda package, recording, and supporting documents did not need to arrive simultaneously to create a valid record. If the calendar feed provided a link that later went dead, the system preserved the file copy under a stable content-based link. This approach meant that the absence of a video URL in the metadata did not prevent the creation of a navigable meeting page. The agenda preview path used the available agenda and extracted subreports to build a summary text from the PDF reliably, rather than just using HTML attachment titles.

This initial phase set the foundation for handling incomplete data sources. By accepting that official links could change while recordings took hours to review, the team built a system that prioritized the availability of source files over the stability of external URLs. The decision to separate the agenda-preview path from the full transcript path allowed the pipeline to handle cases where a meeting had a package but no recording. This separation ensured that the presence of an agenda PDF was sufficient to create a valid record, even if the video recording was missing or the official link was broken.

The retention of these artifacts showed that the system could distinguish between a missing file and a file with a broken link. When the metadata recorded empty video fields, the pipeline did not assume the meeting was invalid. Instead, it proceeded to extract whatever content was available from the agenda PDF. This behavior aligned with the broader goal of making public records durable and accessible even when official sources were unstable. The March 30 example proved that a large agenda PDF could serve as the primary source for a meeting record when no other inputs were present.

Turn long recordings into agenda-linked chapters

The workflow shifted from random chunking to a staged pipeline that aligns transcript rows with agenda timestamps, utilizing retained April 27 artifacts to demonstrate how whole-meeting summaries are derived strictly from transcript evidence rather than agenda text. The April 27, 2026 Owen Sound Council workspace contained a 134-page agenda PDF and an Opus audio file with a recorded duration of over eight hours. The pipeline processed this large input by normalizing sentence cues and timestamp data to create a series of agenda-wise chapters.

Git history: transcript section and heading repair, agenda-aware Grey transcript chunking, whole-meeting synthesis with chunk artifacts, and agenda and transcript page identity.

The move from arbitrary transcript chunks to timestamped agenda-wise chapters addressed the problem of long recordings where content needed to be organized by topic rather than just time. The retained source PDF was 134 pages, and the pipeline produced a 25-entry agenda-wise series that mapped specific sections of the transcript to the official agenda items. This mapping ensured that each chapter corresponded to a specific topic listed on the official meeting agenda.

The staged transcript flow involved two main steps: first, the system extracted sentence-level cues from the audio file and aligned them with the normalized timestamp data; second, it grouped these cues into chapters based on the agenda boundaries. The whole-meeting summary was then derived strictly from this transcript evidence rather than agenda text. This approach prevented the agenda text from driving the content of the article body, which was a critical distinction for maintaining factual accuracy.

The April 27 workspace also retained counts showing that one agenda-aware chunking step was skipped because an existing checkpoint was present. These are observed retained artifacts and counts, not a claim that one run generated every artifact. The system reused checkpoints to avoid redundant processing while ensuring that the final output remained aligned with the original source files. This behavior demonstrated that the pipeline could handle large-scale inputs without losing fidelity to the original meeting structure.

The transcript text of 94,249 bytes and 16,015 words provided the raw material for the chapter generation. The system normalized these cues to create a consistent timeline that matched the agenda items. This normalization process ensured that the timestamps in the transcript aligned with the official agenda schedule. The resulting chapters allowed readers to navigate directly to specific topics within the long recording without having to scroll through hours of unstructured text.

This phase established that long recordings could be transformed into navigable documents by aligning them with the official agenda structure. The April 27 artifacts proved that a single meeting could produce a structured summary that preserved the original topic flow while adding searchability and navigation capabilities. The pipeline did not invent new content; it organized existing transcript evidence into a format that matched the official agenda boundaries. This organization made the long recording accessible to residents who needed to find specific information about particular agenda items.

Use extraction and provenance failures to raise the quality bar

The April agenda-extraction sequence revealed that ingestion contamination could occur when the system reused old files instead of creating new ones. Investigations into cross-meeting bleed identified hardcoded topic rewrites and stale checkpoint reuse as root causes, leading to strict ingestion guards that rejected contaminated subreport bodies and enforced transcript-only evidence for factual content. The May 20 calendar refresh wrote 592 event rows across 78 monthly calendar files, but these numbers described calendar entries, not successful transcript count.

Git history: grounding, extraction, and hook hardening, meeting provenance and publication gates, retries, normalization, and and resumable downloads.

The April agenda-extraction sequence distinguished pruned input from retained PDF counts by removing non-essential text like page headers or footer markers to focus on the main content. The system processed a 134-page agenda PDF and produced a structured summary that excluded irrelevant formatting elements. This pruning step ensured that the extracted content focused on the substantive meeting material rather than document structure artifacts.

The separate May cross-meeting contamination investigation found that data from one meeting incorrectly appeared in another meeting's record because a system reused old files instead of creating new ones. The root cause was artifact reuse, not model creativity. How it got in: a previous run produced garbage output before use, and the pipeline did not fully close the "bleed through cached artifacts" route. This contamination entered through artifact reuse, which meant that content from one meeting could appear in another if the system reused cached files without verifying their provenance.

The source-grounded gates implemented during this phase stopped a process from moving forward unless the current step was verified against its original source file, preventing errors from spreading. The ingestion-layer changes ensured that subreport extraction did not rely only on page-marker or table of contents heuristics. If valid content was missing, the system failed closed rather than generating incorrect summaries. This behavior protected the integrity of the public record by ensuring that every piece of content had a verified source.

The April 25 commit hardened canonical grounding, extraction, and hook flow to prevent these contamination issues. The fix ensured that the pipeline did not reuse artifacts from previous runs without verifying their relevance to the current meeting. This change eliminated the coarse fallback hook behavior that allowed contaminated data to enter the output stream. The system now required fresh evidence for each stage of the pipeline, ensuring that every chapter and summary was grounded in the current meeting's source files.

The May 15 commit further hardened the meeting pipeline by adding checks for attribution, newsworthiness, and publish paths. These checks ensured that only verified content moved forward to publication. The system rejected any subreport body that did not pass the ingestion guard, which prevented contaminated data from appearing in the final record. This approach raised the quality bar by ensuring that every piece of published content had a verified source chain.

The investigation also revealed that agenda text should never drive content for a transcript post. Current bugs where transcript sections collapsed to row 0 were fixed by enforcing transcript-only evidence for factual content. The system now derived chapter boundaries from normalized timestamp cues rather than agenda text alone. This distinction ensured that the content remained faithful to the actual spoken words rather than the written agenda structure.

The May 21 commit added shared pipeline fixes including agenda publish retries, normalization, stage 3, and resumable download capabilities. These improvements allowed the system to handle large agenda packages like the 692-page Owen Sound special Council package or the 1373-page Grey County budget package without losing fidelity. The system could process these large inputs while maintaining strict adherence to the source-grounded gates that prevented contamination.

Keep the source files and recordings reachable

Unstable or missing official links led to mirrored attachments, which ensured that durable public document links remained available even when official sources changed. The July stream discovery concerned two Grey recordings present in the official YouTube Live listing while calendar video fields were empty. To address this gap, a persistent mirror API was deployed that uploaded supporting files to content-addressed storage before publishing.

Related public engineering discussions include Create a council meeting archive browser with searchable minutes, ... and Submit Remote MCP Server: City Council & Public Meeting Transcripts .... These records give concrete examples of how other projects frame similar capacity decisions.

Git history: agenda publishing and archive hardening.

The calendar and YouTube discovery paths widened source coverage by finding recordings that official feeds missed. The July attachment API checks applied to that route and one meeting, where 48 attachments were locally verified for one July Owen meeting. These results confirmed that the mirror API could serve files with expected content types, lengths, cache policies, and nosniff headers. A repeated SHA-256 reused its stored blob, ensuring that the same file always had the same link even if the server changed.

The canonical attachments now mapped to persistent URLs that did not depend on external service availability. The API returned 401 for missing service authorization and 400 for altered SHA-256 or byte-count metadata. Exact replay returned identical JSON, proving that the system could verify file integrity before serving it to users. This verification process ensured that every attachment served was authentic and unaltered.

The reporter-side records showed that the mirror API could handle large files like the 52,428,801-byte file served with the expected content type. The system did not charge for meetings with no video or failed recordings, which aligned with the principle of making public records accessible regardless of official source availability. This approach ensured that residents could access meeting materials even when official links were broken or missing.

The persistent reporter-side fix for both Owen Sound and Grey County meant that every mirrored URL was verified automatically. Post 8526 could be updated to reflect new verification results, ensuring that the public record remained current. The system did not modify posts unless they passed the verification gate, which protected against publishing incorrect or incomplete information.

The discovery paths also found recordings that were archived streams rather than live broadcasts. The system added Live-tab discovery to rerun meetings and ensure that all available recordings were included in the public record. This expansion of source coverage meant that residents could find meeting materials regardless of how they were originally published by the municipality.

The content-addressed storage approach ensured that files had stable identities based on their digital fingerprints rather than server paths. This method prevented broken links from appearing in the public record and ensured that every attachment remained reachable over time. The system did not rely on external service availability for core functionality, which made the public record more resilient to changes in official hosting arrangements.

Carry verification down to speaker turns

The final phase involved correcting speaker attribution errors in candidate chapters and validating that all question rounds passed coverage checks, establishing a reusable pipeline for future public record maintenance. The October 1 session report showed a current 75-entry transcript TOC where misattributions occurred because the system used short name-only cues like "Julie King" without verifying full names. The corrected sentence-level transcript regenerated and aligned 67 candidate-answer chapters, and all 11 question rounds passed full candidate coverage for one checked forum transcript.

Git history: sentence-level speaker attribution policy, and video outcomes grounded in verified chapters.

The October 2 session fixed upstream speaker handling by adding generic handling for short name-only cues. Evidence-backed normalization ensured that future ASR spelling variants did not cause misattribution. The system now distinguished between a meeting-specific replacement and a reusable policy for handling name variations. This distinction ensured that corrections remained specific to the named project while allowing the pipeline to handle similar issues in other contexts.

The October 14 session corrected speaker attribution where someone talked about Carol but it was Ray Botten in this case. The relabel pass hit the GPU housekeeper's capacity gate rather than a code error, so the system checked managed queue safe idle runtimes before retrying. This behavior ensured that corrections did not kill unrelated processes while still fixing the attribution issue.

The improved sentence-level transcript was used to regenerate and align the candidate subchapters. The corrected chapter now began at 01:29:20, with the candidate sequence correctly aligned. Ray Botten now owned specific time slots including 01:04:58 for "Owen Sound completed" and subsequent content. Claire McCormack was correctly attributed to her own segments rather than being mislabeled as Andrii Zvorygin.

The October 19 commit standardized sentence speaker diarization policy to ensure consistent attribution across all meetings. The October 27 commit grounded video article outcomes from verified chapters, ensuring that the published record matched the corrected transcript evidence. These changes ensured that the public record remained accurate even when ASR made initial errors.

The verification records showed that a corrected sentence-level transcript regenerated and aligned 67 candidate-answer chapters. All 11 question rounds passed full candidate coverage for one checked forum transcript. The report was scoped to the checked candidate-forum transcript and its coverage gate, ensuring that claims remained specific to the evidence provided.

This final phase established a reusable pipeline for future public record maintenance. Teams could apply the same speaker attribution corrections and coverage checks to their own meetings. The system did not invent new content; it corrected existing errors based on verified evidence. This approach ensured that the public record remained accurate and trustworthy over time.

The reader takeaway from this chronology is that every stage of the pipeline requires verification against its original source file. Whether correcting speaker attribution or aligning chapters, the process relies on concrete evidence rather than assumptions. The Grey County Reporter and Owen Sound Reporter pipelines demonstrate how a team can build a system that serves the public even when official sources are unstable.

The next step for any team maintaining a similar public-record pipeline is to review these five historical phases and identify which source-grounded gate their organization needs to implement first. Whether it is agenda-aware chunking, contamination guards, attachment mirroring, or speaker verification, each gate adds a layer of reliability that protects the integrity of the public record. By following this sequence, teams can ensure that their archives remain durable, searchable, and faithful to the original source materials.