← All writing
EngineeringDS-2026-015

Building a news feed that helps readers find what matters

In brief

The DanielScience news feed collects technical news, adds short summaries, and moves repeated vulnerability coverage down the list. Readers can scan the results before deciding which original articles to open.

A busy news feed can make it harder to keep up. Several publishers may cover the same security issue, while an unrelated story that deserves attention gets pushed down the page.

The DanielScience news feed brings those stories into one place, adds short summaries, and puts less emphasis on repeated coverage. The aim is to help readers decide what is worth opening, without having to visit every publisher first.

What the feed does

The work happens in three steps. A scheduled job collects articles from publisher feeds, Claude summarizes new material, and rules in the browser decide how to order the results.

For example, several articles might cover the same software vulnerability. The feed keeps those articles available but moves follow-up coverage down the list. That leaves more room for other stories without treating every follow-up as worthless.

News workflow from scheduled collection to summaries, publication, and ranking. Includes technologies and lessons learned.
Articles are collected and summarized ahead of time. Opening the feed does not start a new AI request.

How articles get onto the page

The collector reads RSS and Atom feeds, which are machine-readable lists of a publisher’s recent articles. It has 49 configured sources, including eight Reddit feeds. It removes duplicate article addresses and saves the results for the website.

Some sources fail or limit requests. The collector can skip an unavailable source instead of abandoning the whole update, and it spaces out Reddit requests. The collection job is scheduled daily at 11:00 UTC, though its actual start time can vary.

New articles go to Claude in groups of up to ten for a short summary and category. Existing summaries are reused so the same article does not need to be summarized on every run.

If AI summarization is unavailable, the feed can use the publisher’s description and assign a category from keywords. There is a weakness in the current handling: some failed AI requests are marked as completed, which can prevent a later retry. The feed can continue, but that status needs to be more reliable.

Why AI does not choose the order

Claude writes summaries; JavaScript calculates the ranking. Keeping those jobs separate lets me change how stories are prioritized without asking the model to summarize everything again.

The ranking considers the publisher, topic, age of the article, and signals in its text. Reports of active attacks and available patches can move a story up. Repeated coverage of the same CVE, an identifier for a known vulnerability, moves down after the initial ranking.

This is an editorial aid, not an objective measure of importance. Keywords can miss context, publisher weights reflect judgment, and summaries can be wrong. The original article is still the place to verify details before acting on them.

Engineering details

The site serves saved files rather than asking a live AI service to answer each reader. The browser still runs JavaScript for ranking and display, but the collection and summarization work happens beforehand.

Technology map showing collection, Claude summaries, website generation, GitHub Actions, and development tools.
The technology map separates the collection job, the reader’s browser, and the tools used during development.
  • Collection and summaries: JavaScript runs on Node.js, with rss-parser for feeds and the Anthropic SDK for the Claude Messages API. The configured model is Claude Haiku.
  • Website and browser: Eleventy and Nunjucks generate the pages. HTML and CSS provide the layout, and browser JavaScript ranks and displays the articles.
  • Automation and hosting: GitHub Actions uses YAML and shell commands to run collection and deployment. The deployment builds the site and transfers its files over SSH with rsync to a VPS running Nginx.

The full source is private. This selected file map shows where the main responsibilities sit:

scripts/fetch-news.js
js/layout-components.js
src/news.html
_includes/
.eleventy.js
.github/workflows/
  update-news.yml
  deploy.yml

fetch-news.js collects and summarizes the articles. layout-components.js handles ranking and display, while the templates and Eleventy configuration build the surrounding site.

This short excerpt shows the AI request inside the summarizer. The prompt and response handling are omitted:

const msg = await client.messages.create({
  model: "claude-haiku-4-5-20251001",
  max_tokens: items.length * 160,
  messages: [
    { role: "user", content: prompt }
  ],
});

The response is checked for an array with the expected number of results. The API key comes from GitHub Actions secrets and is not included in the browser’s data files.

The update workflow saves changed article data, and the deployment workflow builds and publishes the site. Deployment has push, manual, scheduled, and update-completion triggers. One limitation is that an update finishing does not by itself prove it succeeded; a success check would make that handoff safer.

I also use Codex and Claude as coding assistants. That development work is separate from the scheduled Claude API call that produces article summaries.

Lessons learned

Keeping the feed available when a source fails is useful, but it is only half the job. The system also needs to record what failed so it can retry the right work later.

  • Reuse completed work: Saved summaries avoid repeated requests for unchanged articles.
  • Keep failures visible: A fallback description should not be mistaken for a successful AI summary.
  • Make ranking understandable: Rules are easier to explain and adjust when they are separate from the summarization step.

I have not measured how much reading time or API cost this saves. Those would be useful things to test, rather than infer from the design.

What I would improve

I’d expand each post’s description to provide a more complete summary, so readers can understand the key details without clicking through. I’d also add a “Read full report” link that opens a pop-up with the full article text, where permissions allow. This would let readers stay on the site and avoid the original publisher’s ads and cookie prompts.

I’d also make search more useful, with better relevance and filters for date, source, category, and CVE. I’d evaluate Elasticsearch for searching an index of the collected articles, or Google Programmable Search for searching publicly indexed pages. Before choosing either, I’d test how well it finds stories in the feed and whether the improvement justifies the added cost and maintenance.

Diagrams and dependency notes

The diagrams are available as a workflow PNG and technology-map PNG. Editable versions are available for the workflow and technology map.

The software bill of materials lists the build dependencies. The audit report and review notes record known dependency findings and the scope of that review; they are not a security certification.

Related posts

Engineering

Building NetHero to investigate calls that connect but have no audio