Skip to content

Add JSON Feed 1.0 and 1.1 parsing - #66

Merged
cardmagic merged 2 commits into
masterfrom
feature/json-feed
Sep 14, 2026
Merged

cardmagic merged 2 commits into
masterfrom
feature/json-feed

Conversation

@cardmagic

@cardmagic cardmagic commented Sep 14, 2026

Copy link
Copy Markdown
Owner

Add JSON Feed 1.0 and 1.1 to SimpleRSS.parse and SimpleRSS.fetch, so an importer can consume RSS, Atom, and JSON through normalized_entries. JSON feeds previously failed at the XML parser guard.

Closes #60. Based on latest master at 99e0882, including #65.

Example: parse and normalize JSON Feed

require "simple-rss"
require "json"

source = JSON.generate(
  version: "https://jsonfeed.org/version/1.1",
  title: "Example",
  authors: [{ name: "Example Editor" }],
  items: [{
    id: 42,
    url: "https://example.com/posts/42",
    content_text: "Hello from JSON Feed",
    date_published: "2026-09-12T10:00:00Z",
    tags: ["ruby", "feeds"],
    attachments: [{
      url: "https://example.com/episode.mp3",
      mime_type: "audio/mpeg",
      duration_in_seconds: 62.5
    }]
  }]
)

feed = SimpleRSS.parse(source)
entry = feed.normalized_entries.first

feed.feed_type                    # => :json_feed
entry.identifier                  # => "42"
entry.raw["id"]                   # => 42 (original value)
entry.content_text                # => "Hello from JSON Feed"
entry.authors.first[:name]         # => "Example Editor" (inherited)
entry.categories                  # => ["ruby", "feeds"]
entry.published_at                 # => a Time
entry.attachments.first[:media_type] # => "audio/mpeg"
entry.attachments.first[:duration_in_seconds] # => 62.5

The same consumer works for all three formats:

entries = %w[rss.xml atom.xml feed.json].flat_map do |path|
  File.open(path) { |source| SimpleRSS.parse(source).normalized_entries }
end
entries.each { |entry| puts [entry.title, entry.url, entry.content_text].compact }

Fetching and source preservation

feed = SimpleRSS.fetch("https://example.com/feed.json", timeout: 10)
feed.next_url   # Pagination metadata; reading it does not fetch another page
feed.raw_json   # Immutable decoded original, including extensions
feed.to_hash    # Ruby object export with compatibility aliases
feed.to_json    # JSON serialization of that object export

SimpleRSS.fetch("https://example.com/feed.json",
                etag: feed.etag, last_modified: feed.last_modified)
# => nil when the server responds with HTTP 304
  • Detection validates the JSON structure and supported version; Content-Type is only a server hint. Accept headers cover all supported formats and remain overridable.
  • Support empty feeds, titleless items, both content forms, inherited 1.0/1.1 authors, tags, images, language, and all attachment metadata. Publication and modification dates remain separate.
  • Supplied expiration flags must be booleans. Invalid required fields reject the feed with SimpleRSSError and a field path. Invalid optional dates/numbers remain raw and produce normalized issues; no IDs or dates are invented.
  • Preserve original JSON with string keys in raw_json and each normalized entry's raw; raw_xml is nil. UTF-8 input accepts one initial BOM before whitespace.
  • XML parsing, raw access, and serialization retain their contracts. JSON to_json is an object export, not a standards exporter; JSON to_xml fails explicitly. XML-only normalization mappings are rejected for JSON.
  • Update README, architecture guidance, and the unreleased changelog. Use only standard library facilities.

Validation

  • Public API regression reproduced the previous Poorly formatted feed error, then passed after implementation.
  • Full suite: 187 tests, 803 assertions, zero failures/errors; three existing external-network omissions.
  • Spec-derived 1.0/1.1 fixtures; equivalent RSS/Atom/JSON core fields; malformed structures, opaque/numeric IDs, dates, inheritance, immutable raw data, extensions, and serialization.
  • Local HTTP tests cover varied/mislabeled Content-Type, relative redirects, custom Accept, conditional 304, and no pagination/article/attachment requests.
  • RuboCop, RBS generation/validation, and Steep pass.
  • Shared digest example runs against RSS, Atom, and JSON fixtures. Built gem includes both new implementation files.

Let importers consume JSON Feed 1.0 and 1.1 through the existing parser,
fetcher, and normalized entry interface. Preserve source metadata and
extensions while recovering optional date and attachment number errors.

Keep XML parsing and serialization compatible. Document required-field
errors, author inheritance, raw JSON snapshots, and object export rules.
Verify format parity, local HTTP behavior, and the shared digest example.

Closes #60
@cardmagic

Copy link
Copy Markdown
Owner Author

@greptileai Please review current head 57d19652a20300266b0a7973218655f3cf6117c2 for #60, including JSON Feed validation, compatibility, author inheritance, raw serialization, and HTTP integration. Please provide an updated confidence score.

Comment thread lib/simple-rss/json_feed.rb
Comment thread lib/simple-rss/json_feed.rb
Comment thread lib/simple-rss/json_feed.rb
Comment thread lib/simple-rss/json_feed.rb
Comment thread lib/simple-rss/json_feed.rb
Comment thread lib/simple-rss/json_feed.rb
Comment thread lib/simple-rss/json_feed.rb
Comment thread lib/simple-rss/json_feed.rb
Comment thread lib/simple-rss/json_feed.rb
Comment thread lib/simple-rss/json_feed.rb
@cardmagic

cardmagic commented Sep 14, 2026

Copy link
Copy Markdown
Owner Author

Reviewed both batches of Hound style threads against .rubocop.yml: this repository requires double quotes and permits 160-character source lines, ABC size 80, cyclomatic complexity 30, method length 60, and perceived complexity 35. The cited lines/methods are within those limits, and the actual RuboCop CI check passes. Resolved those configuration mismatches.

@greptile-apps

greptile-apps Bot commented Sep 14, 2026

Copy link
Copy Markdown

Greptile Summary

Adds JSON Feed 1.0 and 1.1 support while preserving the existing RSS and Atom interfaces.

  • Detects, validates, and parses JSON feeds through SimpleRSS.parse and SimpleRSS.fetch.
  • Normalizes JSON items, authors, tags, dates, URLs, and attachments into immutable normalized entries.
  • Preserves original JSON and extensions while providing compatibility aliases and explicit serialization behavior.
  • Validates expired as a JSON boolean and uses flat guards in JSON normalization, resolving both previous findings.
  • Adds comprehensive parser, normalization, serialization, and HTTP-fetch coverage.

Confidence Score: 5/5

The PR appears safe to merge; both previous findings are resolved and no new actionable issue remains.

The expiration field now rejects every supplied non-boolean value while preserving true, false, and absence correctly. JSON normalization also uses independent guard clauses rather than nesting the JSON path, fully addressing the prior conditional-structure concern.

Important Files Changed

Filename Overview
lib/simple-rss.rb Dispatches JSON bodies to the new parser, exposes JSON metadata, preserves XML behavior, and adds JSON-aware normalization and fetching.
lib/simple-rss/json_feed.rb Validates JSON Feed 1.0/1.1 structures, including boolean expiration values, preserves raw documents, and builds compatibility items.
lib/simple-rss/json_entry_normalizer.rb Maps JSON Feed items into immutable normalized entries with URL, author, date, category, and attachment handling.
lib/simple-rss/normalized_entry.rb Extends normalized entries with JSON Feed-specific URL, image, and language fields.
test/base/json_feed_test.rb Covers valid and malformed feeds, expiration typing, normalization, inheritance, immutable raw data, and serialization contracts.
test/base/normalized_fetch_test.rb Covers JSON response detection, redirects, custom Accept headers, conditional requests, and linked-resource isolation.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[SimpleRSS.parse or fetch] --> B{JSON-like body?}
    B -->|No| C[Existing XML parser]
    B -->|Yes| D[Parse and validate JSON Feed]
    D --> E[Preserve immutable raw JSON]
    D --> F[Build compatibility item view]
    F --> G[Normalize entries]
    G --> H[Resolve URLs and authors]
    G --> I[Parse dates and attachments]
    G --> J[Return immutable NormalizedEntry]
Loading

Reviews (2): Last reviewed commit: "fix: validate JSON Feed expiration flags" | Re-trigger Greptile

Comment thread lib/simple-rss/json_feed.rb
Comment thread lib/simple-rss.rb Outdated
Reject non-boolean expiration values so a string such as "false" cannot
be mistaken for an expired feed. Preserve false in raw and serialized
metadata and cover malformed values with a regression test.

Flatten JSON normalization guards to follow the repository style.
Comment thread lib/simple-rss/json_entry_normalizer.rb
Comment thread lib/simple-rss/json_entry_normalizer.rb
Comment thread lib/simple-rss/json_entry_normalizer.rb
Comment thread lib/simple-rss/json_entry_normalizer.rb
Comment thread lib/simple-rss/json_entry_normalizer.rb
Comment thread lib/simple-rss/json_entry_normalizer.rb
Comment thread lib/simple-rss/json_entry_normalizer.rb
Comment thread lib/simple-rss/json_entry_normalizer.rb
Comment thread lib/simple-rss/json_entry_normalizer.rb
Comment thread lib/simple-rss/json_entry_normalizer.rb
@cardmagic

Copy link
Copy Markdown
Owner Author

@greptileai Please review current head 45f74580579ba6e29d6d07680d8db5cb45651abd and provide an updated confidence score. Both findings are fixed: supplied expired values are validated as booleans (with a reproduced failing regression covering strings, numbers, null, arrays, and objects; true/false/absence and false serialization pass), and JSON normalization uses flat guards. Full suite: 187 tests / 803 assertions, zero failures or errors; RuboCop, RBS, and Steep pass.

@cardmagic
cardmagic merged commit 3c1f114 into master Sep 14, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add JSON Feed 1.0 and 1.1 parsing

2 participants