Guide to Democratizing News Curation, Subscription, and Reading

Virtuoso Universal Server Artificial Intelligence (AI)
, , , ,
1

What is Democratizing News Curation?

Democratizing news curation involves making the process of selecting, subscribing to, and reading news more accessible and personalized for everyone. This is achieved by leveraging open standards such as RSS, OPML, and the Web stack, aided by Large Language Model (LLM)-based AI applications like ChatGPT.

Why Democratize News Curation?

The traditional news curation landscape has been dominated by big tech companies, leading to information silos and limited user control. Democratizing news curation offers several benefits:

  • User Empowerment: Allows individuals to select and curate their preferred news sources.
  • Transparency: Open standards ensure that the process is transparent and not controlled by a single entity.
  • Personalization: AI tools can tailor news feeds to individual preferences, enhancing the reading experience.
  • Interoperability: Open standards like RSS and OPML ensure that news feeds can be easily shared and integrated across different platforms.

How to Democratize News Curation Leveraging Large Language Models (LLMs) that support External Function Integration

This guide walks you through the process of creating a collection of predefined query templates that are ultimately used to configure an OPML and RSS Reader Assistant. This assistant can be used from OpenAI’s native interfaces for ChatGPT (once discovered and enabled via the GPT Store using this link) or through interfaces provided by the [OpenLink AI Layer (OPAL)][https://linkeddata.uriburner.com/chat/], selecting the “OPML & RSS News Reader Assistant”.

OpenLink OPML & RSS News Reader Assistant Demo

Step-by-Step Guide for Building Assistant

  1. Identify News Sources: Determine the news sources you want to include in your feed, such as blogs, news websites, and other content providers that support RSS or Atom feeds.
  2. Use Open Standards: Leverage open standards like RSS and OPML to structure and organize your news feeds.
  3. Integrate AI Tools: Use AI tools like ChatGPT to enhance your news curation process. These tools can help personalize feeds, summarize articles, and provide deeper insights.
  4. Set Up a Feed Reader: Use a feed reader to aggregate and display your curated news feeds. This can be a standalone application or a web-based service.
  5. Customize and Optimize: Continuously refine your news sources and feed settings to better match your interests and preferences.

Example Use Case

Demonstrating the Power of Democratized News Curation

By leveraging tools like RSS, Atom, and OPML, users can showcase how structured data can be uploaded to a knowledge base or knowledge graph. This allows for predefined prompt templates that trigger feed exploration in various modes, all enhanced by AI tools like ChatGPT.

System Architecture

Predefined Prompt Templates for Retrieval Augmented Generation (RAG) based Fine-tuning

To drive feed processing, predefined {prompt}:{action} templates are used. These templates are processed by ChatGPT as part of a Retrieval Augmented Generation (RAG) based prompt processing pipeline. Here are some examples:

  1. Explore the OPML News Feed Knowledge Graph denoted by the URL {url}: {feed-query}

    • Action: Reads an OPML feed denoted by a URL.

  2. Explore the latest edition of the OPML News Feed Knowledge Graph denoted by the URL {url}: {feed-query}

    • Action: Reads an OPML feed denoted by a URL, but in a change-sensitive manner using HTTP cache invalidation.

  3. Explore the RSS News Feed Knowledge Graph denoted by the URL {url}: {feed-query}

    • Action: Reads an RSS feed denoted by a URL.

  4. Explore the latest edition of the RSS News Feed Knowledge Graph denoted by the URL {url}: {feed-query}

    • Action: Reads an RSS feed denoted by a URL, but in a change-sensitive manner using HTTP cache invalidation.

How-To Guides

How to Use RSS Feeds

  1. Find an RSS feed URL.
  2. Use a predefined prompt template to craft a prompt for reading the feed.
  3. Read the feed presented in the prompt response.

How to Use Atom Feeds

  1. Find an Atom feed URL.
  2. Use a predefined prompt template to craft a prompt for reading the feed.
  3. Read the feed presented in the prompt response.

How to Use SPARQL in a predefined prompt template

  1. Write a SPARQL query draft.
  2. Execute the query via a SPARQL endpoint.
  3. Incorporate query into a {prompt}:{query} template
SPARQL Query Template Example – Cached Query Results

Explore the OPML News Feed Knowledge Graph denoted by the URL {url}:

    # DEFINE get:soft "soft"
    # DEFINE input:grab-var "feed"
    # DEFINE input:grab-var "blog"

    PREFIX schema: <http://schema.org/>
    PREFIX sioc: <http://rdfs.org/sioc/ns#>

    SELECT DISTINCT ?feed ?blog ?post ?pubDate ?postTitle 
    FROM <{url}>
    WHERE { 
            ?s a schema:DataFeed ;
               sioc:link ?feed .  
            GRAPH ?g { ?feed schema:mainEntity ?blog. 
                       ?blog schema:dataFeedElement ?post.
                       ?post schema:title ?postTitle ;
                             schema:datePublished ?pubDate. 
                 } 
          } 
    ORDER BY DESC (?pubDate) 
    LIMIT 200
Query Template Example – Change-Sensitive Query Results
    DEFINE get:soft "soft"
    DEFINE input:grab-var "feed"
    DEFINE input:grab-var "blog"

    PREFIX schema: <http://schema.org/>
    PREFIX sioc: <http://rdfs.org/sioc/ns#>

    SELECT DISTINCT ?feed ?blog ?post ?pubDate ?postTitle 
    FROM <{url}>
    WHERE { 
            ?s a schema:DataFeed ;
               sioc:link ?feed .  
            GRAPH ?g { ?feed schema:mainEntity ?blog. 
                       ?blog schema:dataFeedElement ?post.
                       ?post schema:title ?postTitle ;
                             schema:datePublished ?pubDate. 
                 } 
          } 
    ORDER BY DESC (?pubDate) 
    LIMIT 200

Mermaid Entity Relationship Graph Visualization Derived from SPARQL Query
SPARQL Query Visualization

How to Use SQL in a predefined prompt template

  1. Write a SQL query draft.
  2. Execute the query via a SQL endpoint.
  3. Incorporate query into a {prompt}:{query} template
SQL Query Template Example – Cached Query Results

Explore the OPML News Feed Knowledge Graph denoted by the URL {url}:

SELECT news.feed, news.blog, news.post, news.pubDate, news.postTitle
FROM (
    SPARQL 
    # DEFINE get:soft "soft"
    # DEFINE input:grab-var "feed"
    # DEFINE input:grab-var "blog"

    PREFIX schema: <http://schema.org/>
    PREFIX sioc: <http://rdfs.org/sioc/ns#>

    SELECT DISTINCT ?feed ?blog ?post ?pubDate ?postTitle 
    FROM <{url}>
    WHERE { 
            ?s a schema:DataFeed ;
               sioc:link ?feed .  
            GRAPH ?g { ?feed schema:mainEntity ?blog. 
                       ?blog schema:dataFeedElement ?post.
                       ?post schema:title ?postTitle ;
                             schema:datePublished ?pubDate. 
                 } 
          } 
    ORDER BY DESC (?pubDate) 
    LIMIT 200
) AS news
Query Template Example – Change-Sensitive Query Results

Explore the OPML News Feed Knowledge Graph denoted by the URL {url}:

SELECT news.feed, news.blog, news.post, news.pubDate, news.postTitle
FROM (
    SPARQL 
    DEFINE get:soft "soft"
    DEFINE input:grab-var "feed"
    DEFINE input:grab-var "blog"

    PREFIX schema: <http://schema.org/>
    PREFIX sioc: <http://rdfs.org/sioc/ns#>

    SELECT DISTINCT ?feed ?blog ?post ?pubDate ?postTitle 
    FROM <{url}>
    WHERE { 
            ?s a schema:DataFeed ;
               sioc:link ?feed .  
            GRAPH ?g { ?feed schema:mainEntity ?blog. 
                       ?blog schema:dataFeedElement ?post.
                       ?post schema:title ?postTitle ;
                             schema:datePublished ?pubDate. 
                 } 
          } 
    ORDER BY DESC (?pubDate) 
    LIMIT 200
) AS news

Mermaid Entity Relationship Model Visualization Derived from SQL Query
SPARQL Query Visualization

How to Query a News Feed Knowledge Graph using Natural Language

  1. Identify the knowledge graph endpoint.
  2. Write a query using SQL or SPARQL.
  3. Execute the query and analyze the results.

Prompt and Response Example
Query Results

Underlying Query Presentation
SPASQL Query

Example: Querying RSS & OPML News Feeds Knowledge Graphs
  1. Explore the OPML news feed denoted by https://feedland.social/opml?screenname=davewiner&catname=blogroll – this is change insensitive i.e., the result is produced from cache.
  2. Explore the latest edition of the OPML news feed denoted by https://feedland.social/opml?screenname=davewiner&catname=blogroll – this is change sensitive i.e., goes back to the source, courtesy of built-in cache invalidation.
  3. Explore the latest edition of the RSS news source denoted by http://scripting.com/rss.xml
  4. Explore the RSS news source denoted by http://scripting.com/rss.xml

Frequently Asked Questions (FAQ)

Q: What problem is this solving?

  • A: Selecting and curating preferred news sources.

Q: What’s the underlying architecture at play here?

  • A: Loose coupling of Large Language Models (LLMs) and structured data sources like HTML, RSS, Atom, OPML, RDF, etc.

Q: Is this system demonstrating LLM fine-tuning using retrieval augmented generation of responses (RAG)?

  • A: Yes, but with a twist: it can query structured dataspace using SQL, SPARQL, or GraphQL.

Q: How can structured data improve news curation?

  • A: Structured data allows for more accurate and efficient filtering and presentation of news content.

Q: What are the benefits of using open standards for data representation?

  • A: Open standards ensure interoperability, flexibility, and longevity of data.

Q: How does SPARQL differ from SQL?

  • A: SPARQL is used for querying RDF data, while SQL is used for querying relational databases.

Q: What role does RDF play in data interchange?

  • A: RDF provides a standard model for data interchange on the web.

Q: How can AI tools like ChatGPT enhance news reading?

  • A: AI tools can personalize news feeds, summarize articles, and provide deeper insights.

Glossary of Key Terms

  • RSS: Really Simple Syndication, a type of web feed.
  • Atom: Another web feed format.
  • Feed Reader: A tool for reading web feeds.
  • OPML: Outline Processor Markup Language, an XML format for outlines.
  • LLM: Large Language Model, a type of AI model for natural language processing, translation, and code generation.
  • SPARQL: SPARQL Protocol and RDF Query Language for operating on RDF-based structured data.
  • SQL: Structured Query Language, used for managing data in relational databases.
  • GraphQL: A query language for APIs.
  • Knowledge Graph: Machine-computable entities and their relationship types that aids reasoning and inference.
  • RDF: Resource Description Framework, an abstract data definition language that’s loosely coupled with a variety of notations and formats.
  • OPAL: Open Protocol for Accessing Linked Data.
  • Knowledge Base: A centralized repository for information that aids reasoning and inference.
  • Data Space: A conceptual abstraction atop database management systems (DBMS), knowledge base or knowledge graph management systems, and file or document management systems.
  • API: Application Programming Interface, a set of rules for building software.
  • Retrieval Augmented Generation (RAG): A technique for enhancing Large Language Model (LLM) responses through loose coupling with Data Spaces, facilitated by declarative queries that may or may not include full-text or vector search, stored procedures, functions, or application programming interfaces (APIs).

Related