Data Twingler Smart Agent Service

kidehen · January 17, 2024, 10:10pm

The Smart Agent edition of Data Twingler is a sophisticated service that enables querying of public or private data (subject to data access controls) using declarative query languages such as SQL, SPARQL, SPASQL, and GraphQL directly from a ChatGPT instance. It is available as either an OpenLink Personal Assistant (OPAL) module or as a Custom GPT, and it can be accessed via the OpenAI GPT Store.

opal-chatgpt-architecture

Architecture

Data Twingler’s Smart Agent edition offers a sophisticated service for querying both public and private data, adhering to fine-grained access controls. It supports declarative query languages such as SQL, SPARQL, SPASQL, and GraphQL, and can be integrated directly into a ChatGPT instance. This edition is available as an OpenLink Personal Assistant (OPAL) module or as a Custom GPT, accessible through the OpenAI GPT Store.

Architecture Diagram

Features and Benefits

Feature	Benefit
Multi-platform availability as a Custom GPT or OpenLink Personal Assistant (OPAL) Module	Deployment options provide flexibility across diverse teams with a variety of user profiles.
Available for direct use as a Web Service, via an OpenAPI-compliant YAML or JSON document	Enables functionality for a variety of solutions in a loosely coupled fashion, reducing cost-of-ownership and enhancing productivity for users, developers, and integrators.
Multiple query language support (SQL, SPARQL, SPASQL, and GraphQL)	Offers the freedom to choose the best query language for the task at hand, catering to different skill sets.
Data Access with fine-grained access controls	Integrates with sophisticated data governance policies using attribute-based access controls.
Incorporates Web and Internet traversal into the query solution production pipeline	Facilitates the use and integration of data from external sources and knowledge bases, such as the Linked Open Data (LOD) Cloud, which includes Knowledge Graphs curated by domain experts.

Usage

Enabling other smart agents to leverage the sophisticated capabilities of declarative query languages for advanced data access, integration, and management operations. For instance, the creation of query templates that can be used to reduce hallucinations via model fine tuning and/or retrieval augmented generation (RAG) targeting databases and knowledge graphs.

Custom GPT Usage Example

Integration Architecture

Query services for SQL, SPARQL, SPASQL, and GraphQL are mapped to Actions in ChatGPT via a OpenAPI-compliant YAML document that describe invocation arguments, endpoints, and response data formats.

Usage Example via ChatGPT User Interface

You can start by clicking on one of the conversation openers.

Alternatively, you can prompt as per usual e.g., using some of the examples from the next section with regards usage via OPAL.

OPAL Usage Example via Session Transcript

Assistant Response:

Hello! As your Query Processing Configuration Agent, I’m here to help you manage and optimize your query processing settings. Here are the current settings:

SPARQL:

Default Endpoint: Protected SPARQL Query Service

Result Format: application/sparql-results+json

Timeout: 30000

Max Results: 10

Tabulate Results: True

SPARQL-FED:

Default Endpoint: Protected SPARQL Query Service

Service Block Limit: 10

Service Block Order By: True

Tabulate Results: True

SPASQL:

Use Federation: True

Federation Timeout: 30000

Tabulate Results: True

SQL:

Default Schema: Demo

Batch Size: 1000

Query Timeout: 30000

Tabulate Results: True

GraphQL:

Default Endpoint: https://linkeddata.uriburner.com/graphql

Query Depth Limit: 10

Introspection: True

Tabulate Results: True

You can update these settings using the command /update_settings [setting_name] [new_value]. To view the current settings, use the command /show_settings. You can also execute a test query to validate the current settings with /test_query [query_type] [query_content].

Remember, I’m here to provide expert advice on optimizing query performance. So, feel free to ask for any help or guidance.

User Prompt:
Write and execute a sample SQL Query.

Query generated by SQL Processing Service

{
    "sql": "SELECT TOP 10 * FROM Demo.Demo.Customers",
    "max_rows": 10,
    "timeout": 30000
}

Raw data returned by Query Service

[["ALFKI","Alfreds Futterkiste","Maria Anders","Sales Representative","Obere Str. 57","Berlin",null,"12209","Germany",null,"030-0074321","030-0076545"],["ANATR","Ana Trujillo Emparedados y helados","Ana Trujillo","Owner","Avda. de la Constituci?n 2222","M?xico D.F.",null,"05021","Mexico",null,"(5) 555-4729","(5) 555-3745"],["ANTON","Antonio Moreno Taquer?a","Antonio Moreno","Owner","Mataderos  2312","M?xico D.F.",null,"05023","Mexico",null,"(5) 555-3932",null],["AROUT","Around the Horn","Thomas Hardy","Sales Representative","120 Hanover Sq.","London",null,"WA1 1DP","United Kingdom",null,"(171) 555-7788","(171) 555-6750"],["BERGS","Berglunds snabbk?p","Christina Berglund","Order Administrator","Berguvsv?gen  8","Lule?",null,"S-958 22","Sweden",null,"0921-12 34 65","0921-12 34 67"],["BLAUS","Blauer See Delikatessen","Hanna Moos","Sales Representative","Forsterstr. 57","Mannheim",null,"68306","Germany",null,"0621-08460","0621-08924"],["BLONP","Blondel p?re et fils","Fr?d?rique Citeaux","Marketing Manager","24, place Kl?ber","Strasbourg",null,"67000","France",null,"88.60.15.31","88.60.15.32"],["BOLID","B?lido Comidas preparadas","Mart?n Sommer","Owner","C/ Araquil, 67","Madrid",null,"28023","Spain",null,"(91) 555 22 82","(91) 555 91 99"],["BONAP","Bon app'","Laurence Lebihan","Owner","12, rue des Bouchers","Marseille",null,"13008","France",null,"91.24.45.40","91.24.45.41"],["BOTTM","Bottom-Dollar Markets","Elizabeth Lincoln","Accounting Manager","23 Tsawassen Blvd.","Tsawassen","BC","T2F 8M4","Canada",null,"(604) 555-4729","(604) 555-3745"]]

Data Twingler Response:
Here are the top 10 records from the Customers table in the Demo Database:

CustomerID	CompanyName	ContactName	ContactTitle	Address	City	PostalCode	Country	Phone	Mobile
ALFKI	Alfreds Futterkiste	Maria Anders	Sales Representative	Obere Str. 57	Berlin	12209	Germany	030-0074321	030-0076545
ANATR	Ana Trujillo Emparedados y helados	Ana Trujillo	Owner	Avda. de la Constituci?n 2222	M?xico D.F.	05021	Mexico	(5) 555-4729	(5) 555-3745
ANTON	Antonio Moreno Taquer?a	Antonio Moreno	Owner	Mataderos 2312	M?xico D.F.	05023	Mexico	(5) 555-3932
AROUT	Around the Horn	Thomas Hardy	Sales Representative	120 Hanover Sq.	London	WA1 1DP	United Kingdom	(171) 555-7788	(171) 555-6750
BERGS	Berglunds snabbk?p	Christina Berglund	Order Administrator	Berguvsv?gen 8	Lule?	S-958 22	Sweden	0921-12 34 65	0921-12 34 67
BLAUS	Blauer See Delikatessen	Hanna Moos	Sales Representative	Forsterstr. 57	Mannheim	68306	Germany	0621-08460	0621-08924
BLONP	Blondel p?re et fils	Fr?d?rique Citeaux	Marketing Manager	24, place Kl?ber	Strasbourg	67000	France	88.60.15.31	88.60.15.32
BOLID	B?lido Comidas preparadas	Mart?n Sommer	Owner	C/ Araquil, 67	Madrid	28023	Spain	(91) 555 22 82	(91) 555 91 99
BONAP	Bon app’	Laurence

User Prompt:
Write and execute a sample SPARQL Query.

Query generated by SPARQL Processing Service

{
    "query": "SELECT DISTINCT (SAMPLE(?s) AS ?entitySample) (COUNT(*) AS ?count) (?o as ?entityType) WHERE {?s a ?o} GROUP BY ?o ORDER BY DESC (?count) LIMIT 10",
    "format": "application/sparql-results+json"
}

Raw data returned by SPARQL Query Service


{ "head": { "link": [], "vars": ["entitySample", "count", "entityType"] },
  "results": { "distinct": false, "ordered": true, "bindings": [
    { "entitySample": { "type": "uri", "value": "http://purl.org/net/provenance/ns#completedAt" }	, "count": { "type": "typed-literal", "datatype": "http://www.w3.org/2001/XMLSchema#integer", "value": "1421034" }	, "entityType": { "type": "uri", "value": "http://www.w3.org/2002/07/owl#DatatypeProperty" }},
    { "entitySample": { "type": "uri", "value": "http://disqus.com/kidehen#rdf_load_htmlvxc_provenance" }	, "count": { "type": "typed-literal", "datatype": "http://www.w3.org/2001/XMLSchema#integer", "value": "99165" }	, "entityType": { "type": "uri", "value": "http://purl.org/net/provenance/types#QueryResult" }},
    { "entitySample": { "type": "uri", "value": "http://dbpedia.org/resource/Anastasios_Venetsanopoulos" }	, "count": { "type": "typed-literal", "datatype": "http://www.w3.org/2001/XMLSchema#integer", "value": "83442" }	, "entityType": { "type": "uri", "value": "http://www.w3.org/2002/07/owl#Thing" }},
    { "entitySample": { "type": "uri", "value": "http://linkeddata.uriburner.com/about/id/https/www.childrensartmuseum.gr/LD-project/rdfa/inspired-by-artists/painting02.html#mime" }	, "count": { "type": "typed-literal", "datatype": "http://www.w3.org/2001/XMLSchema#integer", "value": "54630" }	, "entityType": { "type": "uri", "value": "http://purl.org/dc/terms/MediaTypeOrExtent" }},
    { "entitySample": { "type": "uri", "value": "https://raw.githubusercontent.com/w3c/hcls-fhir-rdf/gh-pages/ontology/FHIR.ttl#Axiom" }	, "count": { "type": "typed-literal", "datatype": "http://www.w3.org/2001/XMLSchema#integer", "value": "34304" }	, "entityType": { "type": "uri", "value": "http://www.w3.org/2002/07/owl#Axiom" }},
    { "entitySample": { "type": "bnode", "value": "nodeID://b4347873" }	, "count": { "type": "typed-literal", "datatype": "http://www.w3.org/2001/XMLSchema#integer", "value": "30421" }	, "entityType": { "type": "uri", "value": "http://topbraid.org/sxml#TextNode" }},
    { "entitySample": { "type": "uri", "value": "http://finregont.com/fro/cfr/CFR-2015-title12-vol3-part225.ttl#r-1-18-3-327" }	, "count": { "type": "typed-literal", "datatype": "http://www.w3.org/2001/XMLSchema#integer", "value": "5754" }	, "entityType": { "type": "uri", "value": "http://finregont.com/fro/cfr/CFR_FDSys_Schema.ttl#P" }},
    { "entitySample": { "type": "uri", "value": "http://linkeddata.uriburner.com/proxy-iri/001ce81318478dc8a1350eb6d69c945dfa90ccd3" }	, "count": { "type": "typed-literal", "datatype": "http://www.w3.org/2001/XMLSchema#integer", "value": "4407" }	, "entityType": { "type": "uri", "value": "https://corgis-edu.github.io/corgis/datasets/csv/airlines/airlines.csv#class" }},
    { "entitySample": { "type": "uri", "value": "http://finregont.com/fro/usc/USC-2015-title15-chapter2D.ttl#r-1-0-3-0-1-0-0" }	, "count": { "type": "typed-literal", "datatype": "http://www.w3.org/2001/XMLSchema#integer", "value": "3543" }	, "entityType": { "type": "uri", "value": "http://finregont.com/fro/usc/USC_USLM_Schema.ttl#ref" }},
    { "entitySample": { "type": "uri", "value": "https://linkeddata.uriburner.com/proxy-iri/7f80f0442fedf762d4d8d0eba0e221650f375898" }	, "count": { "type": "typed-literal", "datatype": "http://www.w3.org/2001/XMLSchema#integer", "value": "3432" }	, "entityType": { "type": "uri", "value": "https://github.com/CSSEGISandData/COVID-19/raw/master/csse_covid_19_data/csse_covid_19_daily_reports/03-28-2020.csv#class" }} ] } }

Assistant Response:
Here are the top 10 entity types based on the count of their instances:

Entity Sample	Count	Entity Type
http://purl.org/net/provenance/ns#completedAt	1421034	http://www.w3.org/2002/07/owl#DatatypeProperty
http://disqus.com/kidehen#rdf_load_htmlvxc_provenance	99165	http://purl.org/net/provenance/types#QueryResult
http://dbpedia.org/resource/Anastasios_Venetsanopoulos	83442	http://www.w3.org/2002/07/owl#Thing
http://linkeddata.uriburner.com/about/id/https/www.childrensartmuseum.gr/LD-project/rdfa/inspired-by-artists/painting02.html#mime	54630	http://purl.org/dc/terms/MediaTypeOrExtent
https://raw.githubusercontent.com/w3c/hcls-fhir-rdf/gh-pages/ontology/FHIR.ttl#Axiom	34304	http://www.w3.org/2002/07/owl#Axiom
nodeID://b4347873	30421	http://topbraid.org/sxml#TextNode
http://finregont.com/fro/cfr/CFR-2015-title12-vol3-part225.ttl#r-1-18-3-327	5754	http://finregont.com/fro/cfr/CFR_FDSys_Schema.ttl#P
http://linkeddata.uriburner.com/proxy-iri/001ce81318478dc8a1350eb6d69c945dfa90ccd3	4407	https://corgis-edu.github.io/corgis/datasets/csv/airlines/airlines.csv#class
http://finregont.com/fro/usc/USC-2015-title15-chapter2D.ttl#r-1-0-3-0-1-0-0	3543	http://finregont.com/fro/usc/USC_USLM_Schema.ttl#ref
https://linkeddata.uriburner.com/proxy-iri/7f80f0442fedf762d4d8d0eba0e221650f375898	3432	https://github.com/CSSEGISandData/COVID-19/raw/master/csse_covid_19_data/csse_covid_19_daily_reports/03-28-2020.csv#class

Please note that the links are clickable and will take you to the respective resources.

User Prompt:
Write and execute a sample SPARQL-FED Query.

Query generated by SPARQL-FED Processing Service

{
    "query": "PREFIX dbr: <http://dbpedia.org/resource/> PREFIX dbo: <http://dbpedia.org/ontology/> SELECT * WHERE { SERVICE <http://dbpedia.org/sparql> { ?movie rdf:type dbo:Film ; dbo:director dbr:Spike_Lee . } }",
    "format": "application/sparql-results+json"
}

Raw data returned by SPARQL-FED Query Service


{ "head": { "link": [], "vars": ["movie"] },
  "results": { "distinct": false, "ordered": true, "bindings": [
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Rodney_King_(film)" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/School_Daze" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/BlacKkKlansman" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Crooklyn" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Clockers_(film)" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Freak_(film)" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Get_on_the_Bus" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Girl_6" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Miracle_at_St._Anna" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Mo'_Better_Blues" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/The_Original_Kings_of_Comedy" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Malcolm_X_(1992_film)" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Bad_25_(film)" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Bamboozled" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/When_the_Levees_Broke" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/25th_Hour" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/4_Little_Girls" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/American_Utopia_(film)" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Da_5_Bloods" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Da_Sweet_Blood_of_Jesus" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/He_Got_Game" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Sucker_Free_City" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/A_Huey_P._Newton_Story" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Chi-Raq" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Jim_Brown:_All-American" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Joe's_Bed-Stuy_Barbershop:_We_Cut_Heads" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Jungle_Fever" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Summer_of_Sam" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Do_the_Right_Thing" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Kobe_Doin'_Work" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Michael_Jackson's_Journey_from_Motown_to_Off_the_Wall" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Oldboy_(2013_film)" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Red_Hook_Summer" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/She's_Gotta_Have_It" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/She_Hate_Me" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Pass_Over__Pass_Over__1" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/Inside_Man__Inside_Man__1" }},
    { "movie": { "type": "uri", "value": "http://dbpedia.org/resource/All_the_Invisible_Children__Jesus_Children_of_America__1" }} ] } }

Assistant Response:
Here are some movies directed by Spike Lee:

Please note that the links are clickable and will take you to the respective resources.

User Prompt:
Write and execute a sample SPASQL Query.

Query generated by SPASQL Processing Service

{
    "sql": "SELECT movie FROM (SPARQL PREFIX dbr: <http://dbpedia.org/resource/> PREFIX dbo: <http://dbpedia.org/ontology/> SELECT ?movie WHERE { SERVICE <http://dbpedia.org/sparql> { ?movie rdf:type dbo:Film ; dbo:director dbr:Spike_Lee . } }) AS movies",
    "max_rows": 10,
    "timeout": 30000
}

Raw data returned by SPASQL Query Service

[["http://dbpedia.org/resource/Rodney_King_(film)"],["http://dbpedia.org/resource/School_Daze"],["http://dbpedia.org/resource/BlacKkKlansman"],["http://dbpedia.org/resource/Crooklyn"],["http://dbpedia.org/resource/Clockers_(film)"],["http://dbpedia.org/resource/Freak_(film)"],["http://dbpedia.org/resource/Get_on_the_Bus"],["http://dbpedia.org/resource/Girl_6"],["http://dbpedia.org/resource/Miracle_at_St._Anna"],["http://dbpedia.org/resource/Mo'_Better_Blues"]]

Assistant Response:
Here are some movies directed by Spike Lee:

User Prompt:
Write and execute a sample GraphQL Query.

Query generated by GraphQL Processing Service

{
    "query": "query MyQuery { Movies { iri name description director  { name } } }",
    "url": "https://linkeddata.uriburner.com/graphql"
}

Raw data returned by GraphQL Query Service

{"data":{"Movies":[{"iri":"https://www.metacritic.com/movie/the-menu#Movie","name":"The Menu","description":"A couple (Anya Taylor-Joy and Nicholas Hoult) travels to a coastal island to eat at an exclusive restaurant where the chef (Ralph Fiennes) has prepared a lavish menu, with some shocking surprises. ","director":[{"name":"Mark Mylod"}]},{"iri":"https://www.imdb.com/title/tt9114286/#Movie","name":"Black Panther: Wakanda Forever","description":"The people of Wakanda fight to protect their home from intervening world powers as they mourn the death of King T&apos;Challa.","director":[{"name":"Ryan Coogler"}]},{"iri":"https://www.rottentomatoes.com/m/black_panther_wakanda_forever#Movie","name":"Black Panther: Wakanda Forever","description":null,"director":[{"name":"NA"}]}]}}

Assistant Response:
Here are some movies and their details:

The Menu
- Description: A couple (Anya Taylor-Joy and Nicholas Hoult) travels to a coastal island to eat at an exclusive restaurant where the chef (Ralph Fiennes) has prepared a lavish menu, with some shocking surprises.
- Director: Mark Mylod
Black Panther: Wakanda Forever
- Description: The people of Wakanda fight to protect their home from intervening world powers as they mourn the death of King T’Challa.
- Director: Ryan Coogler
Black Panther: Wakanda Forever

Description: Not available
Director: Not available

Frequently Asked Questions

What is the Data Twingler?
The Data Twingler is a component that adds advanced query execution capabilities to ChatGPT, supporting multiple declarative query languages and enabling the creation of smart agents and personal assistants.

How does the Data Twingler integrate with ChatGPT?
It can be integrated as a Plugin, Custom GPT, or as a module within a ChatGPT-based service such as the OpenLink Software Personal Assistant (OPAL).

What query languages are supported by the Data Twingler?
SQL, SPARQL, SPASQL (SPARQL inside SQL), and GraphQL.

Can the Data Twingler access private databases?
Yes, it can access both public and private databases, subject to fine-grained access controls, via protocols such as HTTP, ODBC, or JDBC.

Is the Data Twingler available for direct use as a Web Service?
Yes, it is available for direct use as a Web Service and is compatible with OpenAPI-compliant YAML or JSON description documents. In fact, this is how its exposed as a CustomGPT available from the GPT Store.

How does the Data Twingler service contribute to minimizing hallucination in Large Language Models (LLMs) through Retrieval Augmented Generation?
The Data Twingler service plays a crucial role in this process by facilitating the creation of specialized query templates. These templates effectively map prompts to specific queries targeted at databases or knowledge graphs. This capability is significant because it underpins ChatGPT’s built-in functionality for analyzing sentence similarity. This analysis is a critical step towards the expansion and execution of query templates. By doing so, the Data Twingler aids in ensuring that the responses generated by the LLM are more accurate and grounded in verified information, thereby reducing the likelihood of hallucinations.