Multimodal input is a crucial feature of LLM-based natural language processors, and it’s increasingly becoming accessible to the public. In this article, I’ll demonstrate how the publicly available Bing+ChatGPT effectively processes image input, subsequently generating output in Mermaid Sequence Diagram Notation.
The image used comes from a recent Tweet about titled Top 4 Forms of Authentication Mechanisms.
Transcript
Prompt:
How many distinct boxes (sections) are there in the attached image?
Response:
There are four distinct boxes (sections) in the attached image. Each box represents one of the top 4 most used authentication mechanisms. They are:
- SSH Keys Authentication: This mechanism uses a pair of public and private keys to authenticate a user to a server. The user generates a key pair and sends the public key to the server. The server stores the public key in a file called authorized_keys. When the user tries to log in, the server sends a challenge to the user, who uses the private key to sign it and send it back. The server verifies the signature using the public key and grants access if it matches.
- OAuth Tokens: This mechanism allows a user to grant access to their online resources (such as social media accounts) to a third-party application without sharing their credentials. The user requests an authorization code from the resource provider, which redirects them to the application. The application exchanges the authorization code for an access token, which it can use to access the user’s resources on their behalf.
- SSL Certificates: This mechanism uses digital certificates to establish a secure connection between a client and a server. The server sends its certificate to the client, which verifies its validity using a trusted certificate authority (CA). The client and the server then use the certificate to generate a symmetric encryption key, which they use to encrypt and decrypt the data they exchange.
- HTTP Credentials: This mechanism uses a username and password to authenticate a user to a web server. The user sends their credentials to the server using either a basic or a digest authentication scheme. The server checks the credentials against a database and grants access if they are valid.
I hope this helps you understand the image better. 
Do you have any questions about these authentication mechanisms?
Prompt:
Can you describe what’s in the 3rd box expressed using Mermaid Sequence Notation?
Response:
Sure, I can describe what’s in the 3rd box expressed using Mermaid Sequence Notation. Mermaid Sequence Notation is a way of writing text that can be converted into diagrams, such as sequence diagrams, flowcharts, and Gantt charts¹. A sequence diagram is a type of diagram that shows how different actors interact with each other over time².
The 3rd box in the image is a sequence diagram that explains the SSL Certificates mechanism. SSL Certificates are used to establish a secure connection between a client and a server³. The diagram has four actors: Client, Server, Certificate Authority (CA), and Symmetric Key. The diagram shows the following steps:
- The client sends a request to the server for a secure connection.
- The server sends its certificate to the client, which contains its public key and other information.
- The client verifies the validity of the certificate using a trusted CA, which is a third-party entity that issues and validates certificates.
- If the certificate is valid, the client and the server use the public key to generate a symmetric key, which is a secret key that they both share.
- The client and the server use the symmetric key to encrypt and decrypt the data they exchange.
Using Mermaid Sequence Notation, the 3rd box can be written as:
sequenceDiagram
Client->>Server: Request secure connection
Server->>Client: Send certificate
Client->>CA: Verify certificate
CA-->>Client: Certificate is valid
Client->>Server: Generate symmetric key
Client->>Symmetric Key: Encrypt data
Symmetric Key-->>Client: Decrypt data
Client-->>Server: Exchange data
Click HERE to open Live Mermaid Sequence Diagram.
I hope this helps you understand the 3rd box better. Do you have any questions about Mermaid Sequence Notation or SSL Certificates?
Source: Conversation with Bing, 8/9/2023
(1) Markdown Diagrams · GitHub. Markdown Diagrams · GitHub.
(2) Sequence Diagrams in Markdown with Mermaid.js - The New Dev’s Guide. Sequence Diagrams in Markdown with Mermaid.js.
(3) THIS IS AN AUTOGENERATED FILE. DO NOT EDIT. - GitHub. mermaid/docs/syntax/sequenceDiagram.md at develop · mermaid-js/mermaid · GitHub.
(4) Sequence diagrams - GitHub: Let’s build from here. mermaid-gitbook/content/sequenceDiagram.md at master · mermaidjs/mermaid-gitbook · GitHub.
(5) Blog - Use Mermaid syntax to create diagrams - draw.io. Blog - Use Mermaid syntax to create diagrams.
(6) undefined. https://viewscreen.githubusercontent.com.