AI Session Template Style Guide
Style Guide & Rationale
Below, you will find the guidelines and rationale used in DH session review newsletters. (Don’t worry, not all posts will be session reviews.)
Note that although the template is heavily influenced from traditional scientific paper formats, this approach is NOT meant to represent or impersonate any official scientific body.
The intent is only to provide a consistent, expected format for the reader to follow, and to ensure that I remember to provide all the appropriate context around a given session.
Finally, this style guide, and its matching generic template, are living documents which will evolve over time. If there are a ton of revisions, and version history matters, I will move the template out to GitHub or some other change tracking repository and link to it here.
Current Template Version - 0.9.3 - [Link]
AI Details: This section should display a clear, URL link chain which describes the AI Developer, the AI System, and the AI Version that was in play at the time of the chat session.
Engaging Entities: This section should list out explicitly who is in the chat logs. In almost all cases this will be The Digital Heretic and whatever the chat entity calls itself (ALICE, Assistant, etc), but sometimes we will be reviewing chats supplied by others, so it’s important to discern who is doing the talking.
Original Session Date: Date the session took place.
1: TL;DR
A summary of the paper's main findings and conclusions. This also serves to prequel the rest of the post, so that subscribers can determine if the topic is worth it to them to keep reading.
2: Aims
If the entire session or section of a session was pre-designed to evaluate a particular hypothesis, this section will list any explicit goals going in to the chat.
If there were no Aims going in (e.g. organically discovered interaction review) then Aims will direct the user to the Inbound Context section for details.
3: Methods
3.1: Inbound Context
This may not always be a clean set of goals going in - some session discussions could simply be triggered by an interesting unintended interaction during an unrelated conversation, and as such, context surrounding the spark should be described here.
This session should also link to any CALLBACK sessions from the past that may be related, which would provide further Inbound Context. These callbacks should also be included in the References section at the end.
3.2: Parameters / Expectations
From the goal or trigger described above, this usually leads to some additional pre-wiring that occurs before the discussion continues, to frame the conversation further. These setup interactions should be called out here.
3.3: Impersonated Positions / Personas
If the discussion is fluid, then this section should simply declare “Interacting as self”. But if the session initiates hypothetical debate, role-play, or other techniques that could be mistaken for actually held positions / beliefs, then this section will explicitly call them out.
In many cases, this should be self-evident from the Chat Log, but we reiterate them here to be crystal clear.
4: Results
Observed results from the session relative to the aims, goals, which are salient to the conclusions will be called out here.
5: Commentary / Discussion
The meat-and-potatoes. The forensic review notes citing the session logs and reactions to particular turns in the conversation.
Topics referenced in the Results and Conclusion sections should have entries here in the Discussion that offer reference points for the claims.
6: Conclusion
A summary of the Results, and Conclusions of the study, with feedback on how this relates back to AI behavior, ML training data, implications for humanity, etc.
7: Issues / Callouts
Here, we list out any risks, potential biases, misunderstandings, mistakes, etc observed post-mortem. These callouts will be used to improve future studies.
Observations
Observations of the raw chat log provided by the AI contributor, Cogitator Prime.
Issues/Callouts
Any issues/callouts on the AI/Human interaction worth noting.
Conclusion
This section is intended to review the quality of the AI interaction, not so much to pick apart the merit of the topics discussed.
Opinion
This section occurs as a separate interaction, where the essential question posed by the session is asked of Cogitator for their opinion.
8. References
Various footnotes for more information on topics covered.
8.1 The Session Log
Every session will have a link to download the raw log, if available. However, the logs can and will be edited as necessary to correct any formatting issues, or to mask any sensitive information that would lead to accidental doxxing, etc. These edits will be done ethically and will never alter the actual substance of the conversation.
For example, in the current GP3 web version (Jan 9), when you highlight interactions and paste them out into a text editor, there are two problems:
It pastes the email address of the user preceding each prompt. So, for privacy purposes, the email address will be swapped out with either “Me:”, or whatever “Session Entity” nickname was responsible for conducting the chat.
Second, it doesn’t paste anything that identifies responses. GPT replies simply begin on a new line immediately after the prompt text. Therefore, these responses will be adjusted to include a “Chat:” identifier, or whatever nickname is appropriate for the AI entity (ALICE, Alexa, Siri, etc)
One final format possibility will be callouts to any External Action that may have occurred during the session. These actions will be called out inline with the discussion with clear delineation like “————— Refresh Clicked —————”, and may include things like:
Calling out Error messages that occur in the middle of the session that may not be clear in the chat.
Swapping to new chat windows if a particular test requires trying things in a new, fresh, chat.
Clicking buttons such as “Regenerate Response” or reloading the web page.
8.1 Other External References
This section will contain a list of sources referenced in the chat logs, particularly if they reference acronyms, inside jokes, callbacks, or other topics that may not be clear to the reader or otherwise explained in the chat.
Often, this is mitigated in systems such as ChatGPT, since the bot already has the tendency to reiterate prompts and expand on acronyms, so this section will not seek to contain a comprehensive list of all concepts covered.
Changelog
0.9.3 -
Added section for Cogitator Prime contributions.
0.9.2 -
Implementation of this Changelog section for traceability
Expand “Discussion” section to “Commentary / Discussion” as the forensic walkthrough style will often take more of play by play approach.
Formatting correction to “Issues / Callouts” spacing (was “Issues/Callouts”)
0.9.1 -
Link added to a permanent “bare bones” template page which includes all the sections mentioned here, but without the verbiage, for easier reuse.
Minor Grammar and formatting corrections.