Hello all, I am trying to see if it would be possible to use a local LLM and langchain/plugins/agents in the future to obtain the simple goal of a dynamitic, Ai powered single player roleplaying adventure game that is persistent. To obtain this goal, please see the list of requirements below.

1. 100K plus context tokens – The biggest setback currently for all current LLM is the limited context tokens or “long term memory” I read that there are some models that have up to 100k tokens (Claude) and currently ChatGPT 4 has 36K token content. This sounds very promising but the MPT 7B (65k Token) requires a lot of very expense graphics cards and is not really available. My current custom world (WIP) is 18,098 words long but more than likely be 50K to 100K words when completed.

2. Long Term Memory – I read that the vector data setup is great for pulling context and summarizing to fit into the limited 2K (Local LMM) 8k? (Chatgpt 4 API) This would be great for saving the actual events in the adventure game while it starts and I pulling from the custom world source material to create output prompts. If there is a “good” local LLM with a high context token amount (above 100K) this could be possible (Maybe?) to have a sort of “long term memory”? Private GPT like tool is also great too. You can just throw the 100K word document into the system on top of the 100K plus context tokens.

3. Access to Plugins – As we know most if not all LLM and especially Local LLMs are terrible with math. When would it be possible to Local LLM to actually be proficient in math. The custom world has a lot of formulas for attack, damage, evade percentage and etc. that are a lot more complicated than the DnD 5th edition. Wolfram works well for ChatGPT 4. Zapier would be a great plugin too. For example, all NPCs will have custom stats, personalities and items/equipment. Would it be possible to use plugins/agents to edit a google doc or saved word doc file to record all of these custom NPC character sheets? It would be even better if it is encrypted so the player does not know this information. This would also work with a “save file” as the plugin agent would summarize the adventure story in a google doc or word doc file. The last doc would be the player character sheet and inventory. All of these docs could be pulled by the vector data for context to progress the story. Would it be possible for Local LLM to have access to plugins in the future?

4. NPC Agents – I am not too familiar with LLM agents but would it be possible to have main NPCs in your party to be controlled by NPC agents that are separate from the story teller. These NPC agents will have their own personalities and react to the story teller prompts as the player could.

5. Dynamic and Uncensored story teller – I tried to do many experiments and trial runs with ChatGPT 4 and the censored version really puts a hamper in medieval fantasy role playing. I am not even talking about erotic content. I cannot even roleplay murdering a merchant that I robbed as “He was unarmed and he gave you your gold” I am not asking for much, just a dynamic, creative and uncensored story teller for immersion sake. It is also very important the storyteller doesn’t railroad you and listens to instructions well to prompt you for each action or choices you make.

6. Text to Chat and Text to Image – I think that the Local LLM already have Text to Chat but it would also be cool to have Text to Image included. When the story teller replies with a prompt of the environment, it would be cool if it could also share an image of the prompt for immersion. Stable Diffusion is a good open source tool, would it be possible in the future to have Local LLM API that SD could create an image from the prompt?

Technical Requirements – I currently have pretty decent specs on my computer but I am willing to throw down big bucks if all of these requirements could be ran on one of those A100 or H100 graphics card or a more affordable gaming graphics card or the most expense CPUs for AI. I know that the advancements are phenomenal and I would like to thank all those who worked very hard to bring updates and new models.

TL/DR

When do you think it would be possible to have an Ai powered persistent single player roleplaying adventure game in a customized world that could fulfil all of those requirements (Or alternative better solutions) in order to run this adventure roleplay game? Thank you all in advance!