Cool stuff. Feature: enable history to be a trailing contenxt of n messages.

[syonfox]

First of all nice project, I appreciate a clean cli template for js :)

I think a useful addition would be to allow history.

let messages = [] ;
let n = 5;
let limit = 2000;
onNewMessage(msg) {
  msg.count = gptoken.countTokens(msg.content);
  
  messages.push(msg);

  // prune the convo traling
  if(messages.length < n) {
    let trash = messages.shift();
  }
   
  let len = Math.sum(messages.map(m=>m.count));
  //enforce an input length limit in tokens
  while(len > limit) {
    let trash = messages.shift();
    len -= trash.count;
  }
}

//psudo 
user submit -> addMessage
submit messages -> get response -> addMessage
await Input

this adds to the requested cost but will improve performance. I think you can just pass in the array of messages. should be minimal tweak to API call

[syonfox]

One other idea is for long messages to get the ai to summarize and then save that as the history

[drorm]

History is already saved in a log file: https://github.com/drorm/gish/blob/main/README.md#log-file in JSON format, both the request and the response.
Do you mean, provide a way to retrieve it as in

gish -h

[syonfox]

Sorry for the confusion. That may be useful but I meant a way to support a trailing history context being passed with the requests to openai.

the problem is when engaging in an interactive season there is no history or context being fead to the model. It would be interesting to have a way of defining this.

For example a way of seating a system message and then also providing a trailing history.

For example options

--system "Please be a helpful ai assistant that is funny and tells bad jokes."
--history 5
--max-request 2000

this would result in the request being made with

body: {
  ...
  messages: [
    {role: "system", content: "Please be a helpful ai assistant that is funny and tells bad jokes."},
    {role: "user", content: "lets count one number at a time ill start 1"}
    {role: "assistant", content: "2"}
    {role: "user", content: "3"}
    {role: "assistant", content: "4"}
    {role: "user", content: "5, please summarize what we have done."}
  ]
}

...

Here is an example illustrating the problem with the lack of context.

> whats tha capital of america
As an AI language model, I must inform you that the capital of America is not a specific location as America refers to a continent and not a country. However, the capital of the United States of America is Washington, D.C.
Tokens: 53 Cost: $0.00011 Elapsed: 2.755 Seconds. Tokens and cost are estimates when streaming.
> whats the best thing to do there
As an AI language model, I don't know what you're referring to as "there" but here are some suggestions on what to do in general:
1. Spend time with loved ones - spend quality time with your family and friends or call them if you can't be with them in person.
2. Try a new hobby - pick up a new hobby that you've always wanted to try, such as painting, cooking or dance.
3. Exercise - get active and incorporate exercise into your daily routine to improve physical and mental health.
4. Learn something new - attend a workshop or take an online course to learn something new and stimulating.
5. Volunteer - help others and give back to your community by volunteering at a local organization or charity.
6. Travel - explore new places and cultures by traveling to a different city or country.
7. Relax and take some "me time"- take a break from your busy schedule and relax by reading, meditating or taking a relaxing bubble bath.

Hope this makes sense. I think the psudo code is close, needs to update the interactive request execution or somthing. The system feature would involve extra though.

[drorm]

Ah yes, It's on my roadmap. I was thinking of it as "chat mode". But pretty much the same idea.

I'll probably generalize it so that it'll take arguments like to a printer where you can specify either a range or specific pages to print. Need to think a bit about what the right syntax to support that.

[drorm]

On further thought, it feels like the UI for this could get complicated quite quickly.
What do you think of, instead to:

1. Provide a way to save the history to a file,
2. Use a file as the source to a conversation.
   So say you say gish -h 5, gish creates a file (using your example):

 messages: [
    {role: "user", content: "lets count one number at a time ill start 1"}
    {role: "assistant", content: "2"}
    {role: "user", content: "3"}
    {role: "assistant", content: "4"}
    {role: "user", content: "5, please summarize what we have done."}
  ]

You edit it to your heart's content, and when you exit, gish submits it.

Also, as far as summaries, my inclination is to summarize when needed rather than summarize everything so that we don't waste money on time unless it's needed. So for long content, there could be a summarizing step prior to the editing..

[syonfox]

json would be the most useful for a simple lib. then the end developer would be able to import in their lang of choice. In the editor mode that would be cool if it just happened a new blank message.

Ie on file save it submits and adds one message and the empt space for a new one..

This is where message trimming could be useful. but it could keep extending the file and only submit the last n_responses or max_request_tokens.

[syonfox]

Yeah, it is all an optimization problem at the end of the day. I think there is some threshold LEN such that it is cost-effective to summarize. I think the natural solution will be to get gpt4 access and just submit large texts to that to summarize into an ideal input for 3.5-turbo.

[drorm]

I migrated this conversation to a discussion, and created #5, the initial implementation that I'm looking at.

[drorm]

@syonfox this is now implemented in the dev branch. You're welcome to try it there or wait a bit, till I wrap up 0.2 and deploy it.
Thanks for asking for this. I thought I didn't personally did it, since from the CLI I just got back in the shell history and edit the request, but I am finding myself periodically adding "-c" to continue a chat.

[syonfox]

Sounds good I'll take a quick look if you have a moment to show me around the code what is the logic flow, and where is it implemented. I can just read it too :)

The main things that might be good documentation is

• how to add an option.

• how to add a new interactive mode/ implementation

• where is the history/ chat logic.

• dont spend too much time ill find my way around it eventually.

Is -c mode only one previous request?

My plan is to try to get the streaming to work in a cloud flare worker cleanly (ideally proxy the stream to the browser)

Anyway I think this implementation will be good for a base linux library talk more as things develop
thanks for the cool implementation.

[drorm]

I just release V0.2
The docs are all in the README.
Originally, I was planning on having -c or "chat" in interactive mode take an argument to include multiple history items, but then realized that the typical use case is either you know you're starting a chat, or you want to start a chat on the previous item. In both cases, each gish -c just keeps all the previous ones automatically.
In other words, the requests

10 miles to km
and 20
and 30

result in the last request showing up in the log as:

"messages": [
      {
        "role": "user",
        "content": "10 miles to km"
      },
      {
        "role": "assistant",
        "content": "16.09 kilometers"
      },
      {
        "role": "user",
        "content": "and 20"
      },
      {
        "role": "assistant",
        "content": "32.19 kilometers"
      },
      {
        "role": "user",
        "content": "and 30"
      },
      {
        "role": "assistant",
        "content": "48.28 kilometers"
      }
    ],

Because it carries the previous one which in turn carried the one before.
The log is in ~/.gish/history.json (documented in the README. Helpful to look at to see what gets sent.
Finally, I included some examples in the examples dir on how I use gish. I think it's cool :-).