[FIX]Agents stop outputting "Generating a well-informed response"

[MartinMayday]

Describe the bug

A clear and concise description of what the bug is. Please include what you were expecting to happen vs. what actually happened.
I want to continue a conversation from some days ago. but agent doesn't actually output "Generating a well-informed response"
I've tried:

• reloading browser,
• used another browser,
• /notes please summarise last session

To Reproduce

Steps to reproduce the behavior:
Find conversation in sidebar
continue conversations (user)
assistant *thinks', never outputs "Generating a well-informed response"

Screenshots

Platform

• Server:
  • Cloud-Hosted (https://app.khoj.dev)
  • Self-Hosted Docker
  • Self-Hosted Python package
  • Self-Hosted source code
• Client:
  • Obsidian
  • Emacs
  • [ x] Desktop app
  • Web browser
  • WhatsApp
• OS:
  • Windows
  • [ x] macOS
  • Linux
  • Android
  • iOS

If self-hosted

• Server Version [e.g. 1.0.1]:

Additional context

is there a way to explore logs of conversations?
is there a way to see how many MB sync files are uploaded? - obsidian keeps notifying something like "sync failed due to no more space". Do not know if its related to the issue above.

[ProjectMoon]

I am seeing this happen with ollama due to a timeout on the Khoj side. ollama logs report connection aborted. But everything in the train of thought seems to work up until the last point. Maybe the prompt is so big and the client (Khoj) is not waiting long enough?

[ProjectMoon]

I notice this issue went away immediately after I updated the model in ollama and the database to have a large context size, so it would stop truncating.

Edit: still get some timeout issues though, on subsequent messages. Also some Body already consumed errors in the browser console.

[ProjectMoon]

Also, for me, this seems to be specific to research mode enabled.

Edit: after further testing, this also seems to often be connected to going over the ollama context limit. The Khoj logs say it's truncating to the max prompt limit, but then there's a timeout anyway.

So basically, Khoj doesn't seem to be giving the OpenAI client enough time to ingest the prompt when it's huge.

[debanjum]

Hey @ProjectMoon, thanks for sharing a detailed report.

It maybe a short timeout window or truncation issue on the Khoj side.

Let me try reproduce this issue using Khoj with a small context model on Ollama.

Research mode does result in longer context requirements and hence being more likely to trigger the timeout issue.

If it's a timeout issue, increasing the timeout on decorator to openai api calling funcs in openai/utils.py should fix it (can use different timeouts for openai vs local models)

[ProjectMoon]

I did try editing the code to increase the timeouts. That lead to other strange behavior, which I detailed in the other issue. I'm not sure if I did it right though. I increased the 20 to 200 in gpt.py.

[ProjectMoon]

A few other things to note: I didn't get this behavior when using jina search API. Only when using Searxng and jina without an API key.