Making retrieval really really good is part of the mission of LlamaIndex! Given a natural language input, find the best way to return a set of documents that is relevant to your LLM use case (question-answering, summarization, more complex queries too).
- We integrate with vector db's + ChatGPT Retrieval Plugin
- Submitted a Retrieval PR to langchain here: https://github.com/hwchase17/langchain/pull/2014
- would love to explore further integrations as a plugin in any outer agent system
I want to use llamaindex. My input would be a slack export but I don't want any data to go to openai I want it all to happen locally or within my own EC2 instance. I have seen https://github.com/jerryjliu/llama_index/blob/046183303da416... but it calls hugging face.
My plan was to use https://github.com/cocktailpeanut/dalai with the alpaca model then somehow use llamaindex to input my dataset - a slack export. But it's not too clear how to train the alpaca model.