> ## Documentation Index
> Fetch the complete documentation index at: https://togetherai-migration.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Integrations

> Use Together AI models through partner integrations.

## HuggingFace

*You can use Together AI models with Hugging Face Inference.*

Install the `huggingface_hub` library:

<CodeGroup>
  ```python Python theme={null}
  pip install huggingface_hub>=0.29.0
  ```

  ```typescript typescript theme={null}
  npm install @huggingface/inference
  ```
</CodeGroup>

Chat Completion with Hugging Face Hub library

<CodeGroup>
  ```python Python theme={null}
  from huggingface_hub import InferenceClient

  # Initialize the InferenceClient with together as the provider

  client = InferenceClient(
      provider="together",
      api_key="xxxxxxxxxxxxxxxxxxxxxxxx"  # Replace with your API key (HF or custom)
  )

  # Define the chat messages

  messages = [
      {
          "role": "user",
          "content": "What is the capital of France?"
      }
  ]

  # Generate a chat completion

  completion = client.chat.completions.create(
      model="deepseek-ai/DeepSeek-R1",
      messages=messages,
      max_tokens=500
  )

  # Print the response

  print(completion.choices[0].message)
  ```

  ```typescript TypeScript theme={null}
  import { HfInference } from "@huggingface/inference";

  // Initialize the HfInference client with your API key
  const client = new HfInference("xxxxxxxxxxxxxxxxxxxxxxxx");

  // Generate a chat completion
  const chatCompletion = await client.chatCompletion({
      model: "deepseek-ai/DeepSeek-R1",  // Replace with your desired model
      messages: [
          {
              role: "user",
              content: "What is the capital of France?"
          }
      ],
      provider: "together",  // Replace with together's provider name
      max_tokens: 500
  });

  // Log the response
  console.log(chatCompletion.choices[0].message);
  ```
</CodeGroup>

Learn more in our [Together AI - HuggingFace Guide](/docs/quickstart-using-hugging-face-inference) .

## Vercel AI SDK

*The Vercel AI SDK is a powerful Typescript library designed to help developers build AI-powered applications.*

Install both the Vercel AI SDK and OpenAI's Vercel package.

<CodeGroup>
  ```shell Shell theme={null}
  npm i ai @ai-sdk/openai
  ```
</CodeGroup>

Instantiate the Together client and call the generateText function with Llama 3.1 8B to generate some text.

<CodeGroup>
  ```typescript TypeScript theme={null}
  import { createOpenAI } from "@ai-sdk/openai";
  import { generateText } from "ai";

  const together = createOpenAI({
    apiKey: process.env.TOGETHER_API_KEY ?? "",
    baseURL: "https://api.together.xyz/v1",
  });

  async function main() {
    const { text } = await generateText({
      model: together("meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo"),
      prompt: "Write a vegetarian lasagna recipe for 4 people.",
    });

    console.log(text);
  }

  main();
  ```
</CodeGroup>

Learn more in our [Together AI - Vercel AI SDK Guide](/docs/using-together-with-vercels-ai-sdk) .

## Langchain

*LangChain is a framework for developing context-aware, reasoning applications powered by language models.*

To install the LangChain x Together library, run:

<CodeGroup>
  ```text Shell theme={null}
  pip install --upgrade langchain-together
  ```
</CodeGroup>

Here's sample code to get you started with Langchain + Together AI:

<CodeGroup>
  ```python Python theme={null}
  from langchain_together import ChatTogether

  chat = ChatTogether(model="meta-llama/Llama-3-70b-chat-hf")

  for m in chat.stream("Tell me fun things to do in NYC"):
      print(m.content, end="", flush=True)
  ```
</CodeGroup>

See [this tutorial blog](https://www.together.ai/blog/rag-tutorial-langchain?_gl=1*exkmyi*_gcl_au*MTA3NDk3OTU0MS4xNzM3OTk4MjUw*_ga*MTg5NTkzNDM0LjE3MjgzMzM2MDQ.*_ga_BS43X21GZ2*MTc0NTQ1ODY4OC44MC4xLjE3NDU0NjY2ODYuMC4wLjA.*_ga_BBHKJ5V8S0*MTc0NTQ1ODY4OC42OS4xLjE3NDU0NjY2ODYuMC4wLjA.) for the RAG implementation details using Together and LangChain.

* [LangChain TogetherEmbeddings](https://python.langchain.com/docs/integrations/text_embedding/together)
* [LangChain Together](https://python.langchain.com/docs/integrations/llms/together)

## LlamaIndex

*LlamaIndex is a simple, flexible data framework for connecting custom data sources to large language models (LLMs).*

Install `llama-index`

<CodeGroup>
  ```shell Shell theme={null}
  pip install llama-index
  ```
</CodeGroup>

Here's sample code to get you started with Llama Index + Together AI:

<CodeGroup>
  ```python Python theme={null}
  from llama_index.llms import OpenAILike

  llm = OpenAILike(
      model="mistralai/Mixtral-8x7B-Instruct-v0.1",
      api_base="https://api.together.xyz/v1",
      api_key="TOGETHER_API_KEY",
      is_chat_model=True,
      is_function_calling_model=True,
      temperature=0.1,
  )

  response = llm.complete("Write up to 500 words essay explaining Large Language Models")

  print(response)
  ```
</CodeGroup>

See [this tutorial blog](https://www.together.ai/blog/rag-tutorial-llamaindex?_gl=1*1t16mh2*_gcl_au*MTA3NDk3OTU0MS4xNzM3OTk4MjUw*_ga*MTg5NTkzNDM0LjE3MjgzMzM2MDQ.*_ga_BS43X21GZ2*MTc0NTQ1ODY4OC44MC4xLjE3NDU0NjY2ODYuMC4wLjA.*_ga_BBHKJ5V8S0*MTc0NTQ1ODY4OC42OS4xLjE3NDU0NjY2ODYuMC4wLjA.) for the RAG implementation details using Together and LlamaIndex.

* [LlamaIndex TogetherEmbeddings](https://docs.llamaindex.ai/en/stable/examples/embeddings/together.html)
* [LlamaIndex TogetherLLM](https://docs.llamaindex.ai/en/stable/examples/llm/together.html)

## CrewAI

*CrewAI is an open source framework for orchestrating AI agent systems.*

Install `crewai`

<CodeGroup>
  ```shell shell theme={null}
  pip install crewai
  export TOGETHER_API_KEY=***
  ```
</CodeGroup>

Build an multi-agent workflow:

<CodeGroup>
  ```python Python theme={null}
  import os
  from crewai import LLM, Task, Agent, Crew

  llm = LLM(model="together_ai/meta-llama/Llama-3.3-70B-Instruct-Turbo",
            api_key=os.environ.get("TOGETHER_API_KEY"),
            base_url="https://api.together.xyz/v1"
          )

  research_agent = Agent(
      llm = llm,
      role="Research Analyst",
      goal="Find and summarize information about specific topics",
      backstory="You are an experienced researcher with attention to detail",
      verbose=True  # Enable logging for debugging
  )

  research_task = Task(
      description="Conduct a thorough research about AI Agents.",
      expected_output="A list with 10 bullet points of the most relevant information about AI Agents",
      agent=research_agent
  )

  # Execute the crew
  crew = Crew(
      agents=[research_agent],
      tasks=[research_task],
      verbose=True
  )

  result = crew.kickoff()

  # Accessing the task output
  task_output = research_task.output

  print(task_output)
  ```
</CodeGroup>

Learn more in our [CrewAI guide](/docs/crewai).

## LangGraph

*LangGraph is an OSS library for building stateful, multi-actor applications with LLMs*

Install `langgraph`

<CodeGroup>
  ```shell Shell theme={null}
  pip install -U langgraph langchain-together
  export TOGETHER_API_KEY=***
  ```
</CodeGroup>

Build a tool-using agent:

<CodeGroup>
  ```python Python theme={null}
  import os
  from langchain_together import ChatTogether

  llm = ChatTogether(model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
                     api_key=os.getenv("TOGETHER_API_KEY"))

  # Define a tool
  def multiply(a: int, b: int) -> int:
      return a * b

  # Augment the LLM with tools
  llm_with_tools = llm.bind_tools([multiply])

  # Invoke the LLM with input that triggers the tool call
  msg = llm_with_tools.invoke("What is 2 times 3?")

  # Get the tool call
  msg.tool_calls
  ```
</CodeGroup>

Learn more in our [LangGraph Guide](/docs/langgraph) including code notebooks.

## PydanticAI

*PydanticAI is an agent framework created by the Pydantic team to simplify building agent workflows.*

Install `pydantic-ai`

<CodeGroup>
  ```shell Shell theme={null}
  pip install pydantic-ai
  export TOGETHER_API_KEY=***
  ```
</CodeGroup>

Build PydanticAI agents using Together AI models

<CodeGroup>
  ```python Python theme={null}
  from pydantic_ai import Agent
  from pydantic_ai.models.openai import OpenAIModel
  from pydantic_ai.providers.openai import OpenAIProvider

  # Connect PydanticAI to LLMs on Together
  model = OpenAIModel('meta-llama/Llama-3.3-70B-Instruct-Turbo',
                      provider=OpenAIProvider(
    															base_url="https://api.together.xyz/v1",
                                  api_key=os.environ.get("TOGETHER_API_KEY"),
                                ),
                     )

  # Setup the agent
  agent = Agent(
    					model,
    					system_prompt='Be concise, reply with one sentence.',
  )

  result = agent.run_sync('Where does "hello world" come from?')
  print(result.data)
  ```
</CodeGroup>

Learn more in our [PydanticAI Guide](/docs/pydanticai) including code a notebook.

## DSPy

*DSPy is a framework that enables you to build modular AI systems with code instead of hand-crafted prompting*

Install `dspy`

<CodeGroup>
  ```shell Shell theme={null}
  pip install -U dspy
  export TOGETHER_API_KEY=***
  ```
</CodeGroup>

Build a question answering agent

<CodeGroup>
  ```python Python theme={null}
  import dspy

  #Configure dspy with a LLM from Together AI
  lm = dspy.LM('together_ai/togethercomputer/llama-2-70b-chat',
               api_key=os.environ.get("TOGETHER_API_KEY"),
               api_base="https://api.together.xyz/v1")

  #Configure dspy to use the LLM
  dspy.configure(lm=lm)

  # Gives the agent access to a python interpreter
  def evaluate_math(expression: str):
      return dspy.PythonInterpreter({}).execute(expression)

  # Gives the agent access to a wikipedia search tool
  def search_wikipedia(query: str):

      results = dspy.ColBERTv2(url='http://20.102.90.50:2017/wiki17_abstracts')(query, k=3)
      return [x['text'] for x in results]

  # setup ReAct module with question and math answer signature
  react = dspy.ReAct("question -> answer: float", tools=[evaluate_math, search_wikipedia])

  pred = react(question="What is 9362158 divided by the year of birth of David Gregory of Kinnairdy castle?")

  print(pred.answer)
  ```
</CodeGroup>

Learn more in our [DSPy Guide](/docs/dspy) including code a notebook.

## AutoGen(AG2)

*AG2 (formerly AutoGen) is an open-source framework for building and orchestrating AI agents.*

Install `autogen`

<CodeGroup>
  ```shell Shell theme={null}
  pip install autogen
  export TOGETHER_API_KEY=***
  ```
</CodeGroup>

Build a coding agent

<CodeGroup>
  ```python Python theme={null}
  import os
  from pathlib import Path
  from autogen import AssistantAgent, UserProxyAgent
  from autogen.coding import LocalCommandLineCodeExecutor

  config_list = [
      {
          # Let's choose the Mixtral 8x7B model
          "model": "mistralai/Mixtral-8x7B-Instruct-v0.1",
          # Provide your Together.AI API key here or put it into the TOGETHER_API_KEY environment variable.
          "api_key": os.environ.get("TOGETHER_API_KEY"),
          # We specify the API Type as 'together' so it uses the Together.AI client class
          "api_type": "together",
          "stream": False,
      }
  ]

  # Setting up the code executor
  workdir = Path("coding")
  workdir.mkdir(exist_ok=True)
  code_executor = LocalCommandLineCodeExecutor(work_dir=workdir)

  # Setting up the agents

  # The UserProxyAgent will execute the code that the AssistantAgent provides
  user_proxy_agent = UserProxyAgent(
      name="User",
      code_execution_config={"executor": code_executor},
      is_termination_msg=lambda msg: "FINISH" in msg.get("content"),
  )

  system_message = """You are a helpful AI assistant who writes code and the user executes it.
  Solve tasks using your coding and language skills.
  """

  # The AssistantAgent, using Together.AI's Code Llama model, will take the coding request and return code
  assistant_agent = AssistantAgent(
      name="Together Assistant",
      system_message=system_message,
      llm_config={"config_list": config_list},
  )

  # Start the chat, with the UserProxyAgent asking the AssistantAgent the message
  chat_result = user_proxy_agent.initiate_chat(
      assistant_agent,
      message="Provide code to count the number of prime numbers from 1 to 10000.",
  )
  ```
</CodeGroup>

Learn more in our [Autogen Guide](/docs/autogen) including code a notebook.

## Agno

*Agno is an open-source library for creating multimodal agents.*

Install `agno`

<CodeGroup>
  ```shell Shell theme={null}
  pip install -U agno duckduckgo-search
  ```
</CodeGroup>

Build a search and answer agent

<CodeGroup>
  ```python python theme={null}
  from agno.agent import Agent
  from agno.models.together import Together
  from agno.tools.duckduckgo import DuckDuckGoTools

  agent = Agent(
      model=Together(id="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo"),
      tools=[DuckDuckGoTools()],
      markdown=True
  )
  agent.print_response("What's happening in New York?", stream=True)
  ```
</CodeGroup>

Learn more in our [Agno Guide](/docs/agno) including code a notebook.

## MongoDB

See [this tutorial blog](https://www.together.ai/blog/rag-tutorial-mongodb?_gl=1*13iu8zj*_gcl_au*MTA3NDk3OTU0MS4xNzM3OTk4MjUw*_ga*MTg5NTkzNDM0LjE3MjgzMzM2MDQ.*_ga_BS43X21GZ2*MTc0NTQ1ODY4OC44MC4xLjE3NDU0NjY2ODYuMC4wLjA.*_ga_BBHKJ5V8S0*MTc0NTQ1ODY4OC42OS4xLjE3NDU0NjY2ODYuMC4wLjA.) for the RAG implementation details using Together and MongoDB.

## Pinecone

*Pinecone is a vector database that helps companies build RAG applications.*

Here's some sample code to get you started with Pinecone + Together AI:

<CodeGroup>
  ```python Python theme={null}
  from pinecone import Pinecone, ServerlessSpec
  from together import Together

  pc = Pinecone(
    api_key="PINECONE_API_KEY",
    source_tag="TOGETHER_AI"
  )
  client = Together()

  # Create an index in pinecone
  index = pc.create_index(
      name="serverless-index",
      dimension=1536,
      metric="cosine",
      spec=ServerlessSpec(cloud="aws", region="us-west-2"),
  )

  # Create an embedding on Together AI
  textToEmbed = "Our solar system orbits the Milky Way galaxy at about 515,000 mph"
  embeddings = client.embeddings.create(
      model="togethercomputer/m2-bert-80M-8k-retrieval",
    	input=textToEmbed
  )

  # Use index.upsert() to insert embeddings and index.query() to query for similar vectors
  ```
</CodeGroup>

## Helicone

*Helicone is an open source LLM observability platform.*

Here's some sample code to get started with using Helicone + Together AI:

<CodeGroup>
  ```python Python theme={null}
  import os
  from together import Together

  client = Together(
      api_key=os.environ.get("TOGETHER_API_KEY"),
      base_url="https://together.hconeai.com/v1",
      supplied_headers={
          "Helicone-Auth": f"Bearer {os.environ.get('HELICONE_API_KEY')}",
      },
  )

  stream = client.chat.completions.create(
      model="meta-llama/Llama-3-8b-chat-hf",
      messages=[
          {"role": "user", "content": "What are some fun things to do in New York?"}
      ],
      stream=True,
  )

  for chunk in stream:
      print(chunk.choices[0].delta.content or "", end="", flush=True)
  ```
</CodeGroup>

## Composio

*Composio allows developers to integrate external tools and services into their AI applications.*

Install `composio-togetherai`

<CodeGroup>
  ```shell Shell theme={null}
  pip install together composio-togetherai
  export TOGETHER_API_KEY=***
  export COMPOSIO_API_KEY=***
  ```
</CodeGroup>

Get Together AI models to use integrated tools

<CodeGroup>
  ```python Python theme={null}
  from composio_togetherai import ComposioToolSet, App
  from together import Together

  client = Together()
  toolset = ComposioToolSet()

  request = toolset.initiate_connection(app=App.GITHUB)
  print(f"Open this URL to authenticate: {request.redirectUrl}")

  tools = toolset.get_tools(apps=[App.GITHUB])

  response = client.chat.completions.create(
      tools=tools,
      model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
      messages=[
          {
              "role": "user",
              "content": "Star the repo 'togethercomputer/together-cookbook'",
          }
      ],
  )

  res = toolset.handle_tool_calls(response)
  print(res)
  ```
</CodeGroup>

Learn more in our [Composio Guide](/docs/composio) including code a notebook.

## Pixeltable

See [this tutorial blog](/docs/embeddings-rag#:~:text=Using%20Pixeltable,Together%20and%20Pixeltable.) for the RAG implementation details using Together and Pixeltable.