Integrations

Use Together AI models through partner integrations.

HuggingFace

You can use Together AI models with Hugging Face Inference.

Install the huggingface_hub library:

Python

pip install huggingface_hub>=0.29.0

Chat Completion with Hugging Face Hub library

Python

from huggingface_hub import InferenceClient

# Initialize the InferenceClient with together as the provider

client = InferenceClient(
    provider="together",
    api_key="xxxxxxxxxxxxxxxxxxxxxxxx"  # Replace with your API key (HF or custom)
)

# Define the chat messages

messages = [
    {
        "role": "user",
        "content": "What is the capital of France?"
    }
]

# Generate a chat completion

completion = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-R1",
    messages=messages,
    max_tokens=500
)

# Print the response

print(completion.choices[0].message)

Learn more in our Together AI - HuggingFace Guide .

Vercel AI SDK

The Vercel AI SDK is a powerful Typescript library designed to help developers build AI-powered applications.

Install both the Vercel AI SDK and OpenAI’s Vercel package.

Shell

npm i ai @ai-sdk/openai

Instantiate the Together client and call the generateText function with Llama 3.1 8B to generate some text.

TypeScript

import { createOpenAI } from "@ai-sdk/openai";
import { generateText } from "ai";

const together = createOpenAI({
  apiKey: process.env.TOGETHER_API_KEY ?? "",
  baseURL: "https://api.together.xyz/v1",
});

async function main() {
  const { text } = await generateText({
    model: together("meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo"),
    prompt: "Write a vegetarian lasagna recipe for 4 people.",
  });

  console.log(text);
}

main();

Learn more in our Together AI - Vercel AI SDK Guide .

Langchain

LangChain is a framework for developing context-aware, reasoning applications powered by language models.

To install the LangChain x Together library, run:

Shell

pip install --upgrade langchain-together

Here’s sample code to get you started with Langchain + Together AI:

Python

from langchain_together import ChatTogether

chat = ChatTogether(model="meta-llama/Llama-3-70b-chat-hf")

for m in chat.stream("Tell me fun things to do in NYC"):
    print(m.content, end="", flush=True)

See this tutorial blog for the RAG implementation details using Together and LangChain.

• LangChain TogetherEmbeddings
• LangChain Together

LlamaIndex

LlamaIndex is a simple, flexible data framework for connecting custom data sources to large language models (LLMs).

Install llama-index

Shell

pip install llama-index

Here’s sample code to get you started with Llama Index + Together AI:

Python

from llama_index.llms import OpenAILike

llm = OpenAILike(
    model="mistralai/Mixtral-8x7B-Instruct-v0.1",
    api_base="https://api.together.xyz/v1",
    api_key="TOGETHER_API_KEY",
    is_chat_model=True,
    is_function_calling_model=True,
    temperature=0.1,
)

response = llm.complete("Write up to 500 words essay explaining Large Language Models")

print(response)

See this tutorial blog for the RAG implementation details using Together and LlamaIndex.

• LlamaIndex TogetherEmbeddings
• LlamaIndex TogetherLLM

CrewAI

CrewAI is an open source framework for orchestrating AI agent systems.

Install crewai

shell

pip install crewai
export TOGETHER_API_KEY=***

Build an multi-agent workflow:

Python

import os
from crewai import LLM, Task, Agent, Crew

llm = LLM(model="together_ai/meta-llama/Llama-3.3-70B-Instruct-Turbo",
          api_key=os.environ.get("TOGETHER_API_KEY"),
          base_url="https://api.together.xyz/v1"
        )

research_agent = Agent(
    llm = llm,
    role="Research Analyst",
    goal="Find and summarize information about specific topics",
    backstory="You are an experienced researcher with attention to detail",
    verbose=True  # Enable logging for debugging
)

research_task = Task(
    description="Conduct a thorough research about AI Agents.",
    expected_output="A list with 10 bullet points of the most relevant information about AI Agents",
    agent=research_agent
)

# Execute the crew
crew = Crew(
    agents=[research_agent],
    tasks=[research_task],
    verbose=True
)

result = crew.kickoff()

# Accessing the task output
task_output = research_task.output

print(task_output)

Learn more in our CrewAI guide.

LangGraph

LangGraph is an OSS library for building stateful, multi-actor applications with LLMs

Install langgraph

Shell

pip install -U langgraph langchain-together
export TOGETHER_API_KEY=***

Build a tool-using agent:

Python

import os
from langchain_together import ChatTogether

llm = ChatTogether(model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
                   api_key=os.getenv("TOGETHER_API_KEY"))

# Define a tool
def multiply(a: int, b: int) -> int:
    return a * b

# Augment the LLM with tools
llm_with_tools = llm.bind_tools([multiply])

# Invoke the LLM with input that triggers the tool call
msg = llm_with_tools.invoke("What is 2 times 3?")

# Get the tool call
msg.tool_calls

Learn more in our LangGraph Guide including code notebooks.

PydanticAI

PydanticAI is an agent framework created by the Pydantic team to simplify building agent workflows.

Install pydantic-ai

Shell

pip install pydantic-ai
export TOGETHER_API_KEY=***

Build PydanticAI agents using Together AI models

Python

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIModel
from pydantic_ai.providers.openai import OpenAIProvider

# Connect PydanticAI to LLMs on Together
model = OpenAIModel('meta-llama/Llama-3.3-70B-Instruct-Turbo',
                    provider=OpenAIProvider(
  															base_url="https://api.together.xyz/v1",
                                api_key=os.environ.get("TOGETHER_API_KEY"),
                              ),
                   )

# Setup the agent
agent = Agent(
  					model,
  					system_prompt='Be concise, reply with one sentence.',
)

result = agent.run_sync('Where does "hello world" come from?')
print(result.data)

Learn more in our PydanticAI Guide including code a notebook.

DSPy

DSPy is a framework that enables you to build modular AI systems with code instead of hand-crafted prompting

Install dspy

Shell

pip install -U dspy
export TOGETHER_API_KEY=***

Build a question answering agent

Python

import dspy

#Configure dspy with a LLM from Together AI
lm = dspy.LM('together_ai/togethercomputer/llama-2-70b-chat',
             api_key=os.environ.get("TOGETHER_API_KEY"),
             api_base="https://api.together.xyz/v1")

#Configure dspy to use the LLM
dspy.configure(lm=lm)

# Gives the agent access to a python interpreter
def evaluate_math(expression: str):
    return dspy.PythonInterpreter({}).execute(expression)

# Gives the agent access to a wikipedia search tool
def search_wikipedia(query: str):

    results = dspy.ColBERTv2(url='http://20.102.90.50:2017/wiki17_abstracts')(query, k=3)
    return [x['text'] for x in results]

# setup ReAct module with question and math answer signature
react = dspy.ReAct("question -> answer: float", tools=[evaluate_math, search_wikipedia])

pred = react(question="What is 9362158 divided by the year of birth of David Gregory of Kinnairdy castle?")

print(pred.answer)

Learn more in our DSPy Guide including code a notebook.

AutoGen(AG2)

AG2 (formerly AutoGen) is an open-source framework for building and orchestrating AI agents.

Install autogen

Shell

pip install autogen
export TOGETHER_API_KEY=***

Build a coding agent

Python

import os
from pathlib import Path
from autogen import AssistantAgent, UserProxyAgent
from autogen.coding import LocalCommandLineCodeExecutor

config_list = [
    {
        # Let's choose the Mixtral 8x7B model
        "model": "mistralai/Mixtral-8x7B-Instruct-v0.1",
        # Provide your Together.AI API key here or put it into the TOGETHER_API_KEY environment variable.
        "api_key": os.environ.get("TOGETHER_API_KEY"),
        # We specify the API Type as 'together' so it uses the Together.AI client class
        "api_type": "together",
        "stream": False,
    }
]

# Setting up the code executor
workdir = Path("coding")
workdir.mkdir(exist_ok=True)
code_executor = LocalCommandLineCodeExecutor(work_dir=workdir)

# Setting up the agents

# The UserProxyAgent will execute the code that the AssistantAgent provides
user_proxy_agent = UserProxyAgent(
    name="User",
    code_execution_config={"executor": code_executor},
    is_termination_msg=lambda msg: "FINISH" in msg.get("content"),
)

system_message = """You are a helpful AI assistant who writes code and the user executes it.
Solve tasks using your coding and language skills.
"""

# The AssistantAgent, using Together.AI's Code Llama model, will take the coding request and return code
assistant_agent = AssistantAgent(
    name="Together Assistant",
    system_message=system_message,
    llm_config={"config_list": config_list},
)

# Start the chat, with the UserProxyAgent asking the AssistantAgent the message
chat_result = user_proxy_agent.initiate_chat(
    assistant_agent,
    message="Provide code to count the number of prime numbers from 1 to 10000.",
)

Learn more in our Autogen Guide including code a notebook.

Agno

Agno is an open-source library for creating multimodal agents.

Install agno

Shell

pip install -U agno duckduckgo-search

Build a search and answer agent

python

from agno.agent import Agent
from agno.models.together import Together
from agno.tools.duckduckgo import DuckDuckGoTools

agent = Agent(
    model=Together(id="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo"),
    tools=[DuckDuckGoTools()],
    markdown=True
)
agent.print_response("What's happening in New York?", stream=True)

Learn more in our Agno Guide including code a notebook.

MongoDB

See this tutorial blog for the RAG implementation details using Together and MongoDB.

Pinecone

Pinecone is a vector database that helps companies build RAG applications.

Here’s some sample code to get you started with Pinecone + Together AI:

Python

from pinecone import Pinecone, ServerlessSpec
from together import Together

pc = Pinecone(
  api_key="PINECONE_API_KEY",
  source_tag="TOGETHER_AI"
)
client = Together()

# Create an index in pinecone
index = pc.create_index(
    name="serverless-index",
    dimension=1536,
    metric="cosine",
    spec=ServerlessSpec(cloud="aws", region="us-west-2"),
)

# Create an embedding on Together AI
textToEmbed = "Our solar system orbits the Milky Way galaxy at about 515,000 mph"
embeddings = client.embeddings.create(
    model="togethercomputer/m2-bert-80M-8k-retrieval",
  	input=textToEmbed
)

# Use index.upsert() to insert embeddings and index.query() to query for similar vectors

Helicone

Helicone is an open source LLM observability platform.

Here’s some sample code to get started with using Helicone + Together AI:

Python

import os
from together import Together

client = Together(
    api_key=os.environ.get("TOGETHER_API_KEY"),
    base_url="https://together.hconeai.com/v1",
    supplied_headers={
        "Helicone-Auth": f"Bearer {os.environ.get('HELICONE_API_KEY')}",
    },
)

stream = client.chat.completions.create(
    model="meta-llama/Llama-3-8b-chat-hf",
    messages=[
        {"role": "user", "content": "What are some fun things to do in New York?"}
    ],
    stream=True,
)

for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

Composio

Composio allows developers to integrate external tools and services into their AI applications.

Install composio-togetherai

Shell

pip install together composio-togetherai
export TOGETHER_API_KEY=***
export COMPOSIO_API_KEY=***

Get Together AI models to use integrated tools

Python

from composio_togetherai import ComposioToolSet, App
from together import Together

client = Together()
toolset = ComposioToolSet()

request = toolset.initiate_connection(app=App.GITHUB)
print(f"Open this URL to authenticate: {request.redirectUrl}")

tools = toolset.get_tools(apps=[App.GITHUB])

response = client.chat.completions.create(
    tools=tools,
    model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
    messages=[
        {
            "role": "user",
            "content": "Star the repo 'togethercomputer/together-cookbook'",
        }
    ],
)

res = toolset.handle_tool_calls(response)
print(res)

Learn more in our Composio Guide including code a notebook.

Pixeltable

See this tutorial blog for the RAG implementation details using Together and Pixeltable.