> ## Documentation Index
> Fetch the complete documentation index at: https://togetherai-migration.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Python Library

> Learn how to run inference using our Python API.

In this tutorial, we will teach you how to use the Together Python library to run a chat model. We will be querying the `mistralai/Mixtral-8x7B-Instruct-v0.1` model to list out fun things to do in New York.

## Pre-requisites

* Ensure you have Python installed on your machine.
* Create a [free account](https://api.together.xyz/) to obtain a Together API Key, found in your [account settings](https://api.together.xyz/settings/api-keys).
* Install the Together library with `pip install --upgrade together`:

## Querying a chat model

The model we are using for this guide is `mistralai/Mixtral-8x7B-Instruct-v0.1`. You can browse all available models in [this list](/docs/inference-models). Now that we've chosen our model, let's query it using the Together Python SDK.

<CodeGroup>
  ```python Python theme={null}
  import os
  from together import Together

  client = Together(api_key=os.environ.get("TOGETHER_API_KEY"))

  response = client.chat.completions.create(
      model="mistralai/Mixtral-8x7B-Instruct-v0.1",
      messages=[{"role": "user", "content": "Tell me fun things to do in New York"}],
  )
  print(response.choices[0].message.content)
  ```
</CodeGroup>

### Output

<CodeGroup>
  ```text Output theme={null}
  Sure, here are some fun things to do in New York:

  1. Visit the Statue of Liberty and Ellis Island.
  2. Explore Central Park and visit the Central Park Zoo.
  3. Go shopping on Fifth Avenue.
  4. Visit the Metropolitan Museum of Art.
  5. Watch a Broadway show.
  6. Take a stroll across the Brooklyn Bridge.
  7. Visit the 9/11 Memorial and Museum.
  8. Go ice skating at Rockefeller Center.
  9. Take a ferry to Staten Island.
  10. Visit the American Museum of Natural History.
  11. Explore the neighborhoods of SoHo and Greenwich Village.
  12. Go on a food tour of New York's diverse culinary scene.
  13. Take a helicopter tour of the city.
  14. Visit the Empire State Building and take in the views from the observation deck.
  15. Go to a concert or sporting event at Madison Square Garden.
  16. Take a walking tour of Wall Street and the Financial District.
  17. Visit the Guggenheim Museum.
  18. Take a day trip to the beaches of Fire Island.
  19. Go on a boat ride around Manhattan.
  20. Visit the New York Public Library and see the famous Rose Main Reading Room.

  These are just a few ideas, but there are countless other fun things to do in New York!
  ```
</CodeGroup>

## Streaming tokens from a chat model

If you want to stream the response back, simply specify `stream=True`.

<CodeGroup>
  ```python Python theme={null}
  import os
  from together import Together

  client = Together(api_key=os.environ.get("TOGETHER_API_KEY"))
  stream = client.chat.completions.create(
      model="mistralai/Mixtral-8x7B-Instruct-v0.1",
      messages=[{"role": "user", "content": "tell me about new york"}],
      stream=True,
  )

  for chunk in stream:
      print(chunk.choices[0].delta.content or "", end="", flush=True)
  ```
</CodeGroup>

## Async support

If you want to run several calls in parallel, use our async method.

<CodeGroup>
  ```python Python theme={null}
  import os, asyncio
  from together import AsyncTogether

  async_client = AsyncTogether(api_key=os.environ.get("TOGETHER_API_KEY"))
  messages = [
      "What are the top things to do in San Francisco?",
      "What country is Paris in?",
  ]

  async def async_chat_completion(messages):
      async_client = AsyncTogether(api_key=os.environ.get("TOGETHER_API_KEY"))
      tasks = [
          async_client.chat.completions.create(
              model="mistralai/Mixtral-8x7B-Instruct-v0.1",
              messages=[{"role": "user", "content": message}],
          )
          for message in messages
      ]
      responses = await asyncio.gather(*tasks)

      for response in responses:
          print(response.choices[0].message.content)

  asyncio.run(async_chat_completion(messages))
  ```
</CodeGroup>

## Querying a completion model

Completions are for code and language models [shown here](/docs/inference-models).

<CodeGroup>
  ```python Python theme={null}
  import os
  from together import Together

  client = Together(api_key=os.environ.get("TOGETHER_API_KEY"))

  response = client.completions.create(
      model="codellama/CodeLlama-34b-Python-hf",
      prompt="def bubbleSort(): ",
  )
  print(response.choices[0].text)
  ```
</CodeGroup>

## Querying an image model

To query an image model, use the `.images` method and specify the image model you want to use.

<CodeGroup>
  ```python Python theme={null}
  import os
  from together import Together

  client = Together(api_key=os.environ.get("TOGETHER_API_KEY"))

  response = client.images.generate(
      prompt="space robots",
      model="stabilityai/stable-diffusion-xl-base-1.0",
      steps=10,
      n=4,
  )
  print(response.data[0].b64_json)
  ```
</CodeGroup>

For your reference, here is a link to the [Python Library](https://github.com/togethercomputer/together-python).